JP2013057761A - Distance measuring device, imaging device, and distance measuring method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a distance measuring device capable of measuring a distance by phase difference detection, to a difficult subject in which phase different detection based on pupil color division is hard.SOLUTION: A distance measuring device includes: a color imaging element 22; an imaging optical system 9; a band limiting filter 12 for dividing a band into at least 2 pieces when light passes through a pupil area of the imaging optical system 9; a distance calculating portion 37 for measuring a gravity center position of each band from an image obtained by the imaging element 22, and computing a subject distance on the basis of phase difference amount; a color correlation determining portion 38 for determining whether there is a correlation between images of the bands having different gravity center positions; a pupil position changing portion for changing the gravity center positions; and a system controller 30 for computing the subject distance from a pick-up image of one time when there is a correlation, and changing the gravity center positions and computing the subject distance from pick-up images of at least twice before/after the change when there is not a correlation.

Description

  The present invention relates to a distance measuring device, an imaging device, and a distance measuring method for measuring a distance to a subject based on images of a plurality of bands obtained from light of a plurality of bands that have passed through different pupil regions of an imaging optical system.

  The distance information is used for AF processing by an automatic focus adjustment (AF) mechanism, for creating a stereoscopic image, or for image processing (for example, subject extraction processing, background extraction processing, or blur amount control). The present invention can be used to realize various functions in the imaging apparatus, such as for performing image processing.

  Various methods for obtaining such distance information have been proposed. For example, an active distance measurement method in which illumination light is irradiated and reflected light from a subject is received to perform distance measurement, or a baseline length. The focus lens is driven so as to increase the contrast of images acquired by a plurality of image pickup devices (for example, stereo cameras) arranged in accordance with the principle of triangulation, or the image acquired by the image pickup device itself. Various methods such as a contrast AF method have been proposed.

  However, the active distance measurement method requires a dedicated member for distance measurement such as a light projection device for distance measurement, and the triangular distance measurement method requires a plurality of image pickup devices. Will increase. On the other hand, since the contrast AF method uses an image acquired by the imaging apparatus itself, a dedicated distance measuring member or the like is not required. However, the contrast value peak is obtained by performing multiple imaging while changing the position of the focus lens. Therefore, it takes time to search for a peak corresponding to the in-focus position, and it is difficult to perform high-speed AF.

  Under such a background, as a technology to acquire distance information while suppressing an increase in size and cost, a light beam that passes through the pupil of the lens is divided into a plurality of light beams, and a light beam that passes through one pupil region in the lens The distance information to the subject is obtained by performing a correlation calculation between the pixel signal obtained from the pixel signal and the pixel signal obtained from the light beam that has passed through another pupil region different from the one pupil region in the lens. Techniques to do this have been proposed.

  For example, a technique is known in which a light-shielding film that can perform pupil division is formed on a part of pixels of an image sensor to form a focus detection pixel, and a phase difference is obtained from an image obtained from the focus detection pixel. Yes. However, in this case, it is necessary to form a dedicated image sensor by forming a light shielding film while aligning with pixels using a fine processing technique, and a general-purpose image sensor cannot be used as it is.

  On the other hand, in Japanese Patent Laid-Open No. 11-258889, a movable pupil is inserted into a photographing optical system, and the pupil is moved to obtain an optical image composed of light beams having different pupil regions. High-speed processing in the phase difference method (pupil time-division phase difference AF) that forms an image on an image sensor, detects the phase difference between a plurality of images obtained by different pupil regions, and detects the defocus amount of the photographing lens. In order to reduce the size and the circuit scale, a technique is described in which a signal obtained from one block is used for phase difference detection. In this configuration, although a general-purpose image sensor can be used, it is necessary to acquire a plurality of images at different times in order to detect the phase difference. Therefore, it may take time until the subject is in focus, or the AF accuracy may be reduced in the case of a moving subject. Therefore, this publication describes a technique for performing motion correction on a moving subject, but still requires a plurality of images for phase difference detection.

  Therefore, a technique that can use a general-purpose image sensor and can detect a phase difference from one image has been proposed. For example, in Japanese Patent Laid-Open No. 2001-174696, a pupil color dividing filter having different spectral characteristics (for example, RG and GB) for each partial pupil is interposed in an imaging optical system, and a subject image from the imaging optical system is colored. A technique for performing pupil division by color (for example, R and B) by receiving light with an image sensor is described. In other words, the image signal output from the color image sensor is color-separated, and the relative shift amount between the same subject on each color image is detected, so that it is shifted from the in-focus position to the short distance side or the long distance Two pieces of focusing information, that is, a focusing shift direction that is shifted to the side and a focusing shift amount that is a shift amount from the in-focus position in that direction, are acquired.

  By the way, even if correlation calculation is performed, a contrast evaluation value necessary for phase difference detection may not be obtained. A substitute example of a so-called weak subject is a low-contrast subject, and a subject in accordance with this is a large defocused subject. As a technique corresponding to such a case, Japanese Patent Application Laid-Open No. 2003-344753 discloses that correlation calculation is impossible when any of the contrast evaluation values of the subject image related to pupil division is equal to or less than a threshold value. A technique for enlarging the AF target area until it is determined and correlation calculation is possible is described.

Japanese Patent Application Laid-Open No. 11-258889 JP 2001-174696 A Japanese Patent Laid-Open No. 2003-344753

  Even in the technique described in Japanese Patent Application Laid-Open No. 2001-174696 described above, there is a particular weak subject.

  That is, since the technique described in the publication is, for example, a technique for performing a correlation operation between an R image and a B image, an object having pixel values in both the wavelength band of the R image and the wavelength band of the B image. Otherwise, the correlation calculation cannot be performed. Therefore, for example, in the case of a narrow-band subject, more specifically, a deep red rose of only the R color, an R image can be acquired, but a B image cannot be acquired, so that a correlation calculation cannot be performed.

  Such a poor subject will be described in detail with reference to the drawings according to the embodiment of the present application.

  First, the pupil color dividing filter described in Japanese Patent Laid-Open No. 2001-174696 is configured in the same manner as the band limiting filter 12 shown in FIG. 3, for example. That is, the band limiting filter 12 is configured to divide the pupil of the imaging optical system including the lens into a partial pupil that transmits RG and a partial pupil that transmits GB. Here, the RG filter 12r forms a partial pupil that transmits RG, and the GB filter 12b forms a partial pupil that transmits GB.

  For example, when the normal object OBJrgb at a position farther than the in-focus position is imaged by the optical system having such a configuration, a state in which the optical system is viewed from above is shown in FIG.

  That is, when the subject is located farther than the in-focus position, the in-focus surface is located between the lens 10 and the image sensor 22, and therefore the R blurred image is at a position where the left and right sides of the band limiting filter 12 are reversed. IMGr and B blurred image IMGb are imaged.

  If the image sensor is an image sensor having a Bayer array as shown in FIG. 2, for example, in the case of a normal subject OBJrgb, that is, a subject that reflects light in each of the RGB bands, as shown in FIG. These pixels perform photoelectric conversion to generate a pixel value (hatching in FIG. 5 and FIG. 9 described later indicates that a pixel value has occurred).

  When the correlation between the R image and the B image thus obtained is calculated by the SAD method, a correlation value forming a V-shaped valley as shown in FIG. 6 is obtained, and an image shift amount at the bottom of the valley close to 0 is obtained. The amount of phase difference. Further, when the correlation calculation is performed by the ZNCC method, a correlation value forming an inverted V-shaped peak as shown in FIG. 7 is obtained, and an image shift amount at the peak of the peak close to 1 is obtained as a phase difference amount.

  On the other hand, the subject OBJr having a narrow band such as a crimson rose, which has only the R color (more precisely, the intensity of the R color light is overwhelmingly higher than the intensity of the B color light and the G color light) is described above. FIG. 8 shows a state when an image is formed by the optical system.

  At this time, the R light emitted from the subject OBJr passes through the RG filter 12r but is blocked by the GB filter 12b. Therefore, as shown in FIG. 9, the R pixel performs photoelectric conversion to generate a pixel value, but the B pixel and the G pixel hardly perform photoelectric conversion and generate almost no pixel value.

  Even if the correlation calculation is performed between the R image and the B image obtained in this way by the SAD method, a correlation value in which no clear V-shaped valley as shown in FIG. It is difficult to obtain the bottom of the valley close to, and even if it is obtained, the reliability is extremely low. Similarly, even if the correlation calculation is performed by the ZNCC method, a correlation value in which a clear inverted V-shaped peak as shown in FIG. 11 is not obtained is obtained, and it is difficult to obtain the peak of the peak close to 1. Yes, even if it is obtained, the reliability is extremely low. Thus, when the subject is in a narrow band, it is difficult to detect the phase difference, and even if the phase difference is detected, the reliability of the detected phase difference amount may be lowered. I understand.

  The present invention has been made in view of the above circumstances, and enables distance measurement by phase difference detection even for subjects who are difficult to detect phase difference based on a plurality of images having different wavelength bands and pupil regions. It aims at providing a measuring device, an imaging device, and a distance measuring method.

  In order to achieve the above object, a distance measuring device according to an aspect of the present invention includes an imaging device that receives light and performs photoelectric conversion to generate a color image of a plurality of bands, and the light to the imaging device. An imaging optical system that forms an image, a band limiting filter that divides the band into at least two when the light passes through a pupil region of the imaging optical system, and an image obtained from the imaging element. The distance calculation unit that measures the center of gravity position for each band and calculates the distance to the subject based on the phase difference amount of the center of gravity position, and between the divided images of the bands having different center of gravity positions for each band If it is determined that there is a correlation by a color correlation determination unit that determines whether there is a correlation, a pupil position change unit that changes the position of the center of gravity, and the color correlation determination unit, the image sensor once The distance calculation unit from the captured image of When the distance to the subject is calculated and it is determined that there is no correlation, the pupil position changing unit is caused to change the position of the center of gravity, and at least two times of imaging before and after the change of the position of the center of gravity by the image sensor A control unit that causes the distance calculation unit to calculate the distance to the subject from the image.

  An imaging apparatus according to another aspect of the present invention includes the distance measuring device.

  The distance measurement method according to still another aspect of the present invention divides light into at least two bands, photoelectrically converts the light to generate a plurality of band color images, and each band divided from the color images. The center of gravity position of each of the divided bands is measured, and it is determined whether there is a correlation between the images of the bands having different center of gravity positions, and if it is determined that there is a correlation, The distance to the subject is calculated based on the phase difference amount of the centroid position obtained from the captured image, and when it is determined that there is no correlation, the centroid position is changed, and before and after the change of the centroid position The distance to the subject is calculated based on the phase difference amount of the barycentric position obtained from at least two captured images.

  According to the distance measuring device, imaging device, and distance measuring method of the present invention, distance measurement by phase difference detection is possible even for subjects who are difficult to detect phase difference based on a plurality of images having different wavelength bands and pupil regions. It becomes.

1 is a block diagram illustrating a configuration of an imaging apparatus according to Embodiment 1 of the present invention. FIG. 3 is a diagram for explaining a pixel array of the image sensor in the first embodiment. The figure for demonstrating one structural example of the band-limiting filter in the said Embodiment 1. FIG. In the said Embodiment 1, the top view which shows the mode of a to-be-photographed object light flux when imaging normal object OBJrgb in the far side from a focusing position. FIG. 3 is a diagram illustrating a state of photoelectric conversion of RGB pixels when a normal subject OBJrgb is imaged in the first embodiment. In the said Embodiment 1, the diagram which shows the example of the correlation value calculated by SAD system, shifting the relative position of R image and B image which were obtained by imaging normal object OBJrgb. FIG. 6 is a diagram showing an example of correlation values calculated by the ZNCC method while shifting the relative positions of an R image and a B image obtained by imaging a normal subject OBJrgb in the first embodiment. FIG. 3 is a plan view showing a state of subject light flux condensing when imaging an R-color subject OBJr that is on the far side from the in-focus position in the first embodiment. FIG. 3 is a diagram illustrating a state of photoelectric conversion of RGB pixels when an image of an R-color subject OBJr is imaged in the first embodiment. FIG. 5 is a diagram showing an example of correlation values calculated by the SAD method while shifting the relative positions of the R image and B image obtained by imaging the narrow-band subject OBJr in the first embodiment. FIG. 5 is a diagram showing an example of correlation values calculated by the ZNCC method while shifting the relative positions of the R image and B image obtained by imaging the narrow-band subject OBJr in the first embodiment. In the first embodiment, the subject light flux is collected when the band limiting filter of FIG. 3 is rotated by 180 ° around the optical axis to capture the R-colored subject OBJr on the far side from the in-focus position. FIG. FIG. 4 is a diagram illustrating a color pupil barycentric position g1 of an R color when the band limiting filter is in the position illustrated in FIG. 3 in the first embodiment. FIG. 6 is a diagram illustrating a color pupil barycentric position g2 of R color when the band limiting filter is rotated by 180 ° around the optical axis in the first embodiment. In the said Embodiment 1, the figure which shows the mode before and behind the change of the color pupil gravity center position of R color. FIG. 4 is a diagram showing an R color pupil center of gravity position g2 when the band limiting filter of FIG. 3 is rotated by 120 ° around the optical axis in the first embodiment. FIG. 4 is a diagram showing a color pupil barycentric position g3 of an R color when the band limiting filter of FIG. 3 is rotated by 240 ° around the optical axis in the first embodiment. In the said Embodiment 1, the figure which shows the color pupil gravity center position g1-g3 used as the position of 3 equally around an optical axis. FIG. 4 is a diagram showing a color pupil barycenter position g3 of an R color when the band limiting filter of FIG. 3 is rotated 90 ° around the optical axis in the first embodiment. FIG. 4 is a diagram showing a color pupil barycenter position g4 of the R color when the band limiting filter of FIG. 3 is rotated 270 ° around the optical axis in the first embodiment. In the said Embodiment 1, the figure which shows the color pupil gravity center positions g1-g4 used as the position of 4 equally around an optical axis. The figure which shows the example which provided the gear around the band-limiting filter in the said Embodiment 1, and was able to rotate around an optical axis. 3 is a flowchart illustrating an imaging process in the imaging apparatus according to the first embodiment. 24 is a flowchart showing details of an AF operation performed in the imaging process of FIG. 23 in the first embodiment. 24 is a flowchart showing another example of the AF operation performed in the imaging process of FIG. 23 in the first embodiment. The flowchart which shows the process of the distance calculation performed in AF operation | movement of FIG. 24 or FIG. 25 in the said Embodiment 1. FIG. In the said Embodiment 1, the flowchart which shows the process of the pupil color time division acquisition performed in the distance calculation of FIG. The figure which shows a mode when the band-limiting filter is evacuating from the optical path in the modification of the said Embodiment 1. FIG. The figure which shows a mode when the band-limiting filter is inserted in the optical path in the modification of the said Embodiment 1. FIG. The figure which shows a mode when the band-limiting filter is inserted to the middle in the optical path in the modification of the said Embodiment 1. FIG. The block diagram which shows the structure of the imaging device in Embodiment 2 of this invention. The figure which shows the example of 1 structure of the aperture_diaphragm | restriction in the said Embodiment 2. FIG. FIG. 33 is a diagram illustrating a state in which the first color pupil barycentric position g1 is realized by driving the diaphragm of FIG. 32 in the second embodiment. FIG. 33 is a diagram illustrating a state in which the second color pupil center-of-gravity position g2 is realized by driving the diaphragm of FIG. 32 in the second embodiment. The figure which shows a mode that the aperture_diaphragm | restriction is open | released and the 1st color pupil gravity center position g1 is implement | achieved in the modification of the said Embodiment 2. FIG. The figure which shows a mode that the 2nd color pupil center-of-gravity position g2 is implement | achieved by restrict | squeezing an aperture stop in the modification of the said Embodiment 2. FIG. FIG. 6 is a block diagram illustrating a configuration of an imaging apparatus according to Embodiment 3 of the present invention. In the said Embodiment 3, a figure which shows a mode that the lens barrier is driven and the 1st color pupil gravity center position g1 is implement | achieved. FIG. 10 is a diagram showing a state in which a second color pupil barycentric position g2 is realized by driving a lens barrier in the third embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
[Embodiment 1]

  1 to 27 show Embodiment 1 of the present invention, and FIG. 1 is a block diagram showing a configuration of an imaging apparatus.

  An imaging device including the distance measuring device according to the present embodiment is configured as a digital still camera, for example. However, here, a digital still camera is taken as an example, but the imaging device may be any device as long as it has a color imaging element and has an imaging function. Still cameras, video cameras, mobile phones with cameras, personal digital assistants with cameras (PDAs with cameras), personal computers with cameras, surveillance cameras, endoscopes, digital microscopes (digital microscopes), binoculars with ranging functions, etc. .

  The imaging apparatus includes a lens unit 1 and a body unit 2 that is a main body portion to which the lens unit 1 is detachably attached via a lens mount. Here, a case where the lens unit 1 is detachable will be described as an example, but of course, it may not be detachable.

  The lens unit 1 includes an imaging optical system 9 including a lens 10 and a diaphragm 11, a band limiting filter 12, a lens control unit 14, a lens side communication connector 15, a lens driving unit 16, a filter driving unit 17, And an aperture drive unit 18.

  The body unit 2 includes a shutter 21, an imaging element 22, an imaging circuit 23, an imaging drive unit 24, an image processing unit 25, an image memory 26, a display unit 27, an interface (IF) 28, and a system controller. 30, a sensor unit 31, an operation unit 32, a strobe control circuit 33, a strobe 34, and a body side communication connector 35. In addition, although the recording medium 29 is also described in the body unit 2 of FIG. 1, this recording medium 29 is a memory card (smart media, SD card, xD picture card, etc.) which is detachable with respect to the imaging device. Since it is configured, the configuration may not be unique to the imaging apparatus.

  First, the imaging optical system 9 in the lens unit 1 is for forming a subject image on the imaging element 22. The lens 10 of the imaging optical system 9 includes a focus lens for performing focus adjustment. Although the lens 10 is generally composed of a plurality of lenses in general, only one lens is shown in FIG. 1 for simplicity.

  The diaphragm 11 of the imaging optical system 9 is for adjusting the brightness of the subject image formed on the imaging element 22 by regulating the passage range of the subject light flux passing through the lens 10. When the aperture diameter of the aperture 11 is changed, the pupil region of the imaging optical system 9 is also changed.

  The band limiting filter 12 is disposed on the optical path of the imaging light beam that passes through the imaging optical system 9 and reaches the imaging device 22 (preferably at the position of the diaphragm 11 of the imaging optical system 9 or in the vicinity thereof). The first band (here, the band indicates a wavelength band; the same shall apply hereinafter) in the imaging light flux that attempts to pass through the first area that is a part of the pupil area of The first band limitation that allows light in the third band to pass through and the light in the second band in the imaging light flux that attempts to pass through the second area that is another part of the pupil area of the imaging optical system 9 It is a filter that performs the second band limitation that blocks the light of the first band and the third band and blocks the light.

  FIG. 3 is a diagram for explaining a configuration example of the band limiting filter 12. In the band limiting filter 12 of the configuration example shown in FIG. 3, the pupil region of the imaging optical system 9 is divided into a first region and a second region. In other words, the band limiting filter 12 has a left half of the G (green) component and R (red) when the image pickup device is viewed from the image pickup device 22 in a standard posture (so-called posture in which the camera is held in a normal horizontal position). An RG filter 12r that passes the component and blocks the B (blue) component (that is, the B (blue) component is one of the first band and the second band), and the right half is the G component. The B filter passes through the B component and blocks the R component (that is, the R component is the other one of the first band and the second band). Therefore, the band limiting filter 12 allows the G component included in the light passing through the aperture 11 of the imaging optical system 9 to pass through the entire region (that is, the G component is the third band), and the R component is opened. Only the half area of the aperture is allowed to pass, and the B component is allowed to pass only for the remaining half of the aperture. As described above, the band limiting filter 12 limits the pass region in the pupil region of the imaging optical system 9 by the band (color) of the light to be transmitted, and thus the light of each band divided by the band limit filter 12. The gravity center position (color pupil gravity center position) of the amount of light when passing through the pupil region of the imaging optical system 9 differs for each band (color). Specifically, the gravity center position of the light amount when G passes through the pupil region coincides with the optical axis center of the imaging optical system 9, and the gravity center position of the light amount when R passes through the pupil region is determined by the imaging optical system 9. Eccentric from the optical axis center and shifted to the left in FIG. 3, and the barycentric position of the amount of light when B passes through the pupil region is eccentric from the optical axis center of the imaging optical system 9 and shifted to the right in FIG.

  If the RGB spectral transmission characteristics of the band limiting filter 12 are different from the RGB spectral transmission characteristics of an element filter (see FIG. 2) configured on-chip in the image sensor 22, the RG filter 12r and the GB filter 12b As will be described later, the accuracy of the position information acquired from the images obtained based on the difference in the spatial position is reduced, or a light amount loss due to a mismatch in spectral characteristics occurs. Therefore, it is desirable that the spectral transmission characteristic of the band limiting filter 12 is the same as or as close as possible to the spectral transmission characteristic of the element filter of the image sensor 22.

  The lens control unit 14 controls the lens unit 1. That is, the lens control unit 14 controls the lens driving unit 16, the filter driving unit 17, and the aperture driving unit 18 based on a command received from the system controller 30 via the lens side communication connector 15 and the body side communication connector 35.

  The lens-side communication connector 15 enables communication between the lens control unit 14 and the system controller 30 by connecting the lens unit 1 and the body unit 2 with a lens mount and connecting to the body-side communication connector 35. Connector.

  The lens driving unit 16 drives and focuses the focus lens in the lens 10 based on the control of the lens control unit 14. Further, when the lens 10 is an electric zoom lens, the lens driving unit 16 also drives the zoom lens in the lens 10 based on the control of the lens control unit 14.

  The filter drive unit 17 inserts the band limiting filter 12 on the optical path of the imaging optical system 9 or retracts it from the optical path based on the control of the lens control unit 14.

  The aperture driving unit 18 drives the aperture 11 and changes the aperture diameter based on the control of the lens control unit 14.

  Next, the shutter 21 in the body unit 2 is an optical shutter for adjusting the exposure time of the image sensor 22 by regulating the passage time of the subject light beam reaching the image sensor 22 from the lens 10. Although an optical shutter is used here, an element shutter (electronic shutter) by the image sensor 22 may be used instead of or in addition to the optical shutter.

  The imaging element 22 receives the subject image formed by the imaging optical system 9 for each of a plurality of wavelength bands (for example, but not limited to RGB) in one imaging. A color imaging device that performs photoelectric conversion and outputs an electrical signal, and is configured as a CCD or CMOS, for example. Here, the configuration of the color image sensor may be a single-plate image sensor provided with an on-chip element color filter, or a three-plate system using a dichroic prism that performs color separation into RGB color lights. However, it may be an image sensor of a method capable of acquiring RGB imaging information according to the position in the depth direction of the semiconductor at the same pixel position, or any imaging element that can acquire imaging information of a plurality of wavelength bands. It does n’t matter.

  For example, with reference to FIG. 2, a configuration example of a single-plate color image sensor often used for a general digital still camera will be described. FIG. 2 is a diagram for explaining the pixel arrangement of the image sensor 22. In the present embodiment, the plurality of wavelength band lights transmitted through the element color filter mounted on-chip are R, G, and B. As shown in FIG. An image sensor is configured. Therefore, when the image pickup device 22 has the configuration shown in FIG. 2, only one color component is obtained per pixel, and therefore the image processor 25 performs a demosaicing process. A color image in which three colors of RGB are aligned per pixel is generated.

  The image pickup circuit 23 amplifies (gain adjusts) the image signal output from the image pickup device 22, or performs A / D conversion when the image pickup device 22 is an analog image pickup device and outputs an analog image signal. Thus, a digital image signal (hereinafter also referred to as “image information”) is generated (when the image pickup device 22 is a digital image pickup device, it is already digital when it is input to the image pickup circuit 23). Therefore, A / D conversion is not performed). The imaging circuit 23 outputs an image signal to the image processing unit 25 in a format corresponding to the imaging mode switched by the imaging driving unit 24. Details will be described later.

  The imaging drive unit 24 supplies a timing signal and power to the imaging element 22 and the imaging circuit 23 based on a command from the system controller 30 to cause the imaging element to perform exposure, reading, element shuttering, and the like, and also to the imaging element 22. Control is performed to execute gain adjustment and A / D conversion by the imaging circuit 23 in synchronism with the above operations. The imaging drive unit 24 also performs control to switch the imaging mode of the imaging element 22.

  The image processing unit 25 performs various digital image processing. The image processing unit 25 performs, for example, black level correction, γ correction, defective pixel correction, demosaicing, image information color information conversion processing, image information pixel number conversion processing, and the like. Furthermore, the image processing unit 25 includes a white balance adjustment unit 36 that performs WB (white balance) adjustment. Further, since the band limiting filter 12 limits the passing area in the pupil area of the imaging optical system 9 by the band (color) of the light to pass, the brightness of the passing light varies depending on the band. Therefore, the image processing unit 25 corrects such a difference in brightness between bands (between colors). In addition, when the band limiting filter 12 as shown in FIG. 3 is used, a spatial misalignment occurs between the R image and the B image in the out-of-focus portion, so that the image processing unit 25 also performs a colorization process for correcting the spatial positional deviation and generating a flat image without color deviation.

  The image memory 26 is a memory capable of high-speed writing and reading, and is constituted by, for example, an SDRAM (Synchronous Dynamic Random Access Memory). The image memory 26 is used as a work area for image processing and is also a work area of the system controller 30. Also used as For example, the image memory 26 not only stores the final image processed by the image processing unit 25 but also appropriately stores each intermediate image in a plurality of processing steps by the image processing unit 25.

  The display unit 27 includes an LCD or the like, and an image processed for display by the image processing unit 25 (an image read from the recording medium 29 and processed for display by the image processing unit 25 is also included). Display). Specifically, the display unit 27 performs live view image display, confirmation display when recording a still image, reproduction display of a still image or a moving image read from the recording medium 29, and the like.

  The interface (IF) 28 is detachably connected to the recording medium 29, and transmits information to be recorded on the recording medium 29 and information read from the recording medium 29.

  The recording medium 29 records an image processed for recording by the image processing unit 25 and various data related to the image, and is configured as a memory card or the like as described above.

  The sensor unit 31 is, for example, a camera shake sensor configured by an acceleration sensor or the like for detecting blur of the imaging device, a temperature sensor for measuring the temperature of the imaging device 22, and a brightness for measuring the brightness around the imaging device. Includes brightness sensor, etc. The detection result by the sensor unit 31 is input to the system controller 30. Here, the detection result by the camera shake sensor is used to drive the image pickup device 22 and the lens 10 to perform camera shake correction or to perform camera shake correction by image processing. The detection result by the temperature sensor is used to control the drive clock by the imaging drive unit 24 and to estimate the amount of noise in the image obtained from the image sensor 22. Furthermore, the detection result by the brightness sensor is used, for example, to appropriately control the luminance of the display unit 27 according to the ambient brightness.

  The operation unit 32 changes a power switch for turning on / off the power of the image pickup device, a release button including a two-stage press button for inputting an image pickup operation such as a still image or a moving image, an image pickup mode, and the like. Mode buttons, cross keys used to change selection items, numerical values, and the like. A signal generated by the operation of the operation unit 32 is input to the system controller 30.

  The strobe control circuit 33 controls the light emission amount and the light emission timing of the strobe 34 based on a command from the system controller 30.

  The strobe 34 is a light source that emits illumination light to the subject under the control of the strobe control circuit 33.

  As described above, the body side communication connector 35 is connected between the lens control unit 14 and the system controller 30 when the lens unit 1 and the body unit 2 are coupled by the lens mount and connected to the lens side communication connector 15. It is a connector that enables communication.

  The system controller 30 controls the body unit 2 and also controls the lens unit 1 via the lens control unit 14, and is a control unit that integrally controls the imaging apparatus. The system controller 30 reads a basic control program of the image pickup apparatus from a non-illustrated non-volatile memory such as a flash memory, and controls the entire image pickup apparatus in accordance with an input from the operation unit 32.

  For example, the system controller 30 controls each part in the lens unit 1 via the lens control unit 14, controls and drives the shutter 21, and a camera shake (not shown) based on a detection result by the acceleration sensor of the sensor unit 31. The correction mechanism is controlled to perform camera shake correction or the like. Further, the system controller 30 sets the mode of the imaging device (settings such as a still image mode for capturing a still image and a moving image mode for capturing a moving image) in response to an input from the mode button of the operation unit 32. It is intended to do.

  Further, the system controller 30 includes a distance calculation unit 37, a color correlation determination unit 38, and a contrast AF control unit 39.

  The color correlation determination unit 38 determines whether or not there is a correlation between the images in the bands having different color pupil centroid positions.

  The distance calculation unit 37 calculates the distance to the subject based on the phase difference amounts of images in a plurality of bands having different color pupil centroid positions in the image obtained from the image sensor 22. For example, the distance calculation unit 37 calculates the phase difference amount between the first image in the first band and the second image in the second band obtained from the image sensor 22, and calculates the calculated phase difference amount. Based on this, the distance to the subject is calculated. Specifically, the distance calculation unit 37 calculates the subject distance based on the phase difference amount between the R image and the B image, the phase difference amount between the R image and the G image, or the phase difference amount between the B image and the G image. It is supposed to be.

  The distance information obtained by the distance calculation unit 37 is used, for example, for autofocus (AF), and the system controller 30 controls the lens unit 1 based on the distance information to perform phase difference AF (of course, the distance information is used) Is not limited to AF, and may be used for blur amount control, generation of stereoscopic images, and the like.

  The contrast AF control unit 39 performs contrast AF by driving the focus lens based on the image obtained from the image sensor 22. More specifically, the contrast AF control unit 39 may output an image signal output from the image processing unit 25 (this image signal may be a G image having a high ratio including a luminance component, or may be processed by the above-described colorization process. A contrast value (also referred to as an AF evaluation value) is generated from a luminance signal image related to an image with corrected deviation, and the focus lens in the lens 10 is controlled via the lens control unit 14. is there. That is, the contrast AF control unit 39 extracts a high-frequency component by applying a filter, for example, a high-pass filter, to the image signal and sets it as a contrast value. Then, the contrast AF control unit 39 acquires the contrast value by changing the focus lens position, moves the focus lens in the direction in which the contrast value increases, and further acquires the contrast value. By repeatedly performing such processing, control is performed so that the focus lens is driven to the focus lens position (focus position) at which the maximum contrast value is obtained.

  The system controller 30 is divided by the band limiting filter 12 by controlling a pupil position changing unit (a filter driving unit 17, a diaphragm driving unit 18, or a barrier driving unit 19 (see FIG. 37) as described later). The color pupil centroid position of at least one of the bands is changed (thus, the pupil position changing unit may be called the color pupil centroid position changing unit).

  Then, as will be described later, when the color correlation determination unit 38 determines that there is a correlation, the system controller 30 causes the distance calculation unit 37 to calculate the distance to the subject from one captured image by the image sensor 22, If it is determined that there is no correlation, the pupil position changing unit is caused to change the color pupil centroid position, and the distance calculation unit 37 is obtained from at least two captured images before and after the change of the color pupil centroid position by the image sensor 22. Control to calculate the distance to the subject.

  Further, the system controller 30 determines whether or not the distance calculation unit 37 is appropriate for calculating the subject distance based on the phase difference amount. Control is performed so that contrast AF is performed by the contrast AF control unit without calculating the distance. Details will be described later.

  Next, a case where a normal subject OBJrgb and a narrow-band subject OBJr are imaged by the imaging apparatus having the above-described configuration will be described with reference to FIGS.

  FIG. 4 is a plan view showing a state of subject light beam condensing when an image of a normal subject OBJrgb on the far side from the in-focus position is imaged.

  In the case of a normal subject OBJrgb having light intensity in each of the RGB bands, the light from the subject OBJrgb is blocked by the GB filter 12b while the R component passes through the RG filter 12r, and the B component passes through the GB filter 12b. While passing, it is blocked by the RG filter 12r. Further, the G component passes through both the RG filter 12r and the GB filter 12b.

  When the band limiting filter 12 having the shape shown in FIG. 3 is used, the light emitted from one point on the subject OBJrgb forms a subject image IMGg that forms a circular blur for the G component, A subject image IMGr that forms a semicircular blur for the R component and a subject image IMGb that forms a semicircular blur for the B component are formed. However, the R component blurred subject image IMGr and the B component blurred subject image IMGb coincide with the band limiting filter 12 on the left and right sides when the in-focus state is closer to the in-focus side, and farther than the in-focus position. When the distance side is not in focus, the left and right sides of the band limiting filter 12 are reversed.

  The RGB pixels on the image sensor 22 that have received the light from the imaging optical system 9 perform photoelectric conversion and generate pixel values, as shown by hatching in FIG. FIG. 5 is a diagram showing a state of photoelectric conversion of the RGB pixels when imaging a normal subject OBJrgb.

  As described above, when the R image and the B image thus obtained are subjected to correlation calculation by the SAD method, a correlation value forming a V-shaped valley as shown in FIG. 6 is obtained, and the bottom of the valley close to 0 is obtained. The amount of image shift is the amount of phase difference to be obtained. Further, when the correlation calculation is performed by the ZNCC method, a correlation value forming an inverted V-shaped peak as shown in FIG. 7 is obtained, and an image shift amount at the peak of the peak close to 1 is obtained as a phase difference amount. 6 is a diagram showing an example of correlation values calculated by the SAD method while shifting the relative positions of the R image and the B image obtained by imaging a normal subject OBJrgb, and FIG. It is a diagram which shows the example of the correlation value calculated by the ZNCC system, shifting the relative position of R image and B image which were obtained by imaging to-be-photographed object OBJrgb. In addition, the horizontal axis of the graphs of FIGS. 6 and 7 (and FIGS. 10 and 11 to be described later) represents the image shift amount (unit: pixel) of the correlation calculation target image (for example, the R image and B image described above). The vertical axis indicates the correlation value obtained as a result of the correlation calculation.

  Next, FIG. 8 is a plan view showing a state of subject light flux condensing when the R subject OBJr on the far side from the in-focus position is imaged.

  First, it is assumed that the subject OBJr is a narrow-band subject, such as a crimson rose, whose R color light intensity is overwhelmingly higher than that of B color light and G color light.

  At this time, the R light emitted from the subject OBJr passes through the RG filter 12r but is blocked by the GB filter 12b. Thereby, light emitted from one point on the out-of-focus subject OBJrgb forms a subject image IMGr that forms a semicircular blur for the R component, but hardly any subject image is formed for the G component and the B component. . Therefore, as shown in FIG. 9, the R pixel performs photoelectric conversion to generate a pixel value, but the B pixel and the G pixel do not perform much photoelectric conversion and generate almost no pixel value. FIG. 9 is a diagram showing a state of photoelectric conversion of the RGB pixels when the R subject OBJr is imaged. Since the image sensor 22 outputs image signals not only for the R image but also for the B image and the G image, RGB images are prepared as data.

  However, even if the correlation calculation is performed between the R image and the B image obtained in this way by the SAD method, a correlation value that does not show a clear V-shaped valley as shown in FIG. 10 is obtained. , It is difficult to obtain the bottom of the valley close to 0, and even if it is obtained, the reliability is extremely low. Similarly, even if the correlation calculation is performed by the ZNCC method, a correlation value in which a clear inverted V-shaped peak as shown in FIG. 11 is not obtained is obtained, and it is difficult to obtain the peak of the peak close to 1. Yes, even if it is obtained, the reliability is extremely low. Thus, when the subject is in a narrow band, it is difficult to detect the phase difference, and even if phase difference detection is performed, the reliability of the detected phase difference amount may be reduced. . FIG. 10 is a diagram showing an example of correlation values calculated by the SAD method while shifting the relative positions of the R image and the B image obtained by imaging the narrow band subject OBJr, and FIG. It is a diagram which shows the example of the correlation value computed by the ZNCC system, shifting the relative position of R image and B image which were obtained by image | photographing the object OBJr.

  In such a case, in the present embodiment, the subject distance is calculated based on the correlation calculation between the R image and the G image (or the B image and the G image), or the color pupil centroid position of the R image is changed, When the subject distance is calculated based on the correlation calculation of two or more R images captured before and after the color pupil centroid position change, or when the reliability of the object distance calculation based on the correlation calculation is low, contrast AF is used. The subject distance is calculated. Details will be described later.

  An example of changing the position of the center of gravity of the color pupil will be described with reference to FIGS. FIG. 12 shows the subject light flux collection when the band limiting filter 12 of FIG. 3 is rotated 180 ° around the optical axis and the R subject OBJr on the far side from the in-focus position is imaged. FIG. 13 is a diagram showing a color pupil centroid position g1 of the R color when the band limiting filter 12 is in the position shown in FIG. 3, and FIG. 14 shows the band limiting filter 12 on the optical axis. FIG. 15 is a diagram illustrating the R color pupil center of gravity position g2 when rotated around 180 °, and FIG. 15 is a diagram illustrating a state before and after the change of the R color pupil center of gravity position.

  When the band limiting filter 12 shown in FIG. 3 is rotated by 180 ° around the optical axis, the right semicircle becomes the RG filter 12r and the left semicircle becomes the GB filter 12b as shown in FIG.

  In this case, the subject image IMGr formed by the R color light emitted from one point on the out-of-focus subject OBJrgb is a semicircular blur opposite to the semicircular blur shown in FIG.

  Considering the barycentric position of the amount of light when passing through the pupil of the imaging optical system 9, when the band limiting filter 12 is in the state shown in FIG. 3, the R color pupil barycentric position is g1 shown in FIG. However, when the band limiting filter 12 is in the state shown in FIG. 14, the color pupil center of gravity of the R color is g2 shown in FIG.

  Therefore, as shown in FIG. 15, since two different color pupil centroid positions can be realized even if the subject is a single color (R color in this example), before and after the color pupil centroid position is changed. By performing the correlation calculation based on the two images taken at the same time, the distance to the subject can be obtained.

  Subsequently, with reference to FIGS. 16 to 21, some examples in which the position of the center of gravity of the color pupil is changed will be described. FIG. 16 is a diagram illustrating the color pupil center of gravity position g2 of the R color when the band limiting filter 12 of FIG. 3 is rotated 120 ° around the optical axis, and FIG. 17 is a diagram illustrating the band limiting filter 12 of FIG. FIG. 18 is a diagram illustrating the color pupil centroid position g3 of the R color when rotated by 240 ° around the optical axis, and FIG. 19 is a diagram illustrating the color pupil center of gravity position g3 of the R color when the band limiting filter 12 of FIG. 3 is rotated by 90 ° around the optical axis, and FIG. 20 is a diagram illustrating the band limiting filter 12 of FIG. FIG. 21 is a diagram showing the color pupil centroid position g4 of the R color when rotated around 270 °, and FIG. 21 is a diagram showing the color pupil centroid positions g1 to g4 that are four equal positions around the optical axis.

  The R image of the first color pupil centroid position g1 is acquired at the filter position shown in FIG. 13, the R image of the second color pupil centroid position g2 is acquired at the filter position shown in FIG. 16, and the filter position shown in FIG. As shown in FIG. 18, a parallax image having a color pupil barycentric position at three equal positions around the optical axis can be obtained.

  Various variations are conceivable for the order of R image acquisition at each color pupil center of gravity position and which R image is combined to acquire distance information among the acquired R images, but one example is as follows. become.

  First, the first R image is acquired at the first color pupil centroid position g1 shown in FIG. 18, and then the second R image is acquired at the second color pupil centroid position g2 shown in FIG. Then, correlation calculation is performed based on the first R image and the second R image, and first distance information is acquired.

  Subsequently, a third R image is acquired at the third color pupil centroid position g3 shown in FIG. 18, a correlation operation is performed based on the second R image and the third R image, and the second distance information is obtained. get.

  Thereafter, again, a fourth R image of the first color pupil center of gravity position g1 shown in FIG. 18 is acquired, and correlation calculation is performed based on the third R image and the fourth R image to obtain third distance information. To get. Here, the R image of the first color pupil centroid position g1 shown in FIG. 18 is acquired again because the time difference acquired between the first R image and the third R image is the first R image. This is to prevent the reliability of the correlation calculation from being lowered due to, for example, twice the time difference between the second R image and the second R image.

  In addition, when images are continuously acquired such as live view, distance information may be acquired in the same manner while sequentially changing the color pupil center of gravity position.

  Another example of obtaining distance information is as follows.

  First, the first R image is acquired at the first color pupil centroid position g1 in FIG. 18, and then the second R image is acquired at the second color pupil centroid position g2 in FIG. Then, correlation calculation is performed based on the first R image and the second R image, and first distance information is acquired.

  Subsequently, a third R image is acquired at the third color pupil center of gravity position g3 in FIG. 18, and a correlation calculation is performed based on the second R image and the third R image to acquire second distance information. To do. Further, correlation calculation is performed based on the first R image and the third R image, and third distance information is acquired.

  Then, based on the obtained first distance information, second distance information, and third distance information, distance information that can be estimated to have the highest reliability is calculated. Here, the estimation of distance information with high reliability may be determined based on the degree of correlation of the three distance information, may be obtained by statistical processing, or other means may be used.

  In addition to these, since there are various patterns for obtaining the distance information, an appropriate one may be selected according to the application.

  By combining a plurality of images obtained at the center of gravity of the color pupil as shown in FIG. 18, both the horizontal phase difference amount and the vertical phase difference amount can be detected. Therefore, there is an advantage that the distance information can be calculated with high reliability regardless of whether the subject has a horizontal pattern or a vertical pattern.

  Further, an R image of the first color pupil centroid position g1 is acquired at the filter position shown in FIG. 13, and an R image of the second color pupil centroid position g2 is acquired at the filter position shown in FIG. If the R image of the third color pupil centroid position g3 is acquired at the filter position and the R image of the fourth color pupil centroid position g4 is acquired at the filter position shown in FIG. 20, the optical axis as shown in FIG. It is possible to obtain a parallax image having the color pupil barycentric position at the surrounding four equal positions.

  Various variations are conceivable for the order of image acquisition at this time and which image is used to acquire distance information, but one example is as follows.

  First, the first R image is acquired at the first color pupil centroid position g1 shown in FIG. 20, and then the second R image is acquired at the second color pupil centroid position g2 shown in FIG. Then, correlation calculation is performed based on the first R image and the second R image, and first distance information is acquired.

  Subsequently, the third R image is acquired at the third color pupil centroid position g3 shown in FIG. 20, and then the fourth R image is acquired at the fourth color pupil centroid position g4 shown in FIG. Then, correlation calculation is performed based on the third R image and the fourth R image, and second distance information is acquired.

  The first distance information thus obtained is distance information based on horizontal parallax, and the second distance information is distance information based on vertical parallax. If the first distance information and the second distance information match, any result may be used. If they do not match, for example, an appropriate one may be selected according to the pattern of the subject. good.

  Also in the case of the four color pupil centroid positions, as in the case of the three color pupil centroid positions described above, various patterns can be considered for obtaining the distance information. For example, the images are acquired sequentially in the clockwise direction every 90 °, that is, in the order of the color pupil centroid positions g1, g3, g2, and g4 in FIG. Therefore, it is only necessary to acquire images in an appropriate order according to the application and calculate distance information.

  Next, a configuration for realizing the switching of the plurality of color pupil centroid positions as described above will be described below.

  First, although not shown, as an example, a turret is used. That is, band limiting filters 12 that realize a plurality of color pupil centroid positions are provided at a plurality of rotation angle positions of a rotatable turret plate, respectively, and the turret plate is rotated to set a band limitation filter at a desired color pupil centroid position. 12 may be positioned on the optical path of the imaging optical system 9. At this time, if an opening having the same size as that of the band limiting filter 12 is provided on the turret plate, a state in which the band limiting filter 12 is fully retracted from the optical path can be realized. The rotation position of the turret plate is controlled by the filter driving unit 17 via the lens control unit 14 based on a command from the system controller 30.

  Next, FIG. 22 is a diagram illustrating an example in which a gear is provided around the band limiting filter 12 so as to be rotatable around the optical axis.

  That is, the gear 12a is provided around the band limiting filter 12 having a circular shape. For example, a worm gear 17c is engaged with the gear 12a. The worm gear 17c is attached to the rotating shaft 17b of the motor 17a. Here, the motor 17a, the rotating shaft 17b, and the worm gear 17c are part of the configuration of the filter driving unit 17 that functions as a pupil position changing unit.

  In such a configuration, the band-limiting filter is used while detecting the rotation angle (the motor 17a is configured by a stepping motor and counting the number of pulses, or a sensor such as a photointerrupter for detecting the rotation angle is provided). What is necessary is just to rotate 12 to the desired angle around an optical axis.

  Next, the operation of the imaging device will be described. FIG. 23 is a flowchart illustrating an imaging process in the imaging apparatus.

  This processing is started by turning on the power switch of the imaging apparatus in the imaging mode or by setting the imaging switch after the power switch is turned on.

  First, an image is captured from the image sensor 22 and live view display is executed (step S1). During live view display, an AE (automatic exposure control) operation, an AWB (auto white balance) operation, and an AF (autofocus) operation are repeatedly performed for each frame (or for every predetermined number of frames).

  Then, the system controller 30 determines whether or not the release button of the operation unit 32 is half-pressed (step S2).

  Here, until the release button is pressed halfway, the process returns to step S1 to continue the live view display.

  On the other hand, if it is determined that the release button is half-pressed, the AE operation, AWB operation, and AF operation are performed in order to maintain the exposure value, white balance value, and focus distance at the time when the release button is half-pressed. Is stopped (frozen) (step S3). At this time, the band limiting filter 12 is driven by the filter driving unit 17 so as to be in a state of being retracted from the optical path of the imaging optical system 9. However, depending on the mode setting related to imaging (for example, when continuous AF is set), this operation is not limited to this, and these operations are appropriately performed even after half-pressing the release button.

  Thereafter, the system controller 30 determines whether or not the release button has been fully pressed (step S4).

  If the release button is not fully pressed, the process returns to the half-pressed determination in step S2, and if it is fully pressed, the main imaging operation for reading out the pixel signals of all the pixels of the image sensor 22 is performed (step S2). S5).

  Thereafter, the image signal read from the image sensor 22 is subjected to image processing by the image processing unit 25 via the imaging circuit 23 (step S6), displayed on the display unit 27, and recorded on the recording medium 29 (step). S7) This process ends.

  Next, FIG. 24 is a flowchart showing details of the AF operation performed in the imaging process of FIG.

  When the AF operation is started, the system controller 30 performs a distance calculation process (step S11). The distance calculation process will be described later with reference to FIG.

  The system controller 30 calculates the lens driving amount so as to focus on the distance information with the subject calculated by the distance calculation (step S12).

  Then, the system controller 30 drives the focus lens of the lens 10 via the lens control unit 14 and the lens driving unit 16 (step S13), and returns from the AF operation.

  FIG. 25 is a flowchart showing another example of the AF operation performed in the imaging process of FIG.

  The example of the AF operation shown in FIG. 25 is a modification of the above-described example of the AF operation shown in FIG. 24, and uses the AWB function generally provided in recent imaging apparatuses to (Narrow band subject) is determined efficiently.

  That is, the white balance adjustment unit 36 acquires an evaluation value for grasping the intensity of each RGB color with respect to the captured image. This evaluation value is, for example, an average value of each of the RGB colors in the AWB target area of the image, and it is assumed that AWB1, AWB2, and AWB3 are obtained in descending order.

  In the AF operation shown in FIG. 25, the AF method is selected based on these evaluation values obtained by the white balance adjustment unit 36.

  That is, when the subject has a narrow band (for example, a single color such as R color), the largest evaluation value AWB1 (or the average value of R color when the subject is a single color of R color) is obtained. The value of the obtained (AWB evaluation value) should be a particularly large value compared to the second evaluation value AWB2 and the third evaluation value AWB3.

  Therefore, when this process is started, the system controller 30 acquires the evaluation values AWB1 and AWB2 from the white balance adjustment unit 36, and a value obtained by dividing AWB1 by AWB2 is a predetermined threshold Th1 (for example, 5 or 10). It is determined whether or not (step S10). This determination is a determination as to whether or not it is appropriate to calculate the subject distance based on the phase difference amount.

  Here, when it is determined that the value obtained by dividing AWB1 by AWB2 is less than the predetermined threshold Th1, the processes in steps S11 to S13 similar to the process shown in FIG. 24 are performed.

  On the other hand, when it is determined that the value obtained by dividing AWB1 by AWB2 is equal to or greater than a predetermined threshold Th1 (when it is determined that it is inappropriate to calculate the subject distance based on the phase difference amount), it is narrow. Since there is a high possibility that the subject is in the band, the system controller 30 causes the contrast AF control unit 39 to perform contrast AF (step S14). At this time, the distance calculation unit 37 does not calculate the subject distance.

  Then, when the process of step S13 or step S14 is performed, the process returns from the AF operation.

  In the above description, it is determined whether only one of the RGB colors is high based on the AWB evaluation value. However, if the AE is an AE that uses color information, it may be based on the AE information. Of course, if there is a function for obtaining RGB color correlation among other functions provided in the imaging apparatus, this function may be used.

  If it is determined in step S10 that the threshold value is equal to or greater than the predetermined threshold Th1, contrast AF is performed in the process shown in FIG. 25, but the color pupil centroid position of the color that gives the evaluation value AWB1 is changed. (For example, when the color giving the evaluation value AWB1 is G, the center of gravity of the color pupil cannot be changed with the configuration of the present embodiment, but there are modifications and configurations of some embodiments described later) In some cases, the color pupil centroid position can be changed), and the phase difference AF may be performed by acquiring the image of the color at a different color pupil centroid position.

  Next, FIG. 26 is a flowchart showing distance calculation processing performed in the AF operation of FIG. 24 or FIG.

  When the distance calculation process is started, the system controller 30 controls the filter driving unit 17 via the lens control unit 14 to insert the band-limiting filter 12 in the retracted state on the optical path of the imaging optical system 9 and insert The state (first state) is set (step S21).

  Further, the distance calculation unit 37 sets a distance information acquisition range, so-called AF area (step S22). This distance information acquisition range is set according to the shooting mode set in the imaging apparatus. For example, the distance information acquisition range is an all-pixel area or a target object area (for example, detected by the image processing unit 25). , A face detected by face detection or the center of the screen) or an area manually set by a user.

  And imaging is performed and an image is acquired (step S23).

  Then, the distance calculation unit 37 sets the predetermined two colors in the set distance information acquisition range (in the case where the band limiting filter 12 has the configuration shown in FIG. 3, the predetermined two colors are set in advance). Performs a correlation operation between the R and B) images (step S24). Although this correlation calculation can be performed by various methods such as the SAD method and the ZNCC method, the following description will be made assuming that the correlation calculation is performed by, for example, the ZNCC method.

  Then, the color correlation determination unit 38 determines whether or not there is a correlation between two predetermined colors based on the result of the correlation calculation (step S25). The determination of the presence / absence of the correlation is made, for example, based on whether or not a correlation value equal to or higher than a predetermined threshold (for example, 0.7) is found in the correlation calculation result of the ZNCC method as shown in FIG. 7 or FIG. can do.

  When it is determined that there is a correlation between the two predetermined colors, the distance calculation unit 37 selects a correction table for the two predetermined colors (step S26). Here, the position of the center of gravity of the color pupil of each color changes according to the aperture value of the aperture 11, the driving state of the lens 10, or the insertion state of the band limiting filter 12. Therefore, the distance calculation unit 37 is previously provided with a table that records correction values for performing correction in accordance with the color pupil centroid position of each color that changes with such a combination of parameters. Therefore, here, each parameter is acquired, the table is referred to, and the correction value is acquired.

  Then, based on the acquired correction value, the distance calculation unit 37 acquires distance information of the subject (in the case of performing phase difference AF, the defocus amount of the lens 10) from the acquired phase difference amount (step S27). ).

  On the other hand, if it is determined in step S25 that there is no correlation between the two predetermined colors, the color correlation determination unit 38 is set in step S22 in order to find a combination of correlated colors. Calculate the average value of each color of RGB in the distance information acquisition range (the average value is calculated by dividing the pixel addition value by the number of pixels, so the calculation of the pixel addition value is included), and the highest two average pixel values A color is selected (step S28). Here, the average pixel values of the upper two colors are PV1 and PV2 in order from the largest value. If there is no value overflow or the like, selection may be performed based on the pixel addition value instead of the average value.

  Then, the color correlation determination unit 38 determines whether the selected upper two colors are other than the predetermined two colors (R and B in the above-described example) for which the correlation calculation was performed in step S24 (step S29). .

  Here, when it is determined that the colors are other than the predetermined two colors (in the above-described example, R and G, or B and G), the color correlation determination unit 38 sets PV1 calculated in step S28 to PV2. It is determined whether or not the value divided by is greater than or equal to a predetermined threshold Th2 (for example, a value such as 5) (step S30).

  If it is determined that it is less than the predetermined threshold Th2, there is a possibility that there is a correlation between the upper two color images, and therefore, the same processing as in step S24 described above is performed between these upper two color images. The correlation calculation is performed (step S31).

  Then, the color correlation determination unit 38 determines whether there is a correlation between the upper two colors based on the result of the correlation calculation between the images of the upper two colors (step S32). .

  Here, when it is determined that there is a correlation between the images of the upper two colors, it is considered that the phase difference amount based on the correlation of the upper two colors and thus the distance information can be acquired. The unit 37 selects a correction table for the upper two colors (step S33).

  Then, based on the acquired correction value, the distance calculation unit 37 acquires object distance information (or the defocus amount of the lens 10 when performing phase difference AF) from the acquired phase difference amount (step S34). ).

  On the other hand, if it is determined in step S30 that it is equal to or greater than the predetermined threshold Th2 (in this case, it is determined that there is no correlation between the upper two color images), or the correlation is determined in step S32. If it is determined that there is not, it is determined whether or not the color that gives the highest average pixel value PV1 is G (step S35).

  If it is determined that the color that gives the highest average pixel value PV1 is G, the system controller 30 retracts the band limiting filter 12 from the optical path of the imaging optical system 9 by the filter driving unit 17, for example. Then, the contrast AF control unit 39 is caused to perform contrast AF (step S36). Here, assuming that the color pupil centroid position cannot be changed with G color (for example, in the case of the configuration in which the band-limiting filter 12 is rotated around the optical axis as described above), the color that gives PV1 is Contrast AF is performed when the color is G, but the center of gravity of the color pupil can be changed even when the color is G (in the case of a configuration described in some modified examples and embodiments described later) ) May proceed to step S37 without performing steps S35 and S36.

  If it is determined in step S29 that the upper two colors are the same as the two predetermined colors, or if it is determined in step S35 that the color that gives the highest average pixel value PV1 is not G, FIG. The pupil color time-division acquisition process described with reference to FIG. 6 is performed (step S37). In the example described above, the process of step S37 is performed when the highest color is R or B.

  Thus, when the process of step S27, step S34, step S36, or step S37 is performed and the distance information is acquired, the process returns from this process.

  FIG. 27 is a flowchart showing pupil color time division acquisition processing performed in the distance calculation of FIG.

  When this process is started, first, one color that gives the highest value among the average pixel values of the respective colors calculated in step S28 is selected (step S41). In the example described above, R or B is selected as the highest color. However, if the color pupil center of gravity position of G can be changed in another example, G can also be selected as the highest color.

  Next, the distance calculation unit 37 selects a color pupil barycenter position switching condition (specifically, a rotation angle of the band limiting filter 12 or the like) (step S42). Here, the condition selection for changing the band limiting filter 12 to one or more states different from the first state set in step S21 described above (or, further, from the state different from the first state to the first state). Condition selection for returning to the state of (1) again.

  Then, the pupil position changing unit switches the color pupil centroid position to the nth state (the second state when first entering this process) according to the selected color pupil centroid position switching condition (step S43).

  Thereafter, imaging is performed to acquire an image (step S44). In addition, it is desirable that the camera shake correction by the camera shake correction mechanism described above is performed from the time when the above-described imaging in step S23 is performed until the imaging is performed in step S44.

  Then, the distance calculation unit 37 performs a correlation calculation between the highest color image acquired before the color pupil centroid position is changed and the highest color image acquired after the color pupil centroid position is changed ( Step S45). As a specific example, when the subject is a narrow-band subject OBJr such as the crimson rose described above, here two images (or two or more images) acquired before and after the pupil center of gravity position change may be used. The correlation calculation is performed based on the R color image.

  When performing this correlation calculation, for example, motion correction as described in the above-mentioned Japanese Patent Application Laid-Open No. 11-258889 is performed, and processing for correcting a subject shift based on a difference in imaging timing of two images is performed. You may do it.

  The distance calculation unit 37 selects a correction table in the same manner as in step S26 described above based on the state of the band limiting filter 12 before and after the change of the pupil center of gravity position, the state of the diaphragm 11 and the lens 10 (step S46).

  Then, based on the acquired correction value, the distance calculation unit 37 acquires distance information of the subject (in the case of performing phase difference AF, the defocus amount of the lens 10) from the acquired phase difference amount (step S47). ).

  Thereafter, whether or not to further switch the color pupil centroid position (for example, as described with reference to FIGS. 16 to 21, whether to further switch to the third color pupil centroid position g3 or the fourth color pupil centroid position g4) Whether or not to switch from the second to fourth color pupil barycentric positions g2 to g4 to the first color pupil barycentric position g1 again (step S48). And repeat the process as described above.

  On the other hand, when it is determined that the switching of the color pupil center of gravity position is to be finished, the process returns from the pupil color time division acquisition.

  In the above description, the example in which the image sensor 22 includes the primary color (RGB) on-chip element filter has been described. However, as long as band limitation can be performed, the image sensor 22 is not limited to the primary color filter. For example, a complementary color filter (C (cyan) ) (GB), M (magenta) (RB), Y (yellow) (RG), G (green)). In this case, for example, if the band limiting filter 12 is configured so that the semicircular portion is G (green) and the other semicircular portion is M (magenta) (RB), the subject is almost only R light. In the case of a narrow-band subject that emits R, a plurality of images before and after the movement of the center of gravity of the color pupil obtained from M (magenta) (RB) and Y (yellow) (RG) pixels that can receive R Based on this, it is possible to calculate distance information.

  According to the first embodiment, it is determined whether or not there is a correlation between images in a plurality of bands, and when it is determined that there is a correlation, the distance from one captured image to the subject is calculated. Therefore, distance detection based on the phase difference can be performed at high speed.

  When it is determined that there is no correlation, the center of gravity of the color pupil is changed, and the distance from the at least two captured images before and after the change to the subject is calculated. For example, the subject has a narrow band. Even in this case, distance detection based on the phase difference can be performed.

  Further, whether or not there is a correlation between the images in a plurality of bands is determined by changing the first pixel addition value having the largest value among the pixel addition values for each band in the ranging area to the second value having the second largest value. Since the determination is based on whether or not the value obtained by dividing by the pixel addition value is greater than or equal to a predetermined threshold value, is only the color component of the predetermined band protrudingly larger than the color component of the other band? Whether or not can be determined easily and accurately.

  Since the band limit filter 12 is rotated around the optical axis of the imaging optical system 9 to change the position of the center of gravity of the color pupil, it is not necessary to secure a place for retracting the band limit filter 12, and the lens unit 1 can be downsized. Can be achieved.

  Further, it is determined whether or not it is appropriate to calculate the subject distance based on the phase difference amount, and when it is determined that the calculation is inappropriate, there is a so-called poor subject because contrast AF is performed. Can also perform AF. An appropriate method of phase difference AF and contrast AF can be selected, and AF reliability can be further improved.

  In addition, when the value obtained by dividing the highest white balance evaluation value among the white balance evaluation values for each band by the second highest white balance evaluation value is equal to or greater than a predetermined threshold value, When it is determined that the subject distance calculation based on the calculation is inappropriate, the white balance function already provided in a general imaging device is used effectively to calculate the subject distance based on the phase difference amount. Can be prevented in advance.

In the first embodiment described above, the band limiting filter 12 is driven to change the position of the center of gravity of the color pupil. However, if the band limiting filter 12 is partially shielded by a member other than the band limiting filter 12, the band limiting filter is used. Even if 12 is fixed, the position of the center of gravity of the color pupil can be changed. Several such embodiments and modifications will be described below.
[Modification of Embodiment 1]

  28 to 30 show modifications of the first embodiment of the present invention, FIG. 28 is a diagram showing a state when the band limiting filter 12 is retracted from the optical path, and FIG. 29 is a band limiting. FIG. 30 is a diagram illustrating a state when the filter 12 is inserted into the optical path, and FIG. 30 is a diagram illustrating a state when the band-limiting filter 12 is inserted partway into the optical path.

  In the modified example of the first embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals, description thereof is omitted, and only different points will be mainly described.

  This modification is an example of realizing switching of a plurality of color pupil barycentric positions using a configuration in which the band limiting filter 12 is inserted / retracted in the optical path.

  First, the configuration of the imaging apparatus of the present modification is basically the same as that shown in FIG. 1 of the first embodiment described above.

  However, as described above, the filter driving unit 17 of the present modification changes the position of the band limiting filter 12 so that the band limiting filter 12 is inserted into the optical path of the imaging optical system 9 or retracted from the optical path. Although a mechanism is provided, a mechanism for rotating the band limiting filter 12 around the optical axis as in the first embodiment is not provided.

  The band limiting filter 12 is basically inserted into the optical path of the imaging optical system 9 when distance measurement based on the phase difference is performed, and is retracted from the optical path when distance measurement based on the phase difference is not performed. .

  Further, the band limiting filter 12 shown in this modification is supported by a light shielding frame 12f having a donut disk shape, for example, and does not include the gear 12a as in the first embodiment.

  As shown in FIG. 28, the state where the band limiting filter 12 does not enter the aperture 11 of the imaging optical system 9 is the retracted state. In this state, it is not possible to acquire distance information based on the phase difference based on the image captured by the image sensor 22, but it is possible to capture a normal still image or moving image that does not have a color shift in the blurred portion.

  28, by changing the position of the band limiting filter 12 in the direction perpendicular to the optical axis of the imaging optical system 9, the band limiting filter 12 passes through the aperture of the diaphragm 11 of the imaging optical system 9. Enter the light path.

  Then, as shown in FIG. 29, the center of the band limiting filter 12 is located on the optical axis, and the state where the band limiting filter 12 completely covers the inside of the aperture 11 of the imaging optical system 9 is an insertion state. In this state, a color shift occurs between the R image and the B image in the out-of-focus portion, and distance information based on the phase difference can be acquired. However, as described above, in the case of a narrow-band subject, distance information may not be acquired from only one image.

  Therefore, the filter driving unit 17 functions as a pupil position changing unit to retract the band limiting filter 12 by a predetermined retraction amount, and realizes a state where the color pupil barycentric position is different from FIG. 29 as shown in FIG. It is like that. In the state shown in FIG. 30, the band limiting filter 12 partially covers the inside of the aperture 11 of the imaging optical system 9 by a predetermined amount. The portions other than the portion where the band limiting filter 12 and the aperture 11 overlap are shielded from light by the light shielding frame 12f.

  At this time, in the present modification, the color pupil barycenter position switching condition selected by the distance calculation unit 37 in step S42 is a drive amount (insertion amount) in the insertion direction of the band limiting filter 12 or the like.

  In order to control the insertion amount of the band limiting filter 12, for example, a stepping motor is used as an actuator for inserting the band limiting filter 12 to count the number of pulses, or a sensor for detecting the insertion amount. It is sufficient to use a general technique as appropriate, such as separately providing the control and the like according to the sensor output.

  30 shows an intermediate state between the retracted state shown in FIG. 28 and the inserted state shown in FIG. 29. Further, the over-inserted state moved further in the inserting direction than the inserted state shown in FIG. If realized and correlation calculation is performed between an image captured in an intermediate state and an image captured in an over-inserted state, the amount of change in the center of gravity of the color pupil can be increased, and distance measurement can be performed with higher accuracy. Can also be performed.

  In this case, the signal correction corresponding to the difference in the amount of received light is the same as described above. Also for the G color, it is possible to change the distribution of the transmission region and move the color pupil center of gravity position.

  According to such a modification of the first embodiment, the filter drive that achieves substantially the same effect as the first embodiment described above and inserts / removes the band limiting filter 12 in a direction perpendicular to the optical axis of the imaging optical system 9. Since the insertion position of the band limiting filter 12 is changed using the unit 17 to change the position of the center of gravity of the color pupil, the band limiting filter 12 is preferably provided (or already provided). The insertion / retraction mechanism is used, and a distance detection based on a phase difference with respect to a narrow-band subject can be performed without using a dedicated mechanism for switching the color pupil center of gravity position. Therefore, it is possible to reduce costs and save space as compared with the case where a dedicated mechanism is provided.

  Further, in the configuration of the first embodiment described above, it is impossible to change the color pupil barycentric position for the G color that passes through the entire region of the band limiting filter 12, but according to the configuration of this modification example, The color pupil barycentric position can also be changed for G color (more generally, the color pupil barycentric position in the band limiting filter 12 is not biased). This G color is close to the luminance component, and in the case of the Bayer array, pixels are arrayed on the image sensor 22 at twice the density of the R color and B color, so phase difference information with higher accuracy is acquired. There are advantages that can be done.

Furthermore, when the position of the band limiting filter 12 is changed to change the color pupil center of gravity position, the aperture 11 is driven to change the color pupil center of gravity position as in Embodiment 2 described later, or implementation described later. Compared to the case where the lens barrier 20 is driven to change the position of the center of gravity of the color pupil as in the third embodiment, the control can be performed independently regardless of the driving state of the diaphragm 11 and the lens barrier 20, and the control is free. There is an advantage of high degree.
[Embodiment 2]

  FIGS. 31 to 34 show the second embodiment of the present invention, FIG. 31 is a block diagram showing the configuration of the imaging apparatus, FIG. 32 is a diagram showing a configuration example of the diaphragm 11, and FIG. FIG. 34 is a diagram showing a state in which the first color pupil barycentric position g1 is realized by driving the diaphragm 11, and FIG. 34 is a state in which the second color pupil barycentric position g2 is realized by driving the diaphragm 11. FIG.

  In the second embodiment, parts that are the same as those in the first embodiment are given the same reference numerals and description thereof is omitted, and only differences are mainly described.

  As illustrated in FIG. 31, the imaging apparatus according to the second embodiment is obtained by removing the filter driving unit 17 from the configuration illustrated in FIG. 1 according to the first embodiment. In this embodiment, a plurality of color pupil barycentric positions are switched by using a plurality of diaphragms 11.

  As shown in FIG. 32, the diaphragm 11 is composed of two diaphragm blades 11u and 11d. V-shaped notches are formed in the diaphragm blades 11u and 11d, and the notches are arranged so as to face each other, for example, in the vertical direction. And the opening diameter is changed by enlarging and reducing the rhombus-shaped opening formed by the combination of the V-shaped notches facing each other according to the movement amount of the diaphragm blades 11u and 11d. .

  In such a configuration, the diaphragm drive unit 18 functions as a pupil position changing unit, and only one diaphragm blade 11d of the diaphragm 11 enters the optical path of the imaging light beam, so that, for example, an R transmission region in the band limiting filter 12 As shown in FIG. 33, the first color pupil barycentric position g1 is realized.

  Further, when the diaphragm drive unit 18 causes only the other diaphragm blade 11u of the diaphragm 11 to enter the optical path of the imaging light beam, for example, the distribution of the R transmission region in the band limiting filter 12 is changed, as shown in FIG. The second color pupil center of gravity position g2 is realized.

  In the present embodiment, the color pupil center-of-gravity position switching condition selected by the distance calculation unit 37 in step S42 is the drive amount of each of the diaphragm blades 11u and 11d.

  Further, even if the diaphragm 11 is opened, different color pupil centroid positions can be realized. However, the greater the amount of movement of the color pupil centroid position before and after the change, the higher the accuracy of phase difference detection. Here, the two apertures 11 are driven one by one to realize the first color pupil barycentric position g1 and the second color pupil barycentric position g2.

  According to such a configuration, in the case of a normal subject that is not a narrow band, distance information based on the phase difference in the horizontal direction can be acquired between the R component and the B component, and the subject of the narrow band subject can be acquired. In this case, by driving the diaphragm 11, it is possible to acquire distance information based on the vertical phase difference as indicated by the first color pupil centroid position g1 and the second color pupil centroid position g2.

  Furthermore, even in the case of a normal subject that is not a narrow band, depending on the subject pattern, distance information based on the phase difference in the horizontal direction may not be acquired. Even in such a case, if the center of gravity position of the color pupil is changed to the vertical direction, distance information based on the phase difference in the vertical direction can be acquired, so that the same effect as a so-called cross sensor can be obtained. There is an advantage that information can be obtained. Therefore, the change of the color pupil centroid position is not limited to the case where the subject is in a narrow band.

  In the examples shown in FIGS. 33 and 34, the movement directions of the diaphragm blades 11u and 11d are up and down, so that the first color pupil centroid position g1 and the second color pupil centroid position g2 change. Although the direction is the vertical direction, if the diaphragm blades moving in different directions are used, the change of the color pupil barycentric position in different directions can also be realized.

  According to the second embodiment, the same effect as that of the first embodiment described above can be obtained, and the position of the center of gravity of the color pupil is changed by driving and controlling the diaphragm. Therefore, it is not necessary to add a dedicated mechanism, and the imaging apparatus can be reduced in size and price.

  The configuration of the second embodiment also has an advantage that the G color pupil center of gravity position can be changed.

Furthermore, the same function as that of the cross sensor can be achieved, and the distance information acquisition accuracy can be improved.
[Modification of Embodiment 2]

  FIGS. 35 and 36 show a modification of the second embodiment of the present invention, and FIG. 35 is a diagram showing a state in which the aperture 11 is opened and the first color pupil center of gravity position g1 is realized. FIG. 36 is a diagram illustrating a state in which the second color pupil center-of-gravity position g2 is realized by reducing the stop 11.

  In the modified example of the second embodiment, the same parts as those of the above-described second embodiment are denoted by the same reference numerals, description thereof is omitted, and only different points will be mainly described.

  This modification is an example in which switching of a plurality of color pupil barycentric positions is realized by using a change in the aperture diameter of the diaphragm 11. And the structure of the imaging device of this modification is fundamentally the same as what was shown in FIG. 31 of Embodiment 2 mentioned above. Although FIGS. 35 and 36 illustrate the case where the diaphragm 11 is an ideal circular diaphragm, it does not matter if it is not a circular diaphragm.

  As shown in FIG. 35, the first color pupil center of gravity position g1 when the diaphragm 11 is opened (for example, when the diaphragm value is set to F2), and when the diaphragm 11 is narrowed as shown in FIG. 36 (for example, the diaphragm value). Although the amount of change is relatively small, the second color pupil center of gravity position g2 (when F8) is generally a different position. That is, the aperture driving unit 18 functions as a pupil position changing unit.

  At this time, in the present modification, the color pupil barycentric position switching condition selected by the distance calculation unit 37 in step S42 is the aperture value of the aperture 11.

  Therefore, it is possible to change the color pupil center of gravity position simply by changing the aperture value of the aperture 11. However, in this case, since the amount of light received by the image sensor 22 is different, the signal amplification amounts of the images acquired before and after the color pupil centroid position change are made different so as to correspond to images when the same amount of light is received. Of course, the correction is made.

According to such a modification of the second embodiment, the same effect as that of the second embodiment described above can be obtained, and a special configuration for changing the position of the center of gravity of the color pupil is not necessary. There is an advantage that it is not necessary to use a special driving method such as driving only the blades.
[Embodiment 3]

  FIGS. 37 to 39 show the third embodiment of the present invention, FIG. 37 is a block diagram showing the configuration of the imaging apparatus, and FIG. 38 shows the first color pupil center of gravity position by driving the lens barrier 20. FIG. 39 is a diagram illustrating a state in which g1 is realized, and FIG. 39 is a diagram illustrating a state in which the lens barrier 20 is driven to realize the second color pupil center of gravity position g2.

  In the third embodiment, parts that are the same as those in the first and second embodiments are given the same reference numerals, description thereof is omitted, and only differences are mainly described.

  As shown in FIG. 37, the imaging apparatus of the third embodiment removes the filter driving unit 17 from the configuration shown in FIG. 1 of the above-described first embodiment, and replaces the lens barrier 20 and the barrier driving unit 19 with new ones. It has been added as a configuration.

    In addition to the control described above in the first embodiment, the lens control unit 14 further controls the barrier driving unit 19.

    The barrier driving unit 19 is configured to open and close the lens barrier 20 electrically based on the control of the lens control unit 14.

  The lens barrier 20 is disposed on the front of the optical path of the imaging optical system 9 and is driven by the barrier driving unit 19 so as to shield the imaging optical system 9 and to capture the imaging optical system 9. This is a protective member that can take a state and a distance measurement state in which a part of the imaging optical system 9 is shielded from light and the other part is opened.

    That is, in the present embodiment, switching of a plurality of color pupil barycentric positions is realized using the lens barrier 20.

  As a specific example, the lens barrier 20 includes two light shielding plates 20u and 20d. Here, the light shielding plate 20u covers the front of the imaging optical system 9 from one side (for example, the upper side), and the light shielding plate 20d covers the front of the imaging optical system 9 from the other side (for example, the lower side).

  The light shielding plates 20u and 20d both cover the imaging optical system 9 with light shielding, thereby realizing a protection state.

  Further, the imaging state is realized by retreating both the light shielding plates 20u and 20d from the optical path of the light incident on the imaging optical system 9.

  Furthermore, the barrier drive unit 19 functions as a pupil position changing unit, and not only protects and captures the above-described protection state and imaging state, but also shields a part of the imaging optical system 9 from the lens barrier 20 serving as a protection member. The center of gravity of the color pupil is changed by taking a state for distance measurement in which a part of the color pupil is opened. As a typical example, this state for distance measurement is realized by only one of the two light shielding plates 20u and 20d covering the imaging optical system 9 with light shielding (however, the present invention is not limited to this typical example). Of course). As shown in FIG. 38, the first color pupil barycentric position g1 is realized by only the light shielding plate 20d covering the imaging optical system 9 with light shielding. As shown in FIG. 39, only the light shielding plate 20u covers the imaging optical system 9 with light shielding, thereby realizing the second color pupil barycentric position g2.

  At this time, in the present embodiment, the color pupil barycenter position switching condition selected by the distance calculation unit 37 in step S42 is the driving amount of each of the light shielding plates 20u and 20d.

  The color pupil barycentric position in the shooting state is also different from the first color pupil barycentric position g1 shown in FIG. 38 and the second color pupil barycentric position g2 shown in FIG. Since the greater the amount of movement, the higher the accuracy of phase difference detection, the two optical pupil centroid positions are realized by covering the imaging optical system 9 with one of the light shielding plates 20u, 20d. .

  Further, in the example shown in FIGS. 38 and 39, the movement direction of the light shielding plates 20u and 20d is the vertical direction, and therefore the change in the first color pupil centroid position g1 and the second color pupil centroid position g2. Although the direction is the vertical direction, if a light-shielding plate that moves in a different direction is used, a change in the position of the center of gravity of the color pupil in a different direction can also be realized.

  According to the third embodiment, the same effects as those of the first and second embodiments described above are obtained, and a part of the imaging optical system 9 is shielded from the lens barrier 20 and the other part is opened. Since the position of the center of gravity of the color pupil is changed by taking the state, a mechanism already provided in a general imaging device can be used, and there is no need to add a dedicated mechanism. Miniaturization and cost reduction can be achieved.

  Also, the configuration of the third embodiment has an advantage that the G color pupil center of gravity position can be changed.

  In the above description, some embodiments and modifications have been described for the configuration of changing the center of gravity when light in a specific band passes through the pupil region of the imaging optical system 9, but the present invention is not limited thereto. .

  For example, in the above description, an example in which one of the diaphragm 11 and the lens barrier 20 is used has been described, but both of them may be used.

  Or the structure using the color selective transmission element divided | segmented into the several segment as the band-limiting filter 12 can be considered. Here, the color selective transmission element includes a plurality of members capable of rotating the polarization transmission axis corresponding to the color (wavelength) and members capable of selectively controlling whether or not the polarization transmission axis rotates, such as an LCD. It is an element that can be changed in color distribution, realized by combination. As a specific example of this color selective transmission element, there is a color switch manufactured by Color Link Co., Ltd. disclosed as “SID '00 Digest, Vol. 31, p. 92” in SID 2000 in April 2000 AD.

  Then, the color selective transmission element is divided into left and right segments, the left segment passes through the G and R components and the B component is blocked, and the right segment passes through the G and B components. By adopting a GB state in which components are blocked, the same configuration as the optical filter of FIG. 3 can be realized. On the contrary, different color pupil barycentric positions can be realized by causing the left segment to take the GB state and the right segment to take the RG state.

  Furthermore, a mechanism that is provided as a standard in recent imaging apparatuses, such as a camera shake correction mechanism, a mechanism that removes dust adhering to the front side of the imaging element 22, and various types that are detachably provided on the optical path. The band limiting filter 12 may be used to change the distribution of the transmission region for each band and to move the center of gravity of the color pupil of the transmitted light.

  Thus, the means for changing the color pupil center of gravity position is not limited to a specific technique, and various techniques can be widely applied.

  In the above description, the imaging apparatus including the distance measuring device has been mainly described. However, the distance measuring method for performing the above-described processing, the distance measuring program for performing the above-described processing, and the computer that records the distance measuring program. It may be a recording medium readable by the above.

  Further, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, you may delete some components from all the components shown by embodiment. Furthermore, the constituent elements over different embodiments may be appropriately combined. Thus, it goes without saying that various modifications and applications are possible without departing from the spirit of the invention.

[Appendix]
According to the above-described embodiment of the present invention described in detail above, the following configuration can be obtained.

(1) A color imaging element that receives light in a plurality of wavelength bands and photoelectrically converts each of the light in a plurality of wavelengths by one imaging, and generates an image in a plurality of bands;
An imaging optical system that forms an image of a subject by a photographic light beam on the imaging element through a pupil region;
The position of at least one of the centroids of the centroids of the amount of light when the light beams of the plurality of bands pass through the pupil regions are arranged on the optical path of the photographing light flux, and at least one other centroid position. A band limiting filter that is different from the position of the center of gravity;
A distance calculation unit that calculates the distance to the subject based on the phase difference amounts of a plurality of images obtained from the image sensor;
A color correlation determination unit that determines whether or not there is a correlation between images in bands of different positions of the center of gravity among the plurality of bands;
A color pupil centroid position changing unit that changes at least a position of a first centroid that is a centroid of one of the plurality of bands;
When it is determined that there is a correlation by the color correlation determination unit, the distance calculation unit is informed to the subject based on a plurality of images determined to have the correlation obtained by causing the image sensor to perform one imaging. When the color pupil barycentric position changing unit changes the position of the first barycentric position, and once before and after the change of the first barycentric position, the distance is calculated. A control unit that controls the distance calculation unit to calculate the distance to the subject based on the two images of the one band obtained by performing each imaging;
A distance measuring device comprising:

(2) The distance measuring device according to (1), wherein the position of the first gravity center is a position decentered from the optical axis center of the imaging optical system.

DESCRIPTION OF SYMBOLS 1 ... Lens unit 2 ... Body unit 9 ... Imaging optical system 10 ... Lens (a focus lens is included)
DESCRIPTION OF SYMBOLS 11 ... Diaphragm 11u, 11d ... Diaphragm blade 12 ... Band-limiting filter 12a ... Gear 12b ... GB filter 12r ... RG filter 12f ... Shading frame 14 ... Lens control part 15 ... Lens side communication connector 16 ... Lens drive part 17 ... Filter drive part (Pupil position change part)
17a ... Motor 17b ... Rotating shaft 17c ... Worm gear 18 ... Aperture drive unit (pupil position changing unit)
19: Barrier driving unit (pupil position changing unit)
20 ... Lens barrier (protective member)
20u, 20d ... light shielding plate 21 ... shutter 22 ... image sensor (color image sensor)
DESCRIPTION OF SYMBOLS 23 ... Imaging circuit 24 ... Imaging drive part 25 ... Image processing part 26 ... Image memory 27 ... Display part 28 ... Interface (IF)
29 ... Recording medium 30 ... System controller (control unit)
DESCRIPTION OF SYMBOLS 31 ... Sensor part 32 ... Operation part 33 ... Strobe control circuit 34 ... Strobe 35 ... Body side communication connector 36 ... White balance adjustment part 37 ... Distance calculation part 38 ... Color correlation determination part 39 ... Contrast AF control part

Claims (11)

An image sensor that receives light, performs photoelectric conversion, and generates a color image of a plurality of bands;
An imaging optical system that forms an image of the light on the imaging device;
A band limiting filter that divides the band into at least two when the light passes through a pupil region of the imaging optical system;
A distance calculation unit that measures the position of the center of gravity for each of the divided bands from the image obtained from the image sensor, and calculates the distance to the subject based on the phase difference amount of the center of gravity position;
A color correlation determination unit that determines whether or not there is a correlation between images of different bands having different center-of-gravity positions for each of the bands;
A pupil position changing unit for changing the position of the center of gravity;
When the color correlation determination unit determines that there is a correlation, the distance calculation unit calculates the distance from the single image captured by the image sensor to the subject, and when it is determined that there is no correlation, the pupil position A control unit that causes the change unit to change the position of the center of gravity, and causes the distance calculation unit to calculate a distance to the subject from at least two captured images before and after the change of the center of gravity position by the imaging device;
A distance measuring device comprising: The distance calculation unit calculates a distance to a subject based on a phase difference amount of ranging areas in the plurality of images.
The color correlation determination unit calculates a pixel addition value in the ranging area for each of the images in the bands having different centroid positions, and calculates a first pixel addition value having the largest value among the calculated values. When a value obtained by dividing by the second pixel addition value having the second largest value is equal to or greater than a predetermined threshold value, it is determined that there is no correlation between images in bands having different centroid positions. The distance measuring device according to claim 1, wherein: The imaging optical system has a diaphragm for changing the pupil region,
The distance measuring device according to claim 2, wherein the pupil position changing unit changes the position of the center of gravity by driving and controlling the diaphragm. The distance measuring device according to claim 2, wherein the pupil position changing unit changes the position of the center of gravity by changing a position of the band limiting filter. The pupil position changing unit is capable of changing a position of the band limiting filter in a direction perpendicular to the optical axis of the imaging optical system, and changing the insertion position of the band limiting filter to change the position of the center of gravity. The distance measuring device according to claim 4, wherein the distance measuring device is changed. The pupil position changing unit is capable of rotating the band limiting filter around the optical axis, and changing the position of the center of gravity by changing a rotation position of the band limiting filter. 4. The distance measuring device according to 4. A protection state in which the imaging optical system is shielded from light, a shooting state in which the imaging optical system is opened, a part of the imaging optical system, and a part of the imaging optical system being shielded. Further comprising a protective member capable of taking the distance measurement state to be opened,
The distance measuring device according to claim 4, wherein the pupil position changing unit changes the position of the center of gravity by causing the protection member to take the state for distance measurement. The imaging optical system includes a focus lens for focus adjustment,
A contrast autofocus control unit that performs contrast autofocus by driving the focus lens based on an image obtained from the image sensor;
The control unit determines whether or not it is appropriate to calculate the subject distance based on the phase difference amount by the distance calculation unit, and when it is determined to be inappropriate, the control unit determines the subject distance by the distance calculation unit. The distance measuring device according to any one of claims 1 to 7, wherein control is performed so that contrast autofocus is performed by the contrast autofocus control unit without performing calculation. A white balance adjustment unit that adjusts the white balance based on the white balance evaluation value of the image obtained from the image sensor;
The control unit is configured such that a value obtained by dividing the first white balance evaluation value that is the highest white balance evaluation value by the second white balance evaluation value that is the second highest white balance evaluation value is a predetermined threshold value or more. 9. The distance measuring device according to claim 8, wherein the distance measuring device determines that the distance calculating unit is inappropriate for calculating a subject distance.   An imaging apparatus comprising the distance measuring device according to any one of claims 1 to 9. Split the light into at least two bands,
Photoelectrically converting the light to generate a multi-band color image;
Measure the center of gravity position for each of the bands divided from the color image,
It is determined whether there is a correlation between any of the images of the bands having different centroid positions for each of the divided bands,
When it is determined that there is a correlation, the distance to the subject is calculated based on the phase difference amount of the barycentric position obtained from one captured image, and when it is determined that there is no correlation, the barycentric position is changed. A distance measuring method comprising: calculating a distance to a subject based on a phase difference amount of the centroid position obtained from at least two captured images before and after the change of the centroid position.

Images