JP2019074623A - Imaging apparatus - Google Patents

Abstract

To achieve high accuracy focus adjustment when detecting a focus by use of beams having different spectral conditions.SOLUTION: An imaging apparatus 100 has: image sensors 101, 102 for imaging a subject image formed by beams from an imaging optical system; an optical element 103 capable of directing a beam in a first spectral state and a beam in a second spectral state different from the first spectral state to the image sensors; and control means 104 for detecting a focus by a phase difference detection method using a signal from the image sensors and adjusting and controlling a focus by using the results of the focus detection. The control means adjusts and controls a focus by using the results of focus detection, first spectral distribution information relating to the spectral distribution of the beam in the first spectral state, and second spectral distribution information relating to the spectral distribution of the beam in the second spectral state.SELECTED DRAWING: Figure 1

Description

  The present invention relates to focusing control of an imaging device having a plurality of imaging elements.

  As an imaging device as described above, Patent Document 1 discloses an imaging device in which an imaging element is provided on each of the transmission side and the reflection side of the half mirror. In such an imaging device, it is also possible to perform focus detection by each imaging device. Patent Document 2 includes two imaging devices that respectively receive visible light and infrared light separated by a color separation prism, and a correction lens for adjusting the image formation position of infrared light, And an imaging device for simultaneously focusing on the same object and imaging with infrared light.

Patent No. 5691440 gazette JP 2004-354714 A

  However, in the half mirror, the spectral transmittance and the spectral reflectance are different according to the incident angle of the light flux which changes according to the image height and the lens pupil distance. For this reason, the spectral state of the light flux is different between the transmission side and the reflection side, and as a result, the focus detection result obtained by the transmission side imaging element is different from the focus detection result obtained by the reflection side imaging element. Similarly, a difference occurs between the focus detection result obtained when the half mirror is disposed in the imaging light path and the focus detection result obtained when the half mirror is retracted outside the imaging light path.

  The present invention provides an imaging device capable of performing high-accuracy focus detection and further focus adjustment control when focus detection is performed using light beams in different spectral states.

  An imaging apparatus according to one aspect of the present invention includes an imaging element for capturing an object image formed by a light flux from an imaging optical system, a light flux in a first spectral state, and a first spectral state with respect to the imaging element. An optical element capable of causing light beams of different second spectral states to enter and focus detection using a phase difference detection method using a signal from an imaging element, and focus adjustment control using the result of the focus detection And control means for The control means includes the result of focus detection, first spectral distribution information on the spectral distribution of the light flux in the first spectral state, and second spectral distribution information on the spectral distribution of the light flux in the second spectral state. It is characterized in that focusing control is performed by using.

  According to another aspect of the present invention, there is provided a focus adjustment control method comprising: an imaging device for capturing an object image formed by a light beam from an imaging optical system; a light beam in a first spectral state with respect to the imaging device; And an optical element capable of causing a light beam of a second spectral state different from the first spectral state to be incident, performing focus detection of the phase difference detection method using a signal from the imaging element, and performing the focal point detection. The present invention is applied to an imaging apparatus that performs focusing control using the result of detection. The focus adjustment control method comprises the steps of acquiring first spectral distribution information on spectral distribution of light flux in a first spectral state, and second spectral distribution information on spectral distribution of light flux in a second spectral state. It is characterized by having the step of acquiring, and performing the focus adjustment control using the result of focus detection, the first spectral distribution information, and the second spectral distribution information.

  A computer program that causes a computer of an imaging device to execute processing corresponding to the above-described focus adjustment control method also constitutes another aspect of the present invention.

  According to the present invention, high-accuracy focus detection and focus adjustment control can be performed when focus detection is performed using light beams in different spectral states.

1 is a block diagram showing the configuration of a camera that is Embodiment 1 of the present invention. The figure which shows the structure of the image pick-up element used for the said camera. FIG. 6 is a diagram showing a relative positional relationship between an exit pupil and a focus detection pupil in Embodiment 1. FIG. 2 is a diagram showing a focus detection area in Embodiment 1. 3 is a flowchart showing imaging processing in Embodiment 1. 6 is a flowchart showing AF processing in Embodiment 1. 6 is a flowchart showing subject information extraction processing in the first embodiment. FIG. 6 is a diagram showing an example of subject information obtained by each image sensor in the first embodiment. FIG. 3 is a diagram showing an AF evaluation band and a captured image evaluation band in Embodiment 1. FIG. 8 is another view showing an AF evaluation band and a captured image evaluation band in Embodiment 1. 6 is a flowchart showing BP amount calculation processing in the first embodiment. FIG. 6 is a diagram showing an example of aberration information of the imaging lens in Embodiment 1. FIG. 6 is a graph showing the angular dependence of the spectral transmittance of the half mirror in Example 1; FIG. 6 is a diagram showing an incident angle to a half mirror in the first embodiment. FIG. 6 is a diagram showing object information extraction timing in the first embodiment. The block diagram which shows the structure of the camera which is Example 2 of this invention. 6 is a flowchart showing an imaging process in Embodiment 2. 6 is a flowchart showing an AF process in Embodiment 2. 10 is a flowchart showing BP amount calculation processing in the second embodiment. FIG. 7 is a diagram showing a retracted state of a half mirror of the camera of the second embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows the configuration of a lens-interchangeable digital camera body 100 and an imaging lens 500 as an imaging apparatus (camera) according to a first embodiment of the present invention. The imaging lens 500 is configured to be attachable to and detachable from the camera body 100. The camera is not limited to the interchangeable lens camera as in this embodiment, and may be a lens integrated camera in which a lens is fixed to the camera body.

  The luminous flux transmitted through the imaging optical system in the imaging lens 500 is incident on a beam splitter 103 as luminous flux splitting means fixed in the camera body 100 and is split into two luminous fluxes. In the present embodiment, the beam splitter 103 is configured by a half mirror as a beam splitting unit that splits an incident beam into a transmitted beam (first beam) and a reflected beam (second beam). The transmitted beam from the beam splitter 103 is directed to the first imaging device 101, and the reflected beam from the beam splitter 103 is directed to the second imaging device 102. The beam splitter 103 may not be a half mirror as long as it can split an incident light beam similarly to the half mirror.

  The imaging surface of the first imaging element 101 (hereinafter referred to as a first imaging surface) and the imaging surface of the second imaging element 102 (hereinafter referred to as a second imaging surface) are optically viewed from the imaging lens 500. It is in the equivalent position. In other words, the first imaging surface and the second imaging surface are each positioned on an imaging surface optically conjugate to the subject via the imaging lens 500.

  An object image having a brightness according to the transmittance and reflectance of the beam splitter 103 is formed on the first and second imaging surfaces. It is desirable that the beam splitter (half mirror) 103 is ideally flat and have a uniform refractive index in the region through which the light flux passes, but in reality it has a certain degree of waviness and refractive index distribution. For this reason, there is a possibility that the image quality of the image obtained by capturing the subject image formed by the light flux transmitted and reflected by the beam splitter 103 may be degraded.

  When the half mirror is formed of thin glass, an image obtained by capturing an object image formed by the reflected light beam is better than an image obtained by capturing the object image formed by the transmitted light beam. The degree of deterioration of the image quality is large.

  For this reason, in this embodiment, the first imaging element 101 on the transmission side is used as an imaging element for high resolution recording, that is, an imaging element for mainly capturing a still image. On the other hand, the second imaging element 102 on the reflection side is used as an imaging element for capturing a moving image having a smaller number of recording pixels than a still image. However, as described later, a moving image may be captured by the first imaging element 101, and a still image may be captured by the second imaging element 102.

  The first and second imaging elements 101 and 102, each of which is a CMOS area sensor, are configured by pixels arranged in a matrix that converts an object image into an electrical signal. The pixel information converted into the electrical signal is subjected to various correction processing for obtaining an image signal and a focus detection signal by the camera CPU 104, and processing for converting the obtained image signal into a live view image and a recorded image. In the present embodiment, the camera CPU 104 performs the above processing and the like, but a dedicated circuit may be provided for each processing.

  The imaging element drive unit 114 moves the second imaging element 102 in the optical axis direction (the direction in which the optical axis of the imaging optical system extends). The first and second imaging elements 101 and 102 are disposed on the optically conjugate imaging plane as described above, but due to the assembly error of the camera body 100 and the difference in the best imaging plane due to the influence of the aberration described later. It is arranged with an error with respect to the conjugate imaging plane. Therefore, when an image is simultaneously obtained through the first and second imaging elements 101 and 102, the first and second imaging elements can be provided only by a focusing lens (third lens group 503), which will be described later, provided on the imaging lens 500. The focus adjustment can not be performed with high accuracy for 101 and 102. Therefore, in the present embodiment, the imaging element driving unit 114 is provided. The imaging device driving unit 114 may move the second imaging device 102 in the optical axis direction as described above, or may move the beam splitter 103 in the same direction. Further, an optical filter such as an infrared cut filter or an optical low pass filter is integrally disposed in each of the first and second imaging elements 101 and 102.

  The operation unit 105 includes various operation members such as a switch operated by the user to set an imaging mode, an imaging condition, and the like, a dial, and the like. The storage medium 106 is configured by a flash memory, and records captured images (still images and moving images).

  The in-finder display 107 includes a display device 108 such as an organic EL element or a liquid crystal element and an eyepiece lens 109, and allows a user looking through the eyepiece lens 109 to observe an image displayed on the display device 108. The external display 110 is configured of an organic EL element, a liquid crystal element, and the like, and is provided on the back surface of the camera body 100. Various setting states of the camera body 100, a live view image, a captured image, and the like are displayed on the in-finder display 107 and the external display 110.

  The focal plane shutter 111 is disposed in front of the first imaging device 101, and controls the exposure time of the first imaging device 101 at the time of capturing a still image. The shutter drive unit 112 is constituted by an actuator such as a motor and its drive circuit, and drives the shutter blades of the focal plane shutter 111. Each of the camera body 100 and the imaging lens 500 is provided with a camera mount and a lens mount for mounting the imaging lens 500 on the camera body 100. The camera side communication terminal 113 provided in the camera mount unit and the lens side communication terminal 508 provided in the lens mount unit enable communication between the camera CPU 104 and the lens CPU 507 described later.

  The imaging optical system in the imaging lens 500 is configured as follows as a zoom lens whose focal length is variable. A light flux from the subject passes through the first lens group 501, the second lens group 502, the iris diaphragm 505, and the third lens group 503 to form a subject image. The second lens group 502 is a variator which moves in the direction of the optical axis to change magnification. The third lens group 503 is a focusing lens (focusing element) which moves in the optical axis direction to perform focusing. The third lens group 503 is moved by a focus drive unit 504 including an actuator such as a stepping motor. The iris diaphragm 505 has a plurality of diaphragm blades for adjusting the amount of light. The diaphragm drive unit 506 includes a diaphragm actuator and its drive circuit, and drives a plurality of diaphragm blades in the opening and closing direction.

  The lens CPU 507 communicates with the camera CPU 104 via the lens communication terminal 508 and the camera communication terminal 113 to transmit and receive various information, and controls the focus driver 504 and the diaphragm driver 506 based on an instruction from the camera CPU 104. Do.

  In this embodiment, the open F number of the imaging lens 500 is a constant value (for example, F2) regardless of the zoom state and the focus state. An exit pupil distance which is a distance between an exit pupil of the imaging lens 500 and each imaging surface (each imaging element) changes in accordance with the zoom state and the focus state.

  2A and 2B show the configuration of the first imaging element 101. FIG. In the present embodiment, the first imaging device 101 and the second imaging device 102 have different pixel pitches and the number of photoelectric conversion units corresponding to one microlens, but the other configurations are the same. . The configuration of the second imaging element 102 will be described later.

  FIG. 2A shows a plurality of pixels in the vicinity of the center (image height 0) of the first imaging device 101 (first imaging surface) as viewed from the imaging lens side. Each pixel has a square shape of 4 μm in size in each of the horizontal (x) direction and the vertical (y) direction on the first imaging surface, and the structure of each pixel is basically the same. The first imaging device 101 is an imaging device having 24 million effective pixels, in which pixels are arrayed 6,000 pixels in the horizontal direction and 4000 pixels in the vertical direction. The size of the first imaging plane can be obtained by multiplying the size of each pixel, ie, the pixel pitch, by the number of pixels, and in this embodiment, it is 24 mm in the horizontal direction and 16 mm in the vertical direction. All the pixels are provided with R, G or B color filters in a mosaic manner.

  FIG. 2B shows the cross-sectional structure of one pixel. A pair of photoelectric conversion units 101 a and photoelectric conversion units 101 b are provided in a silicon substrate 101 d that is a base of the CMOS image sensor. Further, in the silicon substrate 101d, a switching transistor (not shown) or the like (not shown) is formed by converting the photoelectrons generated in each photoelectric conversion unit into a voltage and reading it to the outside. The output signal from each photoelectric conversion unit is read out through the plurality of wiring layers 101 e.

  The plurality of wiring layers 101 e are mutually insulated by the transparent interlayer film 101 f. A color separation color filter 101g is provided below the on-chip microlenses 101c. The shape of the micro lens 101c is determined such that the focal position thereof coincides with the upper surface of the pair of photoelectric conversion units 101a and 101b. Therefore, the pair of photoelectric conversion units 101a and 101b are backprojected near the exit pupil of the imaging lens 500 described later via the micro lens 101c, and the backprojected images become focus detection pupils in phase difference detection focus detection, respectively. . The direction in which the pair of focus detection pupils are arranged in the exit pupil of the imaging optical system (the pupil division direction) is the horizontal direction or the vertical direction.

  When performing phase difference detection direction focus detection, an output signal from the photoelectric conversion unit 101a and an output signal from the photoelectric conversion unit 101b are separately processed to generate a pair of phase difference image signals. And camera CPU104 as a focus detection means calculates the defocusing quantity of the to-be-photographed image in a 1st imaging surface from relative deviation | shift amount (image deviation | shift amount) of a pair of phase difference image signal.

  The camera CPU 104 as a control unit adds the output signals from the pair of photoelectric conversion units 101a and 101b to generate a pixel signal for still image (or moving image) recording or a pixel signal for live view image. The live view image is an image displayed in real time on the in-finder display 107 and the external display 110 before capturing for obtaining a recording image. Note that this addition processing may be performed by a dedicated circuit different from the camera CPU 104. Alternatively, the first imaging device 101 may individually read the output signals of the photoelectric conversion units 101a and 101b, add them in the first imaging device 101, and output the result as a pixel signal.

  FIG. 2C shows the configuration of the second imaging element 102, and shows a plurality of pixels in the vicinity of the center (image height 0) in the second imaging surface as viewed from the imaging lens side. Each pixel has a square shape with a size of 12 μm in each of the horizontal (x) direction and the vertical (y) direction on the second imaging surface, and the structure of each pixel is substantially the same. The second imaging element 102 is an imaging element having about 2.67 million effective pixels, in which 2000 pixels are arranged in the horizontal direction and 1333 pixels in the vertical direction. The size of the second imaging surface is 24 mm in the horizontal direction and 16 mm in the vertical direction. All the pixels are provided with R, G or B color filters in a mosaic manner. Since the pixel pitch of the second image sensor 102 is twice that of the first image sensor 101, the total number of pixels of the second image sensor 102 is 1⁄4 of that of the first image sensor 101.

  Each pixel of the second imaging element 102 is configured in the same manner as the first imaging element 101 with respect to the wiring layer, the microlens, and the color separation color filter. Unlike the first imaging device 101, the second imaging device 102 has nine photoelectric conversion units 102a to 102i under one on-chip microlens 102j. As a result, compared to the first imaging device 101, the angular resolution of the light beam to be received can be increased.

  As described above, from the first imaging device 101, the pair of photoelectric conversion units 101a and 101b are back-projected in the vicinity of the exit pupil of the imaging lens, but from the second imaging device 102, nine photoelectric conversion units 102a to 102i Is backprojected. That is, nine backprojected images become focus detection pupils, respectively. When performing phase difference detection direction focus detection, output signals from a pair of photoelectric conversion units among the nine photoelectric conversion units 102a to 102i are individually processed to generate a pair of phase difference image signals. A horizontal direction, a vertical direction, and an oblique direction can be selected as a pupil division direction in which a pair of focus detection pupils are arranged. Then, the camera CPU 104 calculates the defocus amount of the subject image on the second imaging surface from the relative shift amount of the pair of phase difference image signals.

  Also, the camera CPU 104 adds output signals from all or part of the nine photoelectric conversion units 102a to 102i to generate a pixel signal for moving image (or still image) recording or a pixel signal for live view display. . Note that this addition processing may be performed by a dedicated circuit different from the camera CPU 104. Alternatively, the second imaging device 102 may individually read the output signals of the photoelectric conversion units 102 a to 102 i, add them in the second imaging device 102, and output the result as a pixel signal.

  In the present embodiment, the first and second imaging elements 101 and 102 have a first readout mode and a second readout mode. The first readout mode is an all-pixel readout mode in which output signals are read out from all the pixels in order to capture a still image or a moving image for recording. The second readout mode is a thinning readout mode in which only an output signal from a part of all the pixels is read out in order to display a live view image having a smaller number of pixels than a recording still image. Since the number of pixels required to generate a live view image is smaller than the total number of pixels, the signal processing load is reduced by reading out output signals only from the number of pixels thinned out at a predetermined ratio in both x and y directions from the imaging device Contribute to the reduction of power consumption. Further, since the output signal from each photoelectric conversion unit provided in each pixel is read independently in any of the first and second read modes, generation of a pair of phase difference image signals in any read mode Is possible.

  As described above, in the present embodiment, the first imaging element 101 is used for still image capturing, but moving image capturing may be performed. For example, it is possible to record a low resolution moving image generated from the first imaging device 101 in the second readout mode during moving image imaging by the second imaging device 102. In addition, although the second imaging element 102 is used for moving image capturing in the present embodiment, still image capturing is also possible. For example, it is also possible to acquire a specific one frame image as a recording still image while capturing a moving image.

  Next, with reference to FIGS. 3A to 3D, the relationship between the photoelectric conversion units of the first and second imaging elements 101 and 102 and the focus detection pupil will be described. FIGS. 3A and 3B show the conjugate relationship between the exit pupil plane of the imaging optical system and the photoelectric conversion unit disposed at the position of the image height 0 in the first imaging device 101. The exit pupil plane of the imaging optical system and the photoelectric conversion units 101a and 101b of the first imaging element 101 are in a conjugate relationship by the micro lens 101c. The position of the exit pupil of the imaging optical system generally coincides with the position of the iris diaphragm 505.

  The imaging optical system of the present embodiment is a zoom lens, but depending on the optical type, the power of the exit pupil from the exit pupil to the image plane (image pickup surface) and the size of the exit pupil change depending on the magnification change. The imaging optical system shown in FIG. 3A is in an intermediate zoom state where the focal length is between the wide-angle end and the telephoto end. The eccentricity parameter according to the shape of the micro lens 101c and the image height (X coordinate, Y coordinate) is optimized as the exit pupil distance Zep which is the reference at the exit pupil distance at this time.

  In FIG. 3A, the same components as those shown in FIG. 1 are designated by the same reference numerals as in FIG. A lens barrel member 501r holds the first lens group 501, and a lens barrel member 503r holds the focus lens 503. Reference numeral 505a denotes an aperture plate having an opening that determines the aperture of the iris diaphragm 505, and reference numeral 505b denotes an aperture blade for adjusting the aperture diameter. A lens barrel member 501r as a member for limiting a light flux passing through the imaging optical system, an aperture plate 505a, an aperture blade 505b, and a lens barrel member 503r are shown as optical virtual images when observed from the image plane side. Further, the synthetic aperture in the vicinity of the iris diaphragm 505 is defined as an exit pupil (hereinafter referred to as a lens exit pupil) of the imaging optical system, and the distance from the image plane to the lens exit pupil is Zep as described above.

  The two photoelectric conversion units 101a and 101b are back projected as the images EP1a and EP1b on the exit pupil plane of the imaging optical system by the micro lens 101c. In other words, among the exit pupils of the imaging optical system, different focus detection pupils EP1a and EP1b are projected onto the surfaces of the photoelectric conversion units 101a and 101b via the microlenses 101c.

  FIG. 3B shows the backprojected images EP1a and EP1b of the photoelectric conversion units 101a and 101b on the exit pupil plane of the imaging optical system as viewed from the optical axis direction. The first image sensor 101 can output a signal from one of the two photoelectric conversion units 101a and 101b, and also has a pixel that can add and output signals from both of them. The signal output as a result of addition is a signal obtained by photoelectric conversion of all the luminous fluxes passing through the focus detection pupils EP1a and EP1b.

  In FIG. 3A, a light flux L (the outer edge of which is indicated by a straight line in the figure) passing through the imaging optical system is limited by the aperture plate 505a of the stop 505, and the light flux from the focus detection pupils EP1a and EP1b Reaches the pixel without vignetting in the imaging optical system. In FIG. 3B, the cross section (outer edge) of the light flux L shown in FIG. 3A on the exit pupil plane is indicated as TL. Since most of the backprojected images EP1a and EP1b of the two photoelectric conversion units 101a and 101b are included in the circle indicated by TL (that is, the opening of the aperture plate 505a), the backprojected images EP1a and EP1b are included. It can be seen that only a slight vignetting has occurred. At this time, the vignetting states of the backprojected images EP1a and EP1b at the center of the exit pupil plane are symmetrical with respect to the optical axis of the imaging optical system (indicated by dashed dotted line in FIG. 3A), and the photoelectric conversion units 101a and 101b receive light. The amount of light to be emitted is equal to one another.

  FIGS. 3C and 3D show the conjugate relationship between the exit pupil plane of the imaging optical system and the photoelectric conversion unit disposed at the position of image height 0 in the second imaging element 102. FIG. The exit pupil plane of the imaging optical system and the photoelectric conversion units 102a to 102i of the second imaging element 102 are in a conjugate relationship by the microlens 102j. FIG. 3C shows one pixel of the first lens group 501, the lens barrel member 501r, the iris diaphragm 505, the focus lens 503, the lens barrel member 503r, and the second imaging element 102 described above.

  The nine photoelectric conversion units 102a to 102i are back projected as projected images EP2a to EP2i on the exit pupil plane of the imaging optical system by the micro lens 102j. In other words, the image of the focus detection pupil in which the backprojected images EP2a to EP2i are formed is projected on the surface of the photoelectric conversion units 102a to 102i through the micro lens 101c.

  FIG. 3D shows the backprojected images EP2a to EP2i of the photoelectric conversion units 102a to 102i on the exit pupil plane of the imaging optical system as viewed from the optical axis direction. The second image sensor 102 can output signals from each of the nine photoelectric conversion units 102a to 102i, and has pixels that can add and output signals from the nine photoelectric conversion units 102a to 102i. The signal output as a result of addition is a signal obtained by photoelectric conversion of all the luminous fluxes passing through the focus detection pupils EP2a to EP2i.

  Also in FIG. 3C, the light beam L that has passed through the imaging optical system is limited by the aperture plate 505 a of the iris diaphragm 505. Also in FIG. 3 (d), the cross section (outer edge) of the light beam L shown in FIG. 3 (c) on the exit pupil plane is indicated as TL. Since most of the backprojected images EP2a to EP2i of the nine photoelectric conversion units 102a to 102i are included inside the circle indicated by TL (that is, the opening of the aperture plate 505a), the backprojected images EP2a to EP2i It can be seen that only a slight vignetting has occurred. At this time, the vignetting states of the backprojected images EP2a to EP2i are symmetrical with respect to the optical axis of the imaging optical system at the center of the exit pupil plane, and the light amounts received by the photoelectric conversion units 102a to 102i are equal to each other.

  Here, a pair of phase difference image signals used for focus detection of the phase difference detection method will be described. As described above, in the first imaging element 101, the exit pupil of the imaging optical system is pupil-divided by the micro lens 101c and the two photoelectric conversion units 101a and 101b. Then, by combining the output signals from the photoelectric conversion units 101 a of the plurality of focus detection pixels in the focus detection area arranged in the same pixel row of the first image sensor 101, one of the pair of phase difference image signals A phase difference image signal (hereinafter referred to as an A image signal) is generated. Further, by combining output signals from the photoelectric conversion units 101 b of the plurality of focus detection pixels, the other phase difference image signal (hereinafter, referred to as a B image signal) of the pair of phase difference image signals is generated.

  Both of the A image signal and the B image signal are signal addition processing of output signals from the R pixel, B pixel and two G pixels of Bayer arrangement, and are pseudo-calculated as a luminance (Y) signal. However, the A image signal and the B image signal may be generated for each R pixel, each B pixel, and each G pixel.

  By calculating the phase difference which is a relative shift amount between the A image signal and the B image signal generated in this manner by correlation calculation, it is possible to calculate the defocus amount in the focus detection area using the phase difference. .

  Similarly, the C image signal and the D image signal as a pair of phase difference image signals are generated from the output signal of the second imaging element 102 as well. Since the second imaging device 102 has a larger number of pupil divisions than the first imaging device 101, a plurality of methods of generating C and D image signals can be considered. For example, output signals from the photoelectric conversion units 102a, 102d, and 102g are added to form one addition signal, and the addition signals are connected between a plurality of focus detection pixels to generate a C image signal. Further, output signals from the photoelectric conversion units 102c, 102f, and 102i are added to form one addition signal, and the addition signals are connected between a plurality of focus detection pixels to generate a D image signal.

  As another generation method, for example, only the output signal from the photoelectric conversion unit 102 d is connected between a plurality of focus detection pixels to generate a C image signal, and only the output signal from the photoelectric conversion unit 102 f is connected to perform D An image signal may be generated. Further, as another generation method, the output signals from the six photoelectric conversion units 102a, 102b, 102d, 102e, 102g, and 102h on the left side are added to form one addition signal, and the addition signals are mutually added between a plurality of focus detection pixels. The C image signal may be generated by joining together. In this case, the output signals from the six photoelectric conversion units 102b, 102c, 102e, 102f, 102h, and 102i on the right side are added to form one addition signal, and the addition signals are joined together among a plurality of focus detection pixels. Generate an image signal.

  Note that the number of photoelectric conversion units used to generate each of the pair of phase difference image signals may not be the same. In the region where the image height is high in each imaging element, vignettes occur in the arriving light flux, so the photoelectric conversion unit used to generate the pair of phase difference image signals may be selected in consideration of vignette. Even if there is a difference in the amount of light received, that is, the magnitude of the output signal between the photoelectric conversion units used to generate the pair of phase difference image signals, the magnitude of the output signal can be made equal by shading correction described later ,No problem.

  In the following, both the A and B image signals obtained from the first imaging element 101 and the C and D image signals obtained from the second imaging element 102 are configured to generate a phase difference in the horizontal direction. Will be described. However, the second imaging element 102 can also generate C and D image signals in which a phase difference occurs in the vertical direction. Therefore, in this embodiment, not only the C and D image signals but also the E and F image signals from the second image sensor 102 as a pair of phase difference image signals in which a phase difference occurs in the vertical direction. Generate The photoelectric conversion units used to generate the E image and F image signals are also variously selected similarly to the photoelectric conversion units used to generate the C image and D image signals.

  As described above, each of the first imaging device 101 and the second imaging device 102 has not only a function as an imaging sensor but also a function as a focus detection sensor for a phase difference detection method. That is, the camera body 100 of this embodiment can perform imaging surface phase difference AF (focus detection and focus adjustment control) using the first imaging device 101 or the second imaging device 102 as a focus detection sensor.

  Next, the focus detection area in this embodiment will be described with reference to FIG. A plurality of focus detection areas including a plurality of focus detection pixels for performing pupil division in the horizontal direction are set to the first image sensor 101. Further, for the second imaging element 102, a plurality of focus detection areas in which pupil division is performed in each of the horizontal direction and the vertical direction are set.

  A rectangle indicated by a dotted line in FIG. 4 indicates an imaging region 217 used for imaging in the imaging surfaces (all pixel regions) of the first and second imaging elements 101 and 102. Although the sizes of the imaging regions 217 of the first and second imaging elements 101 and 102 are the same here, they may be different from each other. For example, in the case where the first imaging device 101 performs still image imaging and the second imaging device performs moving image imaging, the imaging area aspect ratios of the first and second imaging devices 101 and 102 may be different. Good. In such a case, when displaying a captured image, it is preferable to make the user recognize the imaging region of each imaging element by clearly indicating by a frame.

  In the imaging area 217 of the first imaging element 101, three focus detection areas 218ah, 218bh, and 218ch for performing pupil division in the horizontal direction are provided at the central part and two places on the left and right of the imaging area 217, respectively. On the other hand, in the imaging area 217 of the second imaging element 102, there are three focus detection areas 218av, 218bv, and 218cv that perform pupil division in the vertical direction with the three focus detection areas 218ah, 218bh, and 218ch. Provided at the central part of the

  By obtaining the A image signal and the B image signal using output signals from pixels included in the focus detection areas 218ah, 218bh, and 218ch of the first image sensor 101, focus detection for an object having contrast in the horizontal direction is performed. It can be carried out. Similarly, by obtaining C and D image signals using output signals from pixels included in the focus detection areas 218 ah, 218 b h and 218 ch of the second image sensor 102, a subject having a contrast in the horizontal direction can be obtained. Focus detection can be performed. Furthermore, by using the output signals from the focus detection areas 218av, 218bv, and 218cv of the second imaging element 102, the E image signal and the F image signal are obtained similarly to the C image signal and the D image signal, thereby the vertical direction It is possible to perform focus detection on an object having a contrast. In the following description, the direction in which the subject has contrast is referred to as the contrast direction.

  Further, FIG. 4 shows the display frames 219a, 219b, 219c for displaying the focus detection area on the external display 110 by broken lines. By making the display frames 219a, 219b, 219c substantially the same size as the focus detection area, it is possible to appropriately perform focus detection on the subject arranged in the display frame by the photographer.

  The flowchart of FIG. 5 shows an imaging process in the present embodiment. The camera CPU 104 executes this process in accordance with an imaging control program which is a computer program. In the following description, "S" means a step.

  When the user turns on the power switch provided in the camera body 100, the camera CPU 104 communicates with the lens CPU 507 in S101, and the open F number of the imaging lens 500, the focal length, the exit pupil distance PL, the focus sensitivity, etc. Receive information The focus sensitivity indicates the amount of movement of the image plane with respect to the amount of movement of the focus lens 503.

  Next, in S102, the camera CPU 104 determines whether the imaging mode currently set by the user is a still image / moving image imaging mode in which still image imaging and moving image imaging are simultaneously performed or a moving image imaging mode in which only moving image imaging is performed. If the camera CPU 104 is in the still image / moving image capturing mode, the process proceeds to step S103. If the camera CPU 104 is in the moving image capturing mode, the process proceeds to step S104.

  In S103, the camera CPU 104 drives the first image sensor 101 for still image shooting in the subject information acquisition mode, and drives the second image sensor 102 for moving image shooting in the live view mode.

  Here, the subject information acquisition mode is a mode for acquiring subject information and focus detection information (defocus amount) to be described later. In this mode, when the focus detection area is plural or wide, the first imaging device 101 is driven in the second readout mode described above. On the other hand, when the focus detection area for performing focus detection is limited by the instruction of the user, the subject detection function, or the like, the first imaging device 101 is driven in the first readout mode.

  The live view mode is a mode for generating a live view image to be displayed on the external display 110. Since the number of pixels of the external display 110 in the horizontal and vertical directions is smaller than the number of pixels of the recording image, in the live view mode, the first image sensor 101 is driven in the second readout mode. Also, focus detection can be performed by acquiring a phase difference image signal in the live view mode. In this case, it is preferable to drive in the first readout mode in order to increase the resolution of the phase difference image signal.

  In S104, the camera CPU 104 drives the second imaging element 102 to perform moving image capturing. At this time, the first imaging device 101 for capturing a still image is not driven. This is because the resolution of an image obtained by moving image capturing is limited by the pixel pitch of the second image sensor 102, and therefore, even if information of a higher frequency is obtained from the first image sensor 101, it can not be effectively used. This makes it possible to reduce power consumption. However, more subject information may be obtained by driving the first image pickup device 101 for still image pickup at a higher frame rate than the second image pickup device 102 for moving image pickup.

  Next, in S105, the camera CPU 104 converts the output signal from the second imaging element 102 into a display image signal, transmits it to the external display 110 or the in-finder display 107, and starts displaying a live view image.

  Next, in step S106, the camera CPU 104 determines the brightness of the image signal obtained by driving each image sensor, and performs aperture control at the time of live view image display.

  Next, in step S107, the camera CPU 104 performs an AF process. In this embodiment, focus detection is performed using the first and second imaging elements 101 and 102, and subject information is acquired. Also, the focus detection result (defocus amount) is corrected using the subject information. Furthermore, the camera CPU 104 drives the focus lens 503 based on the corrected focus detection result, and performs in-focus display. Details of AF processing from focus detection to in-focus display will be described later.

  Next, in S108, the camera CPU 104 determines whether the moving image capturing trigger button has been turned on by the user. If the on-operation has been performed, the camera CPU 104 proceeds to S109, performs image processing for a moving image, and starts generating a moving image. Thereby, the generated moving image is recorded. After this, the process proceeds to S110. If the moving image capturing trigger button has not been turned on, the camera CPU 104 skips S109 and proceeds to S110.

  In S110, the camera CPU 104 determines whether the still image capturing trigger button has been turned on by the user. The camera CPU 104 records a still image by the first imaging element 101 in response to the still image capturing trigger button being turned on during moving image live view or moving image recording. When the still image capturing trigger button is turned on, the camera CPU 104 proceeds to S111. If the still image capturing trigger button has not been turned on, the camera CPU 104 skips S111 and proceeds to S112.

  At S111, the camera CPU 104 captures a still image. At this time, the camera CPU 104 transmits an F number (aperture value) for capturing a still image to the lens CPU 507. The lens CPU 507 controls the aperture opening diameter of the iris diaphragm 505 to a diameter corresponding to the F number for imaging a still image through the aperture driving unit 506. Thereafter, the camera CPU 104 controls the opening and closing of the focal plane shutter 111 to control the exposure of the first image sensor 101, reads an output signal from the first image sensor 101, and generates and records a still image for recording. The still image capturing performed here may be single shooting or continuous shooting, but the case where continuous shooting is performed will be described below.

  Next, in step S112, the camera CPU 104 determines whether the moving image capturing trigger button has been turned off by the user. If the off-operation has not been performed, the camera CPU 104 repeats the processing of S105 to S111 to continue the AF processing for moving images and moving image capturing, and also permits the still image capturing interrupt. When the moving image capturing trigger button is turned off, the camera CPU 104 ends this processing.

  In this embodiment, focus detection and imaging are performed using output signals obtained from the first imaging device 101 for capturing a still image and the second imaging device 102 for capturing a moving image. In order to instruct focus detection and imaging, different operation members such as a still image imaging trigger button and a moving image imaging trigger button are prepared. Thus, not only can control start timings of still image capture and moving image capture independently, but also for focus detection, focus detection for still images can be performed at high speed, and focus detection for moving images can be performed slowly (slowly) Different controls can be performed.

  Next, focus detection processing and AF processing (focus adjustment control method) including the focus detection processing will be described using the flowchart of FIG. The camera CPU 104 executes this process according to a focusing control program which is a part of the imaging control program.

  In step S901, the camera CPU 104 sets (selects) one or more focus detection areas on which focus detection is actually performed from the focus detection areas 219a to 219c illustrated in FIG. This setting may be performed in accordance with a user's instruction, or may be performed based on the result of focus detection performed in advance, the result of object recognition such as face recognition, or the like.

  Next, in step S902, the camera CPU 104 acquires a pair of phase difference image signals from focus detection pixels in the set focus detection area. The A and B image signals are acquired from the first imaging element 101, and the C and D image signals and the E and F image signals are acquired from the second imaging element 102.

  Next, in step S903, the camera CPU 104 performs correction processing and filter processing on the phase difference image signal. As the correction process, a process of adjusting the offset and the gain according to the signal output characteristics of each image sensor is performed. In addition, shading correction is also performed to correct the light amount difference between the pair of phase difference image signals due to the influence of the vignette of the imaging lens 500. As filter processing, digital filter processing is performed in accordance with an evaluation band used for focus detection. Generally, evaluation of a high frequency band narrows the detectable defocus region. Therefore, in order to evaluate a plurality of frequency bands, a plurality of filtering processes are performed.

  The A and B image signals obtained from the first imaging device 101 can be evaluated in a higher frequency band because the pixel pitch of the first imaging device 101 is finer than that of the second imaging device 102. Therefore, the filter processing corresponding to the high frequency band and the middle frequency band is performed on the A image and the B image signal. The C and D image signals and the E and F image signals obtained from the second imaging element 102 are subjected to filter processing in the low frequency band and the ultra low frequency band.

  Next, in step S904, the camera CPU 104 performs a correlation operation on the pair of phase difference image signals (A and B image signals, C and D image signals, E and F image signals) to calculate the pair of phase difference image signals. The phase difference between them is calculated, and the defocus amount is further calculated from the phase difference.

  The correlation operation uses, for example, a correlation amount COR (h) shown in the following equation (1).

In the equation (1), a pair of phase difference image signals are represented by S1 (k) and S2 (k) (1 ≦ k ≦ P). NW1 is the number of in-field data indicating the size of the focus detection area. hmax is a shift data number indicating the maximum shift amount when evaluating the correlation amount while shifting the positional relationship between the pair of phase difference image signals S1 (k) and S2 (k) (that is, changing the shift amount h) . After the correlation amount COR (h) for each shift amount h is determined, the shift amount dh for which the correlation amount COR is minimized (that is, the correlation of the pair of phase difference image signals is maximized) is determined. The shift amount dh corresponds to the phase difference.

  The camera CPU 104 multiplies the shift amount dh as the phase difference thus calculated by the focus sensitivity indicating the image plane movement amount with respect to the unit movement amount of the focus lens 503, etc. calculate. Thus, the camera CPU 104 obtains the first defocus amount (first focus detection result) from the A and B image signals, and the second defocus amount (second focus detection from the C and D image signals. Get the result). Further, a third defocus amount (third focus detection result) is acquired from the E image and the F image signal.

  Next, in step S905, the camera CPU 104 extracts (acquires) subject information from output signals obtained from the first and second imaging elements 102, respectively. The subject information is information on spatial frequency characteristics of the subject, spectral distribution (color), contrast direction, and the like. Details of the subject information extraction process will be described later. After extracting the subject information, the camera CPU 104 calculates a best focus (BP) correction amount as a focus adjustment correction amount corresponding to the subject information in S906.

  In step S 907, the camera CPU 104 performs correction to apply the BP correction amount calculated in step S 905 to the first to third defocus amounts calculated in step S 904. Details of the method of calculating the BP correction amount will be described later.

  Next, in S908, a highly reliable defocus amount is selected from the corrected first to third defocus amounts obtained in S907. For example, when there are many high frequency components as subject information, the first defocus amount obtained from the first image sensor 101 is selected. When there are many components in the vertical direction as the contrast direction, the third defocus amount obtained from the second image sensor 102 is selected. Further, when the defocus amount is large, the reliability is higher if the evaluation band is the low frequency band, so it is determined which of the first to third defocus amounts is to be selected.

  Next, in step S909, the camera CPU 104 moves (drives) the focus lens 503 and the second imaging element 102 based on the selected defocus amount. In this embodiment, in addition to focusing by the movement of the focus lens 503, focusing is also possible by the movement of the second image sensor 102. This corresponds to the above-described BP correction value and the fact that the positions of the first and second imaging elements 101 and 102 are not optically conjugate due to their assembly errors.

  The camera CPU 104 moves the focus lens 503 based on the first defocus amount obtained from the first image sensor 101. Furthermore, the excess or deficiency of focusing due to the movement of the focus lens 503 is compensated by the movement of the second imaging element 102 based on the second or third defocus amount obtained from the second imaging element 102. The camera CPU 104 changes the amount of movement of the second imaging element 102 by changing at least one of aberration information of the imaging optical system and a captured image evaluation band (first imaging characteristic information and second imaging characteristic information) described later. Change accordingly.

  The amount of movement of the second image sensor 102 is the difference between the first defocus amount obtained from the first image sensor 101 and the amount of assembly error of the first and second image sensors 101 and 102 and the amount of BP correction. It may be determined from the value. The movement amount of the focus lens 503 may be determined using the second defocus amount.

  In this way, when the in-focus state is obtained, the camera CPU 104 performs a display for informing the user that the in-focus state has been obtained on the in-finder display 107 or the external display 110 in S910. Then, the process ends.

  Next, a method of extracting subject information in S905 of FIG. 6 will be described using FIGS. 7 to 9. The flowchart of FIG. 7 shows subject information extraction processing.

  First, in step S <b> 9051, the camera CPU 104 extracts subject information using the imaging signal (first imaging signal) obtained from the first imaging element 101. The imaging signal referred to here is a signal obtained by photoelectrically converting a light flux in which a pixel included in the set focus detection area has passed the entire exit pupil of the imaging optical system. In other words, it is a signal obtained by adding the output signals from the photoelectric conversion units 101 a and 101 b of each pixel of the first image sensor 101.

  The camera CPU 104 separates mosaic imaging signals corresponding to the Bayer arrangement into R, G and B color signals, and calculates spatial frequency characteristics in the horizontal direction and vertical direction for each color signal. As a method of obtaining spatial frequency characteristics in the horizontal direction and the vertical direction, a two-dimensional FFT may be used. Further, the signal amount (power) of an imaging signal obtained by performing a plurality of digital filter processing in different frequency bands in the vertical direction and the signal amount of an imaging signal obtained by performing the digital filter processing in the horizontal direction You may get it. As a result, it is possible to obtain the color tone, frequency component and contrast direction of the subject for which focus detection is to be performed.

  In the first image sensor 101, since the pupil division direction at the time of focus detection is only the horizontal direction, when focus detection and imaging are performed using only the first image sensor 101, object information in the vertical direction The extraction of may be omitted.

  Next, in step S9052, the camera CPU 104 extracts subject information using the imaging signal (second imaging signal) obtained from the second imaging element 102, as in step S9051. The imaging signal here is a signal obtained by adding the output signals from the photoelectric conversion units 102 a to 102 i of each pixel of the second imaging element 102.

  FIG. 8 shows the characteristics of subject information obtained by the first image sensor 101 and the second image sensor 102. Since the first imaging device 101 has a smaller pixel pitch than the second imaging device 102, by driving in the above-described subject information acquisition mode as necessary, the high frequency range to the low frequency range with respect to the spatial frequency range Subject information can be obtained in a wide frequency band up to In addition, it is possible to set the charge accumulation time short by specializing the use of the first imaging element 101 for acquiring subject information, and the loss of high frequency components of the imaging signal due to the influence of camera shake or subject shake. It is possible to suppress and obtain subject information. Further, with regard to the contrast direction, both horizontal and vertical directions can be acquired by the above-described method.

  However, in the case where correction processing for an imaging signal and digital filter processing are common to the phase difference image signal, calculation of a low frequency band signal not used for focus detection and filter processing in the vertical direction are additionally required. For the items described as “operation required” for the first imaging device 101 in FIG. 8, the calculation result for the second imaging device 102 is substituted. In addition, the amount of computation can be reduced by using the signal of the sum of a pair of phase difference image signals subjected to various types of processing when extracting subject information.

  On the other hand, since the second imaging element 102 has a pixel pitch larger than that of the first imaging element 101, it is impossible to obtain a signal in a high frequency band. However, by using the signal processed for focus detection, it is possible to obtain subject information in the low frequency band and the very low frequency band without increasing the amount of calculation. Also in the contrast direction, since the phase difference image signal in the vertical direction is obtained, it is possible to obtain subject information in the vertical direction without increasing the amount of calculation.

  As described above, in the present embodiment, subject information acquired from the imaging elements is divided according to the characteristics of the first and second imaging elements 101 and 102. Thereby, necessary and sufficient subject information can be acquired without increasing the amount of calculation.

  Next, in step S9053, the camera CPU 104 integrates subject information obtained from the first and second imaging elements 101 and 102. As a result, it is possible to obtain the magnitude relationship of the object information amount of 24 categories multiplied by the number 3 of colors, the number 2 in the contrast direction, and the number 4 of the frequency bands.

  Note that by obtaining the same subject information, for example, a signal in the middle frequency band in the horizontal direction, from the first and second imaging elements 101 and 102, calibration can be performed between the signals obtained from each.

  Next, in step S9054, the camera CPU 104 calculates a frequency band (hereinafter referred to as an AF evaluation band) on which the phase difference image signal used for focus detection is to be evaluated. The camera CPU 104 calculates the AF evaluation band in consideration of subject information, the optical characteristics of the imaging optical system, the sampling frequency of the imaging device, and the digital filter used for evaluation. The specific AF evaluation band calculation method will be described later.

  Further, the camera CPU 104 calculates an evaluation band (hereinafter referred to as a pickup image evaluation band) of an imaging signal used for generating a recording image (pickup image). The camera CPU 104 calculates a captured image evaluation band in consideration of subject information, optical characteristics of an imaging optical system, a sampling frequency of an imaging element, and an evaluation band of a viewer who interferes with a captured image.

  Calculation of the AF evaluation band and the captured image evaluation band will be described using FIGS. 9A to 9C and FIGS. 10A to 10C. The camera CPU 104 calculates an AF evaluation band for each pair of phase difference image signals, and calculates a captured image evaluation band for each imaging element. The AF evaluation band corresponds to focus detection characteristic information related to the characteristics of the focus detection signal, and is information different according to at least one of the evaluation frequency band of the focus detection signal, the contrast direction, and the color. In addition, the captured image evaluation band calculated for the first imaging element 101 corresponds to the first imaging characteristic information, and the captured image evaluation band calculated for the second imaging element 102 is the second imaging. It corresponds to the characteristic information. Each captured image evaluation band is information on the characteristics of each captured signal. Each captured image evaluation band includes the pixel pitch of each imaging element, the number of photoelectric conversion units provided for each pixel, the pupil division direction in each pixel, and the spectral characteristics for each imaging element of the beam splitter 103 (spectral transmittance and This is different information according to the spectral reflectance).

  9 (a) to (c) and 10 (a) to (c) all show the intensity of the signal for each spatial frequency, the horizontal axis shows the spatial frequency, and the vertical axis shows the intensity.

  FIG. 9A shows the spatial frequency characteristic I of the subject. F1, F2, F3 and F4 on the horizontal axis indicate spatial frequencies to be evaluated, and become higher from F1 to F4. The high frequency band in the subject information obtained in S9053 corresponds to F4, the middle frequency band to F3, the low frequency band to F2, and the very low frequency band to F1.

  The subject information exists for each of the R, G, and B colors and for each contrast direction, but here, only the G subject information in the horizontal direction is shown. Further, Nq is a Nyquist frequency determined according to the pixel pitch of the first imaging device 101. The spatial frequencies F1 to F4 and the Nyquist frequency Nq are similarly shown in FIGS. 9 (b) and 9 (c) and 10 (a) to 10 (c) described later.

  The spatial frequency characteristic I of the subject uses the subject information obtained in S9053. In FIG. 9 (a), the spatial frequency characteristic I of the object is drawn as a curve, but in practice it has discrete values corresponding to the spatial frequencies F1, F2, F3 and F4, and I (n) It is expressed as (1 ≦ n ≦ 4).

  FIG. 9B shows the spatial frequency characteristic O in the in-focus state of the imaging optical system. The information of the space frequency characteristic O may be acquired from the lens CPU 507 or may be stored in a memory such as a RAM in the camera CPU 104 (hereinafter referred to as a camera memory). In FIG. 9B, the spatial frequency characteristic O of the imaging optical system is drawn as a curve, but in practice it has discrete values corresponding to the spatial frequencies F1, F2, F3 and F4, and n) (1 <n <4).

  FIG. 9C shows the spatial frequency characteristic L of the optical low pass filter. The information of the spatial frequency characteristic L is stored in the camera memory. In FIG. 9C, the spatial frequency characteristic L of the optical low pass filter is drawn as a curve, but in practice it has discrete values corresponding to the spatial frequencies F1, F2, F3 and F4, and n) (1 <n <4). In the present embodiment, when different optical low pass filters are used between the first and second imaging elements 101 and 102, it is desirable to separately store the spatial frequency characteristics of the optical low pass filters.

  FIG. 10A shows spatial frequency characteristics M1 and M2 related to signal generation. As described above, the readout mode for reading out the output signals from the first and second imaging elements 101 and 102 and generating the focus detection signal or the imaging signal as the phase difference image signal includes the first readout mode and the first readout mode. There is a second read mode. When signal generation is performed in the first readout mode which is the all-pixel readout mode, the spatial frequency characteristic does not change. A spatial frequency characteristic M1 in FIG. 10A is a spatial frequency characteristic when signal generation is performed in the first readout mode.

  On the other hand, when signal generation is performed in the second readout mode which is the thinning readout mode, the spatial frequency characteristic changes. Specifically, in order to improve the S / N by performing signal addition at the time of thinning in the X direction, a low pass effect is generated by the signal addition. A spatial frequency characteristic M2 in FIG. 10A is a spatial frequency characteristic when signal generation is performed in the second readout mode. The spatial frequency characteristic M2 does not take into account the effect of thinning, and exhibits a low pass effect by signal addition.

  In FIG. 10A, the spatial frequency characteristics M1 and M2 related to signal generation are drawn by curves, but in practice they have discrete values corresponding to the spatial frequencies F1, F2, F3 and F4, and It is expressed as M1 (n), M2 (n) (1 ≦ n ≦ 4).

  FIG. 10B shows a spatial frequency characteristic D1 indicating sensitivity for each spatial frequency when viewing a captured image, and a spatial frequency characteristic D2 of a digital filter used when processing an AF evaluation signal. The sensitivity for each spatial frequency when viewing a captured image is affected by the viewer's individual differences, the image size, the viewing distance, the brightness, and other viewing environments. In this embodiment, as a representative value, the sensitivity for each spatial frequency at the time of viewing is set and stored in the camera memory. The viewing distance means the distance from the user to the display device on which the captured image is displayed or the sheet on which the captured image is printed.

  On the other hand, when using the second read mode, aliasing noise of the frequency component of the signal is generated due to the influence of thinning. The spatial frequency characteristic D2 is a spatial frequency characteristic of the digital filter in which the influence is taken into consideration. In FIG. 10 (b), the spatial frequency characteristics D1 and D2 are drawn as curves, but in practice they have discrete values corresponding to the spatial frequencies F1, F2, F3 and F4, which are represented by D1 (n And D 2 (n) (1 ≦ n ≦ 4).

  As described above, by storing various spatial frequency characteristics in advance, the captured image evaluation band W1 which is imaging characteristic information and the AF evaluation band W2 which is focus detection characteristic information can be expressed by the following formulas (2), (3) Calculated using). In the formulas (2) and (3), n is 1 ≦ n ≦ 4.

W1 (n) = I (n) × O (n) × L (n) × M1 (n) × D1 (n) (2)
W2 (n) = I (n) × O (n) × L (n) × M2 (n) × D2 (n) (3)
FIG. 10C shows the captured image evaluation band W1 and the AF evaluation band W2. Quantify the degree of influence for each spatial frequency with respect to the factor that determines the in-focus state of the captured image by performing the calculations represented by the equations (2) and (3) Can. Similarly, it is possible to quantify how much the error of the defocus amount as the focus detection result has for each spatial frequency. In this embodiment, six captured image evaluation bands W1 and AF evaluation bands W2 are calculated for each of R, G, and B colors and in the horizontal and vertical directions as contrast directions.

  Although each frequency characteristic was explained to Nyquist frequency Nq of the 1st image sensor 101 in Drawing 10 (a)-(c), the frequency characteristic of the 2nd image sensor 102 is computed similarly. For example, the spatial frequency characteristics relating to signal generation are changed with respect to the characteristics for the first imaging device 102 in accordance with the difference in pixel pitch. In addition, spatial frequency characteristics indicating sensitivity for each spatial frequency when viewing a captured image and spatial frequency characteristics of a digital filter used when processing an AF evaluation signal, the second image sensor 102 for the first image sensor 101 Make corrections according to the ratio of the Nyquist frequency. As a result, different results as the captured image evaluation band W1 and the AF evaluation band W2 shown in FIG. 10C can be obtained for the first imaging element 101 and the second imaging element 102.

  Although FIG. 9A to FIG. 9C and FIG. 10A to FIG. 10C have been described using four spatial frequencies F1 to F4, the larger the number of spatial frequencies having data, the larger the captured image evaluation band W1. And the spatial frequency characteristics of the AF evaluation band W2 can be accurately reproduced. Therefore, the correction value can be calculated with higher accuracy.

  The camera CPU 104 that has finished calculating the captured image evaluation band W1 and the AF evaluation band W2 in S9054 ends the present processing.

  Next, a method of calculating the BP correction amount in S906 of FIG. 6 will be described using FIGS. 11 to 14. The flowchart of FIG. 11 shows the BP amount calculation process. In step S9061, the camera CPU 104 acquires the F number set in the aperture control performed in step S106 of FIG.

  Next, in step S9062, the camera CPU 104 obtains information of the focus detection area set in step S901, and calculates an image height of the focus detection area on the imaging device.

  Thereafter, in step S9063, the camera CPU 104 acquires aberration information of the imaging optical system. The aberration information of the imaging optical system is information obtained by communication from the lens CPU 507 in response to the request of the camera CPU 104, and is information on the imaging position of the imaging optical system for each color of the subject, for each contrast direction and for each spatial frequency. . The information on the imaging position for each color of the subject is mainly information on chromatic aberration. The information on the imaging position in each contrast direction of the subject is mainly information on astigmatism. The information on the imaging position for each spatial frequency of the subject is mainly information on spherical aberration.

  The aberration information of the imaging optical system stored in a memory (hereinafter referred to as a lens memory) (not shown) in the imaging lens 500 will be specifically described using FIGS. 12 (a) and 12 (b). FIG. 12A shows the position of the focus lens 503 at which the defocus MTF for each spatial frequency, which is a characteristic of the imaging optical system, becomes a maximum value. The horizontal axis indicates the spatial frequency, and F1, F2, F3 and F4 correspond to F1, F2, F3 and F4 shown in FIGS. 9 (a) to (c) and 10 (a) to (c). There is. Nq is the Nyquist frequency of the first imaging device 102, and Nq2 is the Nyquist frequency of the second imaging device 102. The vertical axis indicates the position of the focusing lens 503 at which the defocus MTF is at the maximum value (hereinafter referred to as the defocus MTF peak position).

  FIG. 12A shows the defocus MTF peak position MTF_GH when the color is G and the contrast direction is the horizontal direction, and the defocus MTF peak position MTF_GV when the color is G and the contrast direction is the vertical direction. There is. When defocus MTF peak positions corresponding to red and blue are included, information on defocus MTF peak positions under a total of six different conditions is stored in the lens memory.

  MTF_GH has defocus MTF peak positions PGH1, PGH2, PGH3 and PGH4 for four spatial frequencies F1, F2, F3 and F4. MTF_GV also has defocus MTF peak positions PGV1, PGV2, PGV3 and PGV4 for the four spatial frequencies F1, F2, F3 and F4. These are also true for R and B.

  In the following description, the spatial frequencies F1, F2, F3 and F4 are represented as F (n) (1 ≦ n ≦ 4), and the color is G, R, B and the contrast direction is horizontal. The aberration information is represented as MTF_GH (n), MTF_RH (n), and MTF_BH (n), respectively. Further, the aberration information of the imaging optical system in the case where the color is G, R, B and the contrast direction is the vertical direction is denoted as MTF_GV (n), MTF_RV (n), MTF_BV (n), respectively. In the lens memory, information is stored as aberration information of the imaging optical system according to the zoom state (focal length), the focus adjustment state (position of the focus lens 503), the F number, and the image height of the focus detection area. For example, the zoom state, the focus adjustment state, and the F number are divided into eight zones, and aberration information is stored for each of the image heights divided into three for each divided zone. However, other methods of storing aberration information may be employed.

  In S9063, the camera CPU 104 transmits, to the lens CPU 507, information of the F number and the image height of the focus detection area obtained in advance. The lens CPU 507 selects the F number, the image height of the focus detection area, and the aberration information corresponding to the current zoom state and the focus adjustment state from the lens memory, and transmits the selected information to the camera CPU 104.

  Next, in step S9064, the camera CPU 104 processes the aberration information of the imaging optical system using the optical information of the half mirror (optical element) that is the beam splitter 103. The processing of the aberration information will be described with reference to FIG.

  The half mirror does not have distortion or the like for the second imaging element 102 that the light flux (light flux in the second spectral state) transmitted by the imaging optical system and reflected by the half mirror reaches, and aberration is newly generated by reflection. If it does not occur, it may be used without processing the aberration information of the imaging optical system. However, the light flux (the light flux in the first spectral state) which passes through the imaging optical system and also through the half mirror and reaches the first imaging element 101 is determined by the refractive index and dispersion according to the material of the half mirror. Different refraction occurs depending on the incident angle to the half mirror and the wavelength. Since the aberration information has information on the defocus MTF peak position which is the position of the focus lens 503 corresponding to the maximum value of the defocus MTF, the defocus MTF peak position is offset when the light beam passes through the half mirror. The offset amount can be calculated from the incident angle of the light beam to the half mirror determined by the color (wavelength), the exit pupil distance and the image height.

  FIG. 12B shows MTF_GH2 which is aberration information offset by transmitting through the half mirror with respect to MTF_GH (shown by a broken line) before offset. Although not shown, MTF_GV2, MTF_RH2, MTF_RV2, MTF_BH2 and MTF_BV2 can be calculated by similarly calculating offset amounts of other aberration information.

  Next, in S9065, the camera CPU 104 acquires, from the camera memory, the information of the imaging evaluation band W1 and the AF evaluation band W2 calculated in S9054. As the imaging evaluation band W1, W1_a (n) corresponding to the captured image acquired by the first imaging device 101 and W1_b (n) corresponding to the captured image acquired by the second imaging device 102 are acquired. Do. As the AF evaluation band W2, W2_a (n) for the A and B image signals, W2_b (n) for the C and D image signals, and W2_c (n) for the E and F image signals Get each one. Information of these evaluation bands (coefficients) is acquired in total of six each of three colors (RGB) × two contrast directions (horizontal and vertical directions). For example, six W2_a_RH (n), W2_a_RV (n), W2_a_GH (n), W2_a_GV (n), W2_a_BH (n) and W2_a_BV (n) are acquired as W2_a (n).

  Furthermore, a total of 24 coefficients are obtained by including these four coefficients corresponding to four spatial frequencies, respectively. These 24 coefficients indicate the magnitude relation of the amount of information for each combination of color, spatial frequency and contrast direction. In this embodiment, these 24 coefficients are normalized so that the sum becomes 1 and used as a weighting for subject information.

  Next, in step S9066, the camera CPU 104 performs information on the spectral distribution (first spectral distribution information) according to the spectral transmittance of the half mirror and information on the spectral distribution based on the spectral reflectance of the half mirror (second spectral distribution information Get). The information on the spectral distribution means information indicating the spectral distribution itself, or information that is not the spectral distribution itself but corresponds to the spectral distribution. The angular dependence of the spectral transmittance and the spectral reflectance of the half mirror will be described with reference to FIGS. 13 and 14.

  FIG. 13 shows the spectral transmittance of the half mirror for each light incident angle. The abscissa represents the wavelength, and the ordinate represents the transmittance. The spectral reflectance is a value obtained by subtracting the spectral transmittance from 100%. When the half mirror is formed of a dielectric multilayer film, the transmittance changes depending on the incident angle and wavelength of the light beam. FIG. 13 shows spectral transmittances at incident angles of 30 degrees, 45 degrees, and 60 degrees. In the camera memory, spectral transmittances at incident angles of 30 degrees, 45 degrees, and 60 degrees are stored as spectral transmittances of the three colors of RGB corresponding to the main wavelength of the color filter of the imaging device.

  FIG. 14 shows that the incident angle to the half mirror (beam splitter 103) changes depending on the exit pupil distance and the image height. When the imaging lens 500 is replaceable or a zoom lens as in this embodiment, the exit pupil distance changes according to the replacement and the focal length. In FIG. 14, LPO1 and LPO2 are shown as examples of different exit pupil distances, and the image heights IH1 and IH2 on the first imaging element 101 are reached through the center of the exit pupil of the imaging optical system enclosure. Shows a ray of light.

  As can be seen from the light rays reaching the image height IH1 from the respective exit pupils corresponding to the exit pupil distances LPO1 and LPO2, those light rays have different incident angles with respect to the half mirror. Further, as can be seen from light rays reaching each of the image heights IH1 and IH2 from the exit pupil corresponding to the exit pupil distance LPO1, those light rays have different incident angles with respect to the half mirror. In step S9066, the camera CPU 104 calculates the incident angle of the light beam on the half mirror based on the exit pupil distance and the image height at which the BP correction amount is calculated. Thereafter, the camera CPU 104 calculates the reflectance and transmittance at each wavelength from the obtained incident angle by interpolation calculation or the like.

  Then, the camera CPU 104 obtains the reflectances (%) of R, G and B as Rr, Gr and Br, respectively, and the transmittances (%) as Rt, Gt and Bt as the information of the spectral distribution. As described above, since there is a relationship of Rt = 100% −Rr, either one of Rt and Rr may be calculated.

  In FIG. 14, a light beam passing through the center of the exit pupil has been described as an example, but the actual light beam passes through the opening on the exit pupil determined by the iris diaphragm 505 or the lens frame and the like. Have. Therefore, the incident angle may be calculated using F number corresponding to the size of the opening of the iris diaphragm 505, and vignette information indicating a vignetting state by a lens frame or the like.

  In step S9067, the camera CPU 104 calculates the focal position assumed by the focus detection signal using the spectral transmittance or spectral reflectance of the half mirror, the AF evaluation band W2 and the aberration information of the imaging optical system. The correlation calculation between the A and B image signals obtained from the first imaging element 101 that receives the light flux transmitted through the half mirror (hereinafter referred to as the first correlation calculation) is performed only in the horizontal direction. The camera CPU 104 calculates the focus position P_AF1 assumed by the focus detection signal using the following equation (4).

The focal position using the first correlation calculation is calculated by weighting and adding the aberration information of the imaging optical system based on the spectral distribution of the object and the spectral transmittance of the half mirror by calculation using Equation (4). can do. P_AF1 corresponds to the defocus MTF peak position assumed from the distribution (spectral characteristics) of colors forming the focus detection signal and the distribution (spatial frequency characteristics) of spatial frequencies in the aberration information of the imaging optical system.

  Further, the correlation calculation between the C and D image signals obtained from the second imaging element 102 that receives the light beam reflected by the half mirror (hereinafter referred to as the second correlation calculation) is performed only in the horizontal direction. The camera CPU 104 calculates the focus position P_AF2 assumed by the focus detection signal using the following equation (5). That is, the focal position using the second correlation calculation can be calculated by weighting and adding the aberration information of the imaging optical system based on the spectral distribution of the object and the spectral reflectance of the half mirror.

Further, the correlation calculation between the E image and the F image signal obtained from the second imaging element 102 (hereinafter referred to as the third correlation calculation) is performed only in the vertical direction. The camera CPU 104 calculates the focus position P_AF3 assumed by the focus detection signal using the following equation (6). That is, the focal position using the third correlation operation can be calculated by weighting and adding the aberration information of the imaging optical system based on the spectral distribution of the object and the spectral reflectance of the half mirror.

In Equation (6), since the focus position is calculated using the contrast signal in the vertical direction, the aberration information of the imaging optical system and the information of the AF evaluation band in the vertical direction are used.

  Next, in step S9068, the camera CPU 104 calculates the focal position in the case of assuming an imaging signal, using the spectral transmittance of the half mirror, the imaging evaluation band, and the aberration information of the imaging optical system.

  A still image (hereinafter, referred to as a first captured image) obtained using the first imaging element 101 is obtained by photoelectric conversion of an object image formed by a light beam transmitted through a half mirror. Further, the evaluation of the focusing state is performed by also evaluating the contrast in the horizontal and vertical directions. Therefore, the focal position P_IMG1 assumed in the evaluation of the focusing state in the first captured image is expressed by the following equation (7) Calculated using).

P_IMG1 corresponds to the defocus MTF peak position assumed from the distribution of the color evaluated in the first captured image, the distribution of the spatial frequency, and the distribution of the contrast direction in the aberration information of the imaging optical system.

  A moving image (hereinafter referred to as a second captured image) obtained using the second imaging element 101 is obtained by photoelectric conversion of an object image formed by the light beam reflected by the half mirror. The evaluation of the focusing state is also performed by evaluating the horizontal and vertical contrasts. Therefore, the focal position P_IMG2 assumed in the evaluation of the focusing state in the second captured image is calculated using the following equation (8).

As described above, in steps S9067 and S9068, the camera CPU 104 estimates the defocus MTF peak assumed using the characteristics (color, spatial frequency and contrast direction) of the focus detection signal or the imaging signal and the aberration information of the imaging optical system. Calculate the position.

Next, at S9069, the camera CPU 104 calculates a BP correction amount. The BP correction amount differs depending on which defocus amount as a focus detection result is used for focus adjustment of which captured image. Assuming that the BP correction amount in the case of using the first defocus amount for focusing the first captured image is BP1, and the BP correction amount in the case of using the second imaging image for focus adjustment is BP2, the camera CPU 104 It calculates using following formula (9), (10).
BP1 = P_IMG1-P_AF1 (9)
BP2 = P_IMG2-P_AF1 (10)
Further, assuming that the BP correction amount when using the second defocus amount for focusing of the first captured image is BP3 and the BP correction amount when using for focusing of the second captured image is BP4, the camera CPU 104 These are calculated using the following formulas (11) and (12).
BP3 = P_IMG1-P_AF2 (11)
BP4 = P_IMG2-P_AF2 (12)
Furthermore, assuming that the BP correction amount when using the third defocus amount for focusing of the first captured image is BP5 and the BP correction amount when using for focusing of the second captured image is BP6, the camera CPU 104 These are calculated using the following equations (13) and (14).
BP5 = P_IMG1-P_AF3 (13)
BP6 = P_IMG2-P_AF3 (14)
The camera CPU 104 switches and uses the BP correction amounts BP1 to BP6 calculated in S9069 according to the subject information. For example, when the subject has a contrast in the vertical direction and the reliability of the focus detection result (third defocus amount) obtained by the third correlation operation is high, the BP correction amount for the first image sensor 101 The focus detection result is corrected using BP5 as (first focus adjustment correction amount). Then, using the focus detection result after correction, the amount of movement of the focus lens 503 is calculated. Also, in the same case, the focus detection result is corrected using BP6 as the BP correction amount (second focus adjustment correction amount) for the second imaging element 102, and the second after the correction using the focus detection result. The movement amount of the imaging element 102 is calculated. The amount of movement of the second imaging element 102 is changed in accordance with at least one of the aberration information of the imaging optical system, the first imaging element information, and the second imaging element information.

  The camera CPU 104 sets the first spectral state and the second spectral state in the following combination with respect to the BP correction amount for the first imaging device 101 and the BP correction amount for the second imaging device 102. Do.

  For example, with regard to BP1 and BP2, the first spectral state is a spectral state at the time of focus detection of a light beam which passes through the half mirror and reaches the first imaging element 101, and passes through the half mirror to perform the first imaging The combination with the spectral state at the time of imaging of the light flux reaching the element 101 is used. In addition, the second spectral state is reflected by the half mirror to reach the second imaging element 102, and the spectral state at the time of focus detection of the light flux is reflected by the half mirror to reach the second imaging element 102 The combination of spectral states at the time of imaging. However, the first spectral state and the second spectral state are not limited to the combination described above, and may be a combination for BP1 and BP4 or the like.

  As described above, in this embodiment, the estimated focal positions (P_AF1, P_AF2, P_AF3) are calculated according to the characteristics of the focus detection signals acquired from the first and second imaging elements 101 and 102, respectively. Do. Further, in accordance with the characteristics of the imaging signals acquired from each of the first and second imaging elements 101 and 102, assumed focal positions (P_IMG1 and P_IMG2) are calculated. Then, by calculating the BP correction amount using these assumed focus positions, it is possible to perform focus adjustment control according to the characteristics of the captured image while adopting a highly reliable focus detection result.

  In this embodiment, the second imaging element 102 is configured to be movable in the optical axis direction in preparation for calculating different values as the BP correction amount for still image capturing and the BP correction amount for moving image capturing. That is, focusing is performed by the movement of the focus lens 503 and the movement of the second imaging element 102. However, the second imaging element 102 may not necessarily be movable. In this case, for example, an average value (third focus adjustment correction) of the BP correction amount (first focus adjustment correction amount) for still image capturing and the BP correction amount (second focus adjustment correction amount) for moving image capturing Amount may be used to move the focus lens 503 to perform focus adjustment. As a result, although the focus adjustment accuracy is reduced, a mechanism for moving the second image sensor 102 is not necessary.

  In addition, when the second imaging element 102 is not moved, the recording priority of the captured image obtained from the first and second imaging elements 101 and 102 is determined by the setting of the user or the camera main body 100, and the recording priority The BP correction amount (third focus adjustment correction amount) may be set in accordance with. For example, when the still image obtained by the first imaging device 101 has a high priority, the BP correction amount for the first imaging device 101 and the BP correction amount for the second imaging device 102 are weighted by 4: 1. The focus adjustment control may be performed by calculating (setting) the correction amount.

  As described above, the magnitude of the BP correction amount is determined by the aberration information of the imaging optical system. Therefore, the magnitude of the aberration amount may be determined from the aberration information, and the calculation of the BP correction amount (first or second focus adjustment correction amount) may be omitted if the aberration amount is small.

  Next, the extraction timing of the subject information using the first and second imaging elements 101 and 102 performed by the camera CPU 104 in S9051 and S9052 will be described using FIG. FIG. 15 shows an example in which the vertical synchronization signals V1, V2, V3, V4,... Of the first and second imaging elements 101, 102 are synchronized. As indicated by Ea1, Ra1, Ea2, Ra2,..., The first imaging element 101 repeats charge accumulation and readout at the boundaries of vertical synchronization timings V1, V2,. As indicated by Eb1, Rb1, Eb2, Rb2,..., The second imaging element 102 repeats charge storage and readout at boundaries of vertical synchronization timings V1, V2,.

  Since the second imaging element 102 is for moving image capturing, the charge accumulation time is set to be longer than that of the first imaging element 101 for capturing a still image. However, since the second imaging device 102 has a smaller number of pixels than the first imaging device 101, the readout time of the output signal is short. The first imaging element 101 functions as a subject information extraction unit, and therefore, is driven in a short charge accumulation time to reduce the influence of camera shake and subject shake.

  After reading the output signals from the first and second imaging elements 101 and 102, the camera CPU 104 performs detection D1, D2, D3,... Of subject information using the acquired output signals. Then, when detection of subject information is completed, the camera CPU 104 performs calculation C1, C2, C3,... Of BP correction amount using the obtained subject information.

  By performing such processing, subject information at approximately the same timing can be obtained through the first and second imaging elements 101 and 102, so it is difficult to be affected by changes in subject information due to time lag, and a highly reliable subject Information can be obtained.

  By narrowing the extraction area of the subject information in the first image sensor 101, the drive rate (frame rate) of the first image sensor 101 may be doubled to obtain more information. At this time, half of the data can not be synchronized with the information obtained from the second imaging device 102, but the reliability is higher by determining continuity from the information obtained from the first imaging device 101. Subject information can be extracted. In addition, when the drive rate of the second image sensor 102 is high, the drive of the first image sensor 101 may be thinned to lengthen the extraction interval of the subject information. Although this reduces the subject information to be obtained, it is possible to reduce the power consumption.

  As described above, in the present embodiment, the focal position is calculated by the weighting operation using the spectral transmittance and the spectral reflectance of the half mirror. As a result, it is possible to perform focus detection and focus adjustment control in consideration of the difference in focus position due to the difference in spectral state between the reflection side and the transmission side of the half mirror.

  In the present embodiment, the case where the first focus adjustment correction amount is calculated from the first spectral distribution information and the second focus adjustment correction amount is calculated from the second spectral distribution information has been described. However, when the first spectral distribution information and the second spectral distribution information are close to each other, one of the first and second focus adjustment correction amounts may be calculated.

  Next, a camera according to a second embodiment of the present invention will be described. In the first embodiment, the first spectral state is a combination of the spectral state at the time of focal point detection on the transmission side of the half mirror and the spectral state at the time of imaging, and the second spectral state is at the spectral state at focal point detection on the reflective side. First and second focus adjustment correction amounts were calculated as a combination with the spectral state at the time of imaging.

  On the other hand, in the present embodiment, the first spectral state is a combination of the spectral state at the time of focus detection when passing through the half mirror and the spectral state at the time of imaging when the half mirror is retracted from the light path. Further, the second spectral state is a combination of the spectral state at the time of focus detection when the half mirror retracts and the spectral state at the time of imaging.

  FIG. 16 shows the configuration of the camera body 1500 and the imaging lens 500 of this embodiment. Components in FIG. 16 common to those in Embodiment 1 (FIG. 1) are assigned the same reference numerals as in Embodiment 1. The camera body 1500 of the present embodiment does not have the second imaging element 102 and the display device 108 in Embodiment 1, but has a pentaprism 1501. The pentaprism 1501 guides the light beam reflected by the half mirror 103 to the eyepiece lens 109. The pentaprism 1501 and the eyepiece lens 109 constitute an optical finder. Further, in Example 1, a half mirror as the beam splitter 103 fixed in the optical path from the imaging lens 500 is provided, but in this example, it is an optical element that can be moved back and forth (inserted and removed) with respect to the optical path. A half mirror 1502 is provided.

  A general imaging device guides a light flux to the AF dedicated sensor and the in-finder display 107 by the half mirror 1502, but this embodiment does not have the AF dedicated sensor. For this reason, in the present embodiment, while observing the subject image through the in-finder display 107, focus detection is performed using the light flux transmitted through the half mirror 1502 by the first imaging device 101.

  The flowchart of FIG. 17 shows an imaging process in the present embodiment. The camera CPU 104 executes this process in accordance with an imaging control program which is a computer program. The same steps as those of the first embodiment (FIG. 5) are denoted by the same reference numerals as those of the first embodiment.

  In S1601 after S101, the camera CPU 104 determines whether the set imaging mode is a finder imaging mode or a live view imaging mode. The camera CPU 104 proceeds to S1602 when the finder imaging mode is set, and proceeds to S1604 when the live view imaging mode is set.

  The operation of the camera CPU 104 in the finder imaging mode will be described. In S1602, the camera CPU 104 places the half mirror 1502 in the optical path (mirror down) in order to guide the light flux from the imaging lens 500 to the pentaprism 1501.

  Subsequently, in S107, the camera CPU 104 performs an AF process while the mirror is down. The AF processing will be described later.

  Next, in S1603, as shown in FIG. 20, the camera CPU 104 retracts the half mirror 1502 out of the optical path, and guides the light flux from the imaging lens 500 to the first imaging element 101 without transmitting the half mirror 1502.

  Subsequently, at S111, the camera CPU 104 performs still image capturing. By imaging the still image by retracting the half mirror 1502 out of the optical path, the image quality of the obtained still image may be degraded due to the expansion of the aberration of the imaging optical system by the half mirror 1502 or the light amount may be reduced. It is prevented.

  The operation of the camera CPU 104 in the live view imaging mode will be described. In step S1604, the camera CPU 104 retracts the half mirror 1502 out of the optical path (mirrors up). By guiding the light flux from the imaging lens 500 to the first imaging device 101 without transmitting the half mirror 1502, a good live view image can be obtained.

  Subsequently, in S107, the camera CPU 104 performs an AF process with the mirror up. Then, the process proceeds to step S111 to perform still image capturing.

  The focus detection process and the AF process including the same will be described with reference to the flowchart of FIG. The same steps as those of the first embodiment (FIG. 6) are denoted by the same reference numerals as those of the first embodiment.

  The camera body 1500 of this embodiment does not have the second imaging element 102, and thus does not perform the subject information extraction process described in the first embodiment. Further, since the camera body 1500 does not have the imaging element driving unit 114, the driving of the focus lens 503 and the second imaging element 102 in S909 of FIG. 6 is the driving of the focus lens 503 in S1601. The other processes are the same as in the first embodiment.

  The flowchart of FIG. 19 shows the BP amount calculation process in this embodiment. The same steps as those of the first embodiment (FIG. 11) are denoted by the same reference numerals as those of the first embodiment. In the first embodiment, since the spectral state is different between the reflection side and the transmission side of the beam splitter (half mirror) 103, the camera CPU 104 acquires information of spectral distribution based on the spectral transmittance of the half mirror 1502 in S9066.

  On the other hand, in this embodiment, the spectral state (first spectral state) of the light beam transmitted through the half mirror 1502 in the finder imaging mode and the spectral state of the light beam not transmitted through the half mirror 1502 retracted in the live view imaging mode ( The second spectral state is different. For this reason, in S1801, the camera CPU 104 acquires information on the spectral distribution of the light flux transmitted through the half mirror 1502 (first spectral distribution information: hereinafter referred to as transmission spectral distribution information), and uses this to obtain the BP correction amount ( The first focus adjustment correction amount is calculated. Specifically, in the finder imaging mode, the camera CPU 104 calculates the BP correction amount according to the combination of the spectral state at the time of focus detection of the light flux transmitted through the half mirror and the spectral state at the time of imaging with the half mirror retracted. .

  In step S9067, the camera CPU 104 calculates the focus position when the focus detection signal is assumed, using the transmission spectral distribution information acquired in step S1801.

  Then, in step S9068, the camera CPU 104 does not cause a change due to the half mirror 1502 in the spectral state when the half mirror 1502 is retracted, so the focal position in the case where the imaging signal is assumed is used without using the transmission spectral distribution information. calculate.

On the other hand, in the live view imaging mode, the camera CPU 104 calculates the BP correction amount according to the combination of the spectral state at the time of focus detection when the half mirror 1502 is retracted and the spectral state at the time of imaging. In the live view imaging mode, each of AF and imaging is performed in a spectral state in which the half mirror 1502 is retracted. Therefore, in S9067 and S9068, the camera CPU 104 does not use the transmission spectral distribution information (in other words, uses the second spectral distribution information different from the transmission spectral distribution information) and assumes the focus detection signal. The position and the focal position assuming the imaging signal are calculated. The processes other than the above are the same as in the first embodiment.
(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

  The embodiments described above are only representative examples, and various modifications and changes can be made to the embodiments when the present invention is implemented.

100 Camera Body 101 First Image Sensor 102 Second Image Sensor 104 Camera CPU
500 imaging lens 503 focus lens (third lens group)
507 lens CPU

Claims (9)

An image pickup element for picking up an object image formed by a light flux from the image pickup optical system;
An optical element capable of causing a light flux in a first spectral state and a light flux in a second spectral state different from the first spectral state to be incident on the imaging element;
Control means for performing focus detection of a phase difference detection method using a signal from the imaging element and performing focus adjustment control using a result of the focus detection;
The control means
The result of the focus detection, first spectral distribution information on the spectral distribution of the light flux in the first spectral state, and second spectral distribution information on the spectral distribution of the light flux in the second spectral state An imaging device that performs the focusing control using the   The image pickup apparatus according to claim 1, wherein the first spectral distribution information and the second spectral distribution information are information according to an incident angle of a light beam from an imaging optical system to the optical element. .   The first spectral distribution information and the second spectral distribution information each include information on the spectral distribution at the time of focus detection and information on the spectral distribution at the time of imaging. The imaging device according to. The control means
A first focus adjustment correction amount is acquired using the first spectral distribution information, and a second focus adjustment correction amount is acquired using the second spectral distribution information.
The focus adjustment control is performed using the result of the focus detection, the first focus adjustment correction amount, and the second focus adjustment correction amount. The imaging device according to. The optical element splits the light flux from the imaging optical system into a first light flux in the first spectral state and a second light flux in the second spectral state.
The imaging device includes a first imaging device for capturing a subject image formed by the first light flux, and a second imaging device for capturing a subject image formed by the second light flux,
Using the result of the focus detection using the signal from at least one of the first and second imaging elements, the first spectral distribution information, and the second spectral distribution information, the first The imaging apparatus according to any one of claims 1 to 4, wherein the focusing control is performed to obtain an in-focus state with respect to the second imaging element and the second imaging element.   The control means moves the focusing element included in the imaging optical system to perform the focusing control for obtaining the in-focus state with respect to the first imaging element, and moves the second imaging element to perform the focusing control. The imaging apparatus according to claim 5, wherein the focusing control is performed to obtain a focusing state for the second imaging element.   The optical element can be disposed in the optical path from the imaging optical system and outside the optical path, and when disposed in the optical path, causes the light flux in the first spectral state to be incident on the imaging element. The imaging device according to any one of claims 1 to 4, wherein the light beam in the second spectral state is made incident on the imaging device when the light beam is arranged outside the optical path. An imaging device for capturing an object image formed by a light beam from an imaging optical system, and a light beam in a first spectroscopic condition and a light beam in a second spectroscopic condition different from the first spectroscopic condition with respect to the imaging device And an optical element capable of making the light incident thereon, wherein focus detection using a phase difference detection method is performed using a signal from the image pickup element, and focus adjustment control is performed using a result of the focus detection. A focusing control method,
Acquiring first spectral distribution information related to the spectral distribution of the light flux in the first spectral state;
Acquiring second spectral distribution information on the spectral distribution of the light flux in the second spectral state;
A focus adjustment control method comprising: performing the focus adjustment control using a result of the focus detection, the first spectral distribution information, and the second spectral distribution information.   A focusing control program that is a computer program that causes a computer of an imaging device to execute processing corresponding to the focusing control method according to claim 8.