JP2017032978A - Control device, imaging device, control method, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a control device capable of preventing a difference between a detected focus position based on a focus detection signal and an optimum focus position based on an imaging signal, and enabling a highly accurate focus detection.SOLUTION: A control device comprises generation means 121a for generating a first focus detection signal and a second focus detection signal on the basis of a plurality of kinds of color signals from a first pixels group and a second pixel group receiving luminous flux passing pupil partial areas different from each other, of an imaging optical system and calculation means 121b for calculating a defocus amount by a phase difference detection method using the first focus detection signal and the second focus detection signal. The generation means 121a, regarding the first pixel group and the second pixel group, generates the first focus detection signal and the second focus detection signal by combining the plurality of kinds of color signals so that centers of gravity of the color signals in a pupil division direction match each other.SELECTED DRAWING: Figure 12

Description

  The present invention relates to an imaging apparatus that performs focus detection using a phase difference detection method.

  2. Description of the Related Art Conventionally, an imaging apparatus that performs focus detection by a phase difference detection method (imaging surface phase difference method) using a focus detection signal from an image sensor is known.

  Patent Document 1 discloses an imaging device using a two-dimensional imaging element in which one microlens and a plurality of divided photoelectric conversion units are formed for one pixel. The divided photoelectric conversion unit is configured to receive a pupil partial region having a different exit pupil of the photographing lens through one microlens, and performs pupil division. The amount of image shift can be calculated based on the respective focus detection signals received by the divided photoelectric conversion units (focus detection pixels), and focus detection can be performed using a phase difference detection method. Patent Document 2 discloses an imaging apparatus that generates a captured image by adding respective focus detection signals received by divided photoelectric conversion units.

  Patent Document 3 discloses an imaging apparatus having a pair of focus detection pixels partially arranged in a part of an array of a plurality of imaging pixels. The pair of focus detection pixels are configured to receive different regions of the exit pupil of the photographing lens by a light shielding layer having an opening, and perform pupil division. The amount of image shift can be calculated from the focus detection signals of the pair of focus detection pixels, and focus detection by the phase difference detection method can be performed. According to focus detection using the imaging surface phase difference method, the focus detection pixels of the image sensor can simultaneously detect the defocus direction and the defocus amount, and high-speed focus control is possible.

US Pat. No. 4,410,804 JP 2001-083407 A JP 2000-156823 A

  However, in the imaging plane phase difference method, the spatial frequency band of the focus detection signal for performing focus detection and the spatial frequency band of the imaging signal for generating a captured image may be different from each other. In this case, there is a difference between the detected focus position based on the focus detection signal and the best focus position based on the imaging signal, and it is difficult to perform highly accurate focus detection.

  Therefore, the present invention reduces a difference between a detection focus position based on a focus detection signal and a best focus position based on an imaging signal, and enables a highly accurate focus detection control apparatus, imaging apparatus, control method, A program and a storage medium are provided.

  The control device according to one aspect of the present invention is based on a plurality of types of color signals from the first pixel group and the second pixel group that receive light beams that pass through different pupil partial regions of the imaging optical system. A generating unit configured to generate a one focus detection signal and a second focus detection signal; and a calculation unit configured to calculate a defocus amount by a phase difference detection method using the first focus detection signal and the second focus detection signal. The generating unit generates the first focus detection signal by combining the color signals so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other with respect to the first pixel group; With respect to the second pixel group, the color signals are synthesized so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other to generate the second focus detection signal.

  An imaging device according to another aspect of the present invention includes an imaging device having a first pixel group and a second pixel group that receive light beams that pass through different pupil partial regions of the imaging optical system, the first pixel group, Based on a plurality of types of color signals from the second pixel group, the first focus detection signal and the second focus detection signal are generated, and the first focus detection signal and the second focus detection signal are used. Calculating means for calculating a defocus amount by a phase difference detection method, wherein the generating means relates to the first pixel group so that centroids in the pupil division direction of the plurality of types of color signals coincide with each other. The color signals are combined to generate the first focus detection signal, and the color signals are combined so that the centroids of the plurality of types of color signals in the pupil division direction coincide with each other with respect to the second pixel group. The second focus test To generate a signal.

  A control method according to another aspect of the present invention is based on a plurality of types of color signals from the first pixel group and the second pixel group that receive light beams that pass through different pupil partial regions of the imaging optical system. Generating a first focus detection signal and a second focus detection signal; and calculating a defocus amount by a phase difference detection method using the first focus detection signal and the second focus detection signal; In the step of generating the first focus detection signal and the second focus detection signal, the color signals are synthesized so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other with respect to the first pixel group. Then, the first focus detection signal is generated, and the color signal is synthesized with respect to the second pixel group so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other. Generating a detection signal.

  A control method according to another aspect of the present invention is a control method for acquiring a plurality of detection signals by a phase difference detection method in order to calculate distance information, and for a viewpoint image including a plurality of types of color signals. Performing a first process to obtain a synthesized signal obtained by synthesizing the color signals; and performing a second process on the synthesized signal to generate the detection signal, wherein the first process comprises: This is a process of synthesizing the color signals so that the centroids in the viewpoint direction of the viewpoint images coincide.

  According to another aspect of the present invention, there is provided a program based on a plurality of types of color signals from the first pixel group and the second pixel group that receive light beams that pass through different pupil partial regions of the imaging optical system. A step of generating a one focus detection signal and a second focus detection signal and a step of calculating a defocus amount by a phase difference detection method using the first focus detection signal and the second focus detection signal are executed on a computer. In the step of generating the first focus detection signal and the second focus detection signal, the centroids in the pupil division direction of the plurality of types of color signals are coincident with each other with respect to the first pixel group. The color signals are combined to generate the first focus detection signal, and the centroids of the plurality of types of color signals in the pupil division direction are mutually related with respect to the second pixel group. By combining the color signal to generate the second focus detection signal for match.

  A storage medium according to another aspect of the present invention stores the program.

  Other objects and features of the present invention are illustrated in the following examples.

  According to the present invention, a control device, an imaging device, and a control method capable of highly accurate focus detection by reducing a difference between a detection focus position based on a focus detection signal and a best focus position based on an imaging signal. , A program, and a storage medium can be provided.

It is a block diagram of the imaging device in each embodiment. It is a figure which shows the pixel arrangement | sequence in Example 1,2. It is a figure which shows the pixel structure in Example 1,2. It is explanatory drawing of the image pick-up element and pupil division function in each Example. It is explanatory drawing of the image pick-up element and pupil division function in each Example. FIG. 6 is a relationship diagram between a defocus amount and an image shift amount in each embodiment. It is a flowchart which shows the 1st focus detection process in each Example. It is explanatory drawing of the 1st pixel addition process in each Example. It is explanatory drawing of the shading by the pupil shift | offset | difference of the 1st focus detection signal in each Example, and a 2nd focus detection signal. It is explanatory drawing of the filter frequency band in each Example. It is a flowchart which shows the 2nd focus detection process in each Example. It is explanatory drawing of the 2nd pixel addition process in Example 1. FIG. It is a flowchart which shows the focus control in each Example. It is explanatory drawing of the 2nd pixel addition process in Example 2. FIG. FIG. 10 is a diagram illustrating a pixel array in Example 3. 6 is a diagram illustrating a pixel structure in Embodiment 3. FIG. It is a flowchart which shows the focus control in each Example.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  First, with reference to FIG. 1, a schematic configuration of the imaging apparatus according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram of an imaging apparatus 100 (camera) in the present embodiment. The imaging apparatus 100 is a digital camera system that includes a camera body and an interchangeable lens (imaging optical system or photographing optical system) that can be attached to and detached from the camera body. However, the present embodiment is not limited to this, and can also be applied to an imaging apparatus in which a camera body and a lens are integrally configured.

  The first lens group 101 is disposed in the forefront (subject side) of the plurality of lens groups constituting the photographing lens (imaging optical system), and can advance and retreat in the direction of the optical axis OA (optical axis direction). The lens barrel is held in a state. The aperture / shutter 102 (aperture) adjusts the aperture to adjust the amount of light during shooting, and also functions as an exposure time adjustment shutter during still image shooting. The second lens group 103 has a zoom function of moving forward and backward in the optical axis direction integrally with the diaphragm / shutter 102 and performing a zooming operation in conjunction with the forward / backward movement of the first lens group 101. The third lens group 105 is a focus lens group that performs focus adjustment (focus operation) by moving back and forth in the optical axis direction. The optical low-pass filter 106 is an optical element for reducing false colors and moire in the captured image.

  The image sensor 107 performs photoelectric conversion of a subject image (optical image) via an imaging optical system, and includes, for example, a CMOS sensor or a CCD sensor and its peripheral circuits. As the image sensor 107, for example, a two-dimensional single-plate color sensor in which a Bayer array primary color mosaic filter is formed on-chip on a light receiving pixel having m pixels in the horizontal direction and n pixels in the vertical direction is used. It is done.

  The zoom actuator 111 performs a zooming operation by moving the first lens group 101 and the second lens group 103 along the optical axis direction by rotating (driving) a cam cylinder (not shown). The aperture shutter actuator 112 controls the aperture diameter of the aperture / shutter 102 to adjust the amount of light (photographing light amount), and also controls the exposure time during still image shooting. The focus actuator 114 adjusts the focus by moving the third lens group 105 in the optical axis direction.

  The electronic flash 115 is an illumination device used to illuminate a subject. As the electronic flash 115, a flash illumination device including a xenon tube or an illumination device including an LED (light emitting diode) that continuously emits light is used. The AF auxiliary light unit 116 projects an image of a mask having a predetermined opening pattern onto a subject via a light projection lens. As a result, the focus detection capability for a dark subject or a low-contrast subject can be improved.

  The CPU 121 is a control device (control means) that performs various controls of the imaging device 100. The CPU 121 includes a calculation unit, a ROM, a RAM, an A / D converter, a D / A converter, a communication interface circuit, and the like. The CPU 121 reads and executes a predetermined program stored in the ROM, thereby driving various circuits of the imaging apparatus 100 and controlling a series of operations such as focus detection (AF), shooting, image processing, or recording. To do. Note that the communication interface circuit can perform communication using a wireless technology such as a wireless LAN as well as communication using a cable such as a USB or a wired LAN.

  The CPU 121 includes a generation unit 121a, a calculation unit 121b, and a focus control unit 121c. The generating unit 121a is based on a plurality of types of color signals from a plurality of photoelectric conversion units (first pixel group and second pixel group) that receive light beams that pass through different pupil partial regions of the imaging optical system. A first focus detection signal and a second focus detection signal are generated. The calculating means 121b calculates the defocus amount by the phase difference detection method using the first focus detection signal and the second focus detection signal. The focus control unit 121c performs focus control based on the defocus amount. The generation unit 121a can also generate a viewpoint image (parallax image) based on a plurality of types of color signals.

  The electronic flash control circuit 122 controls lighting of the electronic flash 115 in synchronization with the photographing operation. The auxiliary light driving circuit 123 performs lighting control of the AF auxiliary light unit 116 in synchronization with the focus detection operation. The image sensor driving circuit 124 controls the imaging operation such as vertical and horizontal scanning of the image sensor 107 and A / D-converts the acquired image signal and transmits it to the CPU 121. Note that the A / D conversion circuit may be provided in the image sensor 107. The image processing circuit 125 performs processing such as γ (gamma) conversion, color interpolation, or JPEG (Joint Photographic Experts Group) compression of the image data output from the image sensor 107.

  The focus drive circuit 126 performs focus adjustment by driving the focus actuator 114 based on the focus detection result and moving the third lens group 105 along the optical axis direction. The aperture shutter drive circuit 128 drives the aperture shutter actuator 112 to control the aperture diameter of the aperture / shutter 102. The zoom drive circuit 129 drives the zoom actuator 111 according to the zoom operation of the photographer.

  The display 131 includes, for example, an LCD (Liquid Crystal Display). The display 131 displays information related to the shooting mode of the imaging apparatus 100, a preview image before shooting, a confirmation image after shooting, a focus state display image at the time of focus detection, and the like. The operation unit 132 (operation switch group) includes a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, and the like. The release switch has a two-stage switch in a half-pressed state (a state where SW1 is ON) and a full-pressed state (a state where SW2 is ON). The recording medium 133 is, for example, a flash memory that can be attached to and detached from the imaging apparatus 100, and records captured images (image data). Further, a touch panel or the like may be included in the operation unit 132 so that the operation can be performed using the touch panel.

  Next, with reference to FIG. 2 and FIG. 3, the pixel arrangement and the pixel structure of the image sensor 107 in the present embodiment will be described. FIG. 2 is a diagram illustrating a pixel array of the image sensor 107. 3A and 3B are diagrams illustrating the pixel structure of the image sensor 107. FIG. 3A is a plan view of the pixel 200G of the image sensor 107 (viewed from the + z direction), and FIG. ) Is a sectional view taken along line aa (viewed from the -y direction).

  FIG. 2 shows a pixel array (photographing pixel array) of the image sensor 107 (two-dimensional CMOS sensor) in a range of 4 columns × 4 rows. In this embodiment, each imaging pixel (pixels 200R, 200G, and 200B) includes two sub-pixels 201 and 202 (two focus detection pixels). For this reason, in FIG. 2, the arrangement of subpixels is shown in a range of 8 columns × 4 rows.

  As shown in FIG. 2, a pixel group 200 of 2 columns × 2 rows has pixels 200R, 200G, and 200B arranged in a Bayer array. That is, in the pixel group 200, a pixel 200R having a spectral sensitivity of R (red) is at the upper left, a pixel 200G having a spectral sensitivity of G (green) is at the upper right and a lower left, and a pixel 200B having a spectral sensitivity of B (blue). Are located at the bottom right. Each of the pixels 200R, 200G, and 200B (each imaging pixel) includes a sub-pixel 201 (first focus detection pixel) and a sub-pixel 202 (second focus detection pixel) arranged in 2 columns × 1 row. The sub-pixel 201 is a pixel that receives the light beam that has passed through the first pupil partial region of the imaging optical system. The sub-pixel 202 is a pixel that receives the light beam that has passed through the second pupil partial region of the imaging optical system. The plurality of subpixels 201 constitute a first pixel group, and the plurality of subpixels 202 constitute a second pixel group.

As shown in FIG. 2, the image sensor 107 is configured by arranging a large number of 4 columns × 4 rows of imaging pixels (8 columns × 4 rows of subpixels) on the surface, and an imaging signal (subpixel signal). ) Is output. In the imaging device 107 of this embodiment, the period P of pixels (imaging pixels) is 4 μm, and the number N of pixels (imaging pixels) is 5575 columns × 3725 rows = approximately 20.75 million pixels. The imaging element 107 has a sub-pixel column-direction period P _SUB of 2 μm, and the number of sub-pixels N _SUB is 11150 horizontal rows × 3725 vertical rows = about 41.5 million pixels.

As shown in FIG. 3B, the pixel 200G of the present embodiment is provided with a microlens 305 for condensing incident light on the light receiving surface side of the pixel. A plurality of microlenses 305 are two-dimensionally arranged, and are arranged at a position away from the light receiving surface by a predetermined distance in the z-axis direction (the direction of the optical axis OA). Also in the pixel 200G is, _{N H} divided in the x direction (divided into two), _{N V} division (first division) by photoelectric conversion unit 301 and the photoelectric conversion portion 302 is formed in the y-direction. The photoelectric conversion unit 301 and the photoelectric conversion unit 302 correspond to the subpixel 201 and the subpixel 202, respectively. Thus, the image sensor 107 has a plurality of photoelectric conversion units for one microlens, and the microlenses are arranged in a two-dimensional manner. The photoelectric conversion unit 301 and the photoelectric conversion unit 302 are each configured as a photodiode having a pin structure in which an intrinsic layer is sandwiched between a p-type layer and an n-type layer. If necessary, the intrinsic layer may be omitted and a pn junction photodiode may be configured.

  The pixel 200 </ b> G (each pixel) is provided with a G (green) color filter 306 between the microlens 305, the photoelectric conversion unit 301, and the photoelectric conversion unit 302. Similarly, in the pixels 200R and 200B (each pixel), R (red) and B (blue) color filters 306 are provided between the microlens 305, the photoelectric conversion unit 301, and the photoelectric conversion unit 302, respectively. . If necessary, the spectral transmittance of the color filter 306 can be changed for each sub-pixel, or the color filter may be omitted.

  As shown in FIG. 3, the light incident on the pixel 200G (200R, 200B) is collected by the microlens 305, separated by the G color filter 306 (R, B color filter 306), and then photoelectrically Light is received by the converter 301 and the photoelectric converter 302. In the photoelectric conversion unit 301 and the photoelectric conversion unit 302, pairs of electrons and holes are generated according to the amount of received light, and after they are separated by the depletion layer, negatively charged electrons are accumulated in the n-type layer. On the other hand, the holes are discharged to the outside of the image sensor 107 through a p-type layer connected to a constant voltage source (not shown). Electrons accumulated in the n-type layers of the photoelectric conversion unit 301 and the photoelectric conversion unit 302 are transferred to the electrostatic capacitance unit (FD) via the transfer gate based on scanning control by the image sensor driving circuit 124, and voltage Converted to a signal.

  Next, the pupil division function of the image sensor 107 will be described with reference to FIG. FIG. 4 is an explanatory diagram of the pupil division function of the image sensor 107 and shows a state of pupil division in one pixel unit. FIG. 4 shows a cross-sectional view of the pixel structure shown in FIG. 3A as viewed from the + y side, and an exit pupil plane of the imaging optical system. In FIG. 4, in order to correspond to the coordinate axis of the exit pupil plane, the x-axis and y-axis in the cross-sectional view are inverted with respect to the x-axis and y-axis in FIG.

  In FIG. 4, the pupil partial region 501 (first pupil partial region) of the sub-pixel 201 (first focus detection pixel) includes a light receiving surface of the photoelectric conversion unit 301 whose center of gravity is decentered in the −x direction, and a micro lens 305. It is a substantially conjugate relationship. For this reason, the pupil partial area 501 represents a pupil area that can be received by the sub-pixel 201. The center of gravity of the pupil partial area 501 of the sub-pixel 201 is eccentric to the + x side on the pupil plane. Further, the pupil partial region 502 (second pupil partial region) of the sub-pixel 202 (second focus detection pixel) is substantially omitted via the microlens 305 and the light receiving surface of the photoelectric conversion unit 302 whose center of gravity is decentered in the + x direction. It is a conjugate relationship. Therefore, the pupil partial area 502 represents a pupil area that can be received by the sub-pixel 202. The center of gravity of the pupil partial area 502 of the sub-pixel 202 is eccentric to the −x side on the pupil plane. The pupil region 500 is a pupil region that can receive light by the entire pixel 200G when all of the photoelectric conversion units 301 and 302 (subpixels 201 and 202) are combined.

  FIG. 5 is an explanatory diagram of the image sensor 107 and the pupil division function. Light beams that have passed through different pupil partial regions 501 and 502 in the pupil region of the imaging optical system are incident on the image pickup surface 800 of the image pickup device 107 at different angles from each pixel of the image pickup device 107 and are divided by 2 × 1. The subpixels 201 and 202 receive the light. In this embodiment, an example is described in which the pupil region is divided into two pupils in the horizontal direction. However, the present invention is not limited to this, and pupil division may be performed in the vertical direction as necessary. .

  In this embodiment, the image sensor 107 passes through a first focus detection pixel of the first color that receives a light beam passing through the first pupil partial region of the imaging optical system (photographing lens), and the first pupil partial region. A first focus detection pixel of the second color that receives the luminous flux to be received. The image sensor 107 passes through the second focus detection pixel of the first color that receives the light beam passing through the second pupil partial region different from the first pupil partial region of the imaging optical system, and the second pupil partial region. The second focus detection pixel of the second color that receives the light beam is included. The imaging element 107 includes a plurality of imaging pixels that receive a light beam that passes through a pupil region that is a combination of the first pupil partial region and the second pupil partial region of the imaging optical system. In this embodiment, each imaging pixel (pixel 200) is composed of a first focus detection pixel (subpixel 201) and a second focus detection pixel (subpixel 202). If necessary, the imaging pixel, the first focus detection pixel, and the second focus detection pixel may be configured by different pixels. At this time, the first focus detection pixel and the second focus detection pixel are partially (discretely) arranged in a part of the imaging pixel array.

  In the present embodiment, the imaging apparatus 100 collects the light reception signals of the first focus detection pixels (sub-pixels 201) of each pixel of the image sensor 107 to generate a first focus detection signal, and the second focus detection pixel of each pixel. Focus detection is performed by collecting the light reception signals of (sub-pixel 202) and generating a second focus detection signal. Further, the imaging apparatus 100 generates an imaging signal (captured image) having a resolution of N effective pixels by adding the signals of the first focus detection pixel and the second focus detection pixel for each pixel of the image sensor 107. . As described above, the imaging apparatus 100 includes image generation means (CPU 121) that generates a captured image from a signal obtained by adding the pixels included in each of the first pixel group and the second pixel group for each microlens.

  Next, referring to FIG. 6, the relationship between the defocus amount and the image shift amount of the first focus detection signal acquired from the sub-pixel 201 of the image sensor 107 and the second focus detection signal acquired from the sub-pixel 202. Will be described. FIG. 6 is a relationship diagram between the defocus amount and the image shift amount. In FIG. 6, the image sensor 107 is disposed on the imaging surface 800, and the state in which the exit pupil of the imaging optical system is divided into two pupil partial areas 501 and 502 is shown as in FIGS. 4 and 5. ing.

  The defocus amount d is the distance from the imaging position of the subject to the imaging surface 800 | d |, the negative pin state where the imaging position is closer to the subject than the imaging surface 800 (d <0), and imaging. A rear pin state in which the position is on the opposite side of the subject from the imaging surface 800 is defined as a positive sign (d> 0). A defocus amount d = 0 is established in a focused state where the imaging position of the subject is on the imaging surface 800 (focus position). In FIG. 6, a subject 801 in a focused state (d = 0) and a subject 802 in a front pin state (d <0) are shown. The front pin state (d <0) and the rear pin state (d> 0) are collectively referred to as a defocus state (| d |> 0).

  In the front pin state (d <0), the luminous flux that has passed through the pupil partial area 501 (or the pupil partial area 502) out of the luminous flux from the subject 802 is condensed once. Thereafter, the light beam spreads over a width Γ 1 (Γ 2) centered on the gravity center position G 1 (G 2) of the light beam, resulting in a blurred image on the imaging surface 800. The blurred image is received by the sub-pixel 201 (sub-pixel 202) constituting each pixel arranged in the image sensor 107, and a first focus detection signal (second focus detection signal) is generated. Therefore, the first focus detection signal (second focus detection signal) is recorded as a subject image in which the subject 802 is blurred by the width Γ1 (Γ2) at the gravity center position G1 (G2) on the imaging surface 800. The blur width Γ1 (Γ2) of the subject image increases substantially proportionally as the defocus amount d magnitude | d | increases. Similarly, the magnitude | p | of the subject image displacement amount p (= difference G1-G2 in the center of gravity of the light beam) between the first focus detection signal and the second focus detection signal is also equal to the defocus amount d. As the magnitude | d | increases, it generally increases proportionally. The same applies to the rear pin state (d> 0), but the image shift direction of the subject image between the first focus detection signal and the second focus detection signal is opposite to the front pin state.

  As described above, in this embodiment, the magnitude of the defocus amount of the imaging signal obtained by adding the first focus detection signal and the second focus detection signal or the first focus detection signal and the second focus detection signal increases. Accordingly, the amount of image shift between the first focus detection signal and the second focus detection signal increases.

  Next, focus detection in the present embodiment will be described. The imaging apparatus 100 of the present embodiment performs first focus detection and second focus detection as focus detection. The first focus detection is a phase difference detection type focus detection in which the signal period of the focus detection signal in the pupil division direction is large and the spatial frequency band is low. The second focus detection is a phase difference detection type focus detection in which the signal cycle of the focus detection signal in the pupil division direction is small and the spatial frequency band is high. In this embodiment, the column direction of the image sensor 107 is the pupil division direction, and the row direction of the image sensor 107 is a direction orthogonal to the pupil division direction. First focus detection is performed to perform focus adjustment from the large defocus state to the small defocus state, and second focus detection is performed to perform focus adjustment from the small defocus state to the vicinity of the best focus position.

  Next, the first focus detection of the phase difference detection method in the present embodiment will be described in detail. When performing the first focus detection, the CPU 121 (calculation unit 121b) of the imaging device 100 relatively shifts the first focus detection signal and the second focus detection signal to express the first correlation amount (first correlation). 1 evaluation value) is calculated. Then, the CPU 121 calculates the image shift amount based on the shift amount at which the correlation (signal matching degree) becomes good. As the magnitude of the defocus amount of the imaging signal increases, there is a relationship in which the magnitude of the image shift amount between the first focus detection signal and the second focus detection signal increases. The focus detection is performed by converting the shift amount into the first defocus amount.

  With reference to FIG. 7, the first focus detection process of the phase difference detection method will be described. FIG. 7 is a flowchart showing the first focus detection process, and corresponds to step S100 in FIG. Each step in FIG. 7 is mainly executed by the CPU 121 (generating unit 121a, calculating unit 121b) and the image processing circuit 125.

  First, in step S <b> 110, the CPU 121 sets a focus detection area for performing focus adjustment from the effective pixel area of the image sensor 107. Then, the CPU 121 (generating unit 121a) generates a light reception signal (output signal) of the first focus detection pixel for each of R, G, and B colors (first color, second color, and third color) included in the focus detection region. Based on this, a first focus detection signal (A image signal) is generated (acquired) for each of R, G, and B colors. Similarly, the CPU 121 detects the second focus for each of the R, G, and B colors based on the light reception signal (output signal) of the second focus detection pixel for each of the R, G, and B colors included in the focus detection area. A signal (B image signal) is generated (acquired).

  Subsequently, in step S120, the CPU 121 (generating unit 121a) converts a color signal (RGB signal) into a luminance signal (Y signal) for each of the first focus detection signal and the second focus detection signal of R, G, B. A first pixel addition process is performed to convert to. As a result, a processed first focus detection signal and second focus detection signal are generated.

  Here, with reference to FIG. 8, the 1st pixel addition process in a present Example is demonstrated. FIG. 8 is an explanatory diagram of the first pixel addition process. In FIG. 8, it is assumed that the first focus detection signal of the j-th Bayer array in the column direction (pupil division direction) and the i-th Bayer array (direction orthogonal to the pupil division direction) is A (i, j). Here, the first focus detection signal is shown for each of R, G, and B colors. The first focus detection signal of R (first color) is RA (i, j) = A (i, j). The first focus detection signals of G (second color) are GA (i, j + 1) = A (i, j + 1), GA (i + 1, j) = A (i + 1, j). The first focus detection signal of B (third color) is BA (i + 1, j + 1) = A (i + 1, j + 1). Similarly, the second focus detection signal of the Bayer array that is jth in the column direction (pupil division direction) and i-th in the row direction (direction orthogonal to the pupil division direction) is B (i, j). The second focus detection signal is also shown for each of R, G, and B colors. The second focus detection signal of R (first color) is RB (i, j) = B (i, j). The second focus detection signals of G (second color) are GB (i, j + 1) = B (i, j + 1), GB (i + 1, j) = B (i + 1, j). The second focus detection signal of B (third color) is BB (i + 1, j + 1) = B (i + 1, j + 1).

  As a result of the first pixel addition process in step S120 of FIG. 7, the first focus converted into the Y signal from the first focus detection signal A (i, j) of the Bayer array as represented by the following expression (1A). The detection signal Y1A (i, j) can be calculated. Similarly, by the first pixel addition process, the second focus detection signal Y1B converted into the Y signal from the second focus detection signal B (i, j) of the Bayer array as represented by the following expression (1B). (I, j) can be calculated.

In formulas (1A) and (1B), i = 2m and j = 2n (m and n are integers). Note that i and j are multiples of 2 because the pixels of this embodiment are arranged in a Bayer array. For this reason, in the case of an array different from the Bayer array, it is preferable to match i or j with the periodicity of the array. For example, when color filters are arranged in the horizontal direction with a period of four pixels, i or j is preferably a multiple of four.

  In the first focus detection process of the present embodiment, the signal period in the column direction (pupil division direction) of the focus detection signal converted into a Y signal is greater than the signal period in the column direction (pupil division direction) of the focus detection signals in the Bayer array. Is also big. In the first focus detection process of this embodiment, the signal period of the focus detection signal in the pupil division direction is large and the spatial frequency is set so that the focus detection can be stably performed from the large defocus state to the small defocus state. The first pixel addition process is performed so that the band is lowered.

  Subsequently, in step S130, the CPU 121 and the image processing circuit 125 perform shading correction processing (optical correction processing) on each of the first focus detection signal and the second focus detection signal. Here, with reference to FIG. 9, the shading by the pupil shift | offset | difference of a 1st focus detection signal and a 2nd focus detection signal is demonstrated. FIG. 9 is an explanatory diagram of shading due to pupil misalignment of the first focus detection signal and the second focus detection signal. Specifically, FIG. 9 illustrates a pupil partial region 501 of the sub-pixel 201 (first focus detection pixel), a pupil partial region 502 of the sub-pixel 202 (second focus detection pixel) at the peripheral image height of the image sensor 107, and The relationship of the exit pupil 400 of the imaging optical system is shown.

  FIG. 9A shows a case where the exit pupil distance Dl (the distance between the exit pupil 400 and the imaging surface of the image sensor 107) of the imaging optical system is equal to the set pupil distance Ds of the image sensor 107. In this case, the pupil partial area 501 and the pupil partial area 502 divide the exit pupil 400 of the imaging optical system substantially equally.

  On the other hand, when the exit pupil distance Dl of the imaging optical system is shorter than the set pupil distance Ds of the image sensor 107 as shown in FIG. A pupil shift occurs between the exit pupil 400 and the entrance pupil of the image sensor 107. For this reason, the exit pupil 400 of the imaging optical system is non-uniformly divided into pupils. Similarly, as shown in FIG. 9C, when the exit pupil distance Dl of the imaging optical system is longer than the set pupil distance Ds of the image sensor 107, the image forming optical system has a peripheral image height of the image sensor 107. A pupil shift occurs between the exit pupil 400 of the image sensor and the entrance pupil of the image sensor 107. For this reason, the exit pupil 400 of the imaging optical system is non-uniformly divided into pupils. As pupil division becomes nonuniform at the peripheral image height, the intensity of the first focus detection signal and the second focus detection signal also become nonuniform. For this reason, the intensity | strength of either one of a 1st focus detection signal and a 2nd focus detection signal becomes large, and the shading by which the other intensity | strength becomes small arises.

  In step S130 of FIG. 7, the CPU 121 determines the first shading correction coefficient of the first focus detection signal and the F value of the imaging lens (imaging optical system) and the exit pupil distance according to the image height of the focus detection area. A second shading correction coefficient of the second focus detection signal is generated. Then, the CPU 121 (image processing circuit 125) multiplies the first focus detection signal by the first shading correction coefficient, multiplies the second focus detection signal by the second shading correction coefficient, and outputs the first focus detection signal and the second focus detection signal. A detection signal shading correction process (optical correction process) is performed.

  When performing the first focus detection of the phase difference detection method, the CPU 121 determines the defocus amount (first defocus amount) based on the correlation (the degree of signal coincidence) between the first focus detection signal and the second focus detection signal. Is detected (calculated). When shading due to pupil shift occurs, the correlation (the degree of signal coincidence) between the first focus detection signal and the second focus detection signal may decrease. For this reason, in the present embodiment, in the first focus detection of the phase difference detection method, the correlation (signal coincidence) between the first focus detection signal and the second focus detection signal is improved to improve the focus detection performance. Therefore, it is preferable to perform shading correction processing (optical correction processing).

  Subsequently, in step S140, the CPU 121 and the image processing circuit 125 perform a first filter process on the first focus detection signal and the second focus detection signal. FIG. 10 is an explanatory diagram of the first filter process, and an example of a pass band in the first filter process of the present embodiment is indicated by a broken line. In the present embodiment, focus detection in a large defocus state is performed by the first focus detection of the phase difference detection method. For this reason, the pass band in the first filter processing is configured to include a low frequency band. If necessary, when performing the focus adjustment from the large defocus state to the small defocus state, the pass band of the first filter process in the first focus detection process may be adjusted according to the defocus state. .

  Subsequently, in step S150, the CPU 121 (calculating unit 121b) performs a first shift process for relatively shifting the first focus detection signal and the second focus detection signal after the first filter process in the pupil division direction. Then, the CPU 121 calculates a first correlation amount (first evaluation value) indicating the degree of coincidence of signals.

Here, the first focus detection signal after the first filter processing in the column direction (pupil division direction) and the i-th row direction (direction orthogonal to the pupil division direction) is dY1A (i, j), second Let the focus detection signal be dY1B (i, j). Also, the range of the number j corresponding to the focus detection area is W, and the range of the number i is L. Further, the shift amount by the first shift process is s ₁ , and the shift range of the shift amount s ₁ is Γ1. At this time, the first correlation amount COR1 _even (first evaluation value) is expressed as the following equation (2).

During the calculation of the first correlation amount _{COR1 the even,} to the shift amount _{s 1} for each i rows, columns j + _{s 1} th first focus detection signal _{dY1A (i, j + s 1} ) and the column direction _{j-s 1} th The second focus detection signal dY1B (i, j−s ₁ ) is subtracted correspondingly to generate a shift subtraction signal. Then, the absolute value of the generated shift subtraction signal is calculated, the number j is summed within the range W corresponding to the focus detection area, and the first correlation amount COR1 _even (i, s ₁ ) for each i row is calculated. To do. Further, the first correlation amount COR1 _even (i, s ₁ ) for each i row is summed with the number i in the range L corresponding to the focus detection area for each shift amount, and the first correlation amount COR1 _even (s ₁ ) is calculated.

  Subsequently, in step S160, the CPU 121 (calculating unit 121b) performs a subpixel operation on the first correlation amount (first evaluation value), and calculates a real-value shift amount at which the first correlation amount is the minimum value. Thus, an image shift amount p1 is obtained. Then, the CPU 121 multiplies the image shift amount p1 by the first conversion coefficient K1 corresponding to the image height of the focus detection area, the F value of the imaging lens (imaging optical system), and the exit pupil distance. The defocus amount Def1 is detected (calculated).

  Next, the second focus detection process of the phase difference detection method will be described with reference to FIG. FIG. 11 is a flowchart showing the second focus detection process, and corresponds to step S200 in FIG. Each step of FIG. 11 is mainly executed by the CPU 121 (generating unit 121a, calculating unit 121b) and the image processing circuit 125.

  First, in step S210, the CPU 121 sets a focus detection area for performing focus adjustment from the effective pixel area of the image sensor 107. Then, the CPU 121 (generating unit 121a) generates a light reception signal (output signal) of the first focus detection pixel for each of R, G, and B colors (first color, second color, and third color) included in the focus detection region. Based on this, a first focus detection signal (A image signal) is generated (acquired) for each of R, G, and B colors. Similarly, the CPU 121 detects the second focus for each of the R, G, and B colors based on the light reception signal (output signal) of the second focus detection pixel for each of the R, G, and B colors included in the focus detection area. A signal (B image signal) is generated (acquired).

  Subsequently, in step S220, the CPU 121 (generating unit 121a) converts the color signal (RGB signal) into the luminance signal (Y signal) for each of the first focus detection signal and the second focus detection signal of R, G, B. A second pixel addition process is performed to convert to. As a result, a processed first focus detection signal and second focus detection signal are generated.

  Here, the second pixel addition processing in the present embodiment will be described with reference to FIG. FIG. 12 is an explanatory diagram of the second pixel addition process. In FIG. 12, the first focus detection signal of the Bayer array j-th in the column direction (pupil division direction) and i-th in the row direction (direction orthogonal to the pupil division direction) is A (i, j). Here, the first focus detection signal is shown for each of R, G, and B colors. The first focus detection signal of R (first color) is RA (i, j) = A (i, j). The first focus detection signals of G (second color) are GA (i, j + 1) = A (i, j + 1), GA (i + 1, j) = A (i + 1, j). The first focus detection signal of B (third color) is BA (i + 1, j + 1) = A (i + 1, j + 1). Similarly, the second focus detection signal of the Bayer array that is jth in the column direction (pupil division direction) and i-th in the row direction (direction orthogonal to the pupil division direction) is B (i, j). The second focus detection signal is also shown for each of R, G, and B colors. The second focus detection signal of R (first color) is RB (i, j) = B (i, j). The second focus detection signals of G (second color) are GB (i, j + 1) = B (i, j + 1), GB (i + 1, j) = B (i + 1, j). The second focus detection signal of B (third color) is BB (i + 1, j + 1) = B (i + 1, j + 1).

  As a result of the second pixel addition process in step S220 of FIG. 11, the first focus converted into the Y signal from the first focus detection signal A (i, j) in the Bayer array as represented by the following expression (3A). The detection signal Y2A (i, j), that is, the first luminance signal can be calculated. Similarly, by the second pixel addition process, the second focus detection signal Y2B converted into a Y signal from the second focus detection signal B (i, j) of the Bayer array as represented by the following expression (3B). (I, j), that is, the second luminance signal can be calculated.

In formulas (3A) and (3B), i = 2m, j = 2n, and 2n + 1 (m and n are integers).

  When j = 2n in Expression (3A), the color centroid (i, j) of 2RA (i, j) of R (first color) and GA (i, j-1) + GA (G (second color)) The color centroids (i, j + 1) of i, j + 1) are synthesized so that the color centroids of the respective colors in the pupil division direction (column direction) coincide. Further, the color centroid (i + 1, j) of 2GA (i + 1, j) of G (second color) and the color centroid of BA (i + 1, j−1) + BA (i + 1, j + 1) of B (third color) ( i + 1, j) are also synthesized so as to coincide with the color centroid of each color in the pupil division direction (column direction) described above. At the same time, the color ratio of R (first color): G (second color): B (third color) is 1: 2: 1.

  In j = 2n + 1 in Expression (3A), RA (i, j) + RA (i, j + 2) color centroid (i, j + 1) of R (first color) and 2 GA (i, G) of G (second color) The color centroid (i, j + 1) of j + 1) is synthesized so that the color centroids of the respective colors in the pupil division direction (column direction) coincide. Further, the color centroid (i + 1, j + 1) of G (second color) GA (i + 1, j) + GA (i + 1, j + 2) and the color centroid (i + 1, j + 1) of B (third color) 2BA (i + 1, j + 1). ) Are also combined so as to coincide with the color centroids of the respective colors in the pupil division direction (column direction), and at the same time, R (first color): G (second color): B (third color). The composition is performed so that the color ratio is 1: 2: 1, as described above, the composition is performed at a predetermined color ratio so that the color centroids coincide with each other.

  As described above, in the second pixel addition process, the color ratio of R (first color): G (second color): B (third color) is 1: 2: 1 and a focus detection signal converted into a Y signal is generated so that the color centroids of the respective colors in the pupil division direction coincide with each other. For this reason, in the second focus detection process, the signal period in the column direction (pupil division direction) of the focus detection signal converted into a Y signal is equal to the signal period in the column direction (pupil division direction) of the focus detection signals in the Bayer array, And it is a period of equal intervals. For this reason, a high spatial frequency band can be detected.

  On the other hand, when the color centroids of the respective colors in the pupil division direction do not coincide with each other, the period becomes unequal interval for each color, so it is necessary to stabilize by a low-pass filter, and a high spatial frequency band can be detected stably. difficult.

  In the second focus detection of the present embodiment, the signal cycle in the pupil division direction of the focus detection signal converted into the Y signal is equal to the signal cycle in the pupil division direction of the focus detection signal of the Bayer array, and the Y signal of the first focus detection It is smaller than the signal period in the pupil division direction of the converted focus detection signal. The generation unit 121a combines the signal of the first focus detection pixel of the first color and the signal of the first focus detection pixel of the second color so that the color centroids in the pupil division direction match each other, A first focus detection signal is generated. Further, the generating unit 121a combines the signal of the second focus detection pixel of the first color and the signal of the second focus detection pixel of the second color so that the color centroids in the pupil division direction coincide with each other. The second focus detection signal is generated. Further, in the second focus detection process of the present embodiment, the signal cycle of the focus detection signal in the pupil division direction is small and the space is small so that the focus can be detected with high accuracy from the small defocus state to the vicinity of the best focus position. The second pixel addition process is performed so that the frequency band becomes higher.

  Subsequently, in step S230, the CPU 121 and the image processing circuit 125 perform shading correction processing (optical correction processing) on each of the first focus detection signal and the second focus detection signal. At this time, the CPU 121 determines the first shading correction coefficient and the second focus detection signal of the first focus detection signal according to the image height of the focus detection area, the F value of the imaging lens (imaging optical system), and the exit pupil distance. The second shading correction coefficient is generated. Then, the CPU 121 (image processing circuit 125) multiplies the first focus detection signal by the first shading correction coefficient, multiplies the second focus detection signal by the second shading correction coefficient, and outputs the first focus detection signal and the second focus detection signal. A detection signal shading correction process (optical correction process) is performed.

  When performing the second focus detection of the phase difference detection method, the CPU 121 determines the defocus amount (second defocus amount) based on the correlation (the degree of signal coincidence) between the first focus detection signal and the second focus detection signal. Is detected (calculated). When shading due to pupil shift occurs, the correlation (the degree of signal coincidence) between the first focus detection signal and the second focus detection signal may decrease. For this reason, in the present embodiment, in the second focus detection of the phase difference detection method, the correlation (signal coincidence) between the first focus detection signal and the second focus detection signal is improved to improve the focus detection performance. Therefore, it is preferable to perform shading correction processing (optical correction processing).

  Subsequently, in step S240, the CPU 121 and the image processing circuit 125 perform a second filter process on the first focus detection signal and the second focus detection signal. FIG. 10 is an explanatory diagram of the second filter processing, and an example of a pass band in the second filter processing of the present embodiment is indicated by a solid line. In the present embodiment, focus detection is performed in the vicinity of the best focus position from the small defocus state by the second focus detection of the phase difference detection method. For this reason, the pass band in the second filter processing is configured to include a high frequency band. If necessary, when performing the focus adjustment from the small defocus state to the vicinity of the best focus position, the pass band of the second filter process in the second focus detection process may be adjusted according to the defocus state. Good. An example of a passband adjustment method is horizontal signal addition or thinning.

  Subsequently, in step S250, the CPU 121 (calculating unit 121b) performs a second shift process for relatively shifting the first focus detection signal and the second focus detection signal after the second filter process in the pupil division direction. Then, the CPU 121 calculates a second correlation amount (second evaluation value) representing the degree of coincidence of signals.

Here, the first focus detection signal after the j-th second filter processing in the column direction (pupil division direction) and the i-th row direction (direction orthogonal to the pupil division direction) is dY2A (i, j), second Let the focus detection signal be dY2B (i, j). Also, the range of the number j corresponding to the focus detection area is W, and the range of the number i is L. Further, the shift amount by the second shift process is s ₂ , and the shift range of the shift amount s ₂ is Γ2. At this time, the second correlation amounts COR2 _even and COR2 _odd (second evaluation value) are respectively expressed as the following equations (4A) and (4B).

During the calculation of the second correlation amount _{COR2 the even,} to the shift amount _{s 2} in each row i, column j + _{s 2} th first focus detection signal _{dY2A (i, j + s 2} ) and the column direction _{j-s 2} th The second focus detection signal dY2B (i, j−s ₂ ) is subtracted correspondingly to generate a shift subtraction signal. Then, the absolute value of the generated shift subtraction signal is calculated, the number j is summed within the range W corresponding to the focus detection area, and the second correlation amount COR2 _even (i, s ₂ ) for each i row is calculated. To do. Further, the first correlation amount COR2 _even (i, s ₂ ) for each i row is summed with the number i within the range L corresponding to the focus detection region for each shift amount, and the second correlation amount COR1 _even (s ₂ ) is calculated.

During the calculation of the second correlation amount _{COR2 odd,} to the shift amount _{s 2} in each row i, column j + _{s 2} th first focus detection signal _{dY2A (i, j + s 2} ) and the column direction _{j-s 2} th The second focus detection signal dY2B (i, j-1-s ₂ ) is subtracted in correspondence with each other to generate a shift subtraction signal. Then, the absolute value of the generated shift subtraction signal is calculated, the number j is summed within the range W corresponding to the focus detection region, and the second correlation amount COR2 _odd (i, s ₂ ) for each i row is calculated. To do. Further, the second correlation amount COR2 _odd (i, s ₂ ) for each i row is summed with the number i in the range L corresponding to the focus detection region for each shift amount, and the second correlation amount COR2 _odd (s ₂ ) is calculated. In the present embodiment, the second correlation amount COR2 _odd is a correlation amount in which the shift amount between the first focus detection signal and the second focus detection signal is shifted by a half phase with respect to the second correlation amount COR2 _even .

Subsequently, in step S260, the CPU 121 performs a subpixel operation on each of the second correlation amounts COR2 _even and COR2 _odd (second evaluation value), and calculates a real value shift amount at which the second correlation amount is the minimum value. The average value is calculated to calculate the image shift amount p2. Then, the CPU 121 can calculate the sub-pixel with high accuracy by calculating the image shift amount p2 based on the two second correlation amounts COR2 _even and COR2 _odd that are shifted from each other by a half phase. The second defocus amount Def2 is obtained by multiplying the image shift amount p2 by the second conversion coefficient K2 corresponding to the image height of the focus detection region, the F value of the imaging lens (imaging optical system), and the exit pupil distance. Is detected (calculated).

  As described above, in the second focus detection process, the color ratio of R (first color): G (second color): B (third color) is 1: 2: 1 and a focus detection signal converted into a Y signal is generated so that the color centroids of the respective colors in the pupil division direction coincide with each other. For this reason, in the second focus detection process, the signal period in the column direction (pupil division direction) of the focus detection signal converted into a Y signal is equal to the signal period in the column direction (pupil division direction) of the focus detection signals in the Bayer array, And it is a period of equal intervals. For this reason, a high spatial frequency band can be detected.

  According to the second focus detection process of the present embodiment, the difference between the spatial frequency band of the focus detection signal and the spatial frequency band of the imaging signal for generating the captured image is reduced, and the detection focus calculated from the focus detection signal. The difference between the position and the best focus position of the imaging signal can be reduced. Therefore, according to the second focus detection process of the present embodiment, it is possible to detect a focus with high accuracy from the small defocus state to the vicinity of the best focus position.

  Next, focus control in the present embodiment will be described with reference to FIG. FIG. 13 is a flowchart showing focus control. Each step in FIG. 13 is mainly executed by the CPU 121 (the generation unit 121a, the calculation unit 121b, and the focus control unit 121c). The CPU 121 performs the first focus detection until the absolute value of the defocus amount of the imaging optical system is equal to or less than the first predetermined value, and drives the third lens group 105 (focus lens group) (performs lens driving). Focus adjustment is performed from the large defocus state to the small defocus state of the imaging optical system. Thereafter, the second focus is detected and the lens is driven until the absolute value of the defocus amount of the imaging optical system becomes equal to or less than a second predetermined value (first predetermined value> second predetermined value). The focus is adjusted from the small defocus state to the vicinity of the best focus position.

  First, in step S100, the CPU 121 detects (calculates) the first defocus amount Def1 by the first focus detection. Subsequently, in step S301, the CPU 121 determines whether or not the absolute value (size) | Def1 | of the first defocus amount Def1 calculated in step S100 is equal to or less than a first predetermined value. When the absolute value | Def1 | of the first defocus amount Def1 is larger than the first predetermined value, the CPU 121 performs lens driving according to the first defocus amount Def1, and repeats step S100. On the other hand, when the absolute value | Def1 | of the first defocus amount Def1 calculated in step S100 is equal to or smaller than the first predetermined value, the process proceeds to step S200.

  Subsequently, in step S200, the CPU 121 detects (calculates) the second defocus amount Def2 by the second focus detection. When the absolute value | Def2 | of the second defocus amount Def2 calculated in step S200 is larger than the second predetermined value (first predetermined value> second predetermined value), the CPU 121 responds to the second defocus amount Def2. Then, the lens is driven and step S200 is repeated. On the other hand, when the absolute value | Def2 | of the second defocus amount Def2 calculated in step S200 is equal to or smaller than the second predetermined value, the focus adjustment operation (focus control in this flow) is terminated.

  In the flowchart of FIG. 13, the second focus detection is performed after the first focus detection. However, the present invention is not limited to this, and both the first focus detection and the second focus detection may be processed in parallel. FIG. 17 is a flowchart showing focus control when the first focus detection and the second focus detection are processed in parallel. The same operations as those in FIG. 13 are denoted by the same reference numerals. Each step in FIG. 17 is mainly executed by the CPU 121 (the generation unit 121a, the calculation unit 121b, and the focus control unit 121c).

  The CPU 121 performs first focus detection and second focus detection. Thereafter, it is determined whether or not the absolute value (magnitude) | Def2 | of the second defocus amount Def2 of the imaging optical system, which is the result of the second focus detection, is equal to or smaller than the first predetermined value. When the absolute value | Def2 | of the second defocus amount Def2 is equal to or smaller than the first predetermined value, the CPU 121 determines the absolute value | of the first defocus amount Def1 of the imaging optical system as a result of the first focus detection. It is determined whether Def1 | is equal to or smaller than a second predetermined value. On the other hand, when the second defocus amount Def2 is larger than the first predetermined value, the CPU 121 determines that the absolute value | Def2 | of the second defocus amount Def2 of the imaging optical system as a result of the second focus detection is the second predetermined value. It is determined whether or not it is less than or equal to the value.

  First, in step S100, the CPU 121 detects (calculates) the first defocus amount Def1 by the first focus detection. In parallel, in step S200, the CPU 121 detects (calculates) the second defocus amount Def2 by the second focus detection.

  Subsequently, in step S401, the CPU 121 determines whether or not the absolute value (size) | Def2 | of the second defocus amount Def2 calculated in step S200 is equal to or less than a first predetermined value. When the absolute value | Def2 | of the second defocus amount Def2 is larger than the first predetermined value, the process proceeds to step S402, and the CPU 121 employs the first defocus amount Def1 as the defocus amount Def. On the other hand, when the absolute value | Def2 | of the second defocus amount Def2 calculated in step S200 is equal to or smaller than the first predetermined value, the process proceeds to step S403, and the CPU 121 adopts the second defocus amount Def2 as the defocus amount Def. To do. After step S402 or step S403, the process proceeds to step S404.

  Subsequently, in step S404, the CPU 121 determines whether the absolute value | Def | of the defocus amount Def calculated in step S402 or step S403 is larger than a second predetermined value (first predetermined value> second predetermined value). Determine whether or not. If the absolute value | Def | of the defocus amount Def is larger than the second predetermined value, the process proceeds to step S405, and the CPU 121 performs lens driving according to the defocus amount Def. On the other hand, when the absolute value | Def | of the defocus amount Def is equal to or smaller than the second predetermined value, the focus adjustment operation (focus control in this flow) is ended.

  With the second focus detection of the present embodiment, it is possible to increase the accuracy of focus detection by the phase difference method at the aperture value F on the small aperture side regardless of the defocus state. The larger the aperture value F and the smaller the aperture, the shorter the baseline length, which is the distance between the center of gravity of the pupil part region 501 and the center of gravity of the pupil part region 502, and the amount of change in the image shift amount p is smaller than the defocus amount d. Become. For this reason, in the aperture value F on the small aperture side, the focus detection accuracy may be lowered regardless of the defocus state.

  In the second focus detection of the present embodiment, the column direction (pupil division direction) of the focus detection signal in the second focus detection with respect to the cycle in the column direction (pupil division direction) of the focus detection signal in the first focus detection. ) Period is reduced to 1/2. For this reason, the detection accuracy of the image shift amount calculated from the correlation amount can be doubled. Therefore, with the second focus detection of the present embodiment, the focus detection can be performed with high accuracy regardless of the defocus state at the aperture value F on the small aperture side.

  According to the image pickup apparatus of the present embodiment, it is possible to reduce the difference between the detected focus position based on the focus detection signal and the best focus position based on the image pickup signal, and to perform highly accurate focus detection.

  Next, the second pixel addition process in the second embodiment of the present invention will be described with reference to FIG. FIG. 14 is an explanatory diagram of the second pixel addition process. This embodiment is the same as the first embodiment except for the second pixel addition process, and other description is omitted.

  In FIG. 14, the first focus detection signal of the Bayer array j-th in the column direction (pupil division direction) and i-th in the row direction (direction orthogonal to the pupil division direction) is A (i, j). Here, the first focus detection signal is shown for each of R, G, and B colors. The first focus detection signal of R (first color) is RA (i, j) = A (i, j). The first focus detection signals of G (second color) are GA (i, j + 1) = A (i, j + 1), GA (i + 1, j) = A (i + 1, j). The first focus detection signal of B (third color) is BA (i + 1, j + 1) = A (i + 1, j + 1). Similarly, the second focus detection signal of the Bayer array that is jth in the column direction (pupil division direction) and i-th in the row direction (direction orthogonal to the pupil division direction) is B (i, j). The second focus detection signal is also shown for each of R, G, and B colors. The second focus detection signal of R (first color) is RB (i, j) = B (i, j). The second focus detection signals of G (second color) are GB (i, j + 1) = B (i, j + 1), GB (i + 1, j) = B (i + 1, j). The second focus detection signal of B (third color) is BB (i + 1, j + 1) = B (i + 1, j + 1).

  As a result of the second pixel addition process in step S220 in FIG. 11, the first focus converted into the Y signal from the first focus detection signal A (i, j) in the Bayer array as represented by the following expression (5A). The detection signal Y2A (i, j), that is, the first luminance signal can be calculated. Similarly, by the second pixel addition process, the second focus detection signal Y2B converted into a Y signal from the second focus detection signal B (i, j) of the Bayer array as represented by the following expression (5B). (I, j), that is, the second luminance signal can be calculated.

In formulas (5A) and (5B), i = m and j = n (m and n are integers).

  In the second pixel addition process of the present embodiment, the R (first color): G (second color): B (third color) color ratio is 1: 2 in all pixels from the focus detection signals in the Bayer array. 1 and a focus detection signal converted into a Y signal is generated so that the color centroids of the respective colors in the pupil division direction coincide with each other. For this reason, in the second focus detection process, the signal period in the column direction (pupil division direction) of the focus detection signal converted into a Y signal is equal to the signal period in the column direction (pupil division direction) of the focus detection signals in the Bayer array, And it is a period of equal intervals. For this reason, a high spatial frequency band can be detected.

  According to the image pickup apparatus of the present embodiment, it is possible to reduce the difference between the detected focus position based on the focus detection signal and the best focus position based on the image pickup signal, and to perform highly accurate focus detection.

  Next, with reference to FIG. 15 and FIG. 16, an imaging apparatus according to Embodiment 3 of the present invention will be described. The present embodiment is different from the first embodiment regarding the pixel arrangement of the image sensor 107. Since the other configuration of the present embodiment is the same as that of the first embodiment, description thereof is omitted.

  FIG. 15 is a diagram illustrating a pixel array of the image sensor 107 in the present embodiment. 16A and 16B are diagrams illustrating the pixel structure of the image sensor 107. FIG. 16A is a plan view of the pixel 200G of the image sensor 107 (viewed from the + z direction), and FIG. ) Is a sectional view taken along line aa (viewed from the -y direction).

  FIG. 15 shows a pixel array (photographing pixel array) of the image sensor 107 (two-dimensional CMOS sensor) in a range of 4 columns × 4 rows. In this embodiment, each imaging pixel (pixels 200R, 200G, and 200B) includes four subpixels 201, 202, 203, and 204. For this reason, FIG. 15 shows the arrangement of sub-pixels in a range of 8 columns × 8 rows.

  As shown in FIG. 15, in a pixel group 200 of 2 columns × 2 rows, pixels 200R, 200G, and 200B are arranged in a Bayer array. That is, in the pixel group 200, a pixel 200R having a spectral sensitivity of R (red) is at the upper left, a pixel 200G having a spectral sensitivity of G (green) is at the upper right and a lower left, and a pixel 200B having a spectral sensitivity of B (blue). Are located at the bottom right. Each of the pixels 200R, 200G, and 200B (each imaging pixel) includes subpixels 201, 202, 203, and 204 arranged in 2 columns × 2 rows. In this embodiment, an example in which subpixels are arranged in 2 columns × 2 rows has been shown, but more subpixels can be included, and different numbers of subpixels can be included in the column direction and the row direction. You may make it comprise with a pixel. The sub-pixel 201 is a pixel that receives the light beam that has passed through the first pupil partial region of the imaging optical system. The sub-pixel 202 is a pixel that receives the light beam that has passed through the second pupil partial region of the imaging optical system. The sub-pixel 203 is a pixel that receives the light beam that has passed through the third pupil partial region of the imaging optical system. The sub-pixel 204 is a pixel that receives the light beam that has passed through the fourth pupil partial region of the imaging optical system.

As shown in FIG. 15, the image sensor 107 is configured by arranging a large number of 4 columns × 4 rows of imaging pixels (8 columns × 8 rows of subpixels) on the surface, and an imaging signal (subpixel signal). ) Is output. In the imaging device 107 of this embodiment, the period P of pixels (imaging pixels) is 4 μm, and the number N of pixels (imaging pixels) is 5575 columns × 3725 rows = approximately 20.75 million pixels. The imaging element 107 has a sub-pixel column-direction period P _SUB of 2 μm, and the number of sub-pixels N _SUB is 11150 horizontal rows × 7450 vertical rows = approximately 83 million pixels.

As shown in FIG. 16B, the pixel 200G of this embodiment is provided with a microlens 305 for condensing incident light on the light receiving surface side of the pixel. The microlens 305 is disposed at a position away from the light receiving surface by a predetermined distance in the z-axis direction (the direction of the optical axis OA). Also in the pixel 200G is, _{N H} divided in the x direction (divided into two), _{N V} division (2 divided) photoelectric conversion unit 301, 302, 303 and 304 are formed in the y-direction. The photoelectric conversion units 301 to 304 correspond to the subpixels 201 to 204, respectively.

  In this embodiment, the image sensor 107 shares a single microlens and emits a plurality of light beams that pass through different regions (first to fourth pupil partial regions) of the pupil of the imaging optical system (photographing lens). A plurality of sub-pixels for receiving light are provided. The image sensor 107 includes a first subpixel (a plurality of subpixels 201), a second subpixel (a plurality of subpixels 202), a third subpixel (a plurality of subpixels 203), and a plurality of subpixels. 4 sub-pixels (a plurality of sub-pixels 204) are included.

  In this embodiment, signals of the sub-pixels 201, 202, 203, and 204 are added and read for each pixel of the image sensor 107, thereby generating a captured image having a resolution of N effective pixels. Thus, the captured image is generated for each pixel by combining the light reception signals of a plurality of subpixels (subpixels 201 to 204 in this embodiment).

  In this embodiment, for each pixel of the image sensor 107, a first focus detection signal is generated by adding the signals of the sub-pixels 201 and 203, and a second focus detection is performed by adding the signals of the sub-pixels 202 and 204. A signal can be generated. At this time, the plurality of sub-pixels 201 and 203 constitute a first pixel group, and the plurality of sub-pixels 202 and 204 constitute a second pixel group. By these addition processes, the first focus detection signal and the second focus detection signal corresponding to the pupil division in the horizontal direction can be acquired, and the first focus detection and the second focus detection of the phase difference detection method can be performed. Is possible.

  In this embodiment, for each pixel of the image sensor 107, the first focus detection signal is generated by adding the signals of the subpixels 201 and 202, and the second focus is added by adding the signals of the subpixels 203 and 204. A detection signal can be generated. At this time, the plurality of subpixels 201 and 202 constitute a first pixel group, and the plurality of subpixels 203 and 204 constitute a second pixel group. By these addition processes, the first focus detection signal and the second focus detection signal corresponding to the pupil division in the vertical direction can be acquired, and the first focus detection and the second focus detection of the phase difference detection method can be performed. Is possible.

  According to the image pickup apparatus of the present embodiment, it is possible to reduce the difference between the detected focus position based on the focus detection signal and the best focus position based on the image pickup signal, and to perform highly accurate focus detection.

  Thus, in each embodiment, the control device (CPU 121) includes a generation unit 121a and a calculation unit 121b. The generating unit 121a receives the first focus detection signal and the first focus signal based on a plurality of types of color signals from the first pixel group and the second pixel group that receive light beams that pass through different pupil partial regions of the imaging optical system. A bifocal detection signal is generated. The calculating means 121b calculates the defocus amount by the phase difference detection method using the first focus detection signal and the second focus detection signal. The generation unit 121a generates a first focus detection signal by combining the color signals (performing pixel addition processing) so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other with respect to the first pixel group. To do. Further, the generation unit 121a generates a second focus detection signal by combining the color signals so that the centroids of the plurality of types of color signals in the pupil division direction coincide with each other with respect to the second pixel group.

  Preferably, in the pupil division direction, the signal periods of the first focus detection signal and the second focus detection signal are equal to the arrangement periods of the first pixel group and the second pixel group, respectively. Preferably, the plurality of types of color signals include a first color signal, a second color signal, and a third color signal. The generating unit 121a relates to the first pixel group, the first color signal, the second color signal, the second color signal, and the third color signal so that the centroids in the pupil division direction coincide with each other. Then, the third color signal is combined to generate a first focus detection signal as the first luminance signal (Y2A). In addition, the generation unit 121a relates to the second pixel group so that the first color signal, the second color signal, and the third color signal have the same center of gravity in the pupil division direction. Then, the third color signal is combined to generate a second focus detection signal as the second luminance signal (Y2B).

  Preferably, the first color signal is a signal obtained from the first pixel (for example, RA (i, j) in FIG. 12) in the first pixel group. The second color signal includes a second pixel (GA (i, j−1)), a third pixel (GA (i, j + 1)), a fourth pixel adjacent to the first pixel in the first pixel group. This is a signal obtained from (GA (i + 1, j)). At this time, the centroids in the pupil division direction of the first color signal and the second color signal respectively correspond to the positions of the first pixels. This also applies to the second pixel group. More preferably, the third color signal is obtained from the fifth pixel (for example, BA (i + 1, j−1) in FIG. 12) and the sixth pixel (BA (i + 1, j + 1)) in the first pixel group. Signal. At this time, the center of gravity of the third color signal in the pupil division direction corresponds to the position of the first pixel. This also applies to the second pixel group.

  Preferably, the first color signal is a signal obtained from the first pixel (for example, RA (i, j) in FIG. 14) in the first pixel group. The second color signal is the second pixel (GA (i, j−1)), the third pixel (GA (i, j + 1)), the fourth pixel (GA) adjacent to the first pixel in the first pixel group. (I + 1, j)) and a signal obtained from the fifth pixel (GA (i-1, j)). At this time, the centroids in the pupil division direction of the first color signal and the second color signal respectively correspond to the positions of the first pixels. This also applies to the second pixel group. More preferably, the third color signal includes the sixth pixel (for example, BA (i−1, j−1) in FIG. 14) and the seventh pixel (BA (i + 1, j−1) in the first pixel group. )), The eighth pixel (BA (i-1, j + 1)), and the ninth pixel (BA (i + 1, j + 1)). At this time, the center of gravity of the third color signal in the pupil division direction corresponds to the position of the first pixel. This also applies to the second pixel group.

  Preferably, the first color signal, the second color signal, and the third color signal are a red signal, a green signal, and a blue signal, respectively, and the first pixel group and the second pixel group are Bayer, respectively. Has an array. For each of the first focus detection signal and the second focus detection signal, the synthesis ratio of the first color signal, the second color signal, and the third color signal is 1: 2: 1.

  Preferably, the control device includes a focus control unit 121c that performs focus control based on the defocus amount. The focus control unit 121c performs the focus control based on the first defocus amount when the first defocus amount calculated by the calculation unit 121b in the first focus detection process is larger than the first threshold (first predetermined value). Do. On the other hand, when the first defocus amount is smaller than the first threshold, the focus control unit 121c performs focus control based on the second defocus amount calculated by the calculation unit 121b in the second focus detection process. In the second focus detection process, the generation unit 121a generates the first focus detection signal and the second focus detection signal by synthesizing the color signals so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other. To do. More preferably, in the first focus detection process, the signal period in the pupil division direction of the first focus detection signal and the second focus detection signal generated by the generation unit 121a is the array period of the first pixel group and the second pixel group. Bigger than. More preferably, the spatial frequency bands of the first focus detection signal and the second focus detection signal are higher in the second focus detection process than in the first focus detection process.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  According to each embodiment, a control device, an imaging device, and a control capable of highly accurate focus detection by reducing a difference between a detection focus position based on a focus detection signal and a best focus position based on an imaging signal. Methods, programs, and storage media can be provided.

  As mentioned above, although the preferable Example of this invention was described, this invention is not limited to these Examples, A various deformation | transformation and change are possible within the range of the summary.

  For example, the control device can execute a control method (image processing method) that acquires a plurality of detection signals used in a phase difference detection method to calculate distance information based on a plurality of types of color signals. In this control method, a first process is performed on a viewpoint image (parallax image) including a plurality of types of color signals, a synthesized signal obtained by synthesizing the color signals is obtained, and a second process is performed on the synthesized signal. And generating a detection signal. The first process is a process for synthesizing color signals so that the centroids in the viewpoint direction of the viewpoint images match. Here, the first process includes a process of combining a plurality of types of color signals at different ratios, and the second process includes a process of thinning out in order to vary the frequency band of the signal included in the combined signal. The control method may further include a step of detecting a phase difference using the detection signal. In the second process, the composite signal is processed based on the detected phase difference. Further, the first process may be a process of synthesizing color signals so that the centroids in the direction perpendicular to the viewpoint direction of the viewpoint image coincide.

121 CPU (control device)
121a generation means 121b calculation means

Claims (21)

Based on a plurality of types of color signals from the first pixel group and the second pixel group that receive light beams that pass through different pupil partial regions of the imaging optical system, the first focus detection signal and the second focus detection signal are obtained. Generating means for generating;
Calculating means for calculating a defocus amount by a phase difference detection method using the first focus detection signal and the second focus detection signal;
The generating means includes
The first focus detection signal is generated by combining the color signals so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other with respect to the first pixel group,
The control for generating the second focus detection signal by combining the color signals so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other with respect to the second pixel group apparatus.   The signal period of the first focus detection signal and the second focus detection signal in the pupil division direction is equal to an array period of the first pixel group and the second pixel group, respectively. The control device described. The plurality of types of color signals include a first color signal, a second color signal, and a third color signal,
The generating means includes
Regarding the first pixel group, the first color signal, the second color signal, and the third color signal, the first color signal, the second color, and the third color signal so that the centroids in the pupil division direction coincide with each other. Combining the signal and the third color signal to generate the first focus detection signal as a first luminance signal;
Regarding the second pixel group, the first color signal, the second color signal, and the third color signal, the first color signal, the second color, and the third color signal so that the centroids in the pupil division direction coincide with each other. 2. The control device according to claim 1, wherein the second focus detection signal as a second luminance signal is generated by combining the signal and the third color signal. 3. The first color signal is a signal obtained from the first pixel in each of the first pixel group and the second pixel group,
The second color signal is a signal obtained from a second pixel, a third pixel, and a fourth pixel adjacent to the first pixel in each of the first pixel group and the second pixel group. ,
4. The control device according to claim 3, wherein centroids in the pupil division direction of the first color signal and the second color signal respectively correspond to positions of the first pixels. The third color signal is a signal obtained from a fifth pixel and a sixth pixel in each of the first pixel group and the second pixel group,
The control device according to claim 4, wherein the center of gravity of the third color signal in the pupil division direction corresponds to the position of the first pixel. The first color signal is a signal obtained from the first pixel in each of the first pixel group and the second pixel group,
The second color signal is obtained from a second pixel, a third pixel, a fourth pixel, and a fifth pixel adjacent to the first pixel in each of the first pixel group and the second pixel group. Signal
4. The control device according to claim 3, wherein centroids in the pupil division direction of the first color signal and the second color signal respectively correspond to positions of the first pixels. The third color signal is a signal obtained from the sixth pixel, the seventh pixel, the eighth pixel, and the ninth pixel in each of the first pixel group and the second pixel group,
The control apparatus according to claim 6, wherein the center of gravity of the third color signal in the pupil division direction corresponds to the position of the first pixel. The first color signal, the second color signal, and the third color signal are a red signal, a green signal, and a blue signal, respectively.
Each of the first pixel group and the second pixel group has a Bayer array,
For each of the first focus detection signal and the second focus detection signal, a synthesis ratio of the first color signal, the second color signal, and the third color signal is 1: 2: 1. The control device according to claim 3, wherein the control device is a control device. A focus control unit that performs focus control based on the defocus amount;
The focus control means includes
When the first defocus amount calculated by the calculation means in the first focus detection process is larger than a first threshold value, the focus control is performed based on the first defocus amount;
When the first defocus amount is smaller than the first threshold, the focus control is performed based on the second defocus amount calculated by the calculation unit in the second focus detection process,
In the second focus detection process, the generation unit synthesizes the color signals so that centroids in the pupil division direction of the plurality of types of color signals coincide with each other, and generates the first focus detection signal and the second focus signal. The control apparatus according to claim 1, wherein a focus detection signal is generated.   In the first focus detection process, a signal period in the pupil division direction of the first focus detection signal and the second focus detection signal generated by the generation unit is set between the first pixel group and the second pixel group. The control device according to claim 9, wherein the control device is longer than the array period.   The spatial frequency band of each of the first focus detection signal and the second focus detection signal is higher in the second focus detection process than in the first focus detection process. The control device described in 1. An imaging device having a first pixel group and a second pixel group that receive light beams that pass through different pupil partial regions of the imaging optical system;
Generating means for generating a first focus detection signal and a second focus detection signal based on a plurality of types of color signals from the first pixel group and the second pixel group;
Calculating means for calculating a defocus amount by a phase difference detection method using the first focus detection signal and the second focus detection signal;
The generating means includes
The first focus detection signal is generated by combining the color signals so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other with respect to the first pixel group,
The second focus detection signal is generated by combining the color signals so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other with respect to the second pixel group. apparatus.   The imaging device according to claim 12, wherein the imaging device has a plurality of photoelectric conversion units for one microlens, and the microlenses are arranged in a two-dimensional manner.   The imaging apparatus according to claim 13, further comprising an image generation unit configured to generate a captured image from a signal obtained by adding the pixels included in each of the first pixel group and the second pixel group for each microlens. . Based on a plurality of types of color signals from the first pixel group and the second pixel group that receive light beams that pass through different pupil partial regions of the imaging optical system, the first focus detection signal and the second focus detection signal are obtained. Generating step;
Calculating a defocus amount by a phase difference detection method using the first focus detection signal and the second focus detection signal, and
In the step of generating the first focus detection signal and the second focus detection signal,
The first focus detection signal is generated by combining the color signals so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other with respect to the first pixel group,
The control for generating the second focus detection signal by combining the color signals so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other with respect to the second pixel group Method. A control method for acquiring a plurality of detection signals used in a phase difference detection method to calculate distance information,
Performing a first process on a viewpoint image including a plurality of types of color signals and obtaining a combined signal obtained by combining the color signals;
Performing a second process on the combined signal to generate the detection signal,
The control method according to claim 1, wherein the first process is a process of synthesizing the color signals so that centroids in the viewpoint direction of the viewpoint images coincide with each other. The first process includes a process of combining the plurality of types of color signals at different ratios,
The control method according to claim 16, wherein the second process includes a process of thinning out in order to vary a frequency band of a signal included in the synthesized signal. Further comprising detecting a phase difference using the detection signal;
The control method according to claim 16 or 17, wherein the second process performs a process on the combined signal based on the detected phase difference.   The said 1st process is a process which synthesize | combines the said color signal so that the gravity center in the direction perpendicular | vertical to the viewpoint direction of the said viewpoint image may correspond, The any one of Claims 16 thru | or 18 characterized by the above-mentioned. Control method. Based on a plurality of types of color signals from the first pixel group and the second pixel group that receive light beams that pass through different pupil partial regions of the imaging optical system, the first focus detection signal and the second focus detection signal are obtained. Generating step;
Calculating a defocus amount by a phase difference detection method using the first focus detection signal and the second focus detection signal, and causing a computer to execute the program.
In the step of generating the first focus detection signal and the second focus detection signal,
The first focus detection signal is generated by combining the color signals so that the centroids in the pupil division direction of the plurality of types of color signals coincide with each other with respect to the first pixel group,
A program for generating the second focus detection signal by combining the color signals so that the centroids of the plurality of types of color signals in the pupil division direction coincide with each other with respect to the second pixel group. .   A storage medium storing the program according to claim 20.