JP2016066015A - Focus detector, control method thereof, program, and memory medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a focus detector capable of improving accuracy in focus detection of a phase difference detection method in which an output of an imaging device is used, and a control method of the focus detector.SOLUTION: The focus detector has: a first calculation part that detects a first phase difference between a first image signal based on a luminous flux passing through a part of region of an exit pupil of a photographic optical system, and a third image signal based on the luminous flux passing through a whole region of the exit pupil; a second calculation part that calculates a second phase difference between a second image signal based on the luminous flux passing through another part of region different from the part of the exit pupil, and a fourth image signal based on the luminous flux passing through the whole region of the exit pupil; and a defocus amount calculation part that calculates a defocus amount of the photographic optical system by using a sum obtained by multiplying the first phase difference and the second phase difference by weighting factors and adding the phase differences together.SELECTED DRAWING: Figure 12

Description

  The present invention relates to a focus detection apparatus and a control method thereof, and more particularly to a focus detection apparatus that performs phase difference detection type focus detection based on an output of an image sensor and a control method thereof.

  Patent Document 1 discloses an apparatus that performs focus detection by pupil division using an imaging element in which a microlens is formed in each of two-dimensionally arranged pixels. In this apparatus, the photoelectric conversion unit of each pixel is divided into two regions that receive light beams that have passed through different regions of the exit pupil of the photographing lens through the microlens. A correlation operation is performed on a pair of output signals generated from a plurality of pixels for each divided region to calculate a phase difference (shift amount) between the pair of output signals, and a defocus amount is calculated from the phase difference. it can.

  Further, FIGS. 24 and 25 of Patent Document 1 and Patent Document 2 disclose an imaging device in which some pixels are focus detection pixels for performing pupil division focus detection. Correction is necessary to use the output of the focus detection pixel as the output of the normal pixel, but the number of signals to be read out as the focus detection signal is smaller than the configuration in which the photoelectric conversion unit of each pixel is divided. Processing cost of processing can be suppressed.

JP 2008-52009 A Japanese Patent No. 3592147

  In the configuration using the focus detection pixels, a pair of photoelectric conversion units that receive light beams that have passed through different regions of the exit pupil of the photographing lens through the microlens are arranged in different pixels. That is, the position of the pixel group used for generating a pair of output signals (A image and B image) for detecting the phase difference differs between the A image and the B image. In some cases, the similarity of the B image becomes low, and in such a case, the accuracy of focus detection decreases.

  Further, when the focus detection pixel arrangement interval is wide, it may be impossible to acquire the frequency component in the high frequency band of the subject optical image. Therefore, different aliasing noises are generated in the A image and the B image, and a focus detection error occurs.

  In addition, in a configuration in which a plurality of photoelectric conversion units are provided in each pixel, in addition to reading from each photoelectric conversion unit, addition for generating a normal image signal is necessary, which increases the load of reading processing and arithmetic processing. Therefore, it is considered to reduce the load by reading out the output of one photoelectric conversion unit and adding and reading the outputs of both photoelectric conversion units, and generating the output of the other photoelectric conversion unit by subtraction. It is done. However, since an error occurs due to the subtraction, there is a concern that the phase difference detection accuracy is lowered.

  The present invention aims to improve at least one of the problems of the prior art. Specifically, an object of the present invention is to provide a focus detection apparatus and a control method thereof that can improve the accuracy of focus detection by a phase difference detection method using the output of an image sensor.

  The focus detection apparatus according to the present invention includes a first image signal based on a light beam that has passed through a partial area of an exit pupil of a photographing optical system, and a third image signal based on a light beam that has passed through the entire area of the exit pupil. A first calculating means for detecting a first phase difference between the first pupil and the second pupil, and a second image signal based on a light beam that has passed through a partial area different from the partial area of the exit pupil, A second calculating means for detecting a second phase difference with the fourth image signal based on the light flux that has passed through the entire area; and multiplying the first phase difference and the second phase difference by a weighting coefficient. Defocus amount calculation means for calculating the defocus amount of the photographing optical system using the added sum.

  With such a configuration, according to the present invention, it is possible to provide a focus detection apparatus capable of improving the accuracy of focus detection by the phase difference detection method using the output of the image sensor and a control method thereof.

1 is a block diagram illustrating an example of a functional configuration of a camera system as an example of an imaging apparatus including a focus adjustment device according to an embodiment. FIG. 3 is a diagram illustrating a configuration example of an image sensor according to the first embodiment. The figure which shows the relationship between the photoelectric conversion area | region and exit pupil in 1st Embodiment. 4 is a diagram illustrating an example of a relationship between a focus detection area and pixels used for AF signals in the embodiment. FIG. 6 is a flowchart illustrating a focus adjustment operation in the embodiment. 6 is a flowchart illustrating a defocus amount calculation method according to the first embodiment. FIG. 6 is a diagram illustrating a configuration example of an image sensor according to a second embodiment. The figure which shows the relationship between the photoelectric conversion area | region and exit pupil in 2nd Embodiment. 10 is a flowchart illustrating a defocus amount calculation method according to the third embodiment. The figure which showed the relationship between the exit pupil distance of a photographic lens, and the pupil position which the focus detection pixel is looking at. The figure explaining that the vignetting of the pupil differs greatly between the A image signal for AF and the B image signal for AF when the image height is high. 14 is a flowchart illustrating a defocus amount calculation subroutine according to the fourth embodiment. The flowchart which shows the detailed process of step S7041 in FIG. The flowchart which shows the detailed process of step S7041 in FIG.

● (first embodiment)
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a configuration example of a camera system including a camera and a photographic lens in which a photographic lens can be exchanged, as an example of an imaging device including a focus detection device according to an embodiment of the present invention. In FIG. 1, the camera system includes a camera 100 and an interchangeable photographic lens 300.

  The light beam that has passed through the photographing lens 300 passes through the lens mount 106, is reflected upward by the main mirror 130, and enters the optical viewfinder 104. The optical viewfinder 104 allows the photographer to shoot while observing the subject optical image. In the optical viewfinder 104, some functions of the display unit 54, for example, focus display, camera shake warning display, aperture value display, exposure correction display, and the like are installed.

  A part of the main mirror 130 is composed of a semi-transmissive half mirror, and a part of the light beam incident on the main mirror 130 passes through the half mirror part and is reflected downward by the sub mirror 131 to the focus detection device 105. Incident. The focus detection device 105 is a phase difference detection type focus detection device having a secondary imaging optical system and a line sensor, and outputs a pair of image signals to an AF unit (autofocus unit) 42. The AF unit 42 performs a phase difference detection calculation for a pair of image signals, and obtains the defocus amount and direction of the photographing lens 300. Based on the calculation result, the system control unit 50 controls the focus control unit 342 (described later) of the taking lens 300, such as a focus adjustment process. In this embodiment, the AF detection unit 42 also corrects the focus detection result.

  When the focus adjustment processing of the photographic lens 300 is completed and still image shooting is performed, when an electronic viewfinder display is performed, or when moving image shooting is performed, the main mirror 130 and the sub mirror 131 are moved out of the optical path by a quick return mechanism (not shown). Evacuate. Then, the light beam that passes through the photographing lens 300 and enters the camera 100 can enter the image sensor 14 via the shutter 12 for controlling the exposure amount. After the photographing operation by the image sensor 14 is completed, the main mirror 130 and the sub mirror 131 return to the positions as shown in the figure.

  The image sensor 14 is a CCD or CMOS image sensor, has a configuration in which a plurality of pixels are two-dimensionally arranged, photoelectrically converts a subject optical image for each pixel, and outputs an electrical signal. The electrical signal photoelectrically converted by the image sensor 14 is sent to the A / D converter 16, and the analog signal output is converted into a digital signal (image data). The timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26. The timing generation circuit 18 is controlled by the memory control unit 22 and the system control unit 50. The image processing unit 20 applies predetermined processing such as pixel interpolation processing, white balance adjustment processing, and color conversion processing to the image data from the A / D converter 16 or the image data from the memory control unit 22.

  In the imaging device 14 according to the present embodiment, some pixels are configured as focus detection pixels, and even when the main mirror 130 and the sub mirror 131 are retracted out of the optical path by the quick return mechanism, focus detection by the phase difference detection method is performed. Is possible. Of the image data obtained by the image sensor 14, pixel data used for generating a focus detection signal is converted into focus detection data by the image processing unit 20. Thereafter, the focus detection data is sent to the AF unit 42 via the system control unit 50, and the AF unit 42 adjusts the focus of the photographing lens 300 based on the focus detection data.

  Note that contrast-based AF is also possible in which the image processing unit 20 calculates a contrast evaluation value from image data captured by the image sensor 14 and the system control unit 50 performs focusing on the focus control unit 342 of the imaging lens 300. It is. As described above, the camera 100 according to the present embodiment has a phase difference from the image data obtained by the image sensor 14 even when the main mirror 130 and the sub mirror 131 are retracted from the optical path as in live view display or moving image shooting. Both detection AF and contrast AF are possible. In addition, the camera 100 according to the present embodiment can perform phase difference detection AF using the focus detection device 105 in normal still image shooting in which the main mirror 130 and the sub mirror 131 are in the optical path. Therefore, focus adjustment is possible in any state during still image shooting, live view display, and moving image shooting.

  The memory control unit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing unit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression unit 32. Then, the data of the A / D converter 16 is written into the image display memory 24 or the memory 30 through the image processing unit 20 and the memory control unit 22 or only through the memory control unit 22. Display image data written in the image display memory 24 is displayed on an image display unit 28 including a liquid crystal monitor or the like via a D / A converter 26. An electronic viewfinder function (live view display) can be realized by sequentially displaying a moving image captured by the image sensor 14 on the image display unit 28. The image display unit 28 can turn on / off the display according to an instruction from the system control unit 50. When the display is turned off, the power consumption of the camera 100 can be greatly reduced.

  The memory 30 is used for temporary storage of captured still images and moving images, and has a storage capacity sufficient to store a predetermined number of still images and moving images for a predetermined time. Thereby, even in the case of continuous shooting or panoramic shooting, a large amount of image writing can be performed on the memory 30 at high speed. The memory 30 can also be used as a work area for the system control unit 50. The compression / decompression unit 32 has a function of compressing / decompressing image data by adaptive discrete cosine transform (ADCT) or the like, reads an image stored in the memory 30, performs compression processing or decompression processing, and finishes the processed image data Is written back to the memory 30.

  The shutter control unit 36 controls the shutter 12 based on the photometric information from the photometric unit 46 in cooperation with the aperture control unit 344 that controls the aperture 312 of the photographing lens 300. The interface unit 38 and the connector 122 electrically connect the camera 100 and the photographing lens 300. These have functions of transmitting control signals, status signals, data signals, and the like between the camera 100 and the photographing lens 300 and supplying currents of various voltages. Moreover, it is good also as a structure which transmits not only electrical communication but optical communication, audio | voice communication, etc.

  The photometry unit 46 performs automatic exposure control (AE) processing. The light flux that has passed through the photographic lens 300 is incident on the photometric unit 46 via the lens mount 106, the main mirror 130, and a photometric lens (not shown), whereby the luminance of the subject optical image can be measured. The photometry unit 46 can determine the exposure condition by using a program diagram in which the subject brightness and the exposure condition are associated with each other. The photometry unit 46 also has a light control processing function in cooperation with the flash 48. The system control unit 50 can also perform AE control on the shutter control unit 36 and the aperture control unit 344 of the photographic lens 300 based on the calculation result obtained by calculating the image data of the image sensor 14 by the image processing unit 20. It is. The flash 48 also has an AF auxiliary light projecting function and a flash light control function.

  The system control unit 50 includes a programmable processor such as a CPU or MPU, and controls the operation of the entire camera system by executing a program stored in advance. The nonvolatile memory 52 stores constants, variables, programs, etc. for operating the system control unit 50. The display unit 54 is, for example, a liquid crystal display device that displays an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program by the system control unit 50. One or a plurality of display units 54 are installed at positions in the vicinity of the operation unit of the camera 100 that are easily visible, and are configured by a combination of an LCD, an LED, and the like. Among the display contents of the display unit 54, what is displayed on the LCD or the like includes information on the number of shots such as the number of recorded sheets and the number of remaining shots, information on shooting conditions such as shutter speed, aperture value, exposure correction, and flash. There is. In addition, the remaining battery level, date / time, and the like are also displayed. Further, as described above, a part of the display unit 54 is installed in the optical viewfinder 104.

  The nonvolatile memory 56 is an electrically erasable / recordable memory, and for example, an EEPROM or the like is used. Reference numerals 60, 62, 64, 66, 68, and 70 are operation units for inputting various operation instructions of the system control unit 50, such as a switch, a dial, a touch panel, pointing by gaze detection, a voice recognition device, or the like. Consists of multiple combinations.

  The mode dial 60 can switch and set each function mode such as power-off, auto shooting mode, manual shooting mode, playback mode, and PC connection mode. A shutter switch SW1 62 is turned on when a shutter button (not shown) is half-pressed, and instructs to start operations such as AF processing, AE processing, AWB processing, and EF processing. The shutter switch SW2 64 is turned on when the shutter button is fully pressed, and instructs the start of a series of processing related to photographing. A series of processing related to photographing is exposure processing, development processing, recording processing, and the like. In the exposure process, the signal read from the image sensor 14 is written as image data in the memory 30 via the A / D converter 16 and the memory control unit 22. In the development process, development is performed using computations in the image processing unit 20 and the memory control unit 22. In the recording process, image data is read from the memory 30, compressed by the compression / decompression unit 32, and written as image data on the recording medium 200 or 210.

  The image display ON / OFF switch 66 can set ON / OFF of the image display unit 28. With this function, when photographing using the optical viewfinder 104, power supply can be saved by cutting off the current supply to the image display unit 28 including a liquid crystal monitor or the like. The quick review ON / OFF switch 68 sets a quick review function for automatically reproducing captured image data immediately after shooting. The operation unit 70 includes various buttons and a touch panel. The various buttons include a menu button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, an exposure correction button, and the like.

  The power supply control unit 80 is configured by a battery detection circuit, a DC / DC converter, a switch circuit that switches blocks to be energized, and the like. The presence / absence of a battery, the type of battery, and the remaining battery level are detected, the DC / DC converter is controlled based on the detection result and the instruction of the system control unit 50, and a necessary voltage is included for a necessary period and a recording medium. Supply to each part. The connectors 82 and 84 connect the camera 100 with a power supply unit 86 including a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a lithium ion battery, or an AC adapter.

  The interfaces 90 and 94 have a connection function with a recording medium such as a memory card or a hard disk, and the connectors 92 and 96 make a physical connection with a recording medium such as a memory card or a hard disk. The recording medium attachment / detachment detection unit 98 detects whether or not a recording medium is attached to the connector 92 or 96. Although the present embodiment has been described as having two systems of interfaces and connectors for attaching the recording medium, the interface and connectors may have a single system or a plurality of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard. Furthermore, by connecting various communication cards such as a LAN card to the interface and connector, it is possible to transfer image data and management information attached to the image data to and from other peripheral devices such as a computer and a printer.

  The communication unit 110 has various communication functions such as wired communication and wireless communication. The connector 112 connects the camera 100 to other devices via the communication unit 110, and is an antenna in the case of wireless communication. The recording media 200 and 210 are a memory card, a hard disk, or the like. The recording media 200 and 210 include recording units 202 and 212 configured by a semiconductor memory, a magnetic disk, and the like, interfaces 204 and 214 with the camera 100, and connectors 206 and 216 for connecting with the camera 100.

  Next, the photographing lens 300 will be described. The taking lens 300 is mechanically and electrically coupled to the camera 100 by engaging the lens mount 306 with the lens mount 106 of the camera 100. Electrical coupling is realized by the connector 122 and the connector 322 provided on the lens mount 106 and the lens mount 306. The lens 311 includes a focus lens for adjusting the focusing distance of the photographic lens 300, and the focus control unit 342 adjusts the focus of the photographic lens 300 by driving the focus lens along the optical axis. The diaphragm 312 adjusts the amount and angle of subject light incident on the camera 100.

  The connector 322 and the interface 338 electrically connect the photographing lens 300 to the connector 122 of the camera 100. The connector 322 has a function of transmitting a control signal, a status signal, a data signal, and the like between the camera 100 and the photographing lens 300 and supplying currents of various voltages. The connector 322 may be configured to transmit not only electrical communication but also optical communication, voice communication, and the like.

  The zoom control unit 340 drives the variable power lens of the lens 311 and adjusts the focal length (view angle) of the photographing lens 300. If the taking lens 300 is a single focus lens, the zoom control unit 340 does not exist. The aperture control unit 344 controls the aperture 312 in cooperation with the shutter control unit 36 that controls the shutter 12 based on the photometry information from the photometry unit 46.

  The lens system control unit 346 has a programmable processor such as a CPU or an MPU, for example, and controls the entire operation of the photographing lens 300 by executing a program stored in advance. The lens system control unit 346 has a memory function for storing constants, variables, programs, and the like for operating the photographing lens. The nonvolatile memory 348 stores identification information such as a number unique to the photographing lens, management information, function information such as an open aperture value, a minimum aperture value, and a focal length, and current and past setting values.

  In the present embodiment, lens frame information corresponding to the state of the photographing lens 300 is also stored. This lens frame information is information on the radius of the frame opening that determines the light beam passing through the photographing lens and information on the distance from the image sensor 14 to the frame opening. The diaphragm 312 is included in a frame that determines a light beam that passes through the photographing lens, and an opening of a lens frame component that holds the lens corresponds to the frame. In addition, since the frame for determining the light flux that passes through the photographing lens differs depending on the focus position and zoom position of the lens 311, a plurality of pieces of lens frame information are prepared corresponding to the focus position and zoom position of the lens 311. When the camera 100 performs focus detection using the focus detection means, optimal lens frame information corresponding to the focus position and zoom position of the lens 311 is selected and sent to the camera 100 through the connector 322.

  The above is the configuration of the camera system according to this embodiment including the camera 100 and the photographing lens 300.

  Next, the focus detection operation of the phase difference detection method using the image sensor 14 will be described.

  FIG. 2A is a diagram schematically showing an example of the pixel arrangement of the image sensor 14 in the present embodiment. Among the pixel groups arranged two-dimensionally in the CMOS image sensor, the vertical (Y-axis direction) is 6 rows. A state in which a range of 8 rows (in the X-axis direction) is observed from the photographing lens 300 side is shown. The image sensor 14 has a Bayer array color filter. The even-numbered pixels have green and red color filters in order from the left, and the odd-numbered pixels have blue in order from the left. Green color filters are alternately provided. However, in the image sensor 14 of the present embodiment, a green color filter is provided instead of the original blue color filter for the pixel having the photoelectric conversion unit for focus detection. In the following description, a pixel provided with a blue (or green, red) color filter may be referred to as a blue pixel (or green pixel, red pixel).

  Each pixel is provided with an on-chip microlens 211i, and the rectangle in the on-chip microlens 211i schematically shows the light receiving area of the photoelectric conversion unit. The focus detection photoelectric conversion units 311a and 311b are arranged laterally offset from the center of the pixel. In the following description, a pixel provided with focus detection photoelectric conversion units 311a and 311b may be referred to as a focus detection pixel. The focus detection photoelectric conversion units 311a and 311b are arranged in green pixels provided in place of the original blue pixels. This is because the output of the blue pixel has the lowest influence on the image quality. Note that the present invention does not depend on the color filter pattern of the image sensor. As described above, since the image sensor 14 of this embodiment includes one photoelectric conversion unit for each pixel including the focus detection pixel, one photoelectric conversion signal is read from one pixel.

  Here, generation of an image signal used for focus detection by the phase difference detection method will be described. In this embodiment, four types of image signals are generated. As will be described later, in the present embodiment, the exit pupil of the imaging optical system (imaging lens 300) is divided using the microlens 211i and the photoelectric conversion units 311a and 311b having different bias positions. Then, among the outputs of the pixels 211 arranged in the same pixel row (X-axis direction), an output obtained by connecting the outputs of the plurality of photoelectric conversion units 311a and connecting the outputs of the A image and the plurality of photoelectric conversion units 311b is connected. A B image is obtained by knitting together. As shown in FIG. 2A, the A image and the B image can be obtained from a plurality of blue pixel positions (green pixels) adjacent to each other at a two-pixel pitch in the X-axis direction.

  Further, the photoelectric conversion units 311c that are the plurality of green pixels in the first row in FIG. 2A are adjacent to the photoelectric conversion unit 311a in the X-axis direction in FIG. A GA image is formed by connecting the outputs of 311c. In addition, the photoelectric conversion units 311c that are the plurality of green pixels in the fifth row in FIG. 2A are adjacent to the photoelectric conversion unit 311b in the X-axis direction in FIG. A GB image is formed by connecting the outputs of 311c and knitting them. The photoelectric conversion units 311a and 311b output a signal based on the light beam that has passed through a partial region of the exit pupil of the imaging optical system (imaging lens 300), whereas the photoelectric conversion unit 311c has an imaging optical system (imaging lens). 300), a signal based on the light flux that has passed through the entire area of the exit pupil is output. Thus, the A color image (output of the photoelectric conversion unit 311a), the B image (output of the photoelectric conversion unit 311b), the GA image (output of the photoelectric conversion unit 311c), and the GB image (output of the photoelectric conversion unit 311c) are the same color pixels. By obtaining from the group, it is possible to detect the phase difference with high accuracy.

  Note that the positions and the number of pixels used for generating the A image, the B image, the GA image, and the GB image are determined according to the focus detection area.

  Although details will be described later, since the A image and the GA image generated in this way are images obtained from pixels in the same row, the A image to be originally obtained from the relative image shift amount between the A image and the GA image. And 1/2 of the image shift amount of the B image can be accurately obtained. Similarly, since the B image and the GB image are images obtained from pixels in the same row, the remaining image displacement amount of the A image and the B image that is originally obtained from the relative image displacement amount between the B image and the GB image. Of 1/2 is accurately obtained. Then, by adding these together and performing a correlation calculation, it is possible to detect a defocus amount, that is, a defocus amount in a predetermined area of the A image and the B image. In the present embodiment, a normal pixel signal is obtained from a pixel having a photoelectric conversion region 311c whose position is not deviated from the center of the pixel (which may be referred to as a photographic pixel in the following description). When a captured image is generated, a normal pixel signal at a position corresponding to a focus detection pixel is generated (complemented) using the output of surrounding pixels. Note that, when generating a normal pixel signal, the output of the target focus detection pixel may or may not be used.

  Hereinafter, the plurality of pixels provided with the photoelectric conversion unit 311a used for generating the A image (first image signal) are referred to as a first pixel group, and are used for generating the B image (second image signal). A plurality of pixels provided with the photoelectric conversion unit 311b is referred to as a second pixel group. A plurality of pixels provided with the photoelectric conversion unit 311c, which are used for generating a GA image (third image signal), are referred to as a third pixel group, and are used for generating a GB image (fourth image signal). A plurality of pixels provided with the photoelectric conversion unit 311c is referred to as a fourth pixel group.

  In the present embodiment, the third pixel group and the fourth pixel group are pixel groups adjacent to the first pixel group and the second pixel group in the X-axis direction. However, the third pixel group and the fourth pixel group may be pixel groups adjacent to the first pixel group and the second pixel group in the Y-axis direction. Alternatively, a GA image and a GB image may be generated using pixel values obtained from other pixels. For example, the GA image may be generated from the pixel value calculated as the average value of a plurality of adjacent (for example, four) pixels for each pixel of the first pixel group.

  Basically, in the direction orthogonal to the phase difference detection direction, the distance between the first pixel group and the third pixel group is greater than the distance between the first pixel group and the second pixel group. If the third pixel group is selected so as to shorten the length, the effect of the present invention can be obtained. Similarly, the fourth pixel group may be selected so that the distance between the second pixel group and the fourth pixel group is shorter than the distance between the first pixel group and the second pixel group. . When the pixel values of the third and fourth pixel groups are generated from other pixel values, the virtual pixel positions of the pixels of the third and fourth pixel groups may be selected in the same manner.

  FIG. 2B is a diagram illustrating a configuration example of a readout circuit of the image sensor 14 of the present embodiment. The image sensor 14 has a horizontal scanning circuit 151 and a vertical scanning circuit 153, and a horizontal scanning line 252 and a vertical scanning line 254 are wired at the boundary of each pixel. Signals generated by the photoelectric conversion units 311a, 311b, and 311c are read out through the horizontal scanning line 252 and the vertical scanning line 254.

  FIG. 3 is a diagram for explaining a conjugate relationship between the exit pupil plane of the imaging lens 300 and the photoelectric conversion units 311a and 311b of the pixel 211 disposed in the vicinity of the center of the image plane of the imaging device 14. The exit pupil planes of the photoelectric conversion units 311a and 311b and the imaging lens 300 in the image sensor 14 are designed to have a conjugate relationship by the on-chip microlens 211i. In general, the exit pupil plane of the photographing lens 300 substantially coincides with the plane on which the iris diaphragm for adjusting the light amount is provided.

  On the other hand, the photographing lens 300 of the present embodiment is a zoom lens having a zooming function. Some zoom lenses change the size of the exit pupil and the distance from the image plane to the exit pupil (exit pupil distance) when a zooming operation is performed. FIG. 3 shows a state where the focal length of the photographic lens 300 is at the center between the wide-angle end and the telephoto end. Using the exit pupil distance Zep in this state as a standard value, the optimum design of the eccentric parameter according to the shape of the on-chip microlens and the image height is performed.

  In FIG. 3, the photographic lens 300 includes a first lens group 101, a lens barrel member 101b that holds the first lens group, a third lens group 105, and a lens barrel member 105b that holds the third lens group. . The photographic lens 300 includes an aperture 102, an aperture plate 102a that defines an aperture diameter when the aperture is open, and an aperture blade 102b for adjusting the aperture diameter when the aperture is closed. In FIG. 3, reference numerals 101b, 102a, 102b, and 105b that act as limiting members for the light flux passing through the photographing lens 300 indicate optical virtual images when observed from the image plane. Further, the synthetic aperture in the vicinity of the stop 102 is defined as the exit pupil of the photographing lens 300, and has an exit pupil distance Zep.

  In the lowest layer of the pixel 211, a photoelectric conversion unit 311a (FIG. 3A), a photoelectric conversion unit 311b (FIG. 3B), or a photoelectric conversion unit 311c (not shown) is arranged. On the upper layers of the photoelectric conversion units 311a to 311c, wiring layers 211e to 211g, a color filter 211h, and an on-chip microlens 211i are provided. The photoelectric conversion units 311a to 311c are projected onto the exit pupil plane of the photographing lens 300 by the on-chip microlens 211i. In other words, the exit pupil is projected onto the surfaces of the photoelectric conversion units 311a to 311c via the on-chip microlens 211i.

  FIG. 3C shows projection images EP1a and EP1b of the photoelectric conversion units 311a and 311b on the exit pupil plane. The projected image EP1c for the photoelectric conversion unit 311c is substantially equal to the sum of EP1a and EP1b.

  3A and 3B, the outermost part of the light beam passing through the photographing lens 300 is indicated by L. The outermost L of the light beam is regulated by the aperture plate 102a of the diaphragm, and the vignetting of the projection images EP1a and EP1b hardly occurs in the photographing lens 300. FIG. 3C shows a circle TL formed on the exit surface by the outermost L of the light beam in FIGS. 3A and 3B. Since most of the projection images EP1a and EP1b of the photoelectric conversion units 311a and 311b are present inside the circle TL, it can be seen that almost no vignetting occurs. Since the outermost L of the light beam is defined by the aperture plate 102a of the stop, it can be paraphrased as TL = 102a. At this time, the vignetting state of the projection images EP1a to EP1b is symmetric with respect to the optical axis at the center of the image plane, and the received light amounts of the photoelectric conversion units 311a and 311b are equal. As described above, the imaging device 14 according to the present embodiment has not only an imaging function but also a function as a device that generates a signal used for focus detection by the phase difference detection method.

  FIG. 4A is a diagram illustrating an example of the focus detection area 401 set in the shooting range 400. When focus detection is performed using the output of the pixels of the image sensor 14, the output of the pixels included in the area of the image sensor 14 corresponding to the focus detection area 401 is output in both the contrast detection method and the phase difference detection method. Use. Therefore, it can be said that the focus detection area 401 is set in the image sensor 14. Hereinafter, the focus detection area 401 will be described as a pixel area of the image sensor 14 for ease of explanation and understanding.

  Here, it is assumed that the photoelectric conversion units 311a to 311c are provided in the pixels in the focus detection area 401 according to the rules shown in FIG. Since a focus detection pixel having photoelectric conversion units 311a and 311b deviated in the horizontal (X-axis) direction from the center of the pixel is used, the phase difference of the image signal is detected based on the contrast difference in the horizontal direction of the image in the focus detection area 401. To do.

  The phase difference detected here is generated by the difference in the traveling angle of the pair of light beams, and the phase difference per unit defocus amount is within the region on the exit pupil plane of the light beam that generates the pair of image signals. It is proportional to the center of gravity interval. As described above, the projection image EP1c for the photoelectric conversion unit 311c is substantially equal to the sum of the projection images EP1a and EP1b. Therefore, the barycentric position of the projection image EP1c exists at the center of the pair of barycentric positions of the projection images EP1a and EP1b. Therefore, the phase difference between the pair of image signals (A image and B image) obtained from the photoelectric conversion units 311a and 311b is the same as the pair of image signals (A image (A image (B)) obtained from the photoelectric conversion units 311a (311b) and 311c. B image) and GA image (GB image)) are approximately twice the phase difference.

  Since the projection image EP1c is common to the GA image and the GB image, the light beam for generating the GA image and the light beam for generating the GB image have the same barycentric position on the exit surface. Accordingly, the sum of the phase difference between the A image and the GA image obtained from the outputs of the photoelectric conversion units 311a and 311c and the phase difference between the B image and the GB image obtained from the outputs of the photoelectric conversion units 311b and 311c is the photoelectric conversion unit 311a. , 311b is approximately equal to the phase difference between the A and B images obtained from the output.

  FIG. 4B is a diagram showing pixels used for generating an AF image signal out of the pixels included in the focus detection area 401 and which image signal is generated by the output of each pixel. It is. In FIG. 4B, for each pixel group (first to fourth pixel groups) that generate the same type of image signal, the j-th pixel on the i-th row is referred to as “image signal type” (i, j) (where i and j are integers from 1 to N). For example, in the first pixel group that generates the A image, the first pixel in the first row is represented as A (1,1). Note that the color coding of the photoelectric conversion units in FIG. 4B is intended to facilitate understanding of pixel groups that generate the same type of image signal.

  FIG. 4B shows a case where the pixels used for generating the AF signal among the pixels in the focus detection area 401 are 2 rows and 2 N columns. Not exclusively. The number of rows may be two or more, and the number of columns may be appropriately set within a range in which a phase difference can generally be detected. Note that the number of columns may be dynamically increased when the phase difference cannot be detected or when it is determined that the accuracy is low.

  Next, the focus adjustment operation in the camera 100 will be described using the flowchart shown in FIG. Note that the processing shown in FIG. 5 is a state in which the main mirror 130 and the sub mirror 131 are retracted from the optical path (mirror-up), more specifically, during live view display (when displaying a moving image for display) or during moving image recording (for recording). This is a process that is performed at the time of video shooting. Although the description is given here assuming that the phase difference detection type automatic focus detection using the output of the image sensor 14 is performed, as described above, the contrast detection type automatic focus detection can also be performed.

  In step S501, the system control unit 50 determines whether a focus detection start instruction has been input by operating the SW1 (62), the operation unit 70, and the like. If it is determined that the focus detection start instruction has been input, the process proceeds to step S502. If it is not determined that it is, it waits. Note that the system control unit 50 is not limited to the input of the focus detection start instruction, and the process may proceed to S502 with the start of live view display or moving image recording as a trigger.

  In step S <b> 502, the system control unit 50 acquires various lens information such as the lens frame information and the focus lens position of the photographing lens 300 from the lens system control unit 346 via the interface units 38 and 338 and the connectors 122 and 322.

  In step S503, the system control unit 50 generates AF image signals (A image, B image, GA image, and GB image) from the pixel data in the focus detection area of the frame image data that is sequentially read out. To the image processing unit 20. The AF image signal is sent to the AF unit 42, and processing for correcting a difference in signal level caused by the size of the photoelectric conversion unit being different between the focus detection pixel and the imaging pixel is performed.

  In S <b> 504, the AF unit 42 calculates a shift amount of the image by applying a known correlation calculation or the like to the two pairs of image signals of the A image and the GA image, and the B image and the GB image, and converts it to a defocus amount. . Details of this processing will be described later. The AF unit 42 outputs the defocus amount to the system control unit 50.

  In S505, the system control unit 50 calculates the lens driving amount of the photographing lens 300 based on the defocus amount obtained from the AF unit 42 in S504.

  In step S <b> 506, the system control unit 50 transmits information on the lens driving amount and the driving direction to the focus control unit 342 of the photographing lens 300 via the interface units 38 and 338 and the connectors 122 and 322. The focus control unit 342 drives the focus lens based on the received lens driving amount and driving direction information. Thereby, the focus of the photographic lens 300 is adjusted. Note that the operation of FIG. 5 may be continuously performed even when moving image data after the next frame is read.

  Next, the defocus amount calculation processing performed by the AF unit 42 in S504 of FIG. 5 will be further described with reference to the flowchart shown in FIG. In S5041, the AF unit 42 as the first calculating unit performs a correlation operation between the A image and the GA image generated from the same pixel row (m-th row). The correlation amount COR1 (k) used for the correlation calculation can be calculated by the following equation (1), for example.

  The variable k used in the equation (1) is a shift amount at the time of correlation calculation, and is an integer from −kmax to kmax. After the AF unit 42 obtains the correlation amount COR1 (k) for each shift amount k, the shift amount k that maximizes the correlation between the A image and the GA image, that is, the value of the shift amount k that minimizes the correlation amount COR1. Ask for. Note that the shift amount k at the time of calculating the correlation amount COR1 (k) is an integer, but when obtaining the shift amount k that minimizes the correlation amount COR1 (k), in order to improve the accuracy of the defocus amount, Interpolation processing is performed as appropriate to obtain a value (real value) in units of subpixels.

  In the present embodiment, the shift amount dk at which the sign of the difference value of the correlation amount COR1 changes is calculated as the shift amount k that minimizes the correlation amount COR1 (k).

  First, the AF unit 42 calculates the difference value DCOR1 of the correlation amount according to the following equation (2).

DCOR1 (k) = COR1 (k) -COR1 (k-1) ... (2)
Then, the AF unit 42 obtains the shift amount dk1 in which the sign of the difference amount changes using the correlation amount difference value DCOR1. If the value of k immediately before the sign of the difference amount changes is k1, and the value of k where the sign changes is k2 (k2 = k1 + 1), the AF unit 42 calculates the shift amount dk1 according to the following equation (3). .

dk1 = k1 + | DCOR1 (k1) | / | DCOR1 (k1) -DCOR1 (k2) | ... (3)
As described above, the AF unit 42 serving as the first calculating unit calculates the shift amount dk1 that maximizes the correlation amount between the A image and the GA image in units of subpixels, and ends the process of S5041. The method for calculating the phase difference between the two one-dimensional image signals is not limited to that described here, and any known method can be used.

  In S5042, the AF unit 42 as the second calculation unit calculates the shift amount dk2 that maximizes the correlation for the B image and the GB image generated from the same pixel row (m + 1) by the same method as in S5041. .

  In S5043, the AF unit 42 calculates the sum dk_sum of the two types of shift amounts dk1 and dk2. As described above, the sum dk_sum corresponds to the phase difference between the A image and the B image. Therefore, the AF unit 42 converts the shift amount sum dk_sum into the defocus amount DEF by multiplying the sum of shift amounts dk_sum by, for example, the sensitivity stored in advance in the nonvolatile memory 56. When the calculation of the defocus amount DEF ends, the defocus amount calculation process ends.

  In this embodiment, an A image and a B image, which are signals obtained by photoelectrically converting light beams passing through different areas on the exit pupil of the photographing optical system, are orthogonal to the phase difference detection direction (X axis direction) (Y axis). The pixel group is located away from (direction). For this reason, the positions of the subject optical images sampled by the A and B images are different, and it is not guaranteed that the similarity between the A and B images is high. When obtaining the phase difference between two signals based on the amount of correlation, the higher the degree of similarity between the two signals, the higher the phase difference can be obtained. In the present embodiment, a GA image that can sample substantially the same position as the A image is generated on the subject optical image, and the phase difference between the A image and the GA image is calculated. Further, similarly, the phase difference between the B image and the GB image is calculated. Then, by summing up these two phase difference calculation results, the phase difference between the A image and the B image can be calculated with high accuracy.

  Further, since the phase difference between the A image and the GA image and the phase difference between the B image and the GB image are summed to calculate the phase difference between the A image and the B image, the phase difference per unit defocus amount is It becomes larger than the phase difference between the GA image (or B image and GB image). Therefore, the influence of noise included in the phase difference detection result can be reduced, and highly accurate phase difference detection can be performed.

  With this configuration, even when the similarity between the A image and the B image is low, the phase difference between the A image and the B image can be obtained with high accuracy. Therefore, the degree of freedom of arrangement of the pixel group (first pixel group) that generates the A image signal and the pixel group (second pixel group) that generates the B image signal is improved, and a signal for imaging is generated. In this case, it is possible to arrange the focus detection pixels at positions where correction can be easily performed. As a result, the correction accuracy of the pixel value corresponding to the focus detection pixel is improved, and high image quality can be realized.

  In this embodiment, the phase difference dk1 (first phase difference) obtained using the correlation amount between the A image and the GA image, and the phase difference dk2 (first phase difference) obtained using the correlation amount between the B image and the GB image. 2), and the sum of the phase differences dk1 and dk2 is converted into a defocus amount. However, the defocus amount calculation method is not limited to this. For example, the sum of the correlation amount (first correlation amount) between the A image and the GA image corresponding to the same shift amount k (first correlation amount) and the correlation amount between the B image and GB image (second correlation amount) is calculated, and two correlation amounts are calculated. The defocus amount may be calculated from the shift amount dk that minimizes the sum of. In this case, the phase difference detected from the A image and the B image is reduced, but the difference in the correlation amount can be increased, so that the detection accuracy of the shift amount is improved.

Further, when calculating the defocus amount from the shift amount, the sensitivity is multiplied to the sum of the phase differences dk1 and dk2. However, the sensitivity for each of the phase difference dk1 and the phase difference dk2 is stored in advance in the nonvolatile memory 56, and the defocus amount is calculated by summing the individual phase differences after multiplying the sensitivity. Good. Although the capacity required for storing the sensitivity increases, more accurate focus detection can be performed.
● (Second Embodiment)
Next, a second embodiment of the present invention will be described. The main difference from the first embodiment is the pixel arrangement of the image sensor. The imaging device of the first embodiment has a configuration in which shooting pixels and two types of focus detection pixels are arranged, and one pixel has one photoelectric conversion unit. As described in the first embodiment, when the present invention is applied to an image pickup apparatus using such an image pickup device, the focus detection accuracy can be improved. However, the present invention can also be applied to an imaging apparatus using an imaging device in which two photoelectric conversion units are provided in all pixels and output signals of A and B images can be obtained from all the pixels.

  The configuration of the imaging apparatus described in the first embodiment (FIG. 1), the focus detection region (FIG. 4A), the focus adjustment operation and the defocus amount calculation processing (FIGS. 5 and 6) are as follows. Since it is common also in embodiment, description is abbreviate | omitted.

  The configuration of the image sensor 14 in the present embodiment will be described with reference to FIGS. 7 to 8, the same components as those in FIGS. 2 to 3 are denoted by the same reference numerals, and redundant description is omitted.

  7A is similar to FIG. 2A, among the pixel groups arranged two-dimensionally on the image sensor 14 in the present embodiment, 6 rows in the vertical (Y-axis direction) and 8 columns in the horizontal (X-axis direction). This range is observed from the photographic lens 300 side. However, in the image sensor 14 of the present embodiment, the arrangement of the color filters is the same as the Bayer arrangement. That is, unlike the first embodiment, a blue color filter is provided for all the pixels in the blue position. In this embodiment, since an A (B) image and a GA (GB) image are obtained from the same pixel, there is no need to change the color of the filter.

  In the present embodiment, all the pixels 211 have photoelectric conversion units 211a and 211b that are divided into two in the X-axis direction. The output signal of one photoelectric conversion region and the sum of the output signals of both photoelectric conversion regions are Are separately readable. A signal corresponding to the output signal of the other photoelectric conversion region can be obtained as a difference between the sum of the output signals of both photoelectric conversion regions and the output signal of one photoelectric conversion region. The output signal of the divided photoelectric conversion region can be used for focus detection of a phase difference detection method by a method described later, and also used for generating a 3D (3-Dimensional) image composed of a pair of parallax images. You can also. On the other hand, the sum of the output signals of both photoelectric conversion regions can be used as an output signal of a normal photographing pixel.

  Here, generation of an image signal used for focus detection by the phase difference detection method will be described. In the present embodiment, the exit pupil of the photographing lens 300 is divided by the microlens 211i in FIG. 7A and the divided photoelectric conversion units 211a and 211b. Then, the A image obtained by connecting the outputs of the photoelectric conversion units 211a in the plurality of pixels 211 arranged in the same pixel row (X-axis direction) within the focus detection region, and the output of the photoelectric conversion unit 211b are connected. A B image is obtained by knitting. As described above, the image sensor of this embodiment cannot directly read the output of one of the two photoelectric conversion regions. Therefore, an image signal that requires an output signal of the photoelectric conversion region that cannot be directly read out can be obtained as a difference between the sum of the output signals of the two photoelectric conversion regions and the output signal of the photoelectric conversion region that can be directly read out.

  In the present embodiment, the GA image and the GB image are generated from the sum of the output signals of the two photoelectric conversion regions read from the pixels used for generating the A image and the B image.

  By detecting the relative image shift amount between the A image and the GA image generated in this way and the relative image shift amount between the B image and the GB image by correlation calculation, the defocus amount of the focus detection region, that is, defocusing. The amount can be detected. The basic method is as described in the first embodiment.

  Hereinafter, a plurality of pixels provided with the photoelectric conversion unit 211a used for generating an A image (first image signal) are referred to as a first pixel group, and are used for generating a B image (second image signal). A plurality of pixels provided with the photoelectric conversion unit 211b is referred to as a second pixel group. In the present embodiment, the first pixel group is also used for generating a GA image (third image signal), and the second pixel group is also used for generating a GB image (fourth image signal).

  FIG. 7B is a diagram illustrating a configuration example of a readout circuit in the image sensor 14 of the present embodiment. The imaging device 14 includes a horizontal scanning circuit 151 and a vertical scanning circuit 153, and horizontal scanning lines 152a and 152b and vertical scanning lines 154a and 154b are wired at the boundaries between the pixels. One output of the photoelectric conversion unit and both addition outputs are read out to the outside through these scanning lines.

  In the present embodiment, it is assumed that the outputs of the A image and the GA image are read from the pixels in the odd rows, and the outputs of the B image and the GB image are read from the pixels in the even rows.

  FIG. 8 is a diagram for explaining a conjugate relationship between the exit pupil plane of the photographing lens 300 and the photoelectric conversion units 211 a and 211 b of the pixel 211 disposed in the vicinity of the center of the image plane of the image sensor 14. The photoelectric conversion units 211a and 211b in the pixel 211 and the exit pupil plane of the photographing lens 300 are designed to have a conjugate relationship by the on-chip microlens 211i. The configuration of this embodiment is the same as that of the first embodiment except that each pixel has both of the configurations shown in FIGS. 3A and 3B, and thus a duplicate description is omitted. To do.

  Next, the focus detection method of the phase difference detection method using the output of the image sensor 14 in this embodiment will be described. Also in the present embodiment, as in the first embodiment, the phase difference is detected for each combination of the A image and the GA image, and the B image and the GB image. In the present embodiment, a GA image (shooting signal) that is the sum of the outputs of the photoelectric conversion units 211a and 211b and an A image that is the output of the photoelectric conversion unit 211a are read from the odd-numbered pixel rows. Further, a GB image (shooting signal) that is the sum of the outputs of the photoelectric conversion units 211a and 211b and a B image that is the output of the photoelectric conversion unit 211b are read from the even pixel rows.

  The B image in the odd-numbered pixel row and the A image in the even-numbered pixel row can be calculated as the difference between the GA image and the A image and the difference between the GB image and the B image, respectively. / N is lower than when reading directly. Therefore, it is better not to use the image signal obtained as the difference in order to detect the phase difference with high accuracy. Therefore, in this embodiment, the phase difference is detected using the output of one photoelectric conversion unit that can be read and the sum of the outputs of both photoelectric conversion units.

  Unlike the case of the first embodiment, the imaging signal and one image signal for AF are obtained from the same pixel. Therefore, as shown in FIG. 4C, from the 2 rows and N columns of pixels arranged in the focus detection area 401, A image (A (i, j)), B image (B (i, j)) GA image (GA (i, j)) and GB image (GB (i, j)) (1 ≦ i ≦ 2, 1 ≦ j ≦ N) can be obtained. Similarly to the first embodiment, the shift amounts dk1 and dk2 are obtained for each combination of the A image and the GA image, and the B image and the GB image, and the defocus amount is obtained based on the sum dk_sum.

According to the present embodiment, when an image sensor having a configuration in which the photoelectric conversion area of each pixel is divided is used, the phase difference can be detected with high accuracy while reducing the processing load compared to the individual reading of each photoelectric conversion area. Thus, focus detection accuracy can be improved.
● (Third embodiment)
Next, a third embodiment of the present invention will be described. The main difference from the first embodiment is the difference in the defocus amount calculation method. In the first embodiment, the defocus amount is calculated using the phase difference detection result between the A image and the GA image and the phase difference detection result between the B image and the GB image. Even if the pixels used for generating the B image are far apart, accurate defocus amount detection was realized. However, when the defocus amount is large, the blurring differs between the A image and the GA image (imaging signal), and the degree of coincidence between the two image signals decreases. This is because the F values of the light beams included in the two image signals differ depending on the size and arrangement of the photoelectric conversion units used for generating the respective image signals. Therefore, the focus detection accuracy deteriorates as the defocus amount increases.

  In the present embodiment, when the defocus amount is estimated to be large, the phase difference detection result between the A image and the B image is used. This is because when the defocus amount is large, even if the pixels used to generate the A image and the pixels used to generate the B image are far apart, the subject optical image is largely blurred, so the A image and the B image match. This is because the degree increases. In other words, when the defocus amount is large, the fact that the difference in the sampling positions of the A image and the B image has a small influence on the focus detection result is used.

  The configuration of the imaging apparatus described in the first embodiment (FIG. 1), the configuration of the imaging device 14 (FIGS. 2 and 3), the focus detection region (FIG. 4A), and the focus adjustment operation (FIG. 5). Since the same applies to this embodiment, the description thereof is omitted.

  Hereinafter, the defocus amount calculation method according to the present embodiment will be described with reference to the flowchart shown in FIG. This defocus amount calculation processing can be performed in S504 of FIG. In FIG. 9, the same reference numerals as those in FIG. 6 are assigned to the same operations as those in the first embodiment.

  First, in S6041, the AF unit 42 determines whether or not the current defocus amount is larger than a predetermined threshold value. The current defocus amount is calculated from the defocus amount obtained before the start of the focus adjustment operation, or when the focus detection is repeatedly performed, the previously detected defocus amount and the subsequent lens drive amount. Is an estimate that can be. When the defocus amount is obtained before the start of the focus adjustment operation, focus detection may be performed at regular intervals before the focus detection start condition is satisfied in S501 of FIG. The threshold value used here can be determined in advance through experiments or the like.

  If it is determined that the current defocus amount is greater than the predetermined threshold, the AF unit 42 advances the process to S6042. If it is determined that the current defocus amount is equal to or smaller than the predetermined threshold value, the AF unit 42 executes S5041 and S5042 to obtain the shift amounts dk1 and dk2 in the same manner as in the first embodiment, and the process proceeds to S6044. Proceed.

  In S6042, the AF unit 42 performs a correlation calculation using the A image and the B image obtained from the focus detection pixels (first pixel group and second pixel group) present in different pixel rows. In the correlation calculation, the GA image in the correlation calculation between the A image and the GA image described in the first embodiment can be performed as the B image. Then, the shift amount dk3 that minimizes the correlation amount (COR3) is calculated in the same manner as in the first embodiment, based on the correlation amount difference DCOR3.

  Next, in S6043, the AF unit 42 performs a correlation calculation using a GA image obtained from the same pixel row as the A image and a GB image obtained from the same pixel row as the B image. As in S6042, the AF unit 42 calculates the shift amount dk4 that minimizes the correlation amount (COR4) based on the correlation amount difference DCOR4, and the process advances to S6044.

  In S6044, the AF unit 42 calculates the defocus amount. When the shift amounts dk1 and dk2 are calculated, the AF unit 42 calculates the sum dk_sum of the shift amounts dk1 and dk2 and multiplies the sensitivity stored in advance in the nonvolatile memory 56, as in the first embodiment. Thus, the shift amount is converted into the defocus amount DEF.

  On the other hand, when the shift amounts dk3 and dk4 are calculated, the AF unit 42 calculates the difference dk_dif = dk3−dk4 between the shift amounts dk3 and dk4 and multiplies the sensitivity stored in the nonvolatile memory 56 in advance. Thus, the shift amount is converted into the defocus amount DEF.

  When the calculation of the defocus amount DEF is finished, the AF unit 42 ends the defocus amount calculation subroutine.

  Here, the reason why the shift amount dk4 is subtracted from the shift amount dk3 will be described. The shift amount dk3 is a shift amount between the A image and the B image. The first pixel group that generates the A image and the second pixel group that generates the B image are separated in a direction (Y-axis direction) orthogonal to the direction in which the phase difference is detected (here, the X-axis direction). Therefore, the position where the subject optical image is sampled is different. Therefore, when focus detection is performed on an object having an edge with contrast (brightness difference) in an oblique direction, a phase difference unrelated to the defocus amount occurs between the A image and the B image as a focus detection error. To do. This focus detection error also occurs between the GA image and the GB image for the same reason, but the phase difference (shift amount dk4) between the GA image and the GB image does not increase or decrease depending on the defocus amount. By utilizing this fact and subtracting dk4 from dk3, it is possible to reduce the focus detection error caused by the subject.

As described above, according to the present embodiment, the focus detection result suitable for the magnitude of the defocus amount can be obtained by changing the pair of image signals for performing the correlation calculation according to the magnitude of the defocus amount. it can.
● (Fourth embodiment)
The fourth embodiment of the present invention will be described below. The main difference from the first embodiment is the difference in the defocus amount calculation method. In the embodiment described above, the defocus amount is always calculated by detecting the phase difference between the AF A image signal and the shooting signal (GA image signal), and detecting the phase difference between the AF B image signal and the shooting signal (GB image signal). The results were used. Thereby, even when the pixel for obtaining the AF A image signal and the pixel for obtaining the AF B image signal are distant from each other, focus detection can be performed with high accuracy. However, when the image height for focus detection is high and the aperture value is large (diaphragm diameter is small), the AF A image signal and GA image signal (shooting signal), or the AF B image signal and GB image signal. The baseline length of either (shooting signal) is shortened. This is because when the image height is high, the vignetting of the pupil is greatly different between the AF A image signal and the AF B image signal.

  FIG. 10 is a diagram showing the relationship between the exit pupil distance of the photographing lens and the pupil position viewed by the focus detection pixels.

  1601a represents the pupil of the AF A image signal, and 1601b represents the pupil of the AF B image signal. FIG. 10 illustrates that focus detection is performed by looking at X2 on the entrance pupil plane of the sensor when the image height of focus detection is −x1. As can be seen from this figure, the closer the exit pupil distance ZL of the taking lens is to the entrance pupil distance Zs of the sensor, the closer X2 is to the optical axis. Then, even if x1 changes, since X2 is always near the optical axis, the base lengths of the AF A image signal and the GA image signal (shooting signal), the AF B image signal, and the GB image signal (shooting signal) One of the base line lengths will not become extremely short.

  Conversely, as the exit pupil distance ZL of the photographic lens becomes farther from the entrance pupil distance Zs of the sensor, as | −x1 | increases, X2 also takes a larger value. For this reason, either the baseline length of the AF A image signal and the GA image signal (shooting signal) or the baseline length of the AF B image signal and the GB image signal (shooting signal) tends to become extremely short. In the case of FIG. 10, since ZL is smaller than Zs and x1 is a negative image height, the baseline lengths of the AF B image signal and the GB image signal (shooting signal) are short. On the other hand, when ZL is longer than Zs and x1 is at a negative image height, X2 is inverted to the negative side, so that the baseline lengths of the AF A image signal and the GA image signal (shooting signal) are shortened.

  FIG. 11 shows that when the image height is high (when X2 shown in FIG. 10 is in the position of a circle indicated by a broken line), the vignetting of the pupil is greatly different between the AF A image signal and the AF B image signal. FIG. FIG. 11A shows a state in which the pupil of the AF A image signal (the entrance pupil of the sensor; hereinafter abbreviated as “pupil”) is vignetted by a frame (a general term for a lens frame and a diaphragm frame), and FIG. Shows a state in which the pupil of the AF B image signal is vignetted by the frame. FIG. 11C shows a state in which the pupil of the shooting GA image signal (or GB image signal) is vignetted by the frame. Further, FIG. 11D shows the state of each line image of the AF A image signal, the AF B image signal, and the shooting GA image signal (or GB image signal) in consideration of vignetting due to the frame.

  1401a is the pupil of the AF A image signal, 1401b is the pupil of the AF B image signal, 1401c is the pupil of the imaging GA image signal (or GB image signal), 1402a is the line image of the AF A image signal, and 1402b is AF A line image 1402c of the B image signal for image is a line image of the GA image signal for shooting (or GB image signal). 1403a is the optical axis of the line image of the AF A image signal (the center of gravity of the line image), 1403b is the optical axis of the line image of the AF B image signal, and 1403c is the GA image signal for shooting (or the GB image signal). The optical axis of the line image is shown.

  FIG. 11 shows a case where the image height at which focus detection is performed has a pupil peak of the AF A image signal. In this case, focus detection is performed using the AF A image signal and the AF GA image signal. In this case, since the baseline length is short, the sensitivity is high, and the focus detection accuracy is lowered. On the other hand, when focus detection is performed using the AF B image signal and the GB image signal, the base line length is long and the focus detection accuracy is good. Therefore, rather than adding the focus detection results of the AF A image signal and the GA image signal and the focus detection results of the AF B image signal and the GB image signal to obtain the focus detection result, the AF B image signal and the GB image signal are output. The focus detection accuracy is higher when only the focus detection result is used. When the image height is the reverse of that in FIG. 11 and the image height for focus detection is the peak of the pupil of the AF B image signal, the focus detection result of the AF A signal and the GA image signal is opposite to the above. The focus detection accuracy is higher when using only.

  FIG. 12 is a flowchart illustrating a defocus amount calculation subroutine. In this subroutine, there is a step for performing the same operation as the operation shown in FIG. 6 of the first embodiment, and therefore the same step number as that in FIG.

  When the process proceeds from step S504 of the main routine shown in FIG. 5 of the first embodiment to this subroutine, the process proceeds to step S7041 to determine whether or not the current defocus amount is smaller than a predetermined threshold value. As for the current defocus amount, information on the defocus amount obtained before the start of focus detection, or if the focus detection is repeated, use the defocus amount detected last time, and consider the subsequent lens drive amount, etc. It is a value calculated as follows. When the defocus amount is obtained before the start of focus detection, it is only necessary to always perform focus detection at regular intervals before the focus detection is started in step S501 in FIG.

  In step S7041, it is determined whether to use both the correlation calculation between the A image and the GA image and the correlation calculation between the B image and the GB image. If it is determined that both the correlation calculation between the A image and the GA image and the correlation calculation between the B image and the GB image are used, the process proceeds to the process of performing the same operation as that in steps S5041 and S5042 described in the first embodiment. The shift amounts dk1 and dk2 are calculated.

  If it is determined in step S7041 that one of the correlation calculation between the A image and the GA image and the correlation calculation between the B image and the GB image are used, the process proceeds to step S7042. In step S7042, it is determined which one of the correlation calculation between the A image and the GA image and the correlation calculation between the B image and the GB image is used. If it is determined that the correlation calculation between the A image and the GA image is to be used, the process proceeds to the process of performing the same operation as step S5041 described in the first embodiment, and the shift amount dk1 is calculated. If it is determined that the correlation calculation between the B image and the GB image is to be used, the process proceeds to the process of performing the same operation as step S5042 described in the first embodiment, and the shift amount dk2 is calculated. 0 is substituted for the shift amount for which the correlation calculation has not been performed.

  Here, there are several possible methods for determining whether to use both the correlation calculation of the A image and the GA image and the correlation calculation of the B image and the GB image or only one of them in step S7041. This determination is made in relation to the optical conditions of the taking lens.

  As a first conceivable method, there is a method using the sensitivity of focus detection. As described with reference to FIG. 11, the sensitivity of focus detection increases as the baseline length for focus detection decreases. If the sensitivity is too high, the reliability of the focus detection accuracy decreases, so if the sensitivity exceeds a predetermined threshold, a method that does not perform correlation calculation that calculates the defocus amount using that sensitivity is effective. . Since this method does not require conditional branching other than sensitivity, the system scale can be reduced.

  The next method is the image height for focus detection. As described with reference to FIG. 11, when the image height for focus detection is high, the difference in calculation accuracy between dk1 and dk2 increases. For this reason, a threshold is set for the image height for focus detection. If the image height for focus detection exceeds a predetermined threshold, only one of dk1 and dk2 is used, and if it is less than the predetermined threshold, both dk1 and dk2 are used. There is a method to use. However, if only the image height for focus detection is used, there is a possibility that switching between using both dk1 and dk2 or using one is frequent. Therefore, when it is desired to avoid this (for example, when the subject is followed by servo AF and the AF method is fixed), a method of adding the aperture value and the exit pupil distance of the photographing lens to the condition can be considered. When the aperture value is large, the difference between dk1 and dk2 tends to be large. Therefore, when the aperture value is larger than the predetermined threshold value, the switching based on the image height described above is performed. Also, when the exit pupil distance of the photographing lens is short, the sensitivity for dk1 is compared with the sensitivity for dk2, and when there is a difference (ratio, difference value, etc.) that exceeds a predetermined value, focus detection is performed using one of them. Suppose you do. As described above, there can be a plurality of methods for determining whether to use both the correlation calculation between the A image and the GA image and the correlation calculation between the B image and the GB image or only one of them. This depends on the required specifications. Use it properly.

  FIG. 13 is a flowchart showing the detailed processing of step S7041 in FIG. 12, and uses the exit pupil distance, aperture value, and image height of the taking lens to calculate the correlation between the A image and the GA, and the B image and the GB image. A case is shown in which it is determined whether to use both of the correlation operations or only one of them.

  First, in step S70411, the exit pupil distance ZL of the photographing lens is compared with the entrance pupil distance Zs of the sensor. If the difference is smaller than the predetermined value Zlimit, the defocus amount is added, that is, it is determined that both the correlation calculation between the A image and the GA image and the correlation calculation between the B image and the GB image are used, and the process proceeds to return. On the other hand, if the difference is greater than or equal to Zlimit, the process proceeds to step S70412.

  In step S70412, if the aperture value F is smaller than the predetermined threshold value Flimit, it is determined that the defocus amount is added, and the process proceeds to return, and if it is greater than Flimit, the process proceeds to step S70413. In step S70413, if the absolute value of x1 is smaller than the predetermined threshold value Xlimit, it is determined that the defocus amount is added, and the process proceeds to return, and if it is equal to or greater than Xlimit, the process proceeds to step S70414.

  In step S70414, the exit pupil distance ZL of the photographic lens is compared with the entrance pupil distance Zs of the sensor. If ZL is equal to or greater than Zs, the process proceeds to step S70415, and if smaller than Zs, the process proceeds to step S70416. In step S70415, if x1 is zero or positive, the process proceeds to step S70418, and if negative, the process proceeds to step S70418. In step S70416, if x1 is zero or positive, the process proceeds to step S70417. If x1 is negative, the process proceeds to step S70418.

  In step S70417, since only the shift amount of the AF B image signal and the GB image signal (imaging signal) is used, the sensitivity is set to K1 = 0. In step S70418, the sensitivity K2 = 0 is set in order to use only the shift amounts of the AF A image signal and the GA image signal (imaging signal).

  FIG. 14 is a flowchart showing the detailed processing of step S7041 in FIG. 12, and uses both the correlation calculation between the A image and the GA image and the correlation calculation between the B image and the GB image using the sensitivity K. The case where it is determined whether only one of them is used is shown.

  First, in step S70419a, it is determined whether or not the sensitivity K1 is smaller than a predetermined threshold K_limit. If it is smaller than K_limit, the defocus amount is added, so the process proceeds to RETURN, and if it is greater than K_limit, the process proceeds to step S70417. In step S70417, since only the shift amount of the AF B image signal and the GB image signal (imaging signal) is used, the sensitivity is set to K1 = 0.

  Next, in step S70419b, it is determined whether or not the sensitivity K2 is smaller than a predetermined threshold value K_limit. If it is smaller than K_limit, the defocus amount is added, so the process proceeds to return, and if it is greater than K_limit, the process proceeds to step S70418. In step S70418, the sensitivity K2 = 0 is set in order to use only the shift amounts of the AF A image signal and the GA image signal (imaging signal). The above is the detailed description of the processing in S7041.

  Next, step S7042 in FIG. 12 will be described. In step S7042, it is determined which correlation calculation is performed. In step S7042, if the sensitivity K1 set in step S7041 is not 0, the process proceeds to step S5041, and the shift amount dk1 is calculated. If the sensitivity K1 = 0, the process proceeds to step S5042, and dk2 is calculated.

  In step S7043 in FIG. 12, the defocus amount is calculated. Multiply the calculated shift amounts dk1 and dk2 by the values stored in the nonvolatile memory 56 in advance (or 0 set in the defocus amount addition determination) as the sensitivity K1 and K2 (defocus amount conversion coefficient). The shift amount is converted into the defocus amount Def by calculating the following equation.

Def = K1 × dk1 + K2 × dk2 (4)
In this embodiment, the defocus amount is calculated using Expression (4), but may be calculated using weighting coefficients G1 and G2 as shown in Expression (5).

Def = G1 * K1 * dk1 + G2 * K2 * dk2 (5)
When the calculation of the defocus amount Def ends, the defocus amount calculation subroutine ends.

As described above, by selecting a pair of image signals for which correlation calculation is performed, it is possible to always obtain a focus detection result with high accuracy.
(Other embodiments)
The above-described embodiments are examples for facilitating understanding of the present invention, and the present invention is not limited to the specific configurations described in these embodiments. The scope of the present invention is defined by the description of the claims, and various modifications and changes can be made within the scope.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (13)

Detecting a first phase difference between a first image signal based on a light beam that has passed through a partial area of the exit pupil of the imaging optical system and a third image signal based on a light beam that has passed through the entire area of the exit pupil. First calculating means for
A second image signal based on a light beam that has passed through a partial area different from the partial area of the exit pupil and a second image signal based on a light beam that has passed through the entire area of the exit pupil. Second calculating means for detecting a phase difference between
Defocus amount calculation means for calculating a defocus amount of the photographing optical system, using a sum of the first phase difference and the second phase difference multiplied by a weighting coefficient;
A focus detection apparatus comprising:   The focus detection apparatus according to claim 1, wherein the defocus amount calculation unit determines the weighting coefficient based on an optical condition related to a baseline length for focus detection.   The focus detection apparatus according to claim 2, wherein the optical condition related to the baseline length includes at least one of an image height for focus detection, a pupil distance, an aperture value, and a defocus amount conversion coefficient.   The focus detection apparatus according to claim 3, wherein a defocus amount conversion coefficient is used as the optical condition related to the baseline length.   The first pixel group used for generating the first image signal and the second pixel group used for generating the second image signal have the first and second phase differences in the image sensor. The focus detection apparatus according to claim 1, wherein the focus detection apparatus is separated in a direction orthogonal to a direction to be detected. The distance in the orthogonal direction between the first pixel group and the third pixel group used for generating the third image signal is the distance between the first pixel group and the second pixel group. Smaller than the distance in the orthogonal direction,
The distance in the orthogonal direction between the second pixel group and the fourth pixel group used for generating the four image signals is the distance between the first pixel group and the second pixel group. Smaller than the distance in the orthogonal direction,
The focus detection apparatus according to claim 5.   Each pixel of the third pixel group is adjacent to each pixel of the first pixel group, and each pixel of the fourth pixel group and each pixel of the second pixel group are The focus detection apparatus according to claim 6, wherein the focus detection apparatus is an adjacent pixel.   The focus detection apparatus according to claim 6, wherein the third image signal and the fourth image signal are generated by pixel values generated based on a plurality of pixel values. The first image signal and the third image signal are generated based on the output of the same pixel group;
The second image signal and the fourth image signal are generated based on the output of the same pixel group.
The focus detection apparatus according to claim 6. An image sensor that can read out an image signal based on a light beam that has passed through a partial area of the exit pupil of the imaging optical system, and an image signal based on a light beam that has passed through the entire area of the exit pupil;
The focus detection apparatus according to any one of claims 1 to 9,
An imaging device comprising: Detecting a first phase difference between a first image signal based on a light beam that has passed through a partial area of the exit pupil of the imaging optical system and a third image signal based on a light beam that has passed through the entire area of the exit pupil. A first calculating step,
A second image signal based on a light beam that has passed through a partial area different from the partial area of the exit pupil and a second image signal based on a light beam that has passed through the entire area of the exit pupil. A second calculation step of detecting the phase difference of
A defocus amount calculation step of calculating a defocus amount of the photographing optical system using a sum obtained by multiplying the first phase difference and the second phase difference by a weighting coefficient;
A method for controlling a focus detection apparatus, comprising:   The program for making a computer perform each process of the control method of Claim 11.   A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 11.