JP2018005161A - Control device, image capturing device, control method, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a control device that enables highly accurate focus detection.SOLUTION: A control device (50) includes: acquisition means (50a) for acquiring first and second signals corresponding to light rays passing through different pupil regions of an image capturing optical system; correlation operation means (50b) for computing an amount of correlation between the first and second signals; noise computation means (50c) for computing the correlation noise between the first and second signals; and determination means (50d) configured to determine reliability of the amount of correlation using criteria corresponding to the correlation noise.SELECTED DRAWING: Figure 8

Description

  The present invention relates to an imaging apparatus such as a digital camera or a video camera.

  Patent Document 1 discloses a focus detection apparatus that performs pupil division focus detection using a two-dimensional image sensor in which a microlens is formed in each pixel as one of methods for detecting the focus state of a photographing lens. Has been. In this focus detection apparatus, the photoelectric conversion unit (PD) of each pixel constituting the image sensor is divided into a plurality of light beams, and the divided PD passes through different regions of the pupil of the photographing lens via the microlens. Is configured to receive light. A correlation calculation is performed on a pair of output signals of PDs that receive light beams that have passed through different areas of the pupil of the photographic lens to calculate a phase difference that is a deviation amount. The amount of focus can be calculated.

  Patent Document 2 discloses an imaging apparatus that reads the charge amount of one photoelectric conversion unit (PD) in the same microlens, and then reads the addition value (addition charge amount) of all the PDs in the same microlens. It is disclosed. This imaging device performs focus detection while maintaining the characteristics of the image signal for imaging by generating the charge amount of the other PD from the difference between the added charge amount of all the PDs and the charge amount of one PD.

JP 2008-52009 A JP 2014-182360 A

  However, in the imaging device disclosed in Patent Document 2, noise (correlation noise) having a correlation with each other is included between pixel signals corresponding to respective charge amounts of one PD and the other PD. If focus detection is performed based on a pixel signal including such correlation noise, high-precision focus detection cannot be performed.

  Therefore, the present invention provides a control device, an imaging device, a control method, a program, and a storage medium that can perform focus detection with high accuracy.

  The control device according to one aspect of the present invention includes an acquisition unit that acquires a first signal and a second signal corresponding to light beams that pass through different pupil regions of the imaging optical system, and the first signal and the two signals. Correlation calculation means for calculating a correlation amount; noise calculation means for calculating correlation noise between the first signal and the second signal; and determining reliability related to the correlation amount using a criterion corresponding to the correlation noise. Determination means.

  An imaging apparatus as another aspect of the present invention includes an imaging device having a first photoelectric conversion unit and a second photoelectric conversion unit that receive light beams passing through different pupil regions of an imaging optical system, and the control device. .

  According to another aspect of the present invention, there is provided a control method comprising: obtaining a first signal and a second signal corresponding to light beams passing through different pupil regions of an imaging optical system; and the first signal and the second signal. Calculating a correlation amount; calculating correlation noise between the first signal and the second signal; and determining reliability of the correlation amount according to the correlation noise.

  A program according to another aspect of the present invention causes a computer to execute the control method.

  A storage medium according to another aspect of the present invention stores the program.

  Other objects and features of the present invention are illustrated in the following examples.

  According to the present invention, it is possible to provide a control device, an imaging device, a control method, a program, and a storage medium capable of performing focus detection with high accuracy.

It is a block diagram of the imaging device in each embodiment. It is a circuit diagram of a pixel of an image sensor in each embodiment. It is a pixel array diagram of an image sensor in each embodiment. It is a block diagram of the image pick-up element in each embodiment. It is a timing chart which shows the read-out operation | movement of the imaging device in each embodiment. It is explanatory drawing of the conjugate relationship of the exit pupil plane of the imaging optical system in each embodiment, and the photoelectric conversion part of an image pick-up element. It is explanatory drawing of the focus detection area | region in each embodiment. It is explanatory drawing of the AF signal pair arrange | positioned in the focus detection area | region in each embodiment. It is a flowchart which shows the focus adjustment operation | movement in each embodiment. 6 is a flowchart of a defocus amount calculation subroutine according to the first embodiment. It is a flowchart of the noise amount calculation subroutine in each embodiment. 10 is a flowchart of a defocus amount calculation subroutine according to the second embodiment. It is explanatory drawing of a structure and operation | movement of a comparative example.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  First, with reference to FIG. 11, the configuration and operation of the comparative example and the problems associated therewith will be described. FIG. 11 is an explanatory diagram of the configuration and operation of the comparative example. Specifically, as a comparative example, phase difference detection is performed using an imaging device that has a plurality of photoelectric conversion units (PDs) for one microlens and in which such microlenses are arranged two-dimensionally. A problem in the case of performing focus detection by the method will be described. For example, when the PD is divided into two for one microlens, the pixel signal corresponding to the charge amount of one PD is an A image, the pixel signal corresponding to the charge amount of the other PD is a B image, and A pixel signal corresponding to an added value (added charge amount) of all PDs is defined as an A + B image.

  The imaging device (for example, the imaging device disclosed in Patent Document 2) reads the A image, reads the A + B image, generates the B image by subtracting the A image from the A + B image, and performs the focus detection calculation. Do. Although it is possible to read out the A image and the B image independently of each other and perform a focus detection calculation, an A + B image is also required to generate an image (photographed image). At this time, if an A + B image is generated by adding the A image and the B image, the A image and the B image each include random noise. Therefore, the random noise amount of the A + B image is obtained by adding the A image and the B image. Will become bigger. Since the image quality deteriorates when the amount of random noise increases, it is preferable that the A image and the A + B image are read out independently.

  The detection of the focus state is performed by calculating the shift amount between the A image and the B image from the correlation calculation and associating the image shift amount with the defocus amount of the focus lens. FIG. 11A is a pixel array diagram of the image sensor in the comparative example. In FIG. 11A, 1000 indicates one microlens (same microlens), 1001 indicates an A image PD (first photoelectric conversion unit), and 1002 indicates a B image PD (second photoelectric conversion unit). Yes. As shown in FIG. 11A, the imaging device of the comparative example has a plurality of photoelectric conversion units (A image PD 1001 and B image PD 1002) for one micro lens 1000, and the micro lens 1000 They are arranged in two dimensions.

  Let A (n) and B (n) be the A and B image signals of the pixels (pixel columns) arranged in the horizontal direction. Here, as described above, the B image signal B (n) is generated by subtracting the A image signal A (n) from the A + B image signal. The correlation amount COR (k) used for the correlation calculation is calculated as in the following equation (1).

  In Expression (1), k is a shift amount in the correlation calculation, and is an integer greater than or equal to −kmax and less than or equal to kmax. Thereafter, the value of the shift amount k when the correlation between the A image signal and the B image signal is maximized (that is, when the correlation amount COR is minimized) is obtained. Here, since the resolution becomes coarse when the shift amount k is calculated as an integer, interpolation processing, so-called sub-pixel calculation is appropriately performed. This difference, the A image signal corresponding to the charge of the PD in the first column is S [A (l)], and the A + B image signal is S [A + B (l)]. Further, N [A (l)] is a random noise caused by the readout circuit superimposed on the pixel data when the A image is read out, and a random noise caused by the readout circuit superimposed on the pixel data when the A + B image is read out. Let the noise be N [A + B (l)]. When a B image is generated according to the above-described method, the B image is expressed as the following Expression (2).

B image = (A + B) image−A image = (S [A + B (l)] + N [A + B (l)]) − (S [A (l)] + N [A (l)])
= S [A + B (l)]-S [A (l)] + N [A + B (l)]-N [A (l)] (2)
At this time, the correlation amount COR (s) when the shift amount other than 0 = s is expressed as the following equation (3).

COR (s) = Σ | A (is) −B (i + s) |
= Σ | {S [A (is)] + N [A (is)]}-{S [A + B (i + s)] − S [A (i + s)] + N [A + B (i + s)] − N [ A (i + s)]} |
= Σ | S [A (is)] + S [A (i + s)]-S [A + B (i + s)] + N [A (is)] + N [A (i + s)]-N [A + B (i + s) ] (3)
Further, the correlation amount COR (0) when the shift amount = 0 is expressed as the following equation (4).

COR (0) = Σ | A (i) −B (i) |
= Σ | S [A (i)] + S [A (i)] − S [A + B (i)] + N [A (i)] + N [A (i)] − N [A + B (i)] |
= Σ | 2 × S [A (i)] − S [A + B (i)] + 2 × N [A (i)] − N [A + B (i)] | (4)
Here, the random noise amounts Noise (S) and Noise (0) superimposed on the correlation amount COR when the shift amount other than 0 = s and the shift amount = 0 are respectively expressed by the following equations (5), (6 ).

Noise (S) = Σ | N [A (is)] + N [A (i + s)] − N [A + B (i + s)] | (5)
Noise (0) = Σ | 2 × N [A (i)] − N [A + B (i)] | (6)
N [A (is)], N [A (i + s)], and N [A + B (i + s)] are random noises that are not correlated with each other. For this reason, the random noise amount Noise (s) becomes a substantially constant value except for the shift amount = 0 as shown in FIG. 11B. On the other hand, regarding the random noise amount Noise (0), N [A (i)] and N [A + B (i)] are random noises that are not correlated with each other. However, since N [A (i)] is doubled and integrated, the random noise amount Noise (0) is larger than the random noise amount Noise (s). For this reason, as shown in FIG. 11B, the correlation amount due to random noise increases only when the shift amount = 0. As described above, when a B image is generated from the A + B image and the A image and the correlation calculation is performed, the random noise amount obtained by inverting the sign of the random noise amount superimposed on the A image is superimposed on the B image. For this reason, when the shift amount = 0, the noise amount of Σ | 2 × N [A (i)] | is superimposed on the correlation value, and a peak occurs locally.

  When shooting a subject with low contrast between the A image signal and the B image signal or shooting in a low-luminance environment, the noise N with respect to the signal S becomes relatively large, and the noise becomes dominant in the correlation amount COR. If there is no influence of noise, the amount of correlation COR is minimized when the amount of shift is 0 at the time of focusing (point H in FIG. 11 (c)), and focus detection is performed by detecting this point (point H). Is done appropriately. However, when noise becomes dominant as a component of the correlation amount COR, as shown in FIG. 11D, a result is obtained in which the correlation amount COR increases (correlation decreases) when the shift amount = 0. Point I). As a result, a plurality of points (point J, point K) having a minimum correlation value P (h) are generated, and these points (point J, point K) are recognized as the shift amount of the in-focus position, As a result, erroneous focus detection and hunting are caused.

  Similar to the case where noise having a negative correlation is generated between the A image and the B image, a problem arises when noise having a positive correlation is generated between the A image and the B image. For example, the non-uniformity of sensitivity for each pixel occurs due to non-uniform transmittance for each pixel or a readout circuit, and shares a noise generation source between the A image and the A + B image. At this time, there is a noise component proportional to the output signal amount between the A image and the A + B image.

  Similarly to the above, the A image signal corresponding to the charge of the PD in the l-th column is represented as S [A (l)], and the noise component causing the output difference between the pixels is represented as N [A (l)]. At this time, in the case where no noise other than the positively correlated noise is generated between the A image and the A + B image, the A + B image signal S [A + B (l)] is expressed as g × S [A (l)] (g ≧ 1). N [A + B (l)] can be expressed as g × N [A (l)].

  The B image obtained at this time can be expressed as the following formula (7).

B image = (S [A + B (l)] + N [A + B (l)]) − (S [A (l)] + N [A (l)])
= (G-1) (S [A (l)] + N [A (l)]) (7)
Similarly, when the correlation amount is calculated, the noise amounts when the shift amount = S and the shift amount = 0 are expressed as the following equations (8) and (9), respectively.

Noise (S) = Σ | N [A (is)] − (g−1) × N [A (i + s)] | (8)
Noise (0) = Σ | (2-g) × N [A (i)] | (9)
N [A (is)] and N [A (i + s)] are noises having no correlation with each other and having the same variation as N [A (l)]. For this reason, N [A (is)] − (g−1) × N [A (i + s)] becomes noise having a larger variation than N [A (l)]. In contrast to Noise (S) in which this variation is integrated by Expression (8), Noise (0) is an integrated value of the variation of N [A (l)], and therefore, the frequency of reduction becomes high.

  As described with reference to FIG. 11C, the defocus state is detected by calculating the minimum value of the correlation amount COR. For this reason, a decrease in the correlation amount of the shift amount = 0 due to noise leads to a decrease in detection accuracy. Since positively correlated noise is generated in proportion to the amount of signal, the minimum value due to noise affects the minimum value due to the original subject signal in low contrast, uniformly bright subjects, etc., and the detection accuracy is impaired. There is a case. In addition, when the pattern of the subject is uniform, focus detection is essentially impossible. However, since a local minimum value is generated only by noise components, there is a possibility that it is erroneously determined that the subject is in focus.

  Therefore, each embodiment of the present invention can appropriately grasp the influence of noise and perform focus adjustment using a highly accurate focus detection result even when shooting a low-contrast subject or in a low-brightness environment. A possible imaging device is provided. Each embodiment will be described below.

(First embodiment)
With reference to FIG. 1, an imaging apparatus according to Embodiment 1 of the present invention will be described. FIG. 1 is a block diagram of an imaging apparatus 10 in the present embodiment. The imaging device 10 is a camera system that includes a camera 100 (imaging device main body) having an imaging element and a photographing lens 300 (an interchangeable lens or a lens device) that can be attached to and detached from the camera 100. However, the present embodiment is not limited to this, and can also be applied to an imaging apparatus in which the camera 100 and the photographing lens 300 are integrally configured.

  First, the configuration of the camera 100 will be described. The camera 100 can be mounted with a plurality of types of photographing lenses 300 (and the same type of photographing lenses having different serial numbers). The camera 100 can also be equipped with a photographic lens having a different focal length or open F number or a photographic lens having a zoom function, and is configured to be interchangeable regardless of the same or different photographic lens.

  In the camera 100, the light beam that has passed through the photographing lens 300 passes through the camera mount 106, is reflected upward by the main mirror 130, and enters the optical viewfinder 104. The optical viewfinder 104 allows the photographer to take a picture while observing the subject as an optical image. The optical viewfinder 104 has some functions of the display unit 54, such as a focus display, a camera shake warning display, an aperture value display, and an exposure correction display.

  The main mirror 130 is composed of a semi-transmissive half mirror, and part of the light beam incident on the main mirror 130 passes through the half mirror, is reflected downward by the sub mirror 131, and enters the focus detection device 105. The focus detection apparatus 105 employs a phase difference detection AF mechanism composed of a secondary imaging optical system, converts the obtained optical image into an electrical signal, and sends it to the AF section 42 (autofocus section). The AF unit 42 performs a phase difference detection calculation from this electric signal. Based on the calculation result of the AF unit 42, the system control circuit 50 controls a focus control unit 342, which will be described later, provided in the taking lens 300, such as a focus adjustment process. In the present embodiment, the AF unit 42 also corrects the focus detection result. The AF unit 42 functions as a phase difference calculation unit.

  On the other hand, when the focus adjustment processing of the taking lens 300 is completed and still image shooting, electronic viewfinder display, and moving image shooting are performed, the main mirror 130 and the sub mirror 131 are retracted out of the shooting light beam by a quick return mechanism (not shown). With such a configuration, the light beam that has passed through the photographic lens 300 enters the image sensor 14 that converts an optical image into an electrical signal via the shutter 12 for controlling the exposure amount. After completion of these photographing operations, the main mirror 130 and the sub mirror 131 return to positions (in the photographing light beam) as shown in FIG.

  The electrical signal photoelectrically converted by the image sensor 14 is sent to the A / D converter 16, and the analog signal output is converted into a digital signal (image data). The timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26, and is controlled by the memory control circuit 22 and the system control circuit 50. The image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the image data from the A / D converter 16 or the image data from the memory control circuit 22. The image processing circuit 20 performs predetermined arithmetic processing using the image data.

  The image sensor 14 constitutes a part of the focus detection device, and can perform the phase difference detection AF even in a state where the main mirror 130 and the sub mirror 131 are retracted out of the imaging light beam by the quick return mechanism. Of the obtained image data, image data corresponding to focus detection is converted into focus detection image data by the image processing circuit 20. Thereafter, the image is sent to the AF unit 42 via the system control circuit 50, and the photographing lens 300 is focused. The system control circuit 50 can also perform so-called contrast AF that performs focusing on the focus control unit 342 of the photographing lens 300 based on the calculation result obtained by calculating the image data of the image sensor 14 by the image processing circuit 20. With such a configuration, the main mirror 130 and the sub mirror 131 are retracted out of the imaging light beam during electronic viewfinder observation and moving image shooting. However, both the phase difference detection method AF and the contrast method AF using the image sensor 14 are used together. Focus adjustment. In particular, since phase difference detection AF is performed, high-speed focusing is possible.

  As described above, the camera 100 according to the present embodiment uses the phase difference detection method AF by the focus detection device 105 in normal still image shooting in which the main mirror 130 and the sub mirror 131 are in the shooting optical path. On the other hand, the camera 100 is configured to use both the phase difference detection method AF using the image sensor 14 and the contrast method AF at the time of electronic viewfinder observation and moving image shooting in which the main mirror 130 and the sub mirror 131 are retracted outside the imaging light beam. Has been. Accordingly, focus adjustment can be performed in any of still image shooting, electronic viewfinder, and moving image shooting.

  The memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression circuit 32. Data of the A / D converter 16 is written into the image display memory 24 or the memory 30 through the image processing circuit 20 and the memory control circuit 22 or directly through the memory control circuit 22. The image display unit 28 is composed of a liquid crystal monitor and the like, and the display image data written in the image display memory 24 is displayed on the image display unit 28 via the D / A converter 26. The electronic viewfinder function can be realized by sequentially displaying the image data captured using the image display unit 28. The image display unit 28 can arbitrarily turn on / off the display based on an instruction from the system control circuit 50. When the display is turned off, the power consumption of the camera 100 can be reduced.

  The memory 30 is a storage unit that stores captured still images and moving images, and has a storage capacity sufficient to store a predetermined number of still images and moving images for a predetermined time. Thereby, even in the case of continuous shooting or panoramic shooting, a large amount of image writing can be performed on the memory 30 at high speed. The memory 30 can also be used as a work area for the system control circuit 50. The compression / decompression circuit 32 has a function of compressing / decompressing image data by adaptive discrete cosine transform (ADCT) or the like. The compression / decompression circuit 32 reads an image stored in the memory 30 and performs compression processing or decompression processing. Write to memory 30.

  The shutter control unit 36 controls the shutter 12 based on the photometric information from the photometric unit 46 in cooperation with the aperture control unit 344 that controls the aperture 312 of the photographing lens 300. The interface unit 38 (I / F unit) and the connector 122 electrically connect the camera 100 and the photographing lens 300. These devices have functions of transmitting control signals, status signals, data signals, and the like between the camera 100 and the photographing lens 300 and supplying currents of various voltages. Moreover, it is good also as a structure which transmits not only electrical communication but optical communication, audio | voice communication, etc. The photometry unit 46 performs AE processing. By exposing the light beam that has passed through the photographing lens 300 to the photometry unit 46 via the camera mount 106, the main mirror 130, and a photometric lens (not shown), the exposure state of the image can be measured. The photometry unit 46 also has a light control processing function in cooperation with the flash 48. The system control circuit 50 can also perform AE control on the shutter control unit 36 and the aperture control unit 344 of the photographic lens 300 based on the calculation result obtained by calculating the image data of the image sensor 14 by the image processing circuit 20. It is. The flash 48 also has an AF auxiliary light projecting function and a flash light control function.

  The system control circuit 50 is a control device that controls the entire camera 100. In the present embodiment, the system control circuit 50 includes an acquisition unit 50a, a correlation calculation unit 50b, a noise calculation unit 50c, and a determination unit 50d. The acquisition unit 50a acquires a first signal and a second signal corresponding to light beams that pass through different pupil regions of the imaging optical system (imaging lens 300). The correlation calculating means 50b calculates the correlation amount between the first signal and the second signal. The noise calculating unit 50c calculates correlation noise between the first signal and the second signal. The determination unit 50d determines the reliability related to the correlation amount using a reference according to the correlation noise.

  The memory 52 stores constants, variables, programs, and the like for operating the system control circuit 50. The display unit 54 is a liquid crystal display device that displays an operation state, a message, or the like using characters, images, sounds, or the like according to execution of a program by the system control circuit 50. One or a plurality of the display units 54 are installed at a position near the operation unit of the camera 100 that is easy to visually recognize, and are configured by a combination of, for example, an LCD and an LED. Among the contents displayed on the display unit 54, the contents displayed on the LCD and the like are information on the number of shots such as the number of shots and the number of remaining shots, and shooting conditions such as shutter speed, aperture value, exposure correction, and flash. There is information. The LCD also displays the remaining battery level, date / time, and the like. Further, as described above, the display unit 54 is provided with a part of its functions in the optical viewfinder 104.

  The nonvolatile memory 56 is an electrically erasable / recordable memory, such as an EEPROM. Reference numerals 60, 62, 64, 66, 68, and 70 are operation units for inputting various operation instructions of the system control circuit 50, such as switches, dials, touch panels, pointing by line-of-sight detection, voice recognition devices, and the like. Consists of one or more combinations.

  The mode dial switch 60 is used to switch and set each function mode such as power-off, auto shooting mode, manual shooting mode, playback mode, and PC connection mode. The shutter switch 62 (SW1) is turned on when a shutter button (not shown) is half-pressed, and instructs to start operations such as AF processing, AE processing, AWB processing, and EF processing. The shutter switch 64 (SW2) is turned on when the shutter button is fully pressed, and instructs the start of a series of processing related to photographing. Processing related to photographing includes exposure processing, development processing, recording processing, and the like. During the exposure process, a signal read from the image sensor 14 is written as image data into the memory 30 via the A / D converter 16 and the memory control circuit 22. At the time of development processing, development using calculation by the image processing circuit 20 and the memory control circuit 22 is performed. In the recording process, image data is read from the memory 30, compressed by the compression / decompression circuit 32, and written as image data on the recording medium 150 or the recording medium 160.

  The image display ON / OFF switch 66 is used to set ON / OFF of the image display unit 28. With this function, when photographing using the optical viewfinder 104, power supply can be saved by cutting off the current supply to the image display unit 28 such as a liquid crystal monitor. The quick review ON / OFF switch 68 is used to set a quick review function for automatically reproducing image data taken immediately after photographing. The operation unit 70 includes various buttons and a touch panel. The various buttons include a menu button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, and an exposure correction button.

  The power control unit 80 includes a battery detection circuit, a DC / DC converter, a switch circuit that switches a block to be energized, and the like. The power supply control unit 80 detects the presence / absence of a battery, the type of battery, and the remaining battery level, controls the DC / DC converter based on the detection result and an instruction from the system control circuit 50, and requires a necessary voltage. The period is supplied to each unit including the recording media 150 and 160. The connectors 82 and 84 connect the camera 100 to a power supply unit 86 including a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a lithium ion battery, and an AC adapter.

  The interfaces 90 and 94 (I / F unit) have a connection function with a recording medium such as a memory card or a hard disk. The connectors 92 and 96 make a physical connection with a recording medium such as a memory card or a hard disk. The recording medium attachment / detachment detection unit 98 detects whether or not the recording media 150 and 160 are attached to the connectors 92 and 96, respectively. Although the present embodiment has been described as having two systems of interfaces and connectors to which the recording media 150 and 160 are attached, the interfaces and connectors may have a single system or a plurality of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard. Further, by connecting various communication cards such as a LAN card to the interface and the connector, image data and management information attached to the image data can be transferred to and from other peripheral devices such as a computer and a printer.

  The communication unit 110 has various communication functions such as wired communication and wireless communication. The connector 112 connects the camera 100 to other devices via the communication unit 110, and is an antenna in the case of wireless communication. The recording media 150 and 160 are a memory card or a hard disk. The recording media 150 and 160 include recording units 152 and 162 each including a semiconductor memory or a magnetic disk, interfaces 154 and 164 (I / F units) with the camera 100, and connectors 156 and 166 that connect with the camera 100. ing.

  Next, the photographing lens 300 will be described. The taking lens 300 is configured to be detachable from the camera 100. The lens mount 306 mechanically couples the taking lens 300 with the camera 100 and is exchangeably attached to the camera 100 via the camera mount 106. The camera mount 106 and the lens mount 306 have functions of a connector 122 and a connector 322 that electrically connect the photographing lens 300 to the camera 100. The lens 311 includes a focus lens that focuses the subject. The diaphragm 312 controls the light amount of the photographing light flux.

  The connector 322 and the interface unit 338 (I / F unit) electrically connect the photographing lens 300 to the connector 122 of the camera 100. The connector 322 transmits a control signal, a status signal, a data signal, and the like between the camera 100 and the photographing lens 300, and also has a function of supplying currents of various voltages. The connector 322 may be configured to transmit not only electrical communication but also optical communication and voice communication. The zoom control unit 340 controls zooming of the lens 311 (operation of the zoom lens). The focus control unit 342 controls the operation of the focus lens of the lens 311. When the photographing lens 300 is a single focus lens without a zoom function, the zoom control unit 340 may not be provided. The aperture control unit 344 controls the aperture 312 in cooperation with the shutter control unit 36 that controls the shutter 12 based on the photometry information from the photometry unit 46. The diaphragm control unit 344 and the diaphragm 312 constitute diaphragm aperture adjusting means.

  The lens system control unit 346 controls the entire taking lens 300. The lens system control unit 346 has a memory function for storing constants, variables, programs, and the like for the operation of the photographing lens 300. The non-volatile memory 348 stores identification information such as a number unique to the photographing lens 300, management information, function information such as an open aperture value, a minimum aperture value, and a focal length, and current and past setting values. In the present embodiment, the nonvolatile memory 348 also stores lens frame information corresponding to the state of the taking lens 300. The lens frame information is information on the distance from the imaging element 14 of the frame opening and the radius of the frame opening that determine the light beam passing through the photographing lens 300. The diaphragm 312 is included in a frame that determines a light beam that passes through the photographing lens 300, and an opening of a lens frame component that holds the lens corresponds to the frame. In addition, since the frame for determining the light beam passing through the photographing lens 300 differs depending on the focus position and zoom position of the lens 311, a plurality of frames are prepared corresponding to the focus position and zoom position of the lens 311. When the camera 100 performs focus detection, optimal lens frame information corresponding to the focus position and zoom position of the lens 311 is selected, and the lens frame information is sent to the camera 100 via the connector 322.

  Next, the image sensor 14 will be described in detail with reference to FIGS. 2 (a), 2 (b), and 2 (c). The image sensor 14 performs the phase difference detection method AF similarly to the focus detection device 105. FIG. 2A is a circuit diagram of the pixel 200 of the image sensor 14. FIG. 2B is a pixel array diagram of the image sensor 14.

  The pixel 200 includes photodiodes (PD) 201a and 201b, transfer switches 202a and 202b, a floating diffusion region 203, an amplification unit 204, a reset switch 205, and a selection switch 206. Each switch can be composed of a MOS transistor or the like. Hereinafter, each switch is described as an N-type MOS transistor as an example, but each switch may be a P-type MOS transistor or another switching element.

  As described above, the image sensor 14 according to the present embodiment has two photodiodes in one pixel 200. However, the number of photodiodes provided in each pixel 200 is not limited to the two shown in FIG. 2A, and may be three or more (for example, four). In the present embodiment, the photodiodes 201a and 201b function as focus detection pixels and also function as imaging pixels, as will be described later.

  Photodiodes 201a and 201b receive light that has passed through one microlens 236 (the same microlens) shown in FIG. 2B, and generate photoelectric charges corresponding to the amount of light received by photoelectric conversion. It functions as a part. A signal obtained from the photodiode 201a is referred to as an A signal (first signal), and a signal obtained from the photodiode 201b is referred to as a B signal (second signal). As described above, the imaging device 14 includes the first photoelectric conversion unit (photodiode 201a) and the second photoelectric conversion unit (photodiode 201b) that receive light beams that pass through different pupil regions of the imaging optical system. In addition, the imaging element 14 includes a first photoelectric conversion unit and a second photoelectric conversion unit for one microlens 236, and the microlenses 236 are arranged in a two-dimensional manner.

  The transfer switch 202a is connected between the photodiode 201a and the floating diffusion region 203, and the transfer switch 202b is connected between the photodiode 201b and the floating diffusion region 203. The transfer switches 202a and 202b are elements that transfer charges generated in the photodiodes 201a and 201b to the common floating diffusion region 203, respectively. The transfer switches 202a and 202b are controlled by control signals TX_A and TX_B, respectively.

  The floating diffusion region 203 functions as a charge-voltage converter that temporarily holds the charges transferred from the photodiodes 201a and 201b and converts the held charges into a voltage signal. The amplifying unit 204 is a source follower MOS transistor. The gate of the amplifying unit 204 is connected to the floating diffusion region 203, and the drain of the amplifying unit 204 is connected to a common power source 208 that supplies the power supply potential VDD. The amplifying unit 204 amplifies the voltage signal based on the charge held in the floating diffusion region 203 and outputs it as an image signal. The reset switch 205 is connected between the floating diffusion region 203 and the common power source 208. The reset switch 205 is controlled by the control signal RES and has a function of resetting the potential of the floating diffusion region 203 to the power supply potential VDD. The selection switch 206 is connected between the source of the amplification unit 204 and the vertical output line 207. The selection switch 206 is controlled by the control signal SEL and outputs the image signal amplified by the amplification unit 204 to the vertical output line 207.

  FIG. 2C is a configuration diagram of the image sensor 14. The image sensor 14 includes a pixel array 234, a vertical scanning circuit 209, a current source load 210, a readout circuit 235, common output lines 228 and 229, a horizontal scanning circuit 232, and a data output unit 233.

  The pixel array 234 includes a plurality of pixels 200 arranged in a matrix. In FIG. 2C, for simplification of explanation, n pixels are shown in the horizontal direction and 4 pixels are shown in the vertical direction, but the number of rows and columns of the pixels 200 is arbitrary. Each pixel 200 is provided with any one of a plurality of color filters. In the example shown in FIG. 2C, the colors of the color filters are red (R), green (G), and blue (B). Pixels 200 are arranged according to a Bayer array. Further, the image sensor 14 in the present embodiment has a region (OB) in which a part of the pixel array 234 is shielded from light by the light shielding layer.

  The vertical scanning circuit 209 outputs a control signal to the pixels 200 in each row via the drive signal line 215 provided for each row. In FIG. 2C, one drive signal line 215 is shown for each row for simplification, but a plurality of drive signal lines are actually connected for each row. The pixels 200 in the same column are commonly connected to a vertical output line 207 provided for each column. A signal output from each pixel 200 is input to the readout circuit 235 via the vertical output line 207 and processed by the readout circuit 235. The current source load 210 is connected to the vertical output line 207 of each column.

  The horizontal scanning circuit 232 sequentially selects a column from which a signal is output from the plurality of readout circuits 235 by outputting control signals HSR (0) to HSR (n−1) signals. The read circuit 235 for the selected row outputs the processed signal to the data output unit 233 (output amplifier unit) via the common output lines 228 and 229.

  Next, a specific circuit configuration of the reading circuit 235 will be described. The read circuit 235 includes a clamp capacitor 211, feedback capacitors 214 to 216, an operational amplifier 213, a reference voltage source 212, and switches 217 to 220. The reading circuit 235 includes a comparator 221, Latch_N 222, Latch_S 223, and switches 226 and 227.

  A signal input to the readout circuit 235 through the vertical output line 207 is input to the inverting input terminal of the operational amplifier 213 through the clamp capacitor 211. The reference voltage Vref is supplied from the reference voltage source 212 to the non-inverting input terminal of the operational amplifier 213. The feedback capacitors 214 to 216 are connected between the inverting input terminal and the output terminal of the operational amplifier 213. The switch 217 is also connected between the inverting input terminal and the output terminal of the operational amplifier 213 and has a function of short-circuiting both ends of the feedback capacitors 214 to 216. The switch 217 is controlled by a control signal RES_C. The switches 218 to 220 are controlled by control signals GAIN0 to GAIN2.

  The comparator 221 receives the output signal of the operational amplifier 213 and the ramp signal 224 output from the ramp signal generator 230. Latch_N 222 is a memory element for holding a noise level (N signal). Latch_S 223 is a storage element for holding the A signal and the signal level (S signal) of the AB signal obtained by adding the A signal and the B signal. The output signal of the comparator 221 and the counter value 225 output from the counter 231 are input to Latch_N 222 and Latch_S 223, and are controlled by LATEN_N and LATEN_S, respectively. The output terminals of Latch_N and Latch_S are connected to common output lines 228 and 229 through switches 222 and 223, respectively. The common output lines 228 and 229 are connected to the data output unit 233.

  The switches 226 and 227 are controlled by a control signal HSR (h) (HSR (0) to HSR (n−1)) signal from the horizontal scanning circuit 232. Here, h indicates the column number of the readout circuit 235 to which the control signal line is connected. The signals held in Latch_N 222 and Latch_S 223 are output via the common output lines 228 and 229 and output from the data output unit 233 to the outside. This operation is called horizontal transfer.

  Next, with reference to FIG. 3, the reading operation of the imaging device 10 will be described. FIG. 3 is a timing chart illustrating the reading operation of the imaging apparatus 10. Hereinafter, the reading operation for one row of the image signal will be described with reference to FIG. Each switch is turned on when each control signal is H (High), and each switch is turned off when it is L (Low).

  At time t1, the control signals TX_A and TX_B become H, and the transfer switches 202a and 202b are turned on. At this time, RES is H, and the charges accumulated in the photodiodes 201a and 201b are transferred to the common power source 208 via the transfer switches 202a and 202b and the reset switch 205, and the photodiodes 201a and 201b are reset. The At time t2, the control signals TX_A and TX_B are set to L, and accumulation of photocharges in the photodiodes 201a and 201b is started.

  At time t3 after the photocharge is accumulated for a predetermined time, the control signal SEL of the selection switch 206 becomes H, and the source of the amplifying unit 204 is connected to the vertical output line. At time t4, the control signal RES of the reset switch 205 is set to L to release the reset of the floating diffusion region 203. At this time, a reset signal level potential corresponding to the potential of the floating diffusion region 203 is read to the vertical output line 207 via the amplifier 204 and input to the read circuit 235.

  Thereafter, when the control signal RES_C becomes L at time t5, a voltage based on the difference between the reset signal level read to the vertical output line 207 and the reference voltage Vref is output from the operational amplifier 213. The imaging apparatus 10 is set so that the system control circuit 50 sets any one of the control signals GAIN 0 to 2 to H based on the ISO sensitivity set in the operation unit 70 in advance. The imaging apparatus 10 of this embodiment includes ISO sensitivities 100, 200, and 400. For the ISO sensitivities 100 to 400, the control signals GAIN 0 to 2 become H, respectively. As a result, the corresponding switch among the switches 218 to 219 is turned on. The operational amplifier 213 amplifies the input voltage with an inverting gain determined by the ratio of any one of the clamp capacitor 211 and the feedback capacitors 214 to 216 and outputs the amplified voltage. Here, the random noise component generated in the circuit up to the operational amplifier 213 is also amplified, and the random noise amount of the output signal is different at ISO sensitivities 100, 200, and 400, respectively.

  Next, at time t6, the ramp signal generator 230 starts outputting a ramp signal whose signal level changes in proportion to the passage of time. At the same time, the counter 231 starts counting up from the reset state, and LATEN_N becomes H. The comparator 221 compares the output signal of the operational amplifier 213 with the ramp signal output from the ramp signal generator 230. When the signal level of the input ramp signal increases with time and the signal level exceeds the value of the output signal of the operational amplifier 213, the signal output to Latch_N 222 is inverted from L to H (time t7). Latch_N 222 stores the counter value output from the counter 231 when the signal from the comparator 221 is inverted from L to H while LATEN_N is H. This stored counter value becomes the N signal level. Thereafter, at time t8, the change of the ramp signal is completed and LATEN_N becomes L.

  At time t9, the control signal TX_A becomes H, and the photocharge of the photodiode 201a is transferred to the floating diffusion region 203. Thereafter, the control signal TX_A becomes L at time t10. By this operation, the charge accumulated in the photodiode 201a is transferred to the floating diffusion region 203. Then, a voltage corresponding to the change is output to the reading circuit 235 via the amplification unit 204 and the vertical output line 207. A voltage based on the difference between the reset signal level read to the vertical output line 207 and the reference voltage Vref is output from the operational amplifier 213. The operational amplifier 213 amplifies the input voltage with an inverting gain determined by the ratio of any one of the clamp capacitor 211 and the feedback capacitors 214 to 216 and outputs the amplified voltage.

  Next, at time t11, the ramp signal generator 230 starts outputting the ramp signal. At the same time, the counter 231 starts counting up from the reset state, and LATEN_S becomes H. The comparator 221 compares the output signal of the operational amplifier 213 with the ramp signal output from the ramp signal generator 230. When the signal level of the ramp signal exceeds the value of the output signal of the operational amplifier 213, the signal output to Latch_S223 is inverted from L to H (time t12). Latch_S 223 stores the counter value output from the counter 231 at that time when the signal from the comparator 221 is inverted from L to H while LATEN_S is H. The stored counter value becomes the A signal level. Thereafter, at time t13, the change of the ramp signal is completed, and LATEN_N becomes L.

  Thereafter, during time t14 to t15, the control signal HSR (h) output from the horizontal scanning circuit 232 sequentially changes from L to H and returns to L. Along with this, the switches 226 and 227 are turned on from off and then turned off again. The N signal data and the A signal data held in Latch_N 222 and Latch_S 223 of each column are respectively read to the common output lines 228 and 229 and input to the data output unit 233. The data output unit 233 outputs the difference between the A signal data and N signal data of each column to the outside.

  At time t16, the control signal TX_A becomes H again and the control signal TX_B becomes H again. Thereafter, at time t17, the control signals TX_A and TX_B become L. By this operation, both the photoelectric charges of the photodiodes 201 a and 201 b are transferred to the floating diffusion region 203. Then, a voltage corresponding to the change is output to the reading circuit 235 via the amplification unit 204 and the vertical output line 207. A voltage based on the difference between the reset signal level read to the vertical output line 207 and the reference voltage Vref is output from the operational amplifier 213. The operational amplifier 213 amplifies the input voltage with an inverting gain determined by the ratio of any one of the clamp capacitor 211 and the feedback capacitors 214 to 216 and outputs the amplified voltage.

  Next, at time t18, the ramp signal generator 230 starts outputting the ramp signal. At the same time, the counter 231 starts counting up from the reset state, and LATEN_S becomes H. The comparator 221 compares the output signal of the operational amplifier 213 with the ramp signal output from the ramp signal generator 230. When the signal level of the ramp signal exceeds the value of the output signal of the operational amplifier 213, the signal output to Latch_S223 is inverted from L to H (time t19). Latch_S 223 stores the counter value output from the counter 231 at that time when the signal from the comparator 221 is inverted from L to H while LATEN_S is H. This stored counter value becomes the AB signal level. Thereafter, at time t20, the change of the ramp signal is completed, and LATEN_N becomes L.

  Thereafter, during time t21 to t22, the control signal HSR (h) output from the horizontal scanning circuit 232 sequentially changes from L to H and returns to L. Along with this, the switches 226 and 227 are turned on from off and then turned off again. The N signal data and AB signal data held in Latch_N 222 and Latch_S 223 in each column are read to the common output lines 228 and 229, respectively, and input to the data output unit 233. The data output unit 233 outputs the difference between the AB signal data and the N signal data of each column to the outside.

  Finally, the control signal RES_C is H at time t22, the control signal RES is H at time t23, and the control signal SEL is L at time t24, and the reading operation for one row is completed. By repeating this operation for a predetermined number of rows, an image signal for one screen is acquired.

  The imaging apparatus 10 according to the present embodiment includes a still image mode and a moving image mode. In the still image mode, pixel data for all rows of the image sensor 14 is read. In the moving image mode, pixel data of a row having a three-row cycle is read, and the number of rows to be read is smaller than that in the still image mode. However, the configuration and readout of the still image mode and the moving image mode are not limited to this. With the above operation, an A signal and an AB signal from which noise has been removed are obtained. The A signal is used as a focus detection signal, and the AB signal is used as data constituting the captured image or as a focus detection signal.

  The image sensor 14 of the present embodiment has the following two types of readout modes. The first readout mode is called an all-pixel readout mode and is a mode for capturing a high-definition still image. In this case, signals of all pixels are read out. The second readout mode is called a thinning readout mode, and is a mode for performing only moving image recording or preview image display. In this case, since the number of necessary pixels is smaller than that of all the pixels, the pixel group reads out only the pixels thinned out at a predetermined ratio in both the X direction and the Y direction. Also, when it is necessary to read out at high speed, the thinning-out reading mode is similarly used. When thinning out in the X direction, signals are added to improve the S / N ratio, and thinning out in the Y direction ignores the signal output of the thinned out rows.

  Next, the conjugate relationship between the exit pupil plane of the imaging optical system (imaging lens 300) and the photoelectric conversion unit of the imaging element 14 will be described with reference to FIG. FIGS. 4A and 4B show the exit pupil plane of the imaging optical system and the photoelectric conversion unit of the image sensor 14 disposed near the center of the image plane, that is, the center of the image plane, in the imaging apparatus 10 of the present embodiment. It is explanatory drawing of a conjugate relationship. The photoelectric conversion unit in the imaging element 14 and the exit pupil plane of the imaging optical system are designed to have a conjugate relationship by an on-chip microlens. The exit pupil of the imaging optical system generally coincides with the surface on which the iris diaphragm for adjusting the light amount is placed. On the other hand, the imaging optical system of the present embodiment is a zoom lens having a zooming function. However, depending on the optical type, when the zooming operation is performed, the distance and size of the exit pupil from the image plane change. The imaging optical system shown in FIG. 4 shows a state where the focal length is between the wide-angle end and the telephoto end, that is, in the middle state. Assuming that this is a standard exit pupil distance Zep, an optimum design of an eccentricity parameter according to the shape of the on-chip microlens and the image height (X, Y coordinates) is made.

  In FIG. 4A, reference numeral 101 denotes a first lens group, 101b denotes a lens barrel member that holds the first lens group, 103 denotes a third lens group, and 103b denotes a lens barrel member that holds the third lens group. Reference numeral 102 denotes an aperture, reference numeral 102a denotes an aperture plate that defines an aperture diameter when the aperture is opened, and reference numeral 102b denotes an aperture blade for adjusting the aperture diameter when the aperture is closed. Note that the lens barrel member 101b, the aperture plate 102a, the aperture blade 102b, and the lens barrel member 103b, which act as a restricting member for the light beam passing through the imaging optical system, show optical virtual images when observed from the image plane. Yes. Further, the synthetic aperture in the vicinity of the stop 102 is defined as the exit pupil of the lens, and the distance from the image plane is Zep as described above.

  Reference numeral 2110 denotes a pixel for photoelectrically converting the subject image. The pixel 2110 is a central pixel arranged near the center of the image plane. The pixel 2110 (center pixel) is composed of photoelectric conversion units 2110a and 2110b, wiring layers 2110e to 2110g, a color filter 2110h, and an on-chip microlens 2110i from the bottom layer. The two photoelectric conversion units 2110a and 2110b are projected on the exit pupil plane of the imaging optical system by the on-chip microlens 2110i. In other words, the exit pupil of the imaging optical system is projected onto the surface of the photoelectric conversion unit via the on-chip microlens 2110i.

  FIG. 4B shows a projected image of the photoelectric conversion unit on the exit pupil plane of the imaging optical system. Projected images on the photoelectric conversion units 2110a and 2110b are EP1a and EP1b, respectively. In the present embodiment, the image sensor 14 can obtain an output signal of one of the two photoelectric conversion units 2110a and 2110b and an output signal of the sum of both. The sum output signal of both is a signal obtained by photoelectrically converting a light beam that has passed through both regions of the projection images EP1a and EP1b, which are substantially the entire pupil region of the imaging optical system. The projection image EP1a is called a first pupil region, and the projection image EP1b is called a second pupil region.

  In FIG. 4A, when L indicates the outermost part of the light beams CLa and CLb passing through the imaging optical system, the light beam L is regulated by the aperture plate 102a of the diaphragm, and the projection images EP1a and EP1b are the imaging optical system. There is almost no vignetting. In FIG. 4B, the light beam L in FIG. 4A is shown as TL. Since most of the projected images EP1a and EP1b of the photoelectric conversion units 2110a and 2110b are included in the circle indicated by TL, it can be seen that almost no vignetting has occurred. Since the light beam L is limited only by the aperture plate 102a of the diaphragm, TL can be restated as 102a. At this time, the vignetting state of the projection images EP1a and EP1b is symmetric with respect to the optical axis at the center of the image plane, and the light amounts received by the photoelectric conversion units 2110a and 2110b are equal.

  As described above, the imaging element 14 has not only an imaging function but also a focus detection function. In addition, since the imaging device 14 includes focus detection pixels that receive a light beam obtained by dividing the exit pupil, it is possible to perform phase difference detection AF.

  Next, the focus detection area of the image sensor 14 will be described with reference to FIG. FIG. 5 is an explanatory diagram of the focus detection area and shows the focus detection area 401 in the imaging range 400. The phase difference detection method AF by the image sensor 14 is performed in the focus detection area 401 (focus detection sensor on the imaging surface (on the light receiving surface)). In the focus detection area 401, phase difference detection is performed using the horizontal contrast difference in the imaging range 400. In the focus detection area 401, pixels 2110 are arranged in 1 row and 4N columns. In this embodiment, the pixels used for focus detection are one row and four columns, but the number of rows and the number of columns are not limited to this. The number of rows and the number of columns may be appropriately set within a range in which the phase difference can be detected.

  Next, an AF signal pair arranged in the focus detection area 401 will be described with reference to FIG. FIG. 6 is an explanatory diagram of AF signal pairs arranged in the focus detection area 401, and shows pixels in 1 × 4N columns arranged in the focus detection area 401. In FIG. 6, A (i, j) is used as the pixel for creating the AF A image signal in the i-th row and j-th column, and similarly, the AF B image signal in the i-th row and j-th column is created. The pixel used for this purpose is shown as B (i, j).

  In the present embodiment, the output signals of two pixels are added and used for the purpose of reducing the calculation load, improving the S / N of the output signal, and adjusting the output image size. Assuming that the signal of the k-th AF A image and the AF B image in the i-th row are As (i, k) and Bs (i, k), respectively, the signals As_1 (i, k), Bs_1 (i, k) is expressed as the following equation (10).

As_1 (1, k) = A (1,2 * (k-1) +1) + A (1,2 * (k-1) +2)
Bs_1 (1, k) = B (1,2 * (k-1) +1) + B (1,2 * (k-1) +2)
(1 ≦ k ≦ 2N) (10)
The A image signal for AF (first signal), which is the first image signal group constituted by the output signals of the first pixel group, is a signal sequence As_1 in Expression (10). A B image signal for AF (second signal) which is a second image signal group composed of output signals of the second pixel group is a signal string Bs_1, and an A image signal from an A + B image signal (previous pixel signal). It is generated by subtracting (first signal). In addition, since the readout circuit is shared by the same optical system, the A image signal and the B image signal in the same microlens have a negative or positive correlation with each other in noise components. For this reason, the signal sequences As_1 (m) and Bs_1 (m) in Expression (10) are signal pairs correlated with noise components. In this embodiment, the output signal includes a signal sequence As_1, which is a first image signal group composed of output signals of the first pixel group, and a noise component correlated with the first signal sequence. The correlation amount is calculated on the basis of the signal sequence Bs_1 which is the image signal group.

  Next, the operation of the imaging apparatus 10 (camera 100) will be described with reference to FIG. FIG. 7 is a flowchart showing the focus adjustment operation of the imaging apparatus 10. Each step in FIG. 7 is executed by each unit of the imaging apparatus 10 (camera 100) based on a program stored in the system control circuit 50. This flowchart shows an electronic viewfinder that performs phase difference detection AF using the image sensor 14 when the main mirror 130 and the sub mirror 131 are retracted from the imaging light beam, or a focus adjustment operation during moving image shooting. That is, the focus adjustment operation is performed in parallel with the display for the electronic viewfinder and the moving image recording.

  First, in step S501, the system control circuit 50 determines whether or not focus detection has started (that is, detects whether or not the focus detection start instruction button has been turned ON by an operation of the SW1 or the operation unit 70). When the focus detection start instruction button is turned on and focus detection starts, the process proceeds to step S502. On the other hand, step S501 is repeated until focus detection starts. In the present embodiment, the determination by the focus detection start button is performed, but focus detection may be started by using the transition to electronic viewfinder display or moving image recording as a trigger.

  Subsequently, in step S <b> 502, the system control circuit 50 acquires various lens information such as lens frame information and focus lens position of the photographing lens 300 via the interface units 38 and 338 and the connectors 122 and 322. In step S503, the system control circuit 50 generates a pair of focus detection signals (AF signals) from the sequentially read image data by the combining unit and the connecting unit of the image processing circuit 20. In the present embodiment, AF A and B image signals represented by Expression (10) are generated. These signals are sent to the AF unit 42 to perform light amount correction and the like. The AF unit 42 performs digital filter processing in accordance with the frequency characteristics of a signal to be evaluated at the time of focus detection, and performs extraction of a desired frequency component (spatial frequency band extraction processing). The subsequent correlation calculation is performed using the AF A image signal and the AF B image signal after the digital filter processing.

  Subsequently, in step S504, the AF unit 42 (system control circuit 50) calculates the shift amount of the pair of focus detection signals using a known correlation calculation method and the like, and converts it to the defocus amount (defocus amount calculation). ). Details of this will be described later. Subsequently, in step S505, the system control circuit 50 calculates the lens driving amount of the photographing lens 300 based on the focus detection result calculated in step S504. Subsequently, in step S506, the system control circuit 50 sends the lens driving amount calculated in step S505 to the focus control unit 342 of the photographing lens 300 via the interface units 38 and 338 and the connectors 122 and 322. Then, the system control circuit 50 and the focus control unit 342 perform focus adjustment (lens drive) of the photographing lens 300 by driving the focus lens.

  Next, the calculation of the defocus amount (step S504 in FIG. 7) will be described with reference to FIG. FIG. 8 is a flowchart of a defocus amount calculation subroutine. When the process proceeds from step S504 of the main routine (FIG. 7) to the subroutine of FIG. 8, the process proceeds to step S5041.

  In step S5041, the system control circuit 50 (AF unit 42) performs a correlation calculation using the AF A image signal As_1 and the AF B image signal Bs_1. The correlation amount COR1 (k) used for the correlation calculation is calculated by the above-described equation (1). Thereafter, a value k is obtained when the correlation between the signals of the AF A image and the AF B image is the highest, that is, when the correlation amount COR1 is minimized. Here, if the value k is calculated as an integer, the resolution becomes rough. Therefore, interpolation processing is performed as appropriate, and so-called subpixel calculation is performed. In this embodiment, the difference of the correlation amount COR1 is taken, and the shift amount dk1 in which the sign of the difference amount changes is calculated (detected). The correlation amount difference DCOR1 is calculated as in the following equation (11).

DCOR1 (k) = COR1 (k) −COR1 (k−1) (11)
The system control circuit 50 uses the difference amount DCOR1 of the correlation amount COR1 to detect the shift amount dk1 in which the sign of the difference amount changes. Assuming that k immediately before the sign change is k1 and k immediately after the sign change is k2 (k2 = k1 + 1), the shift amount dk1 is calculated as the following equation (12).

dk1 = k1 + | DCOR1 (k1) | / | DCOR1 (k1) −DCOR1 (k2) | (12)
As described above, the system control circuit 50 calculates the shift amount dk1 between the AF A image and the AF B image of 1 pixel or less, and ends the process of step S5041. There are various known methods for calculating the phase difference described above, and other methods may be used in this embodiment.

  Subsequently, in step S5042, the system control circuit 50 calculates the amount of noise. As described above, when noise having a correlation between the AF A image and the AF B image occurs, the shift amount calculation accuracy performed in step S5041 deteriorates. This is because the influence of correlated noise occurs in the vicinity of the minimum value of the correlation amount COR1. Therefore, in step S5042, two types of noise amounts are calculated in consideration of the gain applied to the AF signal that affects the amount of noise, averaging processing for SN improvement, filter processing, and the like. One of the two types of noise is a situation where noise that is not correlated between the AF A image and the AF B image occurs, that is, the amount of noise that is generated when the shift amount is not close to 0 (uncorrelated noise amount). is there. The other noise amount is a noise amount (correlation noise amount) in a situation where there is a situation correlation in which a noise having no correlation exists between the AF A image and the AF B image when the shift amount is near zero. The amount of correlation noise may be negative or positive. The details of various noise amount calculation methods will be described later.

  Subsequently, in step S5043, the system control circuit 50 determines whether or not the correlation noise amount calculated in step S5042 is smaller than a predetermined amount (determines whether the correlation noise amount is large or small). When the amount of correlated noise is smaller than a predetermined amount, it indicates that most of the noise is composed of uncorrelated noise. In this case, the process proceeds to step S5044, and the system control circuit 50 (determination unit 50d) performs normal reliability determination. In step S5044, the reliability determination is performed using the minimum value of the correlation amount COR1 and the difference DCOR1 of the correlation amount in the vicinity of the minimum value.

  Regarding the magnitude of the minimum value of the correlation amount COR1, it is determined that the smaller the minimum value, the higher the reliability. Ideally, the minimum value of the correlation amount COR1 is 0 when the AF A image and the AF B image have the same shape. However, the A image for AF and the B image for AF have different shapes due to the influence of the diffusion characteristics of light from the subject, the light amount adjustment error, and the influence of noise generated individually for each pixel. For this reason, the minimum value of the correlation amount COR1 is generally a positive value. On the other hand, as the shapes of the AF A image and the AF B image are different, the detection accuracy of the minimum value is degraded, and the focus detection accuracy is degraded. For example, as one of the reliability determinations, if the minimum value of the correlation amount COR1 is smaller than the threshold value Thr1 (determination threshold value as a reference), it is determined that the reliability is high.

  Further, the system control circuit 50 performs reliability determination using the difference DCOR1 of the correlation amount in the vicinity of the shift amount dk1. The larger the correlation amount difference DCOR1 is, the more accurately the shift amount dk1 can be calculated. This is because even if the correlation amount varies due to an error, if the correlation amount difference is large, the influence on the detection of the shift amount is small. From this, when the correlation amount difference DCOR1 is larger than the threshold Thr2, it can be determined that the reliability is high. The threshold value Thr1 and the threshold value Thr2 described above may be changed depending on the amount of uncorrelated noise.

  If it is determined in step S5043 that the amount of correlation noise is greater than the predetermined amount, the process proceeds to step S5045. In step S5045, the system control circuit 50 determines whether the correlation of the correlation noise is positive or negative. If the correlation noise is negative correlation, the process proceeds to step S5046. In step S5046, the system control circuit 50 performs reliability determination (reliability determination for negative correlation noise) when negative correlation noise is present. Specifically, the reliability determination method is the same as that in step S5044, but the threshold value is changed. If a correlated noise amount having a negative correlation is generated in step S5046, the system control circuit 50 changes the threshold value Thr1 based on the difference between the uncorrelated noise amount and the correlated noise amount.

  The uncorrelated noise amount and the correlated noise amount correspond to Noise (S) and Noise (0) in FIG. 11B, and the difference corresponds to the size of the protrusion. Even when the amount of noise due to negative correlation is small, an uncorrelated noise amount occurs, and the minimum value of the correlation amount COR1 cannot be less than or equal to a predetermined value. Therefore, a predetermined value is set as the threshold Thr1 in consideration of the amount of uncorrelated noise. When noise due to negative correlation occurs, the minimum value of the correlation amount COR1 becomes a larger value. For this reason, as the threshold Thr1, a larger value is set in consideration of the amount of negative correlation noise in addition to the amount of uncorrelation noise, and reliability determination is performed. Further, the reliability determination of the correlation amount difference DCOR1 is performed using the threshold value Thr2 as in step S5044. The negative correlation of the correlation noise works in the direction of decreasing the DCOR near the minimum value. For this reason, it is sufficient to set the same value as the threshold value used in step S5044.

  If it is determined in step S5045 that the correlation noise is positive, the process proceeds to step S5047. In step S5047, the system control circuit 50 performs two reliability determinations as in step S5044. When the amount of correlation noise having a positive correlation is large, the minimum value of the correlation amount COR1 is extremely small. Therefore, in addition to determining whether or not the minimum value of the correlation amount COR1 is sufficiently small using the same threshold value Thr1 as in step S5044, it is determined whether or not the minimum value is too small. Specifically, the system control circuit 50 also performs reliability determination that determines that the reliability is high when the minimum value of the correlation amount COR1 is larger than the threshold Thr3 (<Thr1). The value of the threshold Thr3 is changed based on the amount of uncorrelated noise. Although the size (depth) of the valley shape varies depending on the amount of correlation noise, it is not necessary to change the value of the threshold Thr3. When noise due to positive correlation occurs, the minimum value of the correlation amount COR1 is calculated to be smaller than the value assumed from the uncorrelated noise amount. Since the position of the minimum value is detected in the above-described shift amount calculation, the correlation noise has a large minimum value detection error regardless of the amount of noise. Therefore, the threshold value Thr3 is set based on the amount of uncorrelated noise.

  As reliability determination in step S5047, the system control circuit 50 also performs reliability determination using the difference amount DCOR1 of the correlation amount in the vicinity of the shift amount dk1. When the amount of correlation noise having a positive correlation is large, a large change in the amount of correlation is obtained even when the AF signal is uniform, the contrast is small, and the amount of correlation near the local minimum generated by the original signal component is small. It is done. For this reason, in order to calculate the shift amount with high accuracy, it is necessary to change the correlation amount sufficiently larger than the correlation amount change caused by the correlation noise amount. Therefore, a larger value is set as the threshold Thr2 for determining the magnitude of the difference DCOR1 in the vicinity of the minimum value when the amount of correlation noise is large.

  In steps S5044, S5046, and S5047, the above reliability determination is performed using determination threshold values (thresholds Thr1, Thr2, and Thr3) as a reference according to the correlation noise, and the process proceeds to step S5048. In step S5048, the system control circuit 50 determines whether or not there is reliability as a result of the reliability determination. If it is determined that the obtained shift amount dk1 is reliable, the process proceeds to step S5049. In step S5049, the system control circuit 50 converts the obtained shift amount dk1 into a defocus amount. The conversion coefficient used when converting to the defocus amount differs depending on the image height of the detection area where focus detection is performed, the optical conditions of the photographing lens (F value, exit pupil position, vignetting situation), and the like. In the present embodiment, the system control circuit 50 acquires a conversion coefficient to be used from a table stored for each of various conditions.

  On the other hand, if it is determined in step S5048 that the reliability is low, the process proceeds to step S5050. In step S5050, the system control circuit 50 determines the driving direction of the focus lens using statistical data such as the distance distribution of the subject because there is no detection result to be used when driving the focus lens. In general, since the subject often exists on the near side, the system control circuit 50 performs setting for performing search driving on the near side. When step S5049 or step S5050 is completed, this subroutine is terminated, and the process proceeds to step S505 in FIG.

  Next, the calculation of the noise amount (step S5042 in FIG. 8) will be described with reference to FIG. FIG. 9 is a flowchart of a noise amount calculation subroutine. Each step of FIG. 9 is mainly performed by the system control circuit 50 (noise calculation means 50c). When the process proceeds from step S5041 in FIG. 8 to the subroutine in FIG. 9, the process proceeds to step S201.

  In step S <b> 201, the system control circuit 50 acquires information (gain information) related to gain processing performed on the AF A image signal and the AF B image signal. The gain processing applied to the AF A image signal and the AF B image signal includes gain processing in the sensor, gain processing performed after AD conversion, gain processing for adjusting the light amount of two signals, and the like. Conceivable. The amount of noise increases with the gain applied. The gain information G of the signal is calculated as the following formula (13).

G = Gi × Go × Gab (13)
In Expression (13), Gi is a gain applied to the signal in the sensor (imaging device 14), Go is a gain applied to the signal after sensor output, and Gab is applied to the AF A signal signal and the B image signal. The gain is shown. The gain applied to the AF A image signal and the AF B image signal differs for each pixel in order to adjust the light amount. As the gain Gab, an average value of gains applied to each pixel may be used as a representative value.

  Subsequently, in step S202, the system control circuit 50 acquires the signal addition number. Signal addition has the effect of reducing the amount of noise, and it is conceivable to add luminance signals or correlation amounts between signals in different rows or between different frames. The signal addition number Nsum is calculated as in the following equation (14).

Nsum = Npix + Nframe + Ncor (14)
In Expression (14), Npix is the number of additions of signals obtained from pixels in different rows and columns, Nframe is the number of additions between different frames, and Ncor is the number of correlation amounts.

  Subsequently, in step S203, the system control circuit 50 acquires the setting of the digital filter applied to the AF A image signal and the AF B image signal. With the digital filter, the noise amount Noise (0) described with reference to FIG. 11B is diffused to the peripheral shift amount. This is because noise that is correlated only with the pair of pixels diffuses to surrounding pixels by the digital filter. When the number of taps of the digital filter is 2T + 1, when the correlation calculation of Expression (1) is performed, the influence of correlated noise occurs in the range where the shift amount is ± T. For example, when a 3-tap filter [p, q, r] (p + q + r = 1) is applied, the correlation amount COR (0) when the shift amount = 0 expressed by the equation (4) is expressed by the following equation ( 15).

COR (0) = Σ | p × (A (i−1) −B (i−1)) + q × (A (i) −B (i)) + r × (A (i + 1) −B (i + 1)) | (15)
The noise part of equation (15) is 2 × (p × N [A (i−1)] + q × N [A (i)] + r × N [A (i + 1)]) − (p × N [A + B ( i-1)] + q × N [A + B (i)] + r × N [A + B (i + 1)]). When there is no correlation among N [A (i-1)], N [A (i)], and N [A (i + 1)], which are noises of three pixels, averaging is performed by weighted addition using p, q, and r. Processing is performed and the amount of noise is reduced. The noise reduction effect F by the digital filter is calculated as in the following equation (16).

  The noise reduction effect F calculated by Expression (16) can be handled in the same manner as the gain G described above.

  Similarly, the correlation amount COR (1) when the shift amount = 1 is expressed by the following equation (17).

COR (1) = Σ | p × (A (i−2) −B (i)) + q × (A (i−1) −B (i + 1)) + r × (A (i) −B (i + 2)) | (17)
In Expression (17), the shift amount is calculated including a signal correlated to noises A (i) and B (i). Specifically, a correlation occurs in the noise component of r × A (i) −p × B (i). In this way, the influence of noise having a correlation in the vicinity of the shift amount 0 is diffused by the digital filter processing. In step S203, filter settings are acquired, and a correlated noise influence range T and a noise reduction effect F by a digital filter are calculated.

  Subsequently, in step S204, the system control circuit 50 calculates the amount of noise. Here, the amount of noise of the above-described two types of noise (uncorrelated noise and correlated noise) is calculated. For these two types of noise, a reference noise amount is stored in advance in a storage unit such as a memory of the imaging apparatus 10 as a basic characteristic of an output signal of the imaging device. Assuming that the uncorrelated noise reference amount Nn0, the correlated noise reference amounts Np0, NNm0 (Np0 and Nm0 are correlated noise reference amounts having positive and negative correlations, respectively), the uncorrelated noise amount Nn and the correlated noise Nc are: They are calculated as in the following formulas (18) and (19).

  The uncorrelated noise Nn and the correlated noise having a negative correlation (the term after the equation (19)) are mainly dark noise components that do not depend on the received light amount (signal amount). The noise amount is calculated in consideration of the effects of the gain, signal addition, and filter described above. On the other hand, in the correlation noise having a positive correlation (the term before the equation (19)), the non-uniformity of the sensitivity for each pixel is caused by the nonuniformity of the transmittance for each pixel, the readout circuit, etc., and the signal amount is large. In some cases, the noise increases. Therefore, the amount of noise is calculated by multiplying the average value I (the integrated value of at least a part of the signal sequence) of the luminance signals (before digital filter processing) of the AF A image signal and the AF B image signal. When the sign of the correlation noise Nc is positive, it indicates that the correlation noise has a positive correlation. On the other hand, when the sign is negative, it indicates that there is a correlation noise having a negative correlation. When calculating the amount of noise described above, the absolute value of the correlation noise Nc is used. In the present embodiment, the uncorrelated noise has been described as an offset component that is not affected by the amount of received light. However, with respect to the uncorrelated noise, the offset component and the gain component may be considered separately. When the calculation of the noise amount is completed in step S204, this subroutine is terminated, and the process proceeds to step S5043 in FIG.

  As a result, in an imaging apparatus that outputs a signal having a correlation with a noise component between a pair of AF image signals, even when shooting in a low-contrast subject or in a low-luminance environment, the influence of noise is properly grasped. It is possible to adjust the focus using only the highly accurate focus detection result. Although the present embodiment has been described on the assumption that noise having a correlation is generated between pixels sharing the same microlens, the source and nature of the noise are not limited to this. For example, when the floating diffusion, the signal output line, and the amplifier circuit are shared, or when there is unevenness between the areas of the color filter, a correlated noise can be generated between the A image signal and the B image signal. Even in such a case, the effect of this embodiment can be obtained by appropriately calculating the noise amount and performing the reliability determination.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. The present embodiment is different from the first embodiment in that the reliability determination (determination threshold) is made different according to the magnitude of the shift amount dk1 corresponding to the minimum value of the correlation amount COR1. Other configurations and operations in the present embodiment are the same as those in the first embodiment, and a description thereof will be omitted.

  In the first embodiment, the reliability determination is performed by switching the threshold value of the reliability determination depending on the magnitude of the correlation noise amount regardless of the magnitude of the shift amount dk1. As described above, the influence of the correlation noise is diffused by the influence of the digital filter, but occurs in the range where the shift amount is near zero. For this reason, when the absolute value of the shift amount dk1 is sufficiently large, it is not affected by the correlation noise. Therefore, in the present embodiment, the reliability determination is switched according to the magnitude of the shift amount dk1. Thereby, it is not necessary to change the threshold value depending on the magnitude of unnecessary correlation noise, and the calculation load can be reduced. In addition, it is possible to realize highly accurate reliability determination without unnecessarily strict reliability determination.

  With reference to FIG. 10, a method of switching reliability determination based on the magnitude of the shift amount that obtains the minimum value of the correlation amount in this embodiment (calculation of the defocus amount (step S <b> 504 in FIG. 7)) will be described. FIG. 10 is a flowchart of a defocus amount calculation subroutine. When the process proceeds from step S504 of the main routine (FIG. 7) to the subroutine of FIG. 10, the process proceeds to step S5041. In FIG. 10, the same steps as those in FIG. 8 are denoted by the same step numbers. The flowchart in FIG. 10 differs from the flowchart in FIG. 8 in that step S5100 is inserted instead of step S5043 in FIG. Other steps in FIG. 10 are the same as those in FIG.

  In step S5100, the system control circuit 50 determines the magnitude of the correlation noise and the magnitude of the absolute value of the shift amount dk1. The point that the threshold value for reliability determination is changed according to the amount of correlated noise is the same as in the first embodiment. In this embodiment, when the correlation noise amount is smaller than the predetermined amount or the magnitude of the absolute value of the shift amount dk1 is larger than the predetermined value, the process proceeds to step S5044 and normal reliability determination is performed. On the other hand, if the correlation noise is larger than the predetermined amount and the absolute value of the shift amount dk1 is smaller than the predetermined value, it is determined that there is an influence of the correlation noise, and the process proceeds to step S5045. Processes after step S5044 and step S5045 are the same as those in the first embodiment.

  As described above, in this embodiment, when the shift amount dk1 is large, it is determined that there is no influence of the correlation noise and the reliability determination is performed. Accordingly, an appropriate reliability determination threshold can be set, and highly accurate reliability determination can be performed. Further, it is not necessary to set a threshold value due to unnecessary correlation noise, and the calculation load can be reduced.

  As described above, in each embodiment, the control device (system control circuit 50) includes the acquisition unit 50a, the correlation calculation unit 50b, the noise calculation unit 50c, and the determination unit 50d. The acquisition unit 50a includes a first signal (output signal from the first photoelectric conversion unit) and a second signal (from the second photoelectric conversion unit) corresponding to light beams that pass through different pupil regions of the imaging optical system (imaging lens 300). Output signal). The correlation calculating means 50b calculates the correlation amount between the first signal and the second signal. The noise calculating unit 50c calculates correlation noise between the first signal and the second signal. The determination unit 50d determines the reliability related to the correlation amount using a reference according to the correlation noise. Preferably, the first signal corresponds to the output signal of the first photoelectric conversion unit, and the second signal subtracts the output signal of the first photoelectric conversion unit from the output signals of the first photoelectric conversion unit and the second photoelectric conversion unit. Corresponds to the signal obtained.

  Preferably, the determination unit 50d sets a determination threshold value (threshold values Thr1, Thr2, Thr3) according to the correlation noise as the reference, and compares information regarding the correlation amount (such as a minimum value of the correlation amount) with the determination threshold value. To determine reliability. Preferably, the correlation noise is noise having a positive correlation and a negative correlation between the first signal and the second signal. Preferably, the correlation calculation means 50b calculates the correlation amount for each shift amount while shifting the first signal and the second signal, and calculates the shift amount having the highest correlation. Then, the determination unit 50d determines the reliability of the shift amount having the highest correlation as the reliability related to the correlation amount.

  Preferably, the determination unit 50d changes the determination threshold according to the gain processing for the first signal and the second signal (step S201, equation (13)). Preferably, the determination unit 50d changes the determination threshold according to the first signal and the second signal, or signal addition processing (step S202, equation (14)) for the correlation amount. Preferably, the determination unit 50d changes the determination threshold according to the spatial frequency band extraction processing (step S203, equations (15) to (17)) for the first signal and the second signal. Preferably, the determination unit 50d changes the determination threshold according to an integrated value of at least a partial region of at least one of the first signal and the second signal. Preferably, the determination unit 50d changes the determination threshold according to the shift amount having the highest correlation.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  According to the imaging apparatus of each embodiment, it is possible to appropriately grasp the influence of noise even when shooting a low-contrast subject or in a low-brightness environment. Therefore, according to each embodiment, it is possible to provide a control device, an imaging device, a control method, a program, and a storage medium that can perform focus detection with high accuracy.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

50 System control circuit (control device)
50a Acquisition means 50b Correlation calculation means 50c Noise calculation means 50d Determination means

Claims (15)

Obtaining means for obtaining a first signal and a second signal corresponding to light beams passing through different pupil regions of the imaging optical system;
Correlation calculating means for calculating a correlation amount between the first signal and the two signals;
Noise calculating means for calculating correlation noise between the first signal and the second signal;
And a determination unit configured to determine reliability related to the correlation amount using a reference according to the correlation noise. The determination means includes
As the reference, set a determination threshold according to the correlation noise,
The control device according to claim 1, wherein the reliability is determined by comparing information on the correlation amount and the determination threshold.   The control device according to claim 1, wherein the correlation noise is noise having a positive correlation and a negative correlation between the first signal and the second signal. The correlation calculation means calculates the correlation amount for each shift amount while shifting the first signal and the second signal, and calculates the shift amount having the highest correlation,
4. The control device according to claim 1, wherein the determination unit determines the reliability of the shift amount having the highest correlation as the reliability related to the correlation amount. 5.   The control device according to claim 4, wherein the determination unit changes the reference according to the shift amount having the highest correlation.   The control device according to claim 1, wherein the determination unit changes the reference in accordance with a gain process for the first signal and the second signal.   The said determination means changes the said reference according to the signal addition process with respect to the said 1st signal and the said 2nd signal, or the said correlation amount, The any one of Claim 1 thru | or 6 characterized by the above-mentioned. Control device.   8. The control device according to claim 1, wherein the determination unit changes the reference according to a spatial frequency band extraction process for the first signal and the second signal. 9. .   9. The method according to claim 1, wherein the determination unit changes the reference according to an integrated value of at least a partial region of at least one of the first signal and the second signal. The control device according to item. An imaging device having a first photoelectric conversion unit and a second photoelectric conversion unit that receive light beams passing through different pupil regions of the imaging optical system;
Obtaining means for obtaining a first signal and a second signal respectively corresponding to output signals from the first photoelectric conversion unit and the second photoelectric conversion unit;
Correlation calculating means for calculating a correlation amount between the first signal and the two signals;
Noise calculating means for calculating correlation noise between the first signal and the second signal;
An image pickup apparatus comprising: a determination unit that determines reliability of the correlation amount according to the correlation noise.   The image pickup device includes the first photoelectric conversion unit and the second photoelectric conversion unit with respect to one microlens, and the microlens is two-dimensionally arranged. The imaging device described. The first signal corresponds to an output signal of the first photoelectric conversion unit,
The second signal corresponds to a signal obtained by subtracting the output signal of the first photoelectric conversion unit from the output signals of the first photoelectric conversion unit and the second photoelectric conversion unit. Item 12. The imaging device according to Item 10 or 11. Obtaining a first signal and a second signal corresponding to light beams passing through different pupil regions of the imaging optical system;
Calculating a correlation amount between the first signal and the two signals;
Calculating correlation noise between the first signal and the second signal;
And determining the reliability of the correlation amount in accordance with the correlation noise. Obtaining a first signal and a second signal corresponding to light beams passing through different pupil regions of the imaging optical system;
Calculating a correlation amount between the first signal and the two signals;
Calculating correlation noise between the first signal and the second signal;
And causing the computer to execute a step of determining reliability of the correlation amount in accordance with the correlation noise. A storage medium storing the program according to claim 14.