JP6971692B2 - Control device, image pickup device, control method, program, and storage medium - Google Patents

Description

The present invention relates to an image pickup apparatus that adjusts the focus by an image pickup surface phase difference AF.

In recent years, as phase-difference AF (imaging surface phase-difference AF) using an image sensor, a method of acquiring a photographed image and performing focus detection using the pixels of the image sensor has been proposed.

Patent Document 1 discloses a configuration in which a plurality of photodiodes corresponding to one microlens are provided, and each photodiode receives light from different pupil surfaces of the photographing lens. With such a configuration, it is possible to perform imaging surface phase difference AF by comparing the outputs of the two photodiodes. In the imaging surface phase difference AF, after performing bandpass filter processing on two (pair) image signals (A image signal and B image signal) of each pixel, the A image signal and the B image signal are subjected to a predetermined region. The amount of image shift is calculated by performing a correlation calculation, and the amount of defocus is calculated based on the amount of image shift.

In Patent Document 2, the two image signals are subjected to the first filter processing to calculate the defocus amount, and the calculated defocus amount is subjected to the second filter processing to obtain the evaluation value of the defocus amount. The configuration to be generated is disclosed. Further, the focal state is evaluated based on the generated evaluation value, and the focus is adjusted according to the evaluation result.

Japanese Unexamined Patent Publication No. 2001-083407 Japanese Unexamined Patent Publication No. 2014-219549

However, the characteristics of the defocus amount differ depending on the frequency band of the bandpass filter applied to the image signal. For example, when the frequency band is in the low frequency range, the defocus amount can be detected even in a large blur state, but the accuracy of the defocus amount near the in-focus position is low. On the other hand, when the frequency band is in the high frequency range, the accuracy of the defocus amount near the in-focus position is high, but the defocus amount cannot be detected in a large blur state.

In Patent Document 2, an evaluation value of the defocus amount is generated by filtering a frequency band different from the frequency band of the filter processing used for calculating the defocus amount, and the focal state is evaluated using the evaluation value. There is. However, only one frequency band is used as the frequency band of the bandpass filter for detecting the defocus amount. When the frequency band of the filter is set to a low frequency range, the accuracy of the defocus amount near the in-focus position (focus accuracy) becomes low. On the other hand, if the frequency band of the filter is set to a high frequency range, a large blur state cannot be detected. Therefore, it may take time to adjust the focus by moving the focus lens in the direction opposite to the in-focus position.

Therefore, the present invention provides a control device, an image pickup device, a control method, a program, and a storage medium capable of improving focusing accuracy and shortening the focus adjustment time.

The image pickup apparatus as one aspect of the present invention has a first photoelectric conversion unit and a second photoelectric conversion unit that receive light rays passing through different pupil regions of the imaging optical system for one microlens. An image pickup element in which microlenses are arranged in a two-dimensional manner, and an acquisition means for acquiring a first signal and a second signal corresponding to the output signals from the first photoelectric conversion unit and the second photoelectric conversion unit, respectively. , The first signal and the second signal are subjected to a plurality of filtering processes having different bands from each other, and a plurality of defocus amounts and a plurality of reliabilitys are performed based on the first signal and the second signal after each filtering processing. It has a calculation means for calculating the above-mentioned, and a determination means for determining the defocus amount used for the focus adjustment from the plurality of defocus amounts based on the difference between the plurality of defocus amounts.

The control method as another aspect of the present invention includes a first photoelectric conversion unit and a second photoelectric conversion unit that receive light rays passing through different pupil regions of the imaging optical system for one microlens. A step of acquiring a first signal and a second signal corresponding to the output signals from the first photoelectric conversion unit and the second photoelectric conversion unit of the image pickup element in which the microlenses are arranged in two dimensions, respectively. A plurality of filters having different bands are performed on the first signal and the second signal, and a plurality of defocus amounts and a plurality of reliabilitys are obtained based on the first signal and the second signal after each filtering. It has a step of calculating and a step of determining a defocus amount used for focus adjustment from the plurality of defocus amounts based on the difference between the plurality of defocus amounts.

The program as another aspect of the present invention causes a computer to execute the control method.

The storage medium as another aspect of the present invention stores the program.
The control device as another aspect of the present invention includes an acquisition means for acquiring a first signal and a second signal corresponding to light beams passing through different pupil regions of an imaging optical system, and the first signal and the second signal. It has a calculation means for calculating the defocus amount and the reliability based on the above, and a focus control means for controlling the focus lens according to the defocus amount and the reliability, and the focus control means has the focus. The defocus amount is the first mode in which the focus is controlled over the entire movable range of the lens and the second mode in which the focus is controlled within a predetermined range based on the focus lens position set by the user. The conditions for frame addition are changed according to the reliability.
The control device as another aspect of the present invention includes an acquisition means for acquiring a first signal and a second signal corresponding to light beams passing through different pupil regions of an imaging optical system, and the first signal and the second signal. It has a calculation means for calculating the defocus amount and the reliability based on the above, and a focus control means for controlling the focus lens according to the defocus amount and the reliability, and the focus control means has the focus. The defocus amount is the first mode in which the focus is controlled over the entire movable range of the lens and the second mode in which the focus is controlled within a predetermined range based on the focus lens position set by the user. The number of times the focus lens is driven is changed according to the reliability.

Other objects and features of the present invention will be described in the following embodiments.

According to the present invention, it is possible to provide a control device, an image pickup device, a control method, a program, and a storage medium capable of improving focusing accuracy and shortening the focus adjustment time.

It is a block diagram of the image pickup apparatus in 1st Embodiment. It is a figure which shows the pixel composition example of (a) a non-imaging surface phase difference method and (b) an image pickup surface phase difference method. It is a flowchart which shows the operation of the image pickup apparatus in 1st Embodiment. It is a flowchart which shows the AF operation in 1st Embodiment. It is a flowchart which shows the focus detection process in 1st Embodiment. It is explanatory drawing of the focal point detection area in 1st Embodiment. It is explanatory drawing of the AF signal (a pair of image signals) in 1st Embodiment. It is explanatory drawing of the relationship between the shift amount and the correlation amount of the AF signal in 1st Embodiment. It is explanatory drawing of the relationship between the shift amount of the AF signal and the correlation change amount in 1st Embodiment. It is a flowchart which shows the selection of the defocus amount in 1st Embodiment. It is a flowchart which shows the transfer determination from the large Def to the medium Def in the 1st embodiment. It is a flowchart which shows the transfer determination from the medium Def to the small Def in the 1st Embodiment. It is a block diagram of the image pickup apparatus in 2nd Embodiment. It is explanatory drawing of the cam locus. It is a flowchart which shows the operation of the image pickup apparatus in 2nd Embodiment. It is a flowchart which shows the AF operation during zooming in 2nd Embodiment. It is a flowchart which shows the target position selection process in 2nd Embodiment. It is a flowchart which shows the target position selection process in 2nd Embodiment. It is explanatory drawing about the focus control during zooming in 2nd Embodiment. It is a flowchart which shows the normal AF operation in 2nd Embodiment. It is a flowchart which shows the normal AF operation in 2nd Embodiment. It is a flowchart which shows the safety MF operation in 2nd Embodiment. It is a flowchart which shows the selection of the defocus amount in 2nd Embodiment. It is a block diagram of the image pickup apparatus in 3rd Embodiment. It is a flowchart which shows the cam locus follow-up process in 3rd Embodiment. It is a flowchart which shows the AF operation during zooming in 3rd Embodiment. It is a flowchart which shows the AF process in 3rd Embodiment. It is explanatory drawing about the focus control during zooming in 3rd Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[First Embodiment]
First, the configuration of the image pickup apparatus according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram of the image pickup apparatus 10 (interchangeable lens camera system) in the present embodiment. The image pickup device 10 includes a camera body 200 (imaging device body) and a lens device 100 (interchangeable lens) that can be attached to and detached from the camera body 200. The lens device 100 is detachably attached (replaceable) to the camera body 200 via a mount portion (not shown) having an electrical contact unit 106. It should be noted that this embodiment can also be applied to an image pickup device in which a lens device and a camera body are integrally configured.

101 is a photographing lens including a zoom mechanism, 102 is a diaphragm and a shutter for controlling the amount of light, 103 is a focus lens for focusing on an image pickup element 201 described later, 104 is a motor for driving the focus lens 103, and 105 is a lens controller. Is. Reference numeral 201 denotes an image pickup element as a light receiving means (photoelectric conversion means) that converts the reflected light from the subject into an electric signal. Reference numeral 202 denotes an A / D conversion unit including a CDS circuit for removing output noise of the image pickup device 201 and a non-linear amplifier circuit performed before A / D conversion. 203 is an image processing unit, 204 is an AF signal processing unit, and 205 is a format conversion unit. In the present embodiment, the AF signal processing unit 204 has an acquisition unit 204a and a calculation unit 204b.

Reference numeral 206 denotes a high-speed built-in memory (DRAM) such as a random access memory (RAM), which is used as a high-speed buffer as a temporary image storage means or as a work memory for compressing and decompressing an image.
Reference numeral 207 is an image recording unit including a recording medium such as a memory card and its interface, and reference numeral 208 is a timing generator. 209 is a system control unit that controls a system such as a shooting sequence, 210 is a lens communication unit that communicates between the camera body 200 and the lens device 100, 211 is an AE processing unit, and 212 is an image display memory (VRAM). Reference numeral 213 is an image display unit that displays a shooting screen and a focus detection area at the time of shooting, in addition to displaying an image, displaying for assisting operation, and displaying a camera state.

Reference numeral 214 denotes an operation unit for the user to operate the camera body 200, a menu switch for making various settings such as a shooting function of the image pickup apparatus 10 and settings for image reproduction, and an operation mode of a shooting mode and a playback mode. Includes changeover switch and so on. Reference numeral 215 is a shooting mode switch for selecting a shooting mode such as a macro mode or a sports mode. Reference numeral 216 is a main switch for turning on the power to the system (imaging device 10). Reference numeral 217 is a switch (SW1) for performing a shooting standby operation such as AF and AE, and 218 is a switch (SW2) for performing shooting after operating the switch SW1.

The image pickup device 201 includes a CCD sensor, a CMOS sensor, or the like, and photoelectrically converts a subject image (optical image) formed via the image pickup optical system of the lens device 100 to output a pixel signal (image data). That is, the luminous flux incident from the image pickup optical system is imaged on the light receiving surface of the image pickup element 201, and is converted into a signal charge according to the amount of incident light by the pixels (photodiodes) arranged in the image pickup element 201. The signal charge accumulated in each photodiode is sequentially read from the image pickup element 201 as a voltage signal corresponding to the signal charge based on the drive pulse output from the timing generator 208 according to the command of the system control unit 209.

Each pixel of the image sensor 201 used in the present embodiment is provided for two (pair) photodiodes A and B and the pair of photodiodes A and B (sharing photodiodes A and B). It is configured to include one microlens. That is, the image pickup device 201 has a pair of photodiodes (first photoelectric conversion unit and second photoelectric conversion unit) for one microlens, and a plurality of microlenses are arranged two-dimensionally. Each pixel divides the incident light with a microlens to form a pair of optical images on the pair of photodiodes A and B, and the pair of photodiodes A and B are used as a pair of AF signals to be described later. The pixel signal (A image signal and B image signal) is output. Further, by adding the outputs of the pair of photodiodes A and B, an imaging signal (A + B image signal) can be obtained.

AF signals (focus detection) used for AF (imaging surface phase difference AF) by the imaging surface phase difference detection method by synthesizing a plurality of A image signals and a plurality of B image signals output from a plurality of pixels. A pair of image signals as a signal) can be obtained. The AF signal processing unit 204, which will be described later, performs a correlation calculation on the pair of image signals, calculates the phase difference (image shift amount) which is the shift amount of the pair of image signals, and further degenerates the image pickup optical system from the image shift amount. Calculate the focus amount (and defocus direction).

As described above, the image pickup element 201 photoelectrically converts the optical image formed by receiving the light flux passing through the image pickup optical system of the lens apparatus 100 into an electric signal, and outputs the image data (image signal). The image pickup device 201 of the present embodiment is provided with two photodiodes for one microlens, and can generate an image signal used for focus detection by the image pickup surface phase difference AF method. The number of photodiodes (divided PDs) sharing one microlens may be changed, such as providing four photodiodes for one microlens.

FIG. 2A schematically shows a configuration of pixels that do not support the imaging surface phase-difference AF method, and FIG. 2B schematically shows a configuration example of pixels that support the imaging surface phase-difference AF method. In both the pixel configurations of FIGS. 2A and 2B, the Bayer arrangement is used, where R is a red color filter, B is a blue color filter, and Gr and Gb are green color filters, respectively. Shows. In the pixel configuration of FIG. 2B corresponding to the imaging surface phase difference AF, the pixel configuration in the pixel configuration not compatible with the imaging surface phase difference AF method shown in FIG. 2A is within one pixel (pixel shown by a solid line). , Two photodiodes A and B divided into two in the horizontal direction in FIG. 2B are provided. The photodiodes A and B (first photoelectric conversion unit and second photoelectric conversion unit) receive light flux that has passed through different pupil regions of the imaging optical system. As described above, since the photodiode A and the photodiode B receive the light flux that has passed through different regions of the exit pupils of the photographing optical system, the B image signal has parallax with respect to the A image signal. Further, of the above-mentioned imaging signal (A + B image signal) and the pair of parallax image signals, one of the image signals (A image or B image signal) also has parallax. The pixel division method shown in FIG. 2B is an example, such as a configuration in which the pixels are divided in the vertical direction in FIG. 2B, a configuration in which the pixels are divided in the horizontal direction and two in the vertical direction (total of four divisions), and the like. Other configurations may be adopted. Further, a plurality of types of pixels divided by different division methods may be included in the same image pickup device. In the present embodiment, a plurality of photoelectric conversion units are arranged for one microlens, and the pupil-divided light flux is incident on each photoelectric conversion unit, but the present invention is limited to this. It's not a thing. For example, the configuration of the focus detection pixel may be such that one PD is provided under the microlens and the pupil is divided by shading the left and right or the top and bottom with a light shielding layer. Further, a pair of focus detection pixels may be discretely arranged in an array of a plurality of image pickup pixels, and a pair of image signals may be acquired from the pair of focus detection pixels.

The image pickup signal and the AF signal read from the image pickup element 201 are input to the A / D conversion unit 202. The A / D conversion unit 202 performs correlation double sampling for removing reset noise, gain adjustment, and digitization of the imaging signal and the AF signal. The A / D conversion unit 202 outputs the image pickup signal to the image processing unit 203, and outputs the AF signal to the AF signal processing unit 204.

The AF signal processing unit 204 (acquisition means 204a) is an AF signal output from the A / D conversion unit 202 (first signal (A image signal) and second signal (B image signal) as a pair of image signals). To get. Further, the AF signal processing unit 204 (calculation means 204b) performs a correlation calculation based on the AF signal to calculate the image shift amount, and calculates the defocus amount based on the image shift amount. Further, the AF signal processing unit 204 (calculation means 204b) calculates reliability information of the AF signal (matching degree of two images, steepness of two images, contrast information, saturation information, scratch information, etc.). The defocus amount and reliability information (reliability) calculated by the AF signal processing unit 204 are output to the system control unit 209. The details of the correlation calculation by the AF signal processing unit 204 will be described later.

Next, the operation of the image pickup apparatus 10 in the present embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing the operation of the image pickup apparatus 10. Each step in FIG. 3 is mainly executed by each unit based on the command of the system control unit 209.

First, in step S301, the system control unit 209 controls the AE processing unit 211 so as to perform AE processing on the output signal of the image processing unit 203. Subsequently, in step S302, the system control unit 209 determines whether or not the switch 217 (SW1) is on. If the switch 217 (SW1) is on, the process proceeds to step S303. On the other hand, when the switch 217 (SW1) is off, the process returns to step S301.

In step S303, the system control unit 209 performs an AF operation. The details of the AF operation will be described later. Subsequently, in step S304, the system control unit 209 determines whether or not the switch 217 (SW1) is on. If the switch 217 (SW1) is on, the process proceeds to step S305. On the other hand, when the switch 217 (SW1) is off, the process returns to step S301.

In step S305, the system control unit 209 determines whether or not the switch 218 (SW2) is on. If the switch 218 (SW2) is off, the process returns to step S304. On the other hand, when the switch 218 (SW2) is on, the process proceeds to step S306. In step S306, the system control unit 209 performs a shooting operation. After that, the process returns to step S301.

Next, with reference to FIG. 4, the AF operation (step S303 in FIG. 3) in the present embodiment will be described in detail. FIG. 4 is a flowchart showing the AF operation. Each step in FIG. 4 is mainly executed by the AF signal processing unit 204 and the system control unit 209.

First, in step S401, the system control unit 209 controls the AE processing unit 211 so as to perform AE processing on the output signal of the image processing unit 203. Subsequently, in step S402, the AF signal processing unit 204 performs focus detection processing using the pair of image signals, and calculates the defocus amount and reliability. The details of the focus detection process will be described later.

Subsequently, in step S403, the system control unit 209 determines whether or not the reliability calculated in step S402 (reliability of the focus detection result) is higher than the preset second reliability threshold value. If the reliability is higher than the second reliability threshold, the process proceeds to step S404. On the other hand, if the reliability is less than the second reliability threshold, the process proceeds to step S413. If the reliability is less than the second reliability threshold, the accuracy of the defocus amount cannot be guaranteed, but the focus position direction of the subject is set to a value that can be guaranteed.

In step S404, the system control unit 209 determines whether or not the defocus amount (detection defocus amount) calculated in step S402 is equal to or less than the preset second Def amount threshold value. If the detected defocus amount is equal to or less than the second Def amount threshold value, the process proceeds to step S405. On the other hand, when the detected defocus amount is larger than the second Def amount threshold value, the process proceeds to step S412. The second Def amount threshold is such that if the defocus amount is equal to or less than the second Def amount threshold, the focus lens 103 is then moved within the depth of focus within a predetermined number of times (for example, 3 times) by driving the lens by the defocus amount. It is set to a controllable value (eg, 5 times the depth of focus).

In step S405, the system control unit 209 determines whether or not the focus lens 103 is in the stopped state (stopped). If the focus lens 103 is stopped, the process proceeds to step S406. On the other hand, if the focus lens 103 is not stopped, the process proceeds to step S410.

In step S406, the system control unit 209 determines whether or not the reliability calculated in step S402 (reliability of the focus detection result) is higher than the preset first reliability threshold value. If the reliability is higher than the first reliability threshold, the process proceeds to step S407. On the other hand, if the reliability is less than the first reliability threshold, the process proceeds to step S410. The first reliability threshold is set so that the accuracy variation of the defocus amount is within a predetermined range (for example, within the depth of focus) when the reliability is equal to or higher than the first reliability threshold.

In step S407, the system control unit 209 determines whether or not the defocus amount (detection defocus amount) calculated in step S402 is equal to or less than the preset first Def amount threshold value. If the detected defocus amount is equal to or less than the first Def amount threshold value, the process proceeds to step S408. On the other hand, when the detected defocus amount is larger than the first Def amount threshold value, the process proceeds to step S409. The first Def amount threshold is set so that the focus lens 103 is controlled within the depth of focus if the detected defocus amount is equal to or less than the first Def amount threshold.

In step S408, the system control unit 209 determines that the focal state is in the focused state, and ends this flow. On the other hand, in step S409, the system control unit 209 drives the focus lens 103 by the defocus amount (detected defocus amount) calculated in step S402, and then returns to step S402. By performing a series of operations in steps S405 to S409, when the reliability calculated in step S402 is higher than the first reliability threshold value, the system control unit 209 defocuses again with the focus lens 103 stopped. The amount can be calculated.

In step S410, the system control unit 209 drives the focus lens 103 by a predetermined ratio with respect to the defocus amount detected in step S402. Subsequently, in step S411, the system control unit 209 instructs the stop of the focus lens 103 to return to step S402.

In step S412, the system control unit 209 drives the focus lens 103 by a predetermined ratio with respect to the defocus amount detected in step S402, and returns to step S402. The predetermined ratio is set so that the drive amount (lens drive amount) of the focus lens 103 is smaller than the defocus amount (for example, 80%). Further, the lens speed to be set is set to be slower than the speed that can be driven by the set driving amount in, for example, one frame time. As a result, if the detected defocus amount is incorrect, it is possible to avoid exceeding the focus position of the subject. Further, it is possible to drive the next lens while driving the focus lens 103 without stopping (overlap control).

In step S413, the system control unit 209 determines whether or not the out-of-focus condition is satisfied. If the out-of-focus condition is satisfied, the process proceeds to step S414. On the other hand, if the out-of-focus condition is not satisfied, the process proceeds to step S415. The out-of-focus condition is a condition in which the system control unit 209 determines that there is no subject to be focused. The out-of-focus condition is, for example, the case where the lens drive is completed in the entire movable range of the focus lens 103, that is, the case where the focus lens 103 detects both the far side and the near side lens ends and returns to the initial position. Conditions are set.

In step S414, the system control unit 209 determines that the focal state is out of focus, and ends this flow. On the other hand, in step S415, the system control unit 209 determines whether or not the focus lens 103 has reached the far end or the near end of the lens. If the focus lens 103 reaches the lens end, the process proceeds to step S416, while if the focus lens 103 does not reach the lens end, the process proceeds to step S417.

In step S416, the system control unit 209 reverses the drive direction (lens drive direction) of the focus lens 103 and returns to step S402. In step S417, the system control unit 209 drives the focus lens 103 in a predetermined direction and returns to step S402. The drive speed (focus lens speed) of the focus lens 103 is set to, for example, the fastest speed within the range of the lens speed that does not pass the focus position when the defocus amount can be detected.

Next, with reference to FIG. 5, the focus detection process (step S402 in FIG. 4) will be described in detail. FIG. 5 is a flowchart showing the focus detection process. Each step of FIG. 5 is mainly executed by the system control unit 209 or the AF signal processing unit 204 based on the command of the system control unit 209.

First, in step S501, the AF signal processing unit 204 (system control unit 209) sets a focus detection region in an arbitrary range in the image pickup device 201. Subsequently, in step S502, the AF signal processing unit 204 acquires a pair of image signals (A image signal and B image signal) for focus detection from the image pickup device 201 with respect to the focus detection region set in step S501. Subsequently, in step S503, the AF signal processing unit 204 performs row addition averaging processing in the vertical direction with respect to the pair of image signals acquired in step S502. The row addition averaging process can reduce the influence of noise in the image signal. Subsequently, in step S504, the AF signal processing unit 204 performs filter processing for extracting signal components in a predetermined frequency band from the pair of image signals subjected to row addition averaging processing in step S503.

Subsequently, in step S505, the AF signal processing unit 204 calculates the correlation amount based on the pair of image signals filtered in step S504. Subsequently, in step S506, the AF signal processing unit 204 calculates the correlation change amount based on the correlation amount calculated in step S505. Subsequently, in step S507, the AF signal processing unit 204 calculates the image shift amount based on the correlation change amount calculated in step S505. Subsequently, in step S508, the AF signal processing unit 204 calculates the reliability of the image shift amount calculated in step S507. Subsequently, in step S509, the AF signal processing unit 204 converts the image shift amount into the defocus amount.

Subsequently, in step S510, the system control unit 209 determines whether or not the filter processing performed in step S504 has been calculated for each type of filter. If the calculation of the filter processing is completed by the number of types of the filter, the process proceeds to step S511. On the other hand, if the calculation of the filter processing is not completed for the types of filters, the process returns to step S504. In the present embodiment, in the filter processing of step S504, for example, a bandpass filter having three frequency bands (low frequency band, mid frequency band, and high frequency band) having different bands is applied to a pair of image signals subjected to row addition averaging processing. Perform the processing (filter processing) used in the horizontal direction. However, the low frequency filter, the mid frequency filter, and the high frequency filter indicate the relative high and low of the frequency components extracted by each filter, and do not indicate the absolute high and low. In step S510, the system control unit 209 determines whether or not the series of processes of steps S504 to S509 has been performed for all three frequency bands.

In step S511, the system control unit 209 selects the defocus amount. That is, the system control unit 209 (determining means) uses any combination of the defocus amount and the reliability among the combinations of the three defocus amounts and the reliability calculated by the series of processes of steps S504 to S509. Select (decide) whether or not. In the present embodiment, the defocus amount calculated by using the low frequency filter is a large Def, the defocus amount calculated by using the mid frequency filter is a medium Def, and the defocus amount calculated by using the high frequency filter is the defocus amount. Let it be a small Def. The selection of the defocus amount will be described later.

Next, with reference to FIG. 6, the focus detection region (AF region) set in step S501 of FIG. 5 will be described in detail. FIG. 6 is an explanatory diagram of a focus detection region 602 on the pixel array 601 of the image sensor 201. The shift regions 603 on both sides of the focus detection region 602 are regions necessary for the correlation calculation. Therefore, the area 604 in which the focus detection area 602 and the shift area 603 are combined is a pixel area required for the correlation calculation. In FIG. 6, p, q, s, and t are the coordinates in the horizontal direction (x-axis direction), respectively, p and q are the x-coordinates of the start point and the end point of the area 604 (pixel area), respectively, and s and t are the coordinates. The x-coordinates of the start point and the end point of the focus detection region 602 are shown, respectively.

FIG. 7 is an explanatory diagram of AF signals (a pair of image signals) acquired from a plurality of pixels included in the focus detection region 602 of FIG. In FIG. 7, the solid line 701 is one of the pair of image signals (A image signal), and the broken line 702 is the other of the pair of image signals (B image signal). 7 (a) shows the A image signal and the B image signal before the shift, and FIGS. 7 (b) and 7 (c) show the A image signal and the B image signal in the positive direction and from the state of FIG. 7 (a), respectively. It shows the state of shifting in the negative direction. When calculating the correlation amount of a pair of image signals (A image signal shown by the solid line 701 and B image signal shown by the broken line 702), both the A image signal and the B image signal are bit by 1 bit in the direction of the arrow. shift.

Next, a method of calculating the correlation amount will be described. First, as shown in FIGS. 7 (b) and 7 (c), the A image signal and the B image signal are shifted by 1 bit each, and the sum of the absolute values of the differences between the A image signal and the B image signal is calculated. do. The correlation amount COR [i] can be calculated using the following equation (1).

In equation (1), i is the shift amount, ps is the maximum shift amount in the minus direction, q−t is the maximum shift amount in the plus direction, x is the start coordinate of the focus detection region 602, and y is the focus detection region 602. The end coordinates.

Here, the relationship between the shift amount and the correlation amount COR will be described with reference to FIG. 8 (a) and 8 (b) are explanatory diagrams of the relationship between the shift amount and the correlation amount COR. 8 (b) is an enlarged view of the region 802 of FIG. 8 (a). In FIG. 8A, the horizontal axis represents the shift amount and the vertical axis represents the correlation amount COR. Of the regions 802 and 803 near the extreme value in the correlation amount 801 that changes with the shift amount, the degree of coincidence of the pair of image signals (A image signal, B image signal) is the highest in the shift amount corresponding to the smaller correlation amount. ..

Next, a method of calculating the amount of correlation change will be described. In the present embodiment, the difference in the correlation amount every other shift in the waveform of the correlation amount 801 shown in FIG. 8A is calculated as the correlation change amount. The amount of correlation change ΔCOR [i] can be calculated using the following equation (2).

Here, with reference to FIG. 9, the relationship between the shift amount and the correlation change amount ΔCOR will be described. 9 (a) and 9 (b) are explanatory diagrams of the relationship between the shift amount and the correlation change amount ΔCOR. 9 (b) is an enlarged view of the region 902 of FIG. 9 (a). In FIG. 9A, the horizontal axis represents the shift amount and the vertical axis represents the correlation change amount ΔCOR. The correlation change amount 901 that changes with the shift amount changes from plus to minus in the regions 902 and 903. The state in which the amount of correlation change is 0 is called zero cross, and the degree of coincidence between the pair of image signals (A image signal and B image signal) is the highest. Therefore, the shift amount that gives zero cross is the image shift amount.

In FIG. 9B, 904 is a part of the correlation change amount 901. A method of calculating the image shift amount PRD will be described with reference to FIG. 9B. The shift amount (k-1 + α) that gives a zero cross is divided into an integer part β (= k-1) and a decimal part α. The fractional part α can be calculated using the following equation (3) from the similarity relationship between the triangle ABC and the triangle ADE in FIG. 9 (b).

The integer part β can be calculated from FIG. 9B using the following equation (4).

That is, the image shift amount PRD can be calculated from the sum of the decimal part α and the integer part β. As shown in FIG. 9A, when there are a plurality of zero crosses of the correlation change amount ΔCOR, the one having a larger change in the correlation change amount ΔCOR in the vicinity thereof is defined as the first zero cross. This steepness is an index showing the ease of performing AF, and the larger the value, the easier it is to perform high-precision AF. The steepness maxder can be calculated using the following equation (5).

In the present embodiment, when there are a plurality of zero crosses of the correlation change amount, the first zero cross is determined based on the steepness, and the shift amount giving the first zero cross is defined as the image shift amount.

Next, a method of calculating the reliability information (reliability) of the image shift amount will be described. The reliability of the image shift amount can be defined by the degree of coincidence (the degree of coincidence of two images) fnclvl of a pair of image signals (A image signal, B image signal) and the steepness of the above-mentioned correlation change amount ΔCOR. , The reliability may be obtained from any value.
The degree of coincidence between the two images is an index showing the accuracy of the amount of image shift, and here, the smaller the value, the better the accuracy. In FIG. 8B, 804 is a part of the correlation amount 801. The degree of coincidence fnclvl of the two images can be calculated using the following equation (6).

Next, the selection of the defocus amount (step S511 in FIG. 5) will be described with reference to FIG. FIG. 10 is a flowchart showing the selection of the defocus amount. Each step in FIG. 10 is mainly executed by the system control unit 209.

First, in step S1001, the system control unit 209 determines the takeover from the large Def (first defocus amount) to the medium Def (second defocus amount), which will be described later. Subsequently, in step S1002, the system control unit 209 determines whether or not the transfer from the large Def to the medium Def could be performed. If the transfer from the large Def to the medium Def can be performed, the process proceeds to step S1003. On the other hand, if the transfer from the large Def to the medium Def cannot be performed, the process proceeds to step S1007.

In step S1003, the system control unit 209 determines the takeover from the medium def (second defocus amount) to the small def (third defocus amount), which will be described later. Subsequently, in step S1004, the system control unit 209 determines whether or not the transfer from the medium Def to the small Def has been possible. If the transfer from the medium Def to the small Def can be performed, the process proceeds to step S1005. On the other hand, if the transfer from the medium Def to the small Def cannot be performed, the process proceeds to step S1006.

In step S1005, the system control unit 209 selects a small def as the defocus amount used for driving the focus lens 103 (focus control), and ends this flow. Further, in step S1006, the system control unit 209 selects medium Def as the defocus amount used for focus control, and ends this flow. Further, in step S1007, the system control unit 209 selects a large def as the defocus amount used for the focus control, and ends this flow.

Next, with reference to FIG. 11, the takeover determination from the large Def to the medium Def (step S1001 in FIG. 10) will be described. Each step in FIG. 11 is mainly executed by the system control unit 209.

First, in step S1101, the system control unit 209 determines whether or not the difference between the defocus amounts of the large Def and the medium Def is equal to or less than the preset first depth threshold value (first threshold value). When the difference between the defocus amount of the large Def and the defocus amount of the medium Def is equal to or less than the first depth threshold value, the process proceeds to step S1102. On the other hand, if this difference is larger than the first depth threshold value, the process proceeds to step S1104. The first depth threshold is set to, for example, nine times the depth of focus so that the large Def can be appropriately inherited from the medium Def. Further, by setting the first depth threshold value with reference to the depth of focus (making it larger than the depth of focus), a uniform threshold value can be set even if the F value or the focus detection area changes.

In step S1102, in the system control unit 209, the reliability of the medium Def (second reliability) is equal to or higher than the reliability of the large Def (first reliability), and the reliability of the medium Def is the second reliability threshold value. It is determined whether or not it is higher than (threshold value used in the determination in step S403 of FIG. 4). If both of these conditions are satisfied, the process proceeds to step S1103. On the other hand, if at least one of the conditions is not satisfied, the process proceeds to step S1104.

In step S1103, the system control unit 209 determines that the transfer from the large Def to the medium Def can be performed, and ends this flow. On the other hand, in step S1104, the system control unit 209 determines that the transfer from the large Def to the medium Def cannot be performed, and ends this flow. As a result, in the process of moving the focus lens 103 from the large blur state to the small blur state, the transfer from the large Def to the medium Def is performed based on the difference in the defocus amount between the large Def and the medium Def and the reliability of each. It is possible to determine whether or not it is possible to do so.

Next, with reference to FIG. 12, the takeover determination from the medium Def to the small Def (step S1003 in FIG. 10) will be described. Each step in FIG. 12 is mainly executed by the system control unit 209.

First, in step S1201, the system control unit 209 determines whether or not the difference between the defocus amounts of the medium Def and the small Def is equal to or less than the preset second depth threshold value (second threshold value). When the difference between the defocus amount of the medium Def and the small Def is equal to or less than the second depth threshold value, the process proceeds to step S1202. On the other hand, if this difference is larger than the second depth threshold value, the process proceeds to step S1204. The second depth threshold is set to, for example, three times the depth of focus so that the medium Def can be appropriately inherited from the small Def. Further, by setting the second depth threshold value with reference to the depth of focus (making it larger than the depth of focus), a uniform threshold value can be set even if the F value or the focus detection area changes. Further, the second depth threshold value is set to a value smaller than the first depth threshold value used in the determination in step S1101 of FIG. This is because the detection variation of the defocus amount increases as the change from small Def → medium Def → large Def, and as a result, the difference between the medium Def and the large Def becomes larger than the difference between the small Def and the medium Def. be.

In step S1202, in the system control unit 209, the reliability of the small Def is equal to or higher than the reliability of the medium Def, and both the reliability of the small Def and the reliability of the medium Def are higher than the second reliability threshold. Judge whether or not. When all of these conditions are satisfied, the process proceeds to step S1203. On the other hand, if at least one condition is not satisfied, the process proceeds to step S1204.

In step S1203, the system control unit 209 determines that the transfer from the medium Def to the small Def can be performed, and ends this flow. On the other hand, in step S1204, the system control unit 209 determines that the transfer from the medium Def to the small Def cannot be performed, and ends this flow. As a result, in the process of moving the focus lens 103 from the small blur state to the in-focus position, the transfer from the medium Def to the small Def is performed based on the difference in the defocus amount between the medium Def and the small Def and the reliability of each. It is possible to determine whether or not it is possible to do so.

As described above, in the present embodiment, the control device has an AF signal processing unit 204 (acquisition unit 204a, calculation unit 204b) and a determination unit (system control unit 209). The acquisition means acquires a first signal (A image signal) and a second signal (B image signal) corresponding to light fluxes passing through different pupil regions of the imaging optical system. The calculation means performs a plurality of filtering processes (bandpass filtering processing) on the first signal and the second signal having different bands from each other, and a plurality of outputs based on the first signal and the second signal after each filtering processing. Calculate the amount of focus and multiple reliability. The determining means determines the defocus amount used for focus adjustment (focus control) from among the plurality of defocus amounts based on the difference between the plurality of defocus amounts and at least one of the plurality of reliabilitys.

Preferably, the plurality of filtering processes are the first filtering process for extracting the first signal and the second signal in the first frequency band (low frequency band) and the second frequency band (middle frequency band) higher than the first frequency band. It includes a second filter process for extracting a first signal and a second signal. The calculation means has a first defocus amount (large Def) and a first reliability (large Def) based on the first signal and the second signal after the first filter processing as a plurality of defocus amounts and a plurality of reliabilitys. Reliability) is calculated. Further, the calculation means calculates the second defocus amount (medium Def) and the second reliability (medium Def reliability) based on the first signal and the second signal after the second filter processing. More preferably, when the difference between the first defocus amount and the second defocus amount is larger than the first threshold value (first depth threshold value), the calculation means uses the first defocus amount for focus adjustment. It is determined as the focus amount (S1101, S1104). More preferably, the calculation means is used when the second reliability is higher than the first reliability and the second reliability is higher than the predetermined reliability threshold (second reliability threshold). The second defocus amount is determined as the defocus amount used for focus adjustment. On the other hand, the calculation means has a first defocus amount when the second reliability is lower than the first reliability or when the second reliability is lower than a predetermined reliability threshold (second reliability threshold). Is determined as the amount of defocus used for focus adjustment (S1102-S1104).

Preferably, the plurality of filtering processes include a third filtering process for extracting the first signal and the second signal in the third frequency band (high frequency band) higher than the second frequency band. The calculation means has a third defocus amount (small Def) and a third reliability (small Def) based on the first signal and the second signal after the third filter processing as a plurality of defocus amounts and a plurality of reliabilitys. Reliability) is calculated. More preferably, when the difference between the second defocus amount and the third defocus amount is larger than the second threshold value (second depth threshold value), the calculation means uses the second defocus amount for focus adjustment. It is determined as the focus amount (S1201, S1204). More preferably, the calculation means is a case where the third reliability is higher than the second reliability, and each of the second reliability and the third reliability has a predetermined reliability threshold value (second reliability threshold). ), The third defocus amount is determined as the defocus amount used for focus adjustment. On the other hand, when the third reliability is lower than the second reliability, or when the second reliability or the third reliability is lower than the predetermined reliability threshold, the calculation means determines the second defocus amount. It is determined as the amount of defocus used for focus adjustment (S1202 to S1204). More preferably, the second threshold is smaller than the first threshold. Further, preferably, the first threshold value and the second threshold value are set with reference to the depth of focus.

In the present embodiment, in the process of moving the focus lens from the large out-of-focus state to the in-focus position, the defocus amount that can determine the direction of the in-focus position in the large out-of-focus state and the high value in the vicinity of the in-focus position. Use the precision defocus amount. Therefore, according to the present embodiment, it is possible to provide a control device, an image pickup device, a control method, a program, and a storage medium capable of improving the focusing accuracy and shortening the focus adjustment time.

In the present embodiment, the system control unit 209 determines the defocus amount used for the focus adjustment by using both the difference in the defocus amount and the reliability of the defocus amount, but only one of them is used. The amount of defocus used for focus adjustment may be determined. That is, in the transfer determination from the large Def to the medium Def, it may be determined whether to use the large Def or the medium Def by omitting step S1101 or step S1102. Similarly, in the transfer determination from the medium Def to the small Def, it may be determined whether to use the medium Def or the small Def by omitting step S1201 or step S1202.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.

When the subject has low contrast or low brightness, the focusing accuracy is improved by adding multiple frames and moving the lens multiple times, but the focusing time becomes longer. During normal shooting, it is desirable to focus at low contrast or low brightness even if the AF time is long, but it is desirable to prioritize shortening the time to focus rather than focusing accuracy. In some cases. For example, as a manual focus (MF) mode in which the user can manually specify the focus lens position for shooting, a function (safety) that assists focusing by performing AF control in a minute range including the focus lens position specified by the user. There is MF). Since this function is in MF mode, it is desirable to prevent the shooting time lag from becoming long due to AF control.

In addition, when zooming from wide to tele or tele to wide heading, the zoom lens and focus lens are driven and controlled so as to trace the cam locus according to the subject distance, thereby obtaining a good image without out-of-focus. Can be done. However, as shown in FIG. 14, the cam locus for each subject distance becomes denser as it approaches the wide position. Therefore, when zooming from wide to tele, it is not possible to accurately select the cam locus corresponding to the subject distance, and the cam locus selected first may deviate from the cam locus corresponding to the subject distance. As a result, blurring occurs due to zooming. Further, there is known a method of adding an image signal until a predetermined image signal value exceeds a threshold value when detecting a defocus amount by the imaging surface phase difference AF method. However, in this method, until focusing is achieved. The time will be long.

Therefore, in the second to fourth embodiments, the image pickup apparatus capable of improving the focusing accuracy and shortening the AF time by making the AF control different from that in the normal shooting mode during a specific shooting mode or zoom operation. The purpose is to provide. In the second embodiment, an image pickup device in which the lens barrel and the image pickup device are integrated will be described.

<Overall configuration>
First, with reference to FIG. 13, an example of an image pickup device with a lens barrel tied up in the present embodiment is shown. FIG. 13 is a block diagram of the image pickup apparatus 1000 according to the present embodiment. As shown in FIG. 13, the image pickup apparatus 1000 bound to the lens barrel is configured around the lens barrel and the camera control unit 1114 that controls the operation of the entire image pickup apparatus. Further, each operation of the image pickup apparatus 1000 is executed based on a control program stored in an internal ROM, a RAM (not shown), and various data necessary for control.

First, the configuration of the part related to the lens barrel will be described. The lens barrel includes a zoom lens 1101, an aperture 1102, a focus lens 1103, a zoom lens drive unit 1104, an aperture drive unit 1105, and a focus lens drive unit 1106. The zoom lens 1101 is driven by the zoom lens driving unit 1104 to change the focal length. The diaphragm 1102 is driven by the diaphragm drive unit 1105 and controls the amount of light incident on the image pickup device 1107, which will be described later. The focus lens 1103 is driven by the focus lens driving unit 1106, and adjusts the focus formed on the image pickup device 1107, which will be described later. The zoom lens drive unit 1104, the aperture drive unit 1105, and the focus lens drive unit 1106 are controlled by the camera control unit 114 to determine the position of the zoom lens 1101, the opening amount of the aperture 1102, and the position of the focus lens 1103. .. The camera control unit 1114 obtains lens control information from the zoom lens drive unit 1104, the aperture drive unit 1105, and the focus lens drive unit 1106.

Next, the configuration of the part related to the image pickup function that acquires the image pickup signal from the light flux passing through the photographing optical system will be described. The image pickup device 1000 includes an image pickup element 1107, a CDS / AGC circuit 1108, a camera signal processing unit 1109, an AF signal processing unit 1110, a display unit 1111 and a recording unit 1112, a camera control unit 1114, and a camera operation unit 1115. Further, the image pickup apparatus 1000 has a shooting mode switch 1116 for selecting a shooting mode such as a safety MF mode, a switch 1117 (SW1) for performing shooting standby operation such as AF and AE, and a shooting switch 1118 for shooting after operating SW1. It has (SW2). Further, the image pickup apparatus 1000 has an AE processing unit 1119.

The image pickup device 107 is a member as an image sensor, and is composed of a CCD, a CMOS sensor, or the like. The luminous flux that has passed through the photographing optical system of the lens barrel is imaged on the light receiving surface of the image pickup element 107, and is converted into a signal charge according to the amount of incident light by the photodiode. The signal charge accumulated in each photodiode is sequentially read out from the image pickup element 1107 as a voltage signal corresponding to the signal charge based on the drive pulse given from the timing generator 1113 according to the command of the camera control unit 1114. The video signal and AF signal read from the image pickup element 107 are sampled and input to the CDS / AGC circuit 1108 for gain adjustment, and the video signal is sent to the camera signal processing unit 1109 and the image pickup surface phase difference AF signal is sent to the camera signal processing unit 1109. It is output to the AF signal processing unit 1110, respectively.

The camera signal processing unit 1109 performs various image processing on the signal output from the CDS / AGC circuit 1108 to generate a video signal. The display unit 1111 configured by the LCD or the like displays the video signal output from the camera signal processing unit 1109 as an captured image. The recording unit 1112 records the video signal from the camera signal processing unit 1109 on a recording medium such as a magnetic tape, an optical disk, or a semiconductor memory.

The AF signal processing unit 1110 performs a correlation calculation based on the two image signals for AF output from the CDS / AGC circuit 1108, and performs a defocus amount and reliability information (two-image coincidence, two-image steepness, contrast). Information, saturation information, scratch information, etc.) are calculated. Then, the AF signal processing unit 1110 outputs the calculated defocus amount and reliability information to the camera control unit 1114. Further, the camera control unit 114 notifies the AF signal processing unit 110 of a change in the setting for calculating these based on the acquired defocus amount and reliability information. The details of the defocus amount calculation are as described in the first embodiment with reference to FIGS. 5 to 9.

The camera control unit 1114 exchanges information with the entire image pickup apparatus 1000 to perform control. Not only the processing in the image pickup device 1000, but also the user operates such as turning on / off the power, changing the setting, starting recording, starting AF control, checking the recorded image, etc. according to the input from the camera operation unit 1115. Performs various camera functions.

<Reliability of focus detection result>
Next, the reliability of the focus detection result in this embodiment will be described. Similar to the first embodiment, the present embodiment also obtains the reliability from at least one of the concordance degree fnclvl of the two images and the steepness of the correlation change amount ΔCOR. In this embodiment, the reliability is calculated by the AF signal processing unit 1110 in FIG. Basically, the reliability is high when it can be judged that the calculated defocus amount is reliable, and the reliability decreases as it becomes unreliable. The reliability in this embodiment is expressed by a numerical value from 1 to 4, where 1 is the highest reliability and 4 is the lowest reliability. The details of each reliability are as follows.

When the reliability is "1", the contrast between the A image signal and the B image signal is high, and the shapes of the A image signal and the B image signal are similar (the two-image coincidence level is high), or the case is already concerned. This is the case when the subject image is in focus. In this case, the drive is performed with confidence in the defocus amount.

When the reliability is "2", the contrast between the A image signal and the B image signal is high, although the reliability is not as high as "1", and the shapes of the A image signal and the B image signal are similar, or already. The state in which the subject image is located near the in-focus range within a certain error range is shown. In this case, the target position is determined and driven based on the defocus amount.

When the reliability is "3", the correlation obtained by relatively shifting the A image signal and the B image signal is obtained even though the two-image matching degree level calculated by the AF signal processing unit 110 is lower than the predetermined value. There is a certain tendency, and the defocus direction is in a reliable state. For example, it is often determined that the main subject is slightly out of focus.

When the defocus amount and the defocus direction are also unreliable, the reliability is determined to be "4". For example, the contrast between the A image signal and the B image signal is low, and the image matching level is also low. This is often the case when the subject is largely out of focus, and it is difficult to calculate the defocus amount. In this embodiment and the third embodiment described later, the reliability "3" is set as the second reliability threshold value of the first embodiment. In other words, when the reliability is 1 or 2, it is determined that the reliability is larger than the second reliability threshold value, and the takeover determination of the large Def, the medium Def, and the small Def is performed.

Next, the operation of the image pickup apparatus 1000 will be described in detail with reference to FIGS. 15 to 21. FIG. 15 is a flowchart illustrating the operation of the image pickup apparatus 1000. First, in step S1601, the AE processing unit 1119 performs AE processing. Subsequently, in step S1602, the camera control unit 1114 performs the AF operation during zooming, which will be described later. Subsequently, in step S1603, the camera control unit 1114 determines the state of SW1 (117). If the state of SW1 is ON, the process proceeds to step S1604. On the other hand, if the state of SW1 is not ON, the process proceeds to step S1601.

In step S1604, the camera control unit 1114 determines the state of the shooting mode SW116. If the shooting mode is the safety MF mode (second mode), the process proceeds to step S1606. On the other hand, if the shooting mode is not the safety MF mode, the process proceeds to step S1605. In step S1605, the camera control unit 1114 performs a normal AF operation described later (normal mode, first mode). In step S1606, the camera control unit 1114 performs a safety MF operation described later. Here, the normal mode (first mode) is a mode in which the focus control means (camera control unit 1114) performs focus control over the entire movable range of the focus lens. The safety MF mode (second mode) is a mode in which focus control is performed within a predetermined range based on the focus lens position set by the user.

In step S1607, the camera control unit 1114 determines the state of SW1 (117). When the state of SW1 is ON, the process proceeds to step S1608. On the other hand, if the state of SW1 is not ON, the process returns to step S1601. In step S1608, the camera control unit 1114 determines the state of SW2 (118). If the state of SW2 is ON, the process proceeds to step S1609. On the other hand, if the state of SW2 is not ON, the process returns to step S1607. In step S1609, the camera control unit 1114 performs a shooting operation and returns to step S1601.

Next, with reference to FIG. 16, the AF operation during zooming in step S1602 of FIG. 15 will be described. FIG. 16 is a flowchart showing the AF operation during zooming. This process is executed according to the computer program stored in the ROM in the camera control unit 1114. For example, it is executed in the reading cycle (every vertical synchronization period) of the image pickup signal from the image pickup element 1107 for generating a one-field image (hereinafter, also referred to as one frame and one screen). Further, it may be repeated for each of a plurality of vertical synchronization periods (V rates).

First, in step S1701, the AF signal processing unit 1110 confirms whether the AF signal has been updated. If the AF signal has been updated, the process proceeds to step S1702, and the AF signal processing unit 11110 acquires the result (defocus amount and reliability). On the other hand, if the AF signal is not updated in step S1701, the present process is terminated as it is.

Subsequently, in step S1703, the camera control unit 1114 determines whether or not the zoom lens 101 is moving. Specifically, when the photographer operates the zoom operation lever or the like on the camera operation unit 1115, the camera control unit 1114 that detects the operation drives the zoom lens 1101. Therefore, the camera control unit 1114 grasps the moving state of the zoom lens 1101. If the zoom lens 101 is not moving in step S1703, the process proceeds to the process of selecting the target position when the zoom is stopped in step S1704. On the other hand, when the zoom lens 1101 is moving in step S1703, the process proceeds to the zooming target position selection process in step S1705. Both the processes of step S1704 and step S1705 are processes for selecting the target position when moving the focus lens 1103 at the current control timing, and the details will be described later.

If the zoom lens 1101 is moving, the process proceeds to step S1706 after step S1705. In step S1706, the camera control unit 1114 predicts the focus lens position at the next control timing based on the target distance selected in step S1705. Specifically, as shown by the cam locus at each subject distance in FIG. 14, in order to maintain the in-focus state at a certain subject distance, the cam locus at the corresponding subject distance is traced according to the change in the focal length. It is necessary to control the focus lens 1103 so as to. Therefore, in step S1706, the camera control unit 1114 cams at which position the focus lens should be controlled at the next control timing in order to maintain the state in which the target position selected in step S1705 is in focus. Determined based on the trajectory. In this embodiment, since it is premised that the AF control process is executed every vertical synchronization period, the focus lens position advanced by the amount corresponding to that time is specified from the cam locus. Further, the cam locus information in the present embodiment is stored in the ROM in the camera control unit 1114, and the cam locus of the subject distance not held in the ROM can be obtained by interpolation from the adjacent cam loci. .. Subsequently, in step S1707, the camera control unit 1114 controls the focus lens drive unit 1106 and moves the focus lens 1103 with respect to the selected and determined target position.

<Target position selection process when zoom is stopped>
Next, the zoom stop target position selection process in step S1704 of FIG. 16 will be described with reference to FIGS. 17A and 18A. FIG. 17A is a flowchart showing a target position selection process when the zoom is stopped, which is executed by the camera control unit 1114. Each step of FIG. 17A is executed according to a computer program stored in the ROM in the camera control unit 1114, similarly to the AF operation during zooming.

First, in step S1801a of FIG. 17A, the camera control unit 1114 determines whether or not the acquired reliability is "1" based on the reliability acquired in S1702 of FIG. If the reliability is "1" in step S1801a, the process proceeds to step S1806a. On the other hand, if the reliability is not "1", the process proceeds to step S1802a. When the reliability is "1", the accuracy of the detected defocus amount is high, and the focus can be achieved by controlling the focus lens to the target position calculated from the defocus amount. Therefore, in step S1806a, the camera control unit 1114 sets the focus lens position (detection position) calculated based on the detected defocus amount and the current focus lens position as the target position. In this embodiment, the method of moving the focus lens 1103 directly to the detection position is referred to as target drive (first drive). After that, in step S1810a, the search flag is set to OFF and the process ends. In the present embodiment, the target position described later is set to the infinite end or the closest end of the range of motion of the focus lens 1103, and the movement to specify the in-focus position while moving the focus lens 1103 greatly is search drive (second drive). The flag indicating the execution state is called a search flag.

Subsequently, in step S1802a, the camera control unit 1114 determines whether or not the acquired reliability is "2". If the reliability is "2" in step S1802a, the process proceeds to step S1807a. On the other hand, if the reliability is not "2", the process proceeds to step S1803a. When the reliability is "2", it means that the accuracy of the detected defocus amount includes a certain error. Therefore, in step S1807a, the camera control unit 1114 sets a value obtained by multiplying the detected position calculated based on the detected defocus amount and the current focus lens position by the coefficient γ as the target position. The coefficient γ is a numerical value less than 1, and γ assumed in this embodiment is 0.8. Therefore, in step S1807a, a position 80% of the detection position is set as the target position. In this embodiment, the method of driving the focus lens 1103 so as not to reach the detection position (in other words, not to go too far) is called defocus drive (third drive). After that, in step S1810a, the search flag is set to OFF and the process ends.

Subsequently, in step S1803a, the camera control unit 1114 determines whether or not the acquired reliability is "3". If the reliability is "3" in step S1803a, the process proceeds to step S1808a. On the other hand, if the reliability is not "3", the process proceeds to step S1804a. When the reliability is "3", the accuracy of the detected defocus amount itself is low, but the defocus direction indicates a reliable state. Therefore, in step S1808a, the camera control unit 1114 sets the detected end in the defocus direction as the target position and executes the search drive. Then, in the following step S1811a, the search flag is set to ON and the process ends. In step S1804a, the camera control unit 1114 determines whether or not the search flag is OFF. If the search flag is OFF in step S1804a, the process proceeds to step S1809a. On the other hand, when the search flag is ON, the process proceeds to step S1805a.

When the reliability is not "3" in step S1803a, it means that the reliability is "4" and the defocus amount and the defocus direction are also unreliable. In this case, it is difficult to specify the in-focus position only with the information obtained from the AF signal processing unit 1110. Therefore, in step S1809a, the camera control unit 1114 sets the end on the wide driving range side as the target position based on the positional relationship between the current focus lens position and the end on the infinite side and the end on the closest side of the focus lens 1103. Then, search drive is performed. Then, in the following step S1811a, the search flag is set to ON and the process ends.

In step S1805a, the camera control unit 1114 determines whether or not the focus lens 1103 has reached either the infinite end or the near end. When any end is reached in step S1805a, the process proceeds to step S1809a and the target position is set again. As described above, since the end on the wide driving range side is set as the target position, the target position is set on the opposite end when the end is reached. On the other hand, if the end is not reached in step S1805a, the process is terminated as it is and the search drive is continued.

As described above, in the zoom stop target position selection process in the present embodiment, as shown in FIG. 18A, when the reliability is high, the target position is set based on the defocus amount. On the other hand, when the reliability is low, the target position is set at the end of the movable range of the focus lens 103. Therefore, as for the movement of the entire AF, the lower the reliability, the larger the focus fluctuation and the movement to specify the in-focus position.

<Target position selection process when zooming>
Next, with reference to FIGS. 17 (B) and 18 (B), the target position selection process when the zoom is stopped in step S1705 of FIG. 16 will be described. FIG. 17B is a flowchart showing a zoom target position selection process executed by the camera control unit 1114. Each step of FIG. 17B is executed according to the computer program stored in the ROM in the camera control unit 1114, similarly to the AF operation during zooming.

First, in step S1801b of FIG. 17B, the camera control unit 1114 determines whether or not the acquired reliability is "1" based on the reliability acquired in S1702 of FIG. If the reliability is "1" in step S1801b, the process proceeds to step S1804b. On the other hand, if the reliability is not "1", the process proceeds to step S1802b. When the reliability is "1", the accuracy of the detected defocus amount is high, and the focus can be achieved by controlling the focus lens to the target position calculated from the defocus amount. Therefore, in step S1804b, the camera control unit 1114 sets the focus lens position (detection position) calculated based on the detected defocus amount and the current focus lens position as the target position.

Subsequently, in step S1802b, the camera control unit 1114 determines whether or not the acquired reliability is "2". If the reliability is "2" in step S1802b, the process proceeds to step S1805b. On the other hand, if the reliability is not "2", the process proceeds to step S1803b. When the reliability is "2", it means that the accuracy of the detected defocus amount includes a certain error. Therefore, in step S1805b, the camera control unit 1114 sets a value obtained by multiplying the detected position calculated based on the detected defocus amount and the current focus lens position by the coefficient α as the target position. The coefficient α is a numerical value less than 1, and α assumed in this embodiment is 0.8. Therefore, in step S1805b, a position 80% of the detection position is set as the target position.

Subsequently, in step S1803b, the camera control unit 1114 determines whether or not the acquired reliability is "3". If the reliability is "3" in step S1803b, the process proceeds to step S1806b. On the other hand, if the reliability is not "3", the process proceeds to step S1807b. When the reliability is "3", the accuracy of the detected defocus amount itself is low, but the defocus direction indicates a reliable state. Therefore, in step S1806b, the camera control unit 1114 sets a position shifted from the current position by the shift amount β in the defocus direction based on the defocus direction and the current focus lens position as the target position. Note that β assumed in this embodiment is set to about 1 ・ F ・ δ based on the depth of focus F ・ δ (F: F value, δ: permissible circle of confusion diameter).

If the reliability is not "3" in step S1803b, that is, if the reliability is "4", the process proceeds to step S1807b, and the camera control unit 1114 sets the target position to the current focus lens position. After selecting the target position via step S1804b, step S1805b, step S1806b, or step S1807b, the process proceeds to step S1808b. In step S1808b, the camera control unit 1114 calculates an actual lens drive amount (actual drive amount in the figure) based on the target position determined in steps S1803b to S1807b and the current focus lens position. Then, the camera control unit 1114 determines whether or not the actual drive amount is larger than N, F, and δ. If it is determined in step S1808b that the actual drive amount is larger than NF / δ, the process proceeds to step S1809b. On the other hand, when it is determined that the actual drive amount is NF / δ or less, the process is terminated as it is.

In step S1809b, the camera control unit 1114 changes the target position to the current position + NF · δ so that the actual drive amount does not exceed NF · δ. As described above, in steps S1808b and S1809b, the camera control unit 1114 sets the predetermined lens when a focus change larger than the predetermined lens drive amount (NF) occurs regardless of the reliability of the defocus amount. Correct the target position so that it fits in the drive amount. Here, with respect to the predetermined lens drive amount N ・ F ・ δ, the coefficient N in this embodiment is set to 5, and the target position is 5 ・ F ・ δ, that is, when there is a possibility that focus fluctuation of about 5 depths or more occurs. Is corrected to suppress focus fluctuations.

As described above, in the zoom stop target position selection process in the present embodiment, as shown in FIG. 18B, when the reliability of the defocus amount is high, the target position is set based on the defocus amount. On the other hand, if the reliability is low, the target position is set based on the current lens position. FIG. 18C shows the positional relationship between the current time t1 and the focus lens 1103 at the next control timing t2 when the target position is selected by the zoom target position selection process and a series of AF control processes are performed.

P1 at time t1 indicates the current lens position and corresponds to the distance A. The target position in the case of the reliability 1 to 4 in FIG. 18 (B) corresponds to (1) to (4) at the time t1 in FIG. 18 (C). That is, when the reliability is "1", the distance is B, when the reliability is "2", the distance is C, when the reliability is "3", the distance is D, and when the reliability is "4", the distance A is the same as the current position. be. Further, as shown in steps S1808b and S1809b of FIG. 17B, when the target position is larger than the predetermined lens drive amount (NF · δ), it corresponds to the distance E at (5) at time t1. do. In this case, the distance F is corrected to (6) by the target position selection process when the zoom is stopped.

By the zoom stop target position selection process, the target positions (1) to (4) and (6) at the current time t1 are selected and determined according to the reliability of the defocus amount. Then, in step S1706 of FIG. 16, each lens position at the time t2 of the next control timing is calculated from the cam locus, and in step S1707, the focus lens is driven toward the target position at the time t2.

As described above, according to the present embodiment, when the target position is selected based on the defocus amount and its reliability, the focus lens is controlled based on a selection rule different from that in the case of not zooming during zooming. It becomes possible to determine and control the target position. That is, the focus control means (camera control unit 1114) limits the drive range of the focus lens according to the defocus amount and the reliability while the zoom lens is being driven. More specifically, when the focus control means determines that the movement amount of the focus lens becomes larger than the predetermined amount while the zoom lens is being driven, the focus lens is prevented from exceeding the predetermined amount. Correct the target position. This makes it possible to suppress the occurrence of abrupt focus changes during zooming and maintain the continuity of the focus changes. In addition, in the zoom AF operation, even if the reliability is low even though the subject is in the vicinity of focus in a low contrast subject or in low illuminance, the focus tracking performance during the zoom operation can be improved without adding multiple frames. Prioritize.

Next, the normal AF operation in step S1605 of FIG. 15 will be described with reference to FIGS. 19A and 19B. 19 (A) and 19 (B) are flowcharts showing normal AF operation.

First, in step S2001 of FIG. 19A, the AE processing unit 1119 performs AE processing. Subsequently, in step S2002, the camera control unit 1114 initializes the frame addition number to 0. The frame addition number determines how many frames the focus detection results detected so far have been added. Subsequently, in step S2003, the camera control unit 1114 initializes the number of times the lens is driven to zero. The number of times the lens is driven determines how many times the lens is driven in step S2012, which will be described later. Subsequently, in step S2004, the camera control unit 1114 performs a focus detection process to detect the defocus amount and the reliability. The focus detection process will be described later.

Subsequently, in step S2005, the camera control unit 1114 determines whether or not the reliability of the result detected in step S2004 is larger than the second reliability threshold value (that is, whether or not the reliability is "1" or "2". ) Is determined. If the reliability is greater than the second reliability threshold, the process proceeds to step S2006. On the other hand, when the reliability is equal to or less than the second reliability threshold value (that is, when the reliability is neither "1" nor "2"), the process proceeds to step S2025. In step S2006, the camera control unit 1114 determines whether or not the defocus amount detected in step S2004 is equal to or less than the preset second Def amount threshold value. If the detected defocus amount is equal to or less than the second Def amount threshold value, the process proceeds to step S2007. On the other hand, when the defocus amount is larger than the second Def amount threshold value, the process proceeds to step S2023 in FIG. 19B. The second Def amount threshold is to control the focus lens within the depth of focus within a predetermined number of times (for example, 3 times) after that, if the defocus amount is equal to or less than the second Def amount threshold, the lens is driven by the defocus amount within a predetermined number of times (for example, 3 times). Set with a value that allows for (for example, 5 times the depth of focus).

In step S2007 of FIG. 19A, the camera control unit 1114 determines whether or not the focus lens 1103 is in the stopped state. If the focus lens 1103 is in the stopped state, the process proceeds to step S2008. On the other hand, if the focus lens 1103 is not in the stopped state, the process proceeds to step S2015. In step S2008, the camera control unit 1114 determines whether or not the reliability of the result detected in step S2004 is "1". If the reliability is "1", the process proceeds to step S2009. On the other hand, if the reliability is not "1", the process proceeds to step S2015.

In step S2009, the camera control unit 1114 determines whether or not the lens drive count is equal to or greater than the first drive count threshold. If the lens drive count is equal to or greater than the first drive count threshold, the process proceeds to step S2011. On the other hand, when the number of times the lens is driven is smaller than the first threshold value for the number of times the lens is driven, the process proceeds to step S2010. The first drive number threshold value sets the number of times the lens can be driven to the vicinity of the in-focus position (for example, within the depth of focus) when it is equal to or less than the second Def threshold value described above. For example, in the normal AF operation, the drive count threshold value 1 is set to 3 times.

In step S2010, the camera control unit 1114 determines whether or not the defocus amount detected in step S2004 is equal to or less than the preset first Def amount threshold value. When the defocus amount is equal to or less than the first Def amount threshold value, the process proceeds to step S2011. On the other hand, when the defocus amount is larger than the first Def amount threshold value, the process proceeds to step S2012. The first Def amount threshold is set so that if the detected defocus amount is equal to or less than the first Def amount threshold, the focus lens is controlled within the depth of focus.

In step S2011, the camera control unit 1114 determines that the camera is in the focused state, and ends this flow. In step S2012, the camera control unit 1114 drives the focus lens 1103 by the amount of defocus detected in step S2004, and then proceeds to step S2013. In step S2013, the camera control unit 1114 increments the number of times the lens is driven. Subsequently, in step S2014, the camera control unit 1114 initializes the frame addition number to 0, and returns to step S2004.

In step S2015, the camera control unit 1114 determines whether or not the detected defocus amount is equal to or less than the third Def amount threshold value. If the detected defocus amount is equal to or less than the third Def amount threshold value, the process proceeds to step S2016. On the other hand, when the detected defocus amount is larger than the third Def amount threshold value, the process proceeds to step S2019. In step S2016, the camera control unit 1114 drives the focus lens 103 by a predetermined ratio with respect to the defocus amount detected in step S2004. Subsequently, in step S2017, the camera control unit 1114 instructs the focus lens 1103 to stop. Subsequently, in step S2018, the camera control unit 1114 initializes the frame addition number to 0 and returns to step S2004.

In step S2019 of FIG. 19B, the camera control unit 1114 instructs the focus lens 1103 to stop. Subsequently, in step S2020, the camera control unit 1114 increments the frame addition number. Subsequently, in step S2021, the camera control unit 1114 adds the defocus amounts for the plurality of frames detected so far. Here, the frame to be the frame addition is determined to be the frame addition number incremented in step S2020 including this time.

Subsequently, in step S2022, the camera control unit 1114 determines whether or not the frame addition number is equal to or greater than the frame addition number threshold value. If the frame addition number is equal to or greater than the frame addition number threshold value, the process proceeds to step S2009. On the other hand, if the frame addition number is smaller than the frame addition number threshold value, the process returns to step S2004. The frame addition number threshold value is set by the number of frames at which sufficient AF accuracy can be obtained by addition averaging, and is set to, for example, four times. Further, the added defocus amount is averaged using the frame addition number, and in step S2010 immediately after that, the determination is made based on the added average defocus amount.

Subsequently, in step S2023, the camera control unit 1114 drives the focus lens 1103 by a predetermined ratio with respect to the defocus amount detected in step S2004. Here, the predetermined ratio is set so that the lens drive amount is smaller than the defocus amount (for example, 80%). Further, the lens speed to be set is set to be slower than the speed at which the lens can be driven by the lens drive amount set in, for example, one frame time. As a result, it is possible to prevent the subject focus position from being exceeded when the detected defocus amount is incorrect, and it is possible to drive the next lens while driving the lens without stopping (overlap control). ). Subsequently, in step S2020, the camera control unit 1114 initializes the frame addition number to 0 and returns to step S2004.

In step S2025, the camera control unit 1114 initializes the frame addition number to 0. Subsequently, in step S2026, the camera control unit 1114 determines whether or not the out-of-focus condition is satisfied. If the out-of-focus condition is satisfied, the process proceeds to step S2027. On the other hand, if the out-of-focus condition is not satisfied, the process proceeds to step S2028. The out-of-focus condition is a condition for determining that there is no subject to be focused. As a non-focus condition, for example, when the lens drive is completed in the entire movable range of the focus lens 1103, that is, when the focus lens 1103 detects both the far and near lens ends and returns to the initial position. Set the conditions.

In step S2027, the camera control unit 1114 determines that the camera is out of focus, and ends this flow. In step S2028, the camera control unit 1114 determines whether or not the focus lens 1103 has reached the far end or the near end of the lens. When the focus lens 1103 reaches the lens end, the process proceeds to step S2029. On the other hand, if the focus lens 1103 has not reached the lens end, the process proceeds to step S2030. In step S2029, the camera control unit 1114 reverses the driving direction of the focus lens 1103 and returns to step S2004. In step S2030, the camera control unit 1114 drives the focus lens 1103 in a predetermined direction and returns to step S2004. As the focus lens speed, for example, the fastest speed is set in the range of the lens speed that does not pass the focus position when the defocus amount can be detected.

As described above, in the normal AF operation, when the reliability is low even though the subject is in the vicinity of focusing in a low contrast subject or in low illuminance, unlike the zoom AF operation, the focusing performance is better than the AF time. Is prioritized and the addition for multiple frames is performed. Also, the AF control method based on reliability is different from the zoom AF operation.

Next, with reference to FIG. 20, the safety MF operation in step S1606 of FIG. 15 will be described. FIG. 20 is a flowchart showing the safety MF operation.

Since steps S2101 to S2105 are the same as steps S2001 to S2005 during normal AF operation, the description thereof will be omitted. If the reliability is larger than the second reliability threshold, the process proceeds to step S2106. On the other hand, when the reliability is equal to or less than the second reliability threshold, this flow ends. In the normal AF operation, when the reliability is equal to or less than the second reliability threshold value, the lens is driven in a predetermined direction, but in the safety MF operation, it is necessary to take a picture near the current focus lens position. The safety MF operation is terminated without driving the focus lens.

In step S2106, the camera control unit 1114 determines whether or not the defocus amount detected in step S2104 is equal to or less than the preset fourth Def amount threshold value (defocus amount threshold value). If the defocus amount is equal to or less than the fourth Def amount threshold value, the process proceeds to step S2107. On the other hand, when the defocus amount is larger than the fourth Def amount threshold value, this flow ends. The fourth Def amount threshold is different from the second Def amount threshold in the normal AF operation, so that the focus fluctuation becomes difficult to understand on the display unit 1111 even if the focus lens is driven based on the fourth Def amount threshold. Set as a threshold value. As the fourth Def amount threshold value, for example, an amount three times the depth of focus is set.

In step S2107, the camera control unit 1114 determines whether or not the reliability of the result detected in step S2104 is "1". If the reliability is "1", the process proceeds to step S2108. On the other hand, if the reliability is not "1", the process proceeds to step S2113. In step S2108, the camera control unit 1114 determines whether or not the lens drive count is equal to or greater than the second drive count threshold. When the number of times the lens is driven is equal to or greater than the second drive number threshold, this flow ends. On the other hand, if the lens drive count is smaller than the second drive count threshold, the process proceeds to step S2109. The second drive count threshold is set to be smaller than the first drive count threshold in the normal AF operation, and is set to, for example, once. As a result, the number of times the lens is driven can be reduced in the safety MF operation, and the AF time can be shortened. Further, in the safety MF operation, since the focus lens is moved to the vicinity of the in-focus position in advance by the user, the lens can be driven to the in-focus position even if the number of times the lens is driven is small. Since steps S2109 to S2112 are the same as steps S2010 and S2012-S2014 during normal AF operation, details are omitted, but when the detected defocus amount is equal to or less than the first Def amount threshold value, this flow ends. On the other hand, when the defocus amount is larger than the first Def amount threshold value, the focus lens 103 is driven based on the detected defocus amount, the number of lens drives is incremented, the frame addition number is initialized to 0, and the process proceeds to step S2104. return.

In step S2113, the camera control unit 1114 determines whether or not the lens drive count is equal to or greater than the second drive count threshold. When the number of times the lens is driven is equal to or greater than the second drive number threshold, this flow ends. On the other hand, if the lens drive count is smaller than the second drive count threshold, the process proceeds to step S2114. As a result, even when the condition for performing frame addition is met, the number of frame additions and the number of lens drives can be limited to a small number, and the AF time can be shortened.

In step S2114, the camera control unit 1114 increments the frame addition number. Subsequently, in step S2115, the camera control unit 1114 adds the defocus amounts for the plurality of frames detected so far. Here, the frame to be the frame addition is determined to be the frame addition number incremented in step S2114 including this time. Subsequently, in step S2116, the camera control unit 1114 determines whether or not the frame addition number is equal to or greater than the frame addition number threshold value. If the frame addition number is equal to or greater than the frame addition number threshold value, the process proceeds to step S2117. On the other hand, if the frame addition number is smaller than the frame addition number threshold value, the process returns to step S2104. Here, as the frame addition number threshold value, the number of frames from which sufficient AF accuracy can be obtained by averaging is set as in the case of the normal AF operation, and is set to, for example, four times.

In step S2117, the camera control unit 1114 determines whether or not the added average defocus amount obtained by averaging the defocus amount added in step S2115 with the frame addition number is equal to or less than the fourth Def amount threshold. If the added average defocus amount is equal to or less than the fourth Def amount threshold value, the process proceeds to step S2108. On the other hand, when the added average defocus amount is larger than the fourth Def amount threshold value, this flow ends. Further, in step S2109 immediately after the determination in step S2117, the camera control unit 1114 makes a determination based on the added and averaged defocus amount. This enables AF control in a range in which focus fluctuations are difficult to see on the display unit 1111 even when the focus lens is driven.

As described above, in the safety MF operation, when the reliability is low even though the subject is in the vicinity of focusing in a low contrast subject or in low illuminance, the focusing performance is prioritized over the AF time for multiple frames. By adding the above, it is different from the zoom AF operation. Further, since it is the MF mode and it is necessary to shorten the AF time, the number of times the lens is driven after the frame addition is different from the normal AF operation (the number of times is smaller than the normal AF operation). In addition, the AF control method based on reliability is also different from the zoom AF operation and the normal AF operation.

Next, with reference to FIG. 21, selection of the defocus amount in step S511 of FIG. 5 will be described. First, in step S2201, the camera control unit 1114 determines the state (shooting mode) of the shooting mode SW1116. When the shooting mode is the safety MF mode, the process proceeds to step S2206 and a small Def is selected. As described above, when the shooting mode is the safety MF mode, the small Def is always selected on the assumption that the focus lens is set in the vicinity of the in-focus position in advance by the user. On the other hand, if the shooting mode is not the safety MF mode, the process proceeds to step S2202. Since steps S2202 to S2208 in FIG. 21 are the same as steps S1001 to S1007 in FIG. 10, their description will be omitted.

[Third Embodiment]
Next, a third embodiment of the present invention will be described. In the present embodiment, an example in which the present invention is applied in an image pickup device in which the lens barrel and the image pickup device can be separated is shown.

<Overall configuration>
First, with reference to FIG. 22, an example of the image pickup apparatus in this embodiment is shown. FIG. 22 is a block diagram of the image pickup apparatus 2000 according to the present embodiment. As shown in FIG. 22, the image pickup apparatus 2000 includes a lens apparatus 1100 and an image pickup apparatus main body 1200. Further, in the image pickup apparatus 2000, the lens control unit 1217 that controls the operation of the entire lens apparatus 1100 and the camera control unit 1114 that controls the operation of the entire image pickup apparatus main body 1200 communicate information.

First, the configuration of the lens device 1100 will be described. The lens device 1100 includes a zoom lens 1101, an aperture 1102, a focus lens 1103, a zoom lens position detection unit 1204, an aperture drive unit 1105, a focus lens drive unit 1106, a lens control unit 1217, and a lens operation unit 1218. The zoom lens 1101 is manually operated by a zoom ring member (not shown) to change the focal length. At that time, the position of the zoom lens 1101 is detected by the zoom lens position detection unit 1204, and is used for controlling the lens device 1100 and the image pickup device main body 1200. The diaphragm 1102 is driven by the diaphragm drive unit 1105 and controls the amount of light incident on the image pickup device 1107, which will be described later.

The focus lens 1103 is driven by the focus lens driving unit 1106, and adjusts the focus formed on the image pickup device 1107, which will be described later. The aperture drive unit 1105 and the focus lens drive unit 1106 are controlled by the lens control unit 1217, and determine the aperture amount of the aperture 1102 and the position of the focus lens 1103. When there is a user operation by the lens operation unit 1218, the lens control unit 1217 performs control according to the user operation. The lens control unit 1217 controls the aperture drive unit 1105 and the focus lens drive unit 1106 in response to control commands and control information received from the camera control unit 1114, which will be described later, and also transmits lens control information to the camera control unit 1114. do.

The lens control unit 1217 of the lens device 1100 in the present embodiment holds the cam locus information shown in FIG. 14 in a ROM (not shown) provided inside. When the zoom lens position detection unit 1204 detects a change in the position of the zoom lens 1101, the lens control unit 1217 controls the focus lens 1103 so as to maintain the focused state by the cam trajectory tracking process described later. ..

Next, the configuration of the image pickup apparatus main body 1200 will be described. The basic configuration of the image pickup apparatus main body 1200 is common to the configuration of the image pickup apparatus 1000 described in the second embodiment with reference to FIG. However, in this embodiment, the processing content of the camera control unit 1114 is different from that of the second embodiment.

The camera control unit 1114 controls by exchanging information with the entire inside of the image pickup apparatus main body 1200. In addition to the processing in the image pickup device main unit 1200, the user can turn the power on / off, change the settings, start recording, start AF control, check the recorded image, etc. according to the input from the camera operation unit 1115. Perform various operated camera functions. In addition, as described above, the camera control unit 1114 exchanges information with the lens control unit 1217 in the lens device 1100, sends lens control commands and control information, and acquires information in the lens. It is different from the second embodiment.

<Cam trajectory tracking process>
Next, with reference to FIG. 23, the cam trajectory tracking process performed by the lens control unit 1217 will be described. FIG. 23 is a flowchart showing a cam trajectory tracking process executed by the lens control unit 1217. This process is executed according to the computer program stored in the ROM in the lens control unit 1217. This process is periodically performed by the lens control unit 1217, and is performed at a cycle of 10 msec in the present embodiment.

First, in step S2601 of FIG. 23, the lens control unit 1217 acquires the position information of the current zoom lens 1101 from the zoom lens position detection unit 1204. Subsequently, in step S2602, the lens control unit 1217 acquires the current position information of the focus lens 1003 from the focus lens drive unit 1106. Subsequently, in step S2603, the lens control unit 1217 compares the previously acquired zoom lens position with the zoom lens position acquired at the previous control timing, and determines whether or not zooming is in progress based on the change. .. If zooming is in progress in step S2603, the process proceeds to step S2604. On the other hand, if the zoom is not in progress, this flow ends.

In step S2604, the lens control unit 1217 determines the target position of the focus lens 1103 at the current control timing. Specifically, the lens control unit 1217 corresponds to the tracking subject distance at the current focal length based on the zoom lens position acquired in step S2601 and the target subject distance (following subject distance) when tracking the cam trajectory. Specify the position of the focus lens to be used. The tracking subject distance when following the cam locus is updated when the lens control unit 1217 controls the focus lens 1103 via the focus lens drive unit 1106. Therefore, it is updated both when the focus lens 1103 is controlled inside the lens device 1100 by the lens control unit 1217 as in the cam locus tracking process and when the focus lens 1103 is controlled by the instruction of the camera control unit 1114. NS.

In step S2605, the lens control unit 1217 drives the focus lens 1103 by the difference between the focus lens position acquired in step S2602 and the target position determined in step S2604. Subsequently, in step S2606, the lens control unit 1217 stores the subject distance corresponding to the position after the focus lens 1103 is driven in the recording unit 1112 as the tracking subject distance, and ends this flow.

Next, with reference to FIGS. 26 (C) and 26 (D), the movement of the focus lens 1103 when the series of cam trajectory tracking processes up to this point is performed will be described. FIG. 26C is a diagram showing the movement of the focus lens 1103 following based on the cam locus information when the focal length changes, and P2 shows the current position of the focus lens 1103 at a certain control timing. In FIG. 26C, the focus lens 1103 is subjected to cam trajectory tracking processing with the tracking subject distance as the distance A. FIG. 26 (D) is an enlarged view of a part of FIG. 26 (C). When the movement of the focus lens 1103 is captured on a shorter time axis, the focus lens 1103 is controlled to periodically follow the cam locus at a distance A by the cam locus tracking process shown above. By repeating the follow-up control to the cam locus, it is possible to maintain the subject distance even when the focal length changes.

<AF operation during zooming>
Next, with reference to FIG. 24, the AF operation during zooming performed by the camera control unit 1114 will be described. FIG. 24 is a flowchart showing the AF operation during zooming executed by the camera control unit 1114. This process is executed according to the computer program stored in the ROM in the camera control unit 1114. For example, it is executed in the reading cycle (every vertical synchronization period) of the image pickup signal from the image pickup element 1107 for generating a one-field image (hereinafter, also referred to as one frame and one screen). Further, it may be repeated for each of a plurality of vertical synchronization periods (V rates).

In steps S2701 to S2705 of FIG. 24, the target selection process of the focus lens is performed. Since steps S2701 to S2705 are the same as steps S1701 to S1705 in FIG. 16 of the second embodiment, the description thereof will be omitted. After the end of step S2704 or step S2705, the process proceeds to step S2706, the focus lens is driven, and this flow ends. This flow is different from the flow of the operation during zooming (flow of FIG. 16) of the second embodiment in that step S1706 is not provided.

<Target selection process when zoom is stopped>
Since the target selection process when the zoom is stopped in this embodiment is the same as that in the second embodiment, the description thereof will be omitted. FIG. 26A shows the target position set in the zoom stop target position selection process in the present embodiment, which is the same as in FIG. 18A.

<Target position selection process when zooming>
Next, with reference to FIGS. 25 and 26 (B), (C), and (D), the zoom target position selection process in step S2705 of FIG. 24 will be described. FIG. 25 is a flowchart showing the AF processing during zooming executed by the camera control unit 1114. Each step in FIG. 25 is executed according to the computer program stored in the ROM in the camera control unit 1114, similarly to the AF control process.

The zoom target position selection process in the present embodiment ends the zoom target position selection process according to the second embodiment in that the process ends without setting the target position when the reliability is not “3” in step S2803b. Different from. Since steps S2801b to S2806b in the flowchart are the same as steps S1801b to S1806b in FIG. 17B, the description thereof will be omitted.

After selecting the target position via step S2804b, step S2805b, or step S2806b, the process proceeds to step S2807b. Steps S2807b and S2808b are the same as steps S1808b and S2808b in FIG. 17, and if the actual lens drive amount (actual drive amount in the figure) is larger than NF δ, the actual drive amount is NF δ. do. The coefficient N in this embodiment is set to 5, and when there is a possibility that focus fluctuations of 5 · F · δ, that is, about 5 depths or more occur, the target position is corrected and the focus fluctuations are suppressed. After correcting the target position, this flow ends.

As described above, in the AF processing during zooming in the present embodiment, as shown in FIG. 26B, when the reliability of the defocus amount is high, the camera control unit 1114 is based on the detected defocus amount. The focus lens 1103 is controlled to the target position. On the other hand, when the reliability is low, the camera control unit 1114 controls the focus lens 1103 to a target position based on the current lens position. If the reliability is such that the in-focus position cannot be specified, the camera control unit 1114 controls the focus lens 1103 so as not to be explicitly driven.

FIG. 26C is a diagram showing the movement of the focus lens 1003 to follow based on the cam locus information when the focal length changes, and P2 is the current position at a certain control timing. In FIG. 26C, the focus lens 1003 is subjected to cam trajectory tracking processing independently of the instruction of the camera control unit 1114 of the image pickup device main body 1200 by the lens control unit 1217 of the lens device 1100 with the tracking subject distance as the distance A. It shows that it is.

(1) to (6) in FIGS. 26 (C) correspond to the reliability of the defocus amount when the AF processing during zooming shown above is performed by the camera control unit 1114 at the current control timing. The relationship between the drive target positions of the focus lens 1103 is shown. The target positions in the cases of reliability 1 to 4 in FIG. 26 (B) correspond to (1) to (4) in FIG. 26 (C), respectively. That is, when the reliability is "1", the distance is B, when the reliability is "2", the distance is C, when the reliability is "3", the distance is D, and when the reliability is "4", the distance A is the same as the current position. Yes, the focus lens 1103 is not explicitly driven by the camera control unit 1114. Further, similarly to FIG. 18C, when the target position is larger than the predetermined lens drive amount (NF δ), it becomes (5) of FIG. 26 (c), which corresponds to the distance E, and corresponds to (6). It is corrected to the distance F of. In this way, the lens device 1100 that performs cam trajectory tracking processing and the image pickup device main body 1200 that corrects the focus position according to the defocus amount and its reliability operate in conjunction with each other to control focus during zooming. It will be realized.

According to the present embodiment, when selecting a target position based on the amount of defocus and its reliability, the target position for controlling the focus lens is determined based on a selection rule different from that in the case of not zooming during zooming. It becomes possible to control. That is, the focus control means (camera control unit 1114) limits the drive range of the focus lens according to the defocus amount and the reliability while the zoom lens is being driven. More specifically, when the focus control means determines that the movement amount of the focus lens becomes larger than the predetermined amount while the zoom lens is being driven, the focus lens is prevented from exceeding the predetermined amount. Correct the target position. This makes it possible to suppress the occurrence of abrupt focus changes during zooming and maintain the continuity of the focus changes. In the present embodiment, an example in which the zoom lens 1101 is manually operated is shown, but the present invention is not necessarily limited to this. For example, as in the second embodiment, the zoom lens drive unit may be provided inside the lens device 1100, and the lens control unit 1217 may control the zoom lens drive unit. Further, an independent zoom lens drive unit may be detachably attached to the outside of the lens device 1100. In any case, the driving means may be used as long as it can detect the moving state of the zoom lens itself.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described. In the second embodiment and the third embodiment, the configurations mainly considering the difference in the hardware configuration have been described, but the present invention is not limited thereto. For example, the following configuration can be considered.

<Switching of control method suitable for operating conditions>
In the case of controlling focusing with the highest priority regardless of the change in focus even during zooming according to the operating state of the image pickup device or the lens device (lens lens barrel) (focus tracking priority control), the above-mentioned It is also conceivable to switch between control that prioritizes focus continuity. For example, the drive amount of steps S1808b and S1809b in FIG. 17B is not limited, and the defocus amount detected from α of step S1805b and β of step S1806b is positively reflected. As a result, focus tracking priority control can be realized. Further, the drive amount of steps S2007b and S2008b in FIG. 19A is not limited, and the defocus amount detected from α in step S2005b and β in step S2006b is positively reflected. As a result, focus tracking priority control can be realized. In addition, focus tracking priority control can be realized by performing AF processing when zooming is stopped during zooming.

The conditions for switching between the focus tracking priority control and the focus continuity priority control during conventional zooming are as follows, which are common to the configurations of the second embodiment and the third embodiment. .. For example, it is a method of switching according to the speed of the zoom lens. When the zoom speed is fast and the change in the captured image is large, focus tracking priority control is performed, and when the zoom speed is slow and the change in the captured image is gradual. Focus continuity priority control is used. In addition, a method of switching depending on the recording state of the moving image is also conceivable. For example, focus tracking priority control is used during non-recording of moving images in which no captured image remains, and focus continuity priority control is used during recording of moving images in which captured images remain. Further, in the configuration of the second embodiment, the camera control unit 1114 grasps the cam locus information. Therefore, focus tracking priority control is used in a region where the cam locus information is low in density, that is, a region where the drive range of the focus lens is wide, and focus continuity priority control is used in a region where the density is high, that is, a region where the drive range of the focus lens is narrow. The method can be considered. In addition, a method of switching according to the shape of the cam locus information can be considered. For example, focus tracking priority control is used in a region where the shape change is large, and focus continuity priority control is used in a region where the shape change is small.

As described above, according to the present embodiment, it is possible to switch between focus tracking priority control and focus continuity priority control according to various states of the image pickup apparatus, and therefore appropriate control is performed according to the required focus change. It is possible to switch the method.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and modifications can be made within the scope of the gist thereof.

204a Acquisition means 204b Calculation means 209 System control unit (decision means)

Claims (20)

One microlens has a first photoelectric conversion unit and a second photoelectric conversion unit that receive light flux passing through different pupil regions of the image pickup optical system, and the microlenses are arranged two-dimensionally. Image sensor and
An acquisition means for acquiring a first signal and a second signal corresponding to the output signals from the first photoelectric conversion unit and the second photoelectric conversion unit, respectively.
A plurality of filters with different bands are performed on the first signal and the second signal, and a plurality of defocus amounts and a plurality of reliabilitys are obtained based on the first signal and the second signal after each filter processing. Calculation means to calculate and
An image pickup apparatus comprising: a determination means for determining a defocus amount used for focus adjustment from the plurality of defocus amounts based on the difference between the plurality of defocus amounts. The calculation means has a plurality of the calculation means based on the degree of coincidence between the first signal and the second signal after each filtering process and the steepness of the correlation change amount between the first signal and the second signal. The imaging device according to claim 1, wherein the reliability is calculated. The determination means is characterized in that the defocus amount used for focus adjustment is determined from the plurality of defocus amounts based on the difference between the plurality of defocus amounts and the plurality of reliabilitys. Item 2. The image pickup apparatus according to Item 1 or 2. The plurality of filtering processes are
The first filter processing for extracting the first signal and the second signal in the first frequency band, and
A second filter process for extracting a first signal and a second signal in a second frequency band higher than the first frequency band is included.
The calculation means has the plurality of defocus amounts and the plurality of reliabilitys.
The first defocus amount and the first reliability are calculated based on the first signal and the second signal after the first filter processing.
The invention according to any one of claims 1 to 3, wherein the second defocus amount and the second reliability are calculated based on the first signal and the second signal after the second filter processing. Imaging device . When the difference between the first defocus amount and the second defocus amount is larger than the first threshold value, the calculation means determines the first defocus amount as the defocus amount used for the focus adjustment. The image pickup apparatus according to claim 4. The calculation means is
When the difference between the first defocus amount and the second defocus amount is equal to or less than the first threshold value.
When the second reliability is higher than the first reliability and the second reliability is higher than a predetermined reliability threshold, the second defocus amount is used for the focus adjustment. Determined as the amount of defocus to be
When the second reliability is lower than the first reliability, or when the second reliability is lower than the predetermined reliability threshold value, the first defocus amount is used for the focus adjustment. The image pickup apparatus according to claim 5, wherein the focus amount is determined. The plurality of filtering processes include a third filtering process for extracting a first signal and a second signal in a third frequency band higher than the second frequency band.
The calculation means calculates the third defocus amount and the third reliability as the plurality of defocus amounts and the plurality of reliabilitys based on the first signal and the second signal after the third filter processing. The imaging device according to claim 5 or 6, wherein the image pickup apparatus is characterized in that. When the difference between the second defocus amount and the third defocus amount is larger than the second threshold value, the calculation means determines the second defocus amount as the defocus amount used for the focus adjustment. The image pickup apparatus according to claim 7. The calculation means is
When the difference between the second defocus amount and the third defocus amount is equal to or less than the second threshold value,
When the third reliability is higher than the second reliability and each of the second reliability and the third reliability is higher than a predetermined reliability threshold, the third defocus The amount is determined as the defocus amount used for the focus adjustment.
When the third reliability is lower than the second reliability, or when the second reliability or the third reliability is lower than the predetermined reliability threshold, the second defocus amount is applied. The image pickup apparatus according to claim 8, wherein the defocus amount used for focus adjustment is determined. The imaging device according to claim 8 or 9, wherein the second threshold value is smaller than the first threshold value. The imaging device according to any one of claims 8 to 10, wherein the first threshold value and the second threshold value are larger than the depth of focus. One microlens has a first photoelectric conversion unit and a second photoelectric conversion unit that receive light flux passing through different pupil regions of the imaging optical system, and the microlenses are arranged two-dimensionally. A step of acquiring a first signal and a second signal corresponding to the output signals from the first photoelectric conversion unit and the second photoelectric conversion unit of the image pickup element, respectively.
A plurality of filters with different bands are performed on the first signal and the second signal, and a plurality of defocus amounts and a plurality of reliabilitys are obtained based on the first signal and the second signal after each filter processing. Steps to calculate and
A control method comprising: a step of determining a defocus amount used for focus adjustment from the plurality of defocus amounts based on a difference between the plurality of defocus amounts. One microlens has a first photoelectric conversion unit and a second photoelectric conversion unit that receive light flux passing through different pupil regions of the imaging optical system, and the microlenses are arranged two-dimensionally. A step of acquiring a first signal and a second signal corresponding to the output signals from the first photoelectric conversion unit and the second photoelectric conversion unit of the image pickup element, respectively.
A plurality of filters with different bands are performed on the first signal and the second signal, and a plurality of defocus amounts and a plurality of reliabilitys are obtained based on the first signal and the second signal after each filter processing. Steps to calculate and
A program characterized by causing a computer to perform a step of determining a defocus amount used for focus adjustment from the plurality of defocus amounts based on the difference between the plurality of defocus amounts. A storage medium for storing the program according to claim 13. Acquisition means for acquiring the first signal and the second signal corresponding to the luminous flux passing through different pupil regions of the imaging optical system, and
A calculation means for calculating the defocus amount and reliability based on the first signal and the second signal, and
It has a focus control means for controlling a focus lens according to the defocus amount and the reliability.
The focus control means has a first mode in which focus control is performed over the entire movable range of the focus lens, and a second mode in which focus control is performed within a predetermined range based on a focus lens position set by a user. The control device is characterized in that the conditions for frame addition are changed according to the defocus amount and the reliability. Acquisition means for acquiring the first signal and the second signal corresponding to the luminous flux passing through different pupil regions of the imaging optical system, and
A calculation means for calculating the defocus amount and reliability based on the first signal and the second signal, and
It has a focus control means for controlling a focus lens according to the defocus amount and the reliability.
The focus control means has a first mode in which focus control is performed over the entire movable range of the focus lens, and a second mode in which focus control is performed within a predetermined range based on a focus lens position set by the user. and in the defocus amount and the reliability and to that control device and changes the number of times of driving of the focus lens in accordance with the. The control device according to claim 16 , wherein the focus control means reduces the number of times the focus lens is driven in the second mode to be less than the number of times the focus lens is driven in the first mode. In the second mode, the focus control means is described when the reliability is higher than the first reliability threshold value and the defocus amount is larger than the first defocus amount threshold value. The control device according to any one of claims 15 to 17 , wherein no control is performed. The focus control means is characterized in that when the defocus amount and the reliability satisfy the condition of the frame addition, the focus lens is controlled based on the addition average of the defocus amounts of a plurality of frames. Item 15. The control device according to Item 15. The calculation means performs a plurality of filtering processes on the first signal and the second signal having different bands, and a plurality of defocus amounts based on the first signal and the second signal after the filtering processing. Calculate and
The focus control means is characterized in that, in the second mode, the focus lens is controlled based on the defocus amount in the highest band among the plurality of defocus amounts calculated by the calculation means. Item 12. The control device according to any one of Items 15 to 19.

Images