JP7237473B2 - Focus adjustment device and its control method - Google Patents

Description

The present invention relates to a focus adjustment device that performs focus adjustment by a phase difference detection method using an output signal from an image sensor, and a control method thereof.

2. Description of the Related Art Conventionally, some image pickup apparatuses such as electronic still cameras employ an imaging plane phase difference detection method that performs focus detection by a phase difference detection method using an output signal from an image sensor. This imaging plane phase difference detection method has a problem that when a low-contrast subject is photographed, the obtained image signal becomes small, and the follow-up accuracy of focusing deteriorates.

Therefore, in the technique disclosed in Patent Document 1, an evaluation value based on the contrast information of the output signal is calculated, and it is determined whether or not the defocus amount calculated in the past is taken into consideration, and the final defocus amount is determined. is calculated.

JP 2016-57546 A

However, in Patent Document 1, since the evaluation value based on the contrast information is referred to, the evaluation value changes depending on the setting of the imaging device (aperture value, etc.), the brightness of the subject, etc., and the defocus amount calculated in the past is taken into consideration. In some cases, the accuracy of determination as to whether or not it is better may be degraded. For example, when shooting a low-contrast subject such as a person's face with a small aperture setting, the resulting image signal will be smaller than when shooting a high-brightness or high-contrast subject, and the tracking accuracy of focusing will deteriorate. There was a case where it was lost.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to prevent the AF focusing accuracy from deteriorating due to the setting (aperture value, etc.) of an imaging device, the brightness of an object, or the like.

As a technical feature of the present invention, there is provided a control method for a focus adjustment device that performs focus adjustment using an output signal from an image pickup device that picks up an image of a subject, wherein the focal point of the subject image is generated from the output signal of the image pickup device. A generation step of generating a plurality of pieces of focus detection information by processing a pair of image signals having a phase difference according to a state with a plurality of filters having different bands, and reliability of the focus detection information generated in the generation step. and calculating an evaluation value regarding the reliability of the focus detection information based on the setting of the aperture in a moving image shooting mode and when the contrast of the subject satisfies a predetermined condition. and a setting step of not changing the evaluation value when the mode is a mode for moving image shooting and the contrast of the subject does not satisfy a predetermined condition, wherein the setting step is based on the setting of the aperture . and, when changing the evaluation value relating to the reliability of the focus detection information, the evaluation value at the first aperture value is made larger than the evaluation value at the second aperture value smaller than the first aperture value. It is characterized by

According to the present invention, it is possible to prevent deterioration of AF focusing accuracy.

2 is a block diagram showing configurations of a camera and a lens; FIG. It is a figure which shows the pixel structure of an imaging surface phase difference method. FIG. 10 is an explanatory diagram of evaluation values relating to the amount of correlation change between two images; 7 is a flowchart for explaining calculation of a threshold for standard deviation of defocus amounts; 6 is a flowchart for explaining an example of imaging processing; 6 is a flowchart for explaining an example of automatic focus detection processing; 4 is a flowchart for explaining a method of selecting a defocus amount; FIG. 5 is a diagram showing a threshold value of standard deviation of defocus amount with respect to aperture (hereinafter referred to as F value); FIG. 9 is a flowchart for explaining a handover determination from large Def to medium Def in FIG. 8 ; FIG. FIG. 9 is a flowchart for explaining a handover determination from medium Def to small Def in FIG. 8 ; FIG.

Preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings.

Configuration of Imaging Apparatus FIG. 1 is a block diagram showing the configuration of an interchangeable lens camera (hereinafter simply referred to as a camera) according to an embodiment. A camera according to an embodiment is an example of an imaging device equipped with a focus adjustment device to which the present invention is applied, and performs focus adjustment by an imaging plane phase difference detection method using an output signal from an imaging device that captures an image of a subject. . The camera is composed of a lens unit 10 and a camera body 20 . When the lens unit 10 is attached to the camera body 20, the lens control unit 106 that controls the operation of the lens unit 10 and the camera control unit 212 that controls the operation of the entire camera can communicate with each other.

A configuration of the lens unit 10 will be described. The lens unit 10 includes a fixed lens 101 , a diaphragm 102 , a focus lens 103 , a diaphragm drive section 104 , a focus lens drive section 105 , a lens control section 106 and a lens operation section 107 . A fixed lens 101, a diaphragm 102, and a focus lens 103 constitute a photographing optical system. A diaphragm 102 is driven by a diaphragm drive unit 104 to control the amount of light incident on an imaging device 201 of the camera body 20 . A focus lens 103 is driven by a focus lens driving unit 105 and performs focus adjustment for forming an image on an imaging device 201, which will be described later. A diaphragm driver 104 and a focus lens driver 105 are controlled by a lens controller 106 to determine the opening amount of the diaphragm 102 and the position of the focus lens 103 . When a user operation is performed by the lens operation unit 107, the lens control unit 106 performs control according to the user operation. The lens control unit 106 controls the aperture driving unit 104 and the focus lens driving unit 105 according to the control command/control information received from the camera control unit 212 and also transmits lens control information to the camera control unit 212 .

Next, the configuration of the camera body 20 will be described. The camera body 20 is configured to acquire an imaging signal from the light flux that has passed through the imaging optical system of the lens unit 10 . The camera body 20 includes an image sensor 201 , a CDS/AGC/AD converter 202 , an image input controller 203 and an AF (automatic focusing) signal processing section 204 . The camera body 20 also includes a display control section 205 , a display section 206 , a recording medium control section 207 , a recording medium 208 , an SDRAM 209 , a ROM 210 and a flash ROM 211 . The camera body 20 also includes a camera control section 212 having an AF control section 213 , a camera operation section 214 , a timing generator 215 and a bus 21 .

The imaging element 201 is composed of a CCD or CMOS sensor, and the light flux passing through the imaging optical system of the lens unit 10 forms an image on its light receiving surface and is converted into signal charges according to the amount of incident light by a photodiode. The signal charge accumulated in each photodiode is sequentially read as a voltage signal corresponding to the signal charge based on the drive pulse given from the timing generator 215 according to the command from the camera control section 212 . The imaging device 201 holds two photodiodes in one pixel, as shown in FIG. FIG. 2A shows a pixel configuration when the imaging plane phase difference detection method is not adopted, and FIG. 2B shows a pixel configuration when the imaging plane phase difference detection method is adopted. A light beam is separated by a microlens and imaged by two photodiodes of each pixel, so that an imaging signal and an image signal for AF (hereinafter referred to as an AF signal) can be extracted. A signal obtained by adding the signals of the two photodiodes (A+B image signal) becomes an imaging signal. Signals (A image signal, B image signal) of individual photodiodes are respectively used as AF signals, and a pair of AF signals has a phase difference according to the focus state of the subject image.

The CDS/AGC/AD converter 202 performs correlated double sampling for removing reset noise, gain adjustment, and signal digitization on the imaging signal read from the imaging device 201 and the pair of AF signals. conduct. The CDS/AGC/AD converter 202 outputs the imaging signal to the image input controller 203 and the pair of AF signals to the AF signal processing unit 204, respectively. The image input controller 203 stores the imaging signal output from the CDS/AGC/AD converter 202 in the SDRAM 209 . The image signal stored in the SDRAM 209 is displayed on the display unit 206 by the display control unit 205 via the bus 21, and is displayed on the recording medium 208 by the recording medium control unit 207 in the mode of recording the imaging signal. Recorded. The ROM 210 stores control programs executed by the camera control unit 212 and various data necessary for control. The flash ROM 211 stores various setting information related to camera operations such as user setting information.

The AF signal processing unit 204 performs a correlation operation on the pair of AF signals output from the CDS/AGC/AD converter 202, and calculates the amount of image shift and various information (degree of coincidence between two images, contrast information, saturation information, scratch, etc.). information, etc.). The AF signal processing unit 204 outputs the calculated image shift amount and various information to the camera control unit 212 .

A camera control unit 212 controls by exchanging information with each unit of the camera. In addition, according to the input from the camera operation unit 214, various camera functions operated by the user such as power ON/OFF, change of settings, start of recording, start of AF control, confirmation of recorded images, etc. are executed. . It also communicates with the lens controller 106 of the lens unit 10 to send lens control commands and control information, and to acquire information on the lens unit 10 side. Further, the AF control unit 213 of the camera control unit 212 performs focusing control on the subject, although the details will be described later. Based on the image shift amount and various information acquired from the AF signal processing unit 204, the AF control unit 213 notifies the AF signal processing unit 204 of changes in settings for calculating these. For example, when the amount of image shift is large, the area for correlation calculation is set wide, or the types of a plurality of bandpass filters are changed according to the contrast information. A camera operation unit 214 includes operation units such as a shutter button, a mode switch, an electronic dial, and a power switch.

In this embodiment, the imaging signal and the pair of AF signals are extracted from the imaging device 201, but the present invention is not limited to this method. Considering the load of the imaging device 201, for example, an imaging signal and one AF signal are extracted from the imaging device, and another AF signal is generated by taking the difference between the imaging signal and the AF signal. good too.

●Photographing Processing Next, the photographing processing of the present embodiment will be described with reference to the flowchart of FIG.

First, in S501, it is determined whether or not the first switch (SW1) for photographing is on. If SW1 is on, the process proceeds to S502, and if SW1 is off, the determination process of S501 is repeated in a waiting state. In S502, automatic focus detection processing is performed.

Details of the automatic focus detection process will be described later. In S503, focus detection information is acquired, and it is determined whether or not the focus detection result calculated in S502 is within the in-focus range. If the focus detection result is within the in-focus range, the process proceeds to S505, but if the focus detection result is out of the in-focus range, the process proceeds to S504. In S504, a focus lens drive command is transmitted. Accordingly, drive control of the focus lens is performed based on the focus detection result calculated in S502. In S505, it is determined whether or not the second switch (SW2) for photographing is on. If SW2 is on, the process proceeds to S506, and if SW2 is off, the process proceeds to S507. In S506, shooting preparation processing is performed, and in the next step S508, shooting processing is completed after performing recording processing of the captured image data. Also, in S507, it is determined whether or not SW1 is in the ON state. If SW1 is on, the process returns to S505, and if SW1 is off, the process ends.

●Automatic Focus Detection Processing Next, the automatic focus detection processing performed by the AF control unit 213 and the AF signal processing unit 204 in S502 of FIG. 5 will be described with reference to the flowchart shown in FIG.

In S<b>601 , the AF control unit 213 causes the AF signal processing unit 204 to perform correlation calculation on the pair of phase difference image signals acquired from the image sensor 201 . Then, the defocus amount is calculated based on the image shift amount, which is the shift amount Shift at which the correlation amount received from the AF signal processing unit 204 is the minimum value. At this time, correlation calculation is performed using three types of filters with different frequency bands for low, middle, and high frequencies, and three types of defocus amounts for low, middle, and high frequencies are calculated. calculate. Although three types are used here, a plurality of types may be used. Further, the AF control unit 213 causes the AF signal processing unit 204 to calculate the correlation amount of the pair of phase difference image signals for each shift amount Shift. The AF control unit 213 generates a correlation amount waveform for each shift amount Shift received from the AF signal processing unit 204 .

Next, in S602, the amount of correlation change between the two images is calculated. FIG. 3 is a graph exemplifying the amount of correlation change when the focus lens is driven from a state where the degree of blur is large to near the focal point in imaging plane phase difference AF. The horizontal axis represents the degree of blurring of the subject, and the vertical axis represents the correlation change amount MAXDER. The correlation change amount MAXDER can be calculated by the following formula (1).
MAXDER(k)=(COR[k-3]-COR[k-1])
-(COR[k-2]-COR[k]) (1)

k in the above equation is an integer variable for specifying the position, and COR[k] is the amount of correlation between two images at position k. At this time, as in S601, three types of correlation change amounts MAXDER for low, middle, and high frequencies with different filter frequency bands are calculated. As shown in FIG. 9, in the imaging plane phase difference AF, the value of the correlation change amount increases as the degree of blur approaches the in-focus state from the state in which the degree of blur is large.

In S603, the standard deviation Defocus3σ of the defocus amount is calculated based on the correlation change amount MAXDER. The standard deviation Defocus3σ of the defocus amount can be calculated by the following formula (2).
Defocus3σ=K×(A×MAXDER^B) (2)

K in the above formula is a conversion coefficient for converting the amount of image shift into a defocus amount, and A and B are coefficients of a conversion formula for converting the amount of correlation change MAXDER into the standard deviation of the amount of image shift. At this time, the standard deviation Defocus3σ of the three types of defocus amounts is calculated by substituting the three types of correlation change amounts MAXDER for low, middle, and high frequencies for different frequency bands calculated in S603. do.

In S604, in order to calculate the reliability evaluation value Rel representing the reliability of the defocus amount, the threshold of the standard deviation Defocus3σ of the calculated defocus amount is calculated. The reliability evaluation value Rel is determined in four levels of reliability evaluation value Rel3, reliability evaluation value Rel2, reliability evaluation value Rel1, and reliability evaluation value Rel0 in descending order of reliability. Here, when the reliability evaluation value Rel is the reliability evaluation value Rel3, since the reliability of the defocus amount is high, it is set as a condition in which focusing is possible. When the reliability evaluation value Rel is the reliability evaluation value Rel2, the defocus amount for the past m frames and the defocus amount calculated for the current frame are added and averaged. Make it possible to burn. Furthermore, the reliability evaluation value Rel1 means that the direction of the calculated defocus amount is correct, and the reliability evaluation value Rel0 means that the reliability of the defocus amount is the lowest. conditions that make it impossible. If the calculated standard deviation Defocus3σ of the defocus amount is greater than the standard deviation threshold value Def3σTH1, Rel0 is selected. Further, when the calculated standard deviation Defocus3σ of the defocus amount is greater than the standard deviation threshold Def3σTH2 and equal to or less than the standard deviation threshold Def3σTH1, Rel1 is selected. If the calculated standard deviation Defocus3σ of the defocus amount is larger than the standard deviation threshold value Def3σTH3 and equal to or smaller than the standard deviation threshold value Def3σTH2, Rel2 is selected, and Rel3 is selected otherwise. Further, the focus lens speed is determined depending on which reliability evaluation value the value of Defocus3σ obtained during automatic focus adjustment falls within. Here, the magnitude relationship of the focus lens driving amount (the larger the value, the higher the speed) for each standard deviation threshold value is set so as to satisfy the following equation (3). Here, the driving amount is Rel3_SPEED for the reliability evaluation value Rel3, Rel2_SPEED for the reliability evaluation value Rel2, Rel1_SPEED for the reliability evaluation value Rel1, and Rel0_SPEED for the reliability evaluation value Rel0.
Rel3_SPEED<Rel2_SPEED<Rel1_SPEED<Rel0_SPEED (3)

Here, a method for calculating the threshold value of the standard deviation Defocus3σ will be described with reference to the flowchart shown in FIG.

First, in S401, a standard deviation threshold value Def3σTH1 is calculated. Def3σTH1 is set to a predetermined value. In S402, a standard deviation threshold Def3σTH2 is calculated. Def3σTH2 is set to a value smaller than Def3σTH1.

In S403, it is determined whether the moving image mode is selected and whether or not the sharpness evaluation value Sharpness/PB obtained by dividing the sharpness, which is the contrast value of the subject (hereinafter, the sharpness of the image) by the amplitude PB, is equal to or greater than a predetermined threshold. If both determinations are true, the process proceeds to S404; otherwise, the process proceeds to S405.

The sharpness of the image can be calculated by the following formula (4).
Sharpness=Σ(S[k+1]−S[k]) ² /Σ(S[k+1]−S[k])(4)

However, k is an integer variable for specifying the position, and S[k] is the signal value of the phase difference image signal at a certain lens position k. Then, the sharpness evaluation value Sharpness/PB is calculated using the sharpness Sharpness of the image calculated from the above equation (4). The threshold value SPB_TH in the sharpness evaluation value this time is obtained from a value actually measured from a person's face, a low-contrast chart, or the like.

In S404, as shown in FIG. 8, Def3σTH2 is made variable based on the aperture value. In FIG. 8, the vertical axis is the standard deviation threshold and the horizontal axis is the F value. In this embodiment, the following formula (5) for linear interpolation according to the F value is adopted.
Defocus3σ_TH2=A× _XF +B (5)

This interpolation formula obtains A (slope) and B (intercept) in the above formula using the actual measured values of the standard deviation Defocus3σ obtained from various charts.

In S405, a standard deviation threshold value Def3σTH3 is calculated. Def3σTH3 is set to a value smaller than Def3σTH2. In this manner, the focus can be tracked without degrading the responsiveness in control on the open side of still image shooting, and reliability can be prevented from lowering when shooting a low-contrast subject at a small aperture during moving image shooting. Also, along with this, it becomes possible to follow the focus of the subject in the vicinity of the in-focus position (improve the quality of moving image shooting).

In S605, a reliability evaluation value Rel representing the reliability of the calculated defocus amount is calculated. Based on Def3σTH1, Def3σTH2, and Def3σTH3 calculated in S604, the reliability of each of the three types of defocus amounts for low, middle, and high frequencies calculated in S601 is calculated.

In S606, one defocus amount is selected based on the three types of defocus amounts for low, middle, and high frequencies calculated in S601 and the reliability calculated in S605. Here, the defocus amount calculated with the low frequency filter is defined as large Def, the defocus amount calculated with the middle frequency filter is defined as medium Def, and the defocus amount calculated with the high frequency filter is defined as small Def.

●Defocus amount selection method A defocus amount selection method will be described with reference to the flowchart shown in FIG.

First, in S701, a handover determination from large Def to medium Def, which will be described later, is performed, and the process proceeds to S702.

In S702, it is checked whether or not the high Def has been handed over to the medium Def in the determination of S701. In S703, a handover determination from medium def to small def, which will be described later, is performed, and the process proceeds to step S704. In S704, it is checked whether or not the medium Def has been handed over to the small Def.

In S705, small Def is selected as the defocus amount to be used, and this flow ends. In S706, medium Def is selected as the defocus amount to be used, and this flow ends. In S707, large Def is selected as the defocus amount to be used, and this flow ends.

● Determination of Handover from Large Def to Medium Def Next, determination of handover from large Def to medium Def in S701 of FIG. 7 will be described with reference to FIG.

First, in S901, it is determined whether or not the reliability evaluation value of medium Def is Rel2 or higher. By the determination in S901, it is possible to increase the possibility of taking over to medium Def and increase the focusing rate. If the determination is true, the process proceeds to S902, and if not false, the process proceeds to S903. In S902, it is determined whether or not the subject has a periodic pattern. This is to prevent the high Def from being handed over to the medium Def, since the reliability of the defocus amount is low for a subject having a periodic pattern. A waveform indicating the amount of correlation of an object with a periodic pattern periodically has a portion where the amount of correlation, which is a candidate for a focusing point, is minimal. This is performed by calculating the amount of correlation change and comparing the calculated difference in the amount of correlation change between the two images with the threshold. If the determination is true, the process proceeds to S903, and if not false, the process proceeds to S905.

In S903, it is checked whether the difference in defocus amount between the large Def and the medium Def is equal to or less than a preset first predetermined depth threshold. Here, the first predetermined depth threshold is set to, for example, 9 times the depth of focus so that the large Def can be successfully transferred to the medium Def. Further, by setting the predetermined depth threshold based on the focal depth, a uniform threshold can be set even if the F-number or the focus detection area changes.

In S904, it is checked whether the reliability evaluation value of medium def is higher than the reliability evaluation value of large def and whether the reliability evaluation value of medium def is greater than Rel0. If not, proceed to S906.

In S905, it is determined that the high Def can be handed over to the medium Def, and this flow ends. In S906, it is determined that the high Def cannot be handed over to the medium Def, and this flow ends. As a result, in the process of moving the focus lens 103 from the large blur state to the small blur state, the defocus difference between the large Def and the medium Def and the respective reliability evaluation values can be handed over from the large Def to the medium Def.

Determination of Handover from Medium Def to Small Def Next, determination of handover from medium Def to small Def in S703 of FIG. 7 will be described with reference to FIG. First, in S1001, it is checked whether or not the difference in defocus amount between the medium Def and the small Def is equal to or less than a preset second predetermined depth threshold. Here, the second predetermined depth threshold is set to, for example, three times the depth of focus so that the medium Def can be successfully transferred to the small Def. Further, by setting the predetermined depth threshold based on the focal depth, a uniform threshold can be set even if the F-number or the focus detection area changes. Also, the second predetermined depth threshold is set to a value smaller than the first predetermined depth threshold in S901 of FIG. This is because the variation in detection of the defocus amount increases in the order of small Def→medium Def→large Def, so the difference between medium Def and large Def is greater than the difference between small Def and medium Def. . In S1002, it is checked whether the reliability evaluation value of the small Def is equal to or higher than the reliability evaluation value of the medium Def, and whether the reliability evaluation values of both the medium Def and the small Def are higher than Rel0, and if so, the process proceeds to S1003. , otherwise, the process proceeds to S1004. In S1003, it is determined that the medium Def can be handed over to the small Def, and this flow ends. In S1004, it is determined that the medium Def cannot be handed over to the small Def, and this flow ends. As a result, in the process of moving the focus lens 103 from the small blurring state to the in-focus position, the defocus difference between the medium Def and the small Def and the respective reliability evaluation values can be handed over from the medium Def to the small Def.

By performing the above-described processing together with the focus drive amount control in Expression (5), it is possible to improve the focusing accuracy and shorten the AF time. In addition, it is possible to increase the number of shooting scenes that can be accurately focused while reducing the number of shooting scenes that are out of focus and preventing the harmful effect of focusing while the image is out of focus.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

10 lens 20 camera 102 diaphragm 103 focus lens 104 diaphragm driver 105 focus lens driver 106 lens controller 201 image sensor 204 AF signal processor 212 camera controller 213 AF controller

Claims (5)

A focus adjustment device that performs focus adjustment using an output signal from an image sensor that captures an image of a subject,
generating means for generating a plurality of pieces of focus detection information by performing processing using a plurality of filters having different bands on a pair of image signals generated from the output signal of the imaging device and having a phase difference according to the focus state of the subject image; ,
calculation means for calculating an evaluation value relating to the reliability of the focus detection information generated by the generation means;
In the moving image shooting mode, when the contrast of the subject satisfies a predetermined condition, the evaluation value regarding the reliability of the focus detection information is changed based on the aperture setting , and the moving image shooting mode; and setting means that does not change the evaluation value unless the contrast of the object satisfies a predetermined condition ,
When changing the evaluation value relating to the reliability of the focus detection information based on the setting of the aperture, the setting means sets the evaluation value at the first aperture value to a second aperture value smaller than the first aperture value. A focus adjustment device characterized in that the aperture value is made larger than the evaluation value. 3. The calculating means calculates the evaluation value regarding the reliability based on the standard deviation of the focus detection information calculated from the evaluation value indicating the amount of change in the correlation amount of the pair of image signals. 2. The focusing device according to 1. The focus detection information is a defocus amount,
When the evaluation value is the first evaluation value, the direction of the corresponding defocus amount is correct, and when the evaluation value is the second evaluation value higher than the first evaluation value, the corresponding defocus amount is 3. The focus adjustment device according to claim 1, wherein the focus amount can be focused. The plurality of filters with different bands includes a first filter and a second filter with a higher band than the first filter,
In the handover from the first defocus amount generated by the processing by the first filter to the second defocus amount generated by the processing by the second filter, in the case of the second evaluation value, the 4. The focus adjustment device according to claim 3 , wherein a condition for judging whether or not it is possible to take over is different from the case of the first evaluation value. A control method for a focus adjustment device that performs focus adjustment using an output signal from an image sensor that captures an image of a subject,
a generating step of generating a plurality of pieces of focus detection information by performing processing using a plurality of filters having different bands on a pair of image signals generated from the output signal of the imaging element and having a phase difference according to the focus state of the subject image; ,
a calculating step of calculating an evaluation value relating to the reliability of the focus detection information generated in the generating step;
In the moving image shooting mode, when the contrast of the subject satisfies a predetermined condition, the evaluation value regarding the reliability of the focus detection information is changed based on the aperture setting , and the moving image shooting mode; and a setting step of not changing the evaluation value unless the contrast of the subject satisfies a predetermined condition ,
In the setting step, when changing the evaluation value regarding the reliability of the focus detection information based on the setting of the aperture, the evaluation value at the first aperture value is changed to a second aperture value smaller than the first aperture value. A control method for a focus adjustment device, characterized in that the aperture value is made larger than the evaluation value.

Images