JP2019015818A - Control device, imaging device, control method, program, and storage medium - Google Patents

Abstract

To provide a control device capable of achieving faster AF operation while suppressing a decrease in display image quality.SOLUTION: A control device comprises: image generation means (15a) for generating a first image containing a first frequency component and a second image containing a second frequency component lower than the first frequency component; evaluation value generation means (14) for generating a first evaluation value from the first image and a second evaluation value from the second image; amount-of-change calculation means (15b) for calculating an amount of evaluation value change from the first evaluation value to the second evaluation value; and determination means (15c) for determining a focusing degree of the image on the basis of the amount of change.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an imaging apparatus capable of performing automatic focus adjustment.

  In the conventional automatic focus adjustment operation (AF operation), when the evaluation value used for performing focus adjustment after focusing is changed, the determination regarding restart of the focus adjustment operation and the literature relating to the focus adjustment operation are described below. There is.

  Patent Document 1 discloses that in a camera that performs a focusing operation using a contrast method, a lens position between a lens position corresponding to a focus evaluation value approximately equal to a focus evaluation value at the time of restart and a lens position corresponding to a maximum focus evaluation value. It is disclosed that the distance is set to the moving amount of the photographing lens at the time of restart. With such a configuration, the time required for focusing can be reduced, and quick focusing can be achieved.

JP 2003-107329 A

  However, when the evaluation value used for focus adjustment changes after performing focus search control, the camera disclosed in Patent Literature 1 performs focus search control again with a limited range. That is, the camera disclosed in Patent Document 1 does not determine whether or not the state is maintained after reaching the in-focus position. For this reason, since the camera disclosed in Patent Document 1 performs the focus search control even when the in-focus state is maintained, there may be a delay in shifting to the photographing operation and a deterioration in the quality of the display image.

  Therefore, an object of the present invention is to provide a control device, an imaging device, a control method, a program, and a storage medium capable of realizing a high-speed AF operation while suppressing deterioration in display image quality. .

  The control device according to one aspect of the present invention generates an image that generates a first image that includes a first frequency component and a second image that includes a second frequency component that is lower than the first frequency component. Means for generating a first evaluation value from the first image and generating a second evaluation value from the second image; and the second evaluation from the first evaluation value. A change amount calculation unit that calculates a change amount of the evaluation value to the value; and a determination unit that determines the degree of focus of the image based on the change amount.

  An imaging device as another aspect of the present invention includes an imaging device that photoelectrically converts an optical image formed through an imaging optical system, and the control device.

  According to another aspect of the present invention, there is provided a control method for generating a first image including a first frequency component and a second image including a second frequency component lower than the first frequency component. Generating a first evaluation value from the first image, generating a second evaluation value from the second image, and evaluating from the first evaluation value to the second evaluation value Calculating a change amount of the value; and determining a focus degree of the image based on the change amount.

  A program according to another aspect of the present invention causes a computer to execute the control method.

  A storage medium according to another aspect of the present invention stores the program.

  Other objects and features of the present invention are illustrated in the following examples.

  According to the present invention, it is possible to provide a control device, an imaging device, a control method, a program, and a storage medium that can realize a high-speed AF operation while suppressing deterioration in quality of a display image.

It is a block diagram of the imaging device in each embodiment. 3 is a flowchart illustrating an operation of the imaging apparatus according to the first exemplary embodiment. 6 is a flowchart illustrating an operation for determining a blur amount according to the first exemplary embodiment. 6 is an explanatory diagram of a scan AF operation in Embodiment 1. FIG. 3 is a flowchart illustrating an operation for maintaining focus in Embodiment 1. FIG. 5 is a configuration diagram of an image sensor in Example 2. 6 is a flowchart illustrating the operation of the imaging apparatus according to the second embodiment. 12 is a flowchart illustrating an operation for maintaining focus in Embodiment 2. 6 is a flowchart illustrating an operation of imaging surface phase difference AF in Embodiment 2.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  First, with reference to FIG. 1, an imaging apparatus according to Embodiment 1 of the present invention will be described. FIG. 1 is a block diagram of the imaging apparatus 1. In the imaging apparatus 1, 2 is a zoom lens group, and 3 is a focus lens group. Reference numeral 4 denotes a stop as a light amount adjusting means (exposure means) for controlling the amount of light beam transmitted through the imaging optical system including the zoom lens group 2 and the focus lens group 3. A lens barrel 31 includes a zoom lens group 2, a focus lens group 3, a diaphragm 4, and the like. The lens barrel 31 (imaging optical system) is configured integrally with the imaging apparatus main body. However, the present embodiment is not limited to this, and the imaging apparatus may include an imaging apparatus body and a lens barrel (imaging optical system) that can be attached to and detached from the imaging apparatus body.

  Reference numeral 5 denotes an image sensor (solid-state image sensor) that includes a CMOS sensor, a CCD sensor, or the like, and photoelectrically converts a subject image (optical image) formed via an imaging optical system and outputs an electrical signal (image data). . Reference numeral 6 denotes an imaging circuit that generates a predetermined image signal by receiving the electrical signal photoelectrically converted by the imaging sensor 5 and performing various image processing. Reference numeral 7 denotes an A / D conversion circuit that converts an analog image signal generated by the imaging circuit 6 into a digital image signal. Reference numeral 8 denotes a memory (VRAM) such as a buffer memory that receives the output of the A / D conversion circuit 7 and temporarily stores an image signal.

  Reference numeral 9 denotes a D / A conversion circuit that reads an image signal stored in the VRAM 8 and converts it into an analog signal, and converts it into an image signal in a form suitable for reproduction output. Reference numeral 10 denotes an image display device (LCD) such as a liquid crystal display device that displays an image signal output from the D / A conversion circuit 9. A storage memory (semiconductor memory) 12 stores image data. Reference numeral 11 denotes a compression / decompression circuit including a compression circuit and an expansion circuit. The compression / decompression circuit 11 reads the image signal temporarily stored in the VRAM 8 and performs a compression process and an encoding process on the image data in order to make it suitable for storage in the storage memory 12. The compression / decompression circuit 11 performs a decoding process and an expansion process for making the image data stored in the storage memory 12 into a form suitable for reproduction and display.

  Reference numeral 13 denotes an AE processing circuit that receives an output from the A / D conversion circuit 7 and calculates an evaluation value for automatic exposure (AE) processing. Reference numeral 14 denotes a scan AF processing circuit (evaluation value generation means) that calculates an evaluation value for an automatic focus adjustment (AF) process that receives an output from the A / D conversion circuit 7 and generates an AF evaluation value. Reference numeral 15 denotes a CPU (control device) having a built-in memory for controlling the imaging device 1.

  The CPU 15 includes an image generation unit 15a, a change amount calculation unit 15b, and a determination unit 15c. The image generation unit 15a generates a first image including a first frequency component and a second image including a second frequency component lower than the first frequency component. The image generation unit 15a generates a first image and a second image using a filter that extracts a signal in a specific frequency band. Then, the scan AF processing circuit 14 (evaluation value generation means) generates a first evaluation value from the first image and generates a second evaluation value from the second image. The change amount calculation means 15b calculates the change amount of the evaluation value from the first evaluation value to the second evaluation value. The determination unit 15c determines the degree of focus of the image based on the change amount. In this embodiment, the CPU 15 and the scan AF processing circuit 14 constitute a control device.

  Reference numeral 16 denotes a timing generator (TG) that generates a predetermined timing signal. Reference numeral 17 denotes a driver (drive unit) that drives the image sensor 5. Reference numeral 21 denotes an aperture drive motor that drives the aperture 4. Reference numeral 18 denotes a first motor drive circuit that drives and controls the aperture drive motor 21. A focus drive motor 22 drives the focus lens group 3. Reference numeral 19 denotes a second motor drive circuit that drives and controls the focus drive motor 22. A zoom drive motor 23 drives the zoom lens group 2. Reference numeral 20 denotes a third motor drive circuit that drives and controls the zoom drive motor 23.

  An operation switch (operation SW) 24 includes various switch groups. Reference numeral 25 denotes an EEPROM which is an electrically rewritable read-only memory in which programs for performing various controls and data used for performing various operations are stored in advance. 26 is a battery. Reference numeral 28 denotes a strobe light emitting unit. Reference numeral 27 denotes a switching circuit that controls the flash emission of the strobe light emitting unit 28. Reference numeral 29 denotes a display element (LED) that performs warning display or the like.

  Reference numeral 30 denotes a phase difference AF processing circuit. Reference numeral 33 denotes an AF auxiliary light unit, which includes an illuminating unit (light source such as an LED) that illuminates all or a part of the subject when an AF evaluation value is acquired. Reference numeral 32 denotes an AF auxiliary light unit driving circuit for driving the AF auxiliary light unit 33. Reference numeral 35 denotes a shake detection sensor that detects camera shake and the like. A shake detection circuit 34 processes a signal from the shake detection sensor 35. Reference numeral 36 denotes a face detection circuit that receives an output from the A / D conversion circuit 7 and detects a face position, a face size, and the like on the screen. Reference numeral 37 denotes a motion vector detection circuit for obtaining the amount of motion of the subject on the image signal from a plurality of image signals having different acquisition times.

  As the storage memory 12 that is a storage medium for image data and the like, a fixed semiconductor memory such as a flash memory or a semiconductor memory such as a flash memory that can be attached to and detached from the card-shaped or stick-shaped imaging device 1 is applied. Is possible. In addition, various types of memory such as a magnetic storage medium such as a hard disk or a floppy disk can be used as the storage memory 12.

  The operation switch 24 includes a main power switch for activating the imaging apparatus 1 to supply power, a release switch for starting a photographing operation (storage operation), a reproduction switch for starting a reproduction operation, and a zoom lens group 2 of the imaging optical system. It includes a zoom switch that moves and zooms. The release switch includes a first stroke (SW1) for generating an instruction signal for starting an AE process and an AF process performed prior to a shooting operation, and a second stroke (SW2) for generating an instruction signal for starting an actual exposure operation. Consists of a two-stage switch.

  Subsequently, the operation of the imaging apparatus 1 configured as described above will be described. First, the light flux of the subject that has passed through the lens barrel 31 of the image pickup apparatus 1 is imaged on the light receiving surface of the image sensor 5 after the amount of light is adjusted by the diaphragm 4. This subject image is converted into an electrical signal by photoelectric conversion processing by the image sensor 5 and is output to the imaging circuit 6. The imaging circuit 6 performs various signal processing on the input signal to generate a predetermined image signal. This image signal is output to the A / D conversion circuit 7 and converted into a digital signal (image data), and then temporarily stored in the VRAM 8. The image data stored in the VRAM 8 is output to the D / A conversion circuit 9, converted into an analog signal, converted into an image signal in a form suitable for display, and then displayed on the LCD 10 as an image. The image data stored in the VRAM 8 is also output to the compression / decompression circuit 11. The compression circuit in the compression / decompression circuit 11 performs compression processing on the image data to convert it into image data in a form suitable for storage, and stores the converted image data in the storage memory 12.

  When a playback switch (not shown) of the operation switches 24 is operated and turned on, a playback operation is started. At this time, the image data stored in the compressed form in the storage memory 12 is output to the compression / expansion circuit 11, subjected to decoding processing, expansion processing, etc. in the expansion circuit, and then output to the VRAM 8 and temporarily. Memorized. Further, this image data is output to the D / A conversion circuit 9 and converted into an analog signal, converted into an image signal in a form suitable for display, and then displayed on the LCD 10 as an image.

  On the other hand, the image data digitized by the A / D conversion circuit 7 is sent to the AE processing circuit 13, the scan AF processing circuit 14, the face detection circuit 36, and the motion vector detection circuit 37 separately from the VRAM 8 described above. Is also output. The AE processing circuit 13 receives the input digital image signal and performs arithmetic processing such as cumulative addition on the luminance value of the image data for one screen. Thereby, the AE evaluation value corresponding to the brightness of the subject is calculated. This AE evaluation value is output to the CPU 15.

  The scan AF processing circuit 14 receives the input digital image signal, extracts a predetermined frequency component of the image data through a band-pass filter (BPF), and performs arithmetic processing such as cumulative addition to obtain a predetermined band. An AF evaluation value signal corresponding to the component amount is calculated. Specifically, a plurality of (for example, four) band-pass filters (BPF), that is, a plurality of filters having different frequency characteristics are used as predetermined frequency components of image data corresponding to a partial area of the screen designated as the AF area. And performing arithmetic processing such as cumulative addition. Thereby, an AF evaluation value signal (a plurality of evaluation values having different frequency bands for evaluating the focus state) corresponding to the component amount of the predetermined band is calculated.

  The scan AF processing circuit 14 calculates a ratio of a plurality of AF evaluation value signals, and generates a plurality of AF evaluation value ratio signals (ratio signals). For example, a case where four AF evaluation value signals are calculated via four band pass filters will be described. That is, the first AF evaluation value signal calculated through the band-pass filter having the lowest frequency characteristic and the three AF evaluation value signals (second output) calculated through the other three band-pass filters. AF evaluation value signal, third AF evaluation value signal, and fourth AF evaluation value signal). As a result, three AF evaluation value ratio signals (second AF evaluation value signal / first AF evaluation value signal, third AF evaluation value signal / first AF evaluation value signal, and fourth AF evaluation value signal) / First AF evaluation value signal) is generated.

  The area for calculating the AF evaluation value and the AF evaluation value ratio signal (evaluation value ratio) is a central part or one part of any part on the screen, or is adjacent to the central part or any part on the screen. For example, there are a plurality of locations that are distributed, and a plurality of locations that are discretely distributed.

  Further, the four AF evaluation value signals calculated through the band pass filter may be divided by the difference value between the maximum value and the minimum value of the signal obtained without going through the band pass filter (BPF). That is, four AF evaluation value ratio signals (first AF evaluation value signal / difference value between maximum value and minimum value, second AF evaluation value signal / difference value between maximum value and minimum value, third AF) It is also possible to generate an evaluation value signal / a difference value between the maximum value and the minimum value). In this way, the scan AF processing circuit 14 detects the predetermined frequency component from the image signal generated by the imaging sensor 5 in the course of performing the AF process, and the ratio of the detected predetermined frequency component. It plays the role of the ratio signal calculation means. In the present embodiment, the case where four AF evaluation value signals are calculated via four band pass filters has been described, but the present invention is not limited to this. This embodiment can also be applied when at least two AF evaluation value ratio signals are calculated. That is, three or more AF evaluation value signals may be calculated via three or more types of bandpass filters.

  The face detection circuit 36 receives the input digital image signal, searches the image for a part characterizing the face such as eyes and eyebrows, and obtains the position of the person's face on the image. Further, the face detection circuit 36 obtains the size and inclination of the face from the positional relationship such as the interval between the parts characterizing the face.

  When a past image signal of a region for calculating a motion vector is recorded, the motion vector detection circuit 37 calculates a correlation between the image signal and the image signal acquired this time and obtains a motion vector. The image signal is replaced with the image signal acquired this time. If no past image signal is recorded, the motion vector detection circuit 37 records the image signal acquired this time in a predetermined area.

  The TG 16 outputs a predetermined timing signal to the CPU 15, the imaging circuit 6, and the imaging sensor driver 17. The CPU 15 performs various controls in synchronization with this timing signal. The imaging circuit 6 receives a timing signal from the TG 16 and performs various image processing such as separation of color signals in synchronization with the timing signal. The imaging sensor driver 17 receives the timing signal of the TG 16 and drives the imaging sensor 5 in synchronization with this timing signal.

  The CPU 15 controls the first motor drive circuit 18, the second motor drive circuit 19, and the third motor drive circuit 20. As a result, the CPU 15 drives and controls the diaphragm 4, the focus lens group 3, and the zoom lens group 2 via the diaphragm drive motor 21, the focus drive motor 22, and the zoom drive motor 23. That is, the CPU 15 controls the first motor drive circuit 18 based on the AE evaluation value calculated by the AE processing circuit 13 to drive the aperture drive motor 21 so that the aperture amount of the aperture 4 becomes appropriate. AE control to be adjusted is performed. Further, the CPU 15 controls the second motor drive circuit 19 based on the AF evaluation value signal or the AF evaluation value ratio signal calculated by the scan AF processing circuit 14 to drive the focus drive motor 22, and the focus lens group 3. AF control is performed to move to the in-focus position. When a zoom switch (not shown) among the operation switches 24 is operated, the CPU 15 moves the zoom lens group 2 by controlling the third motor drive circuit 20 and driving the zoom drive motor 23 in response to the operation. Then, a zooming operation of the imaging optical system is performed.

  Next, the shooting operation of the imaging apparatus 1 will be described with reference to FIG. FIG. 2 is a flowchart showing the shooting operation of the imaging apparatus 1. Each step in FIG. 2 is executed by each unit of the imaging apparatus 1 based on a command from the CPU 15.

  In this embodiment, the operation of acquiring the AF evaluation value is scanned while driving the focus lens group 3 to a predetermined position, the interval of the position of the focus lens group 3 for acquiring the AF evaluation value is the scan interval, and the AF evaluation value is acquired. The number is called the number of scan points. In this embodiment, the range for acquiring the AF evaluation value is called a scan range, and the region for acquiring an image signal for detecting the in-focus position is called an AF frame (AF region). The scan range is determined by the product of the scan interval and (the number of scan points−1). When the main power switch of the imaging apparatus 1 is in the ON state and the operation mode of the imaging apparatus 1 is the imaging (recording) mode, the imaging process sequence is executed, and imaging is performed by supplying power to the imaging sensor 5 and the like ( Shooting operation).

  First, in step S <b> 201, the CPU 15 displays an image formed on the image sensor 5 through the lens barrel 31 as an image on the LCD 10. In other words, the subject image formed on the image sensor 5 is converted into an electric signal by the photoelectric conversion process of the image sensor 5 and then output to the image pickup circuit 6. The imaging circuit 6 performs various signal processing on the input signal to generate a predetermined image signal. The predetermined image signal is output to the A / D conversion circuit 7, converted into a digital signal (image data), and temporarily stored in the VRAM 8. The image data stored in the VRAM 8 is output to the D / A conversion circuit 9, converted into an analog signal, converted into an image signal in a form suitable for display, and then displayed on the LCD 10 as an image.

  Subsequently, in step S202, the CPU 15 confirms the state of the release switch (determines whether SW1 is in the on state). When the photographer operates the release switch and the CPU 15 confirms that SW1 (first stroke of the release switch) is turned on, the process proceeds to step S203, and after initialization processing, the output of the AE processing circuit 13 is performed. A normal AE process is executed using.

  Subsequently, a scan AF process is performed in step S204. First, the scan AF processing circuit 14 is used to create an image obtained by intentionally reducing (blurred) the resolution using a bandpass filter on the acquired image. Then, a plurality of AF evaluation values (AF evaluation value signals) having different frequency components are obtained based on the original image and the image whose resolution is reduced. The CPU 15 (determination unit 15c) determines the amount of blur based on a plurality of AF evaluation value signals. Next, the scan AF processing circuit 14 has a plurality of AF evaluation values and a plurality of AF evaluation values based on the image signal acquired from the AF frame in the scan range set based on the set AF frame and the blur amount determination result. Find the ratio of. Then, the CPU 15 detects the in-focus position based on the result. When the CPU 15 stops the focus lens group 3 at the in-focus position, the scan AF processing circuit 14 obtains a plurality of AF evaluation values and a ratio of the AF evaluation values at that position, and records them in a predetermined recording area of the CPU 15. Details of the process in step S204 will be described later.

When the in-focus position is obtained in step S204, the process proceeds to step S205, and the CPU 15 performs AF display (AFOK display). Specifically, the CPU 15 performs processing such as displaying an AFOF display by turning on the display element 29 and simultaneously displaying a green frame on the LCD 10. If the in-focus position cannot be obtained in step S204, the process proceeds to step S205 and AFNG display is performed. Specifically, the CPU 15 performs processing such as displaying a yellow frame on the LCD 10 at the same time as performing AFNG display by blinking the display element 29 or the like.
Note that if the in-focus state is not in focus in the scan AF process in step S204 and the ON state of SW1 continues, the process proceeds to step S206. On the other hand, if SW1 is turned off in the case of out-of-focus, the process returns to step S201, and AF is performed again when SW1 is turned on.

  In step S206, the CPU 15 determines whether or not the in-focus state is maintained. In step S207, if there is no movement of the subject in the distance direction and there is no unintentional movement of the focus lens group 3 and the in-focus state is maintained, the process proceeds to step S208 and waits for SW2 to be turned on. . On the other hand, if the in-focus state is not maintained due to movement of the subject in the distance direction, etc., the process proceeds to step S201 (AF again) or step S210 (fine adjustment) depending on the degree.

  In step S208, the CPU 15 confirms the state of the release switch (determines whether SW2 is on). When the CPU 15 confirms that SW2 (second stroke of the release switch) is turned on, the process proceeds to step S209, and the CPU 15 executes an exposure process.

  In step S210, the scan AF processing circuit 14 scans in the vicinity of the current focus lens group 3 (for example, the five scan points in the vicinity) at the same scan interval as the normal scan, and acquires the AF evaluation value (focus fine Adjustment scan). In step S211, the CPU 15 determines whether or not the peak of the scan AF evaluation value is detected in step S210. When the peak is detected, the CPU 15 moves the focus lens group 3 to a position corresponding to the peak, and returns to step S207. On the other hand, if the peak cannot be detected, the process returns to step S201, and the CPU 15 performs AF again when the SW1 is turned on.

  Next, the blur amount determination (focus maintenance determination) performed in step S206 of FIG. 2 will be described. First, an AF frame setting method will be described. Follow any instructions from the photographer. For example, the size of the AF frame can be selected from 18% horizontal by 18% vertical and 10% horizontal by 10% vertical, and the position can be set so that the end of the AF frame does not protrude from the screen. And

  When the imaging apparatus 1 automatically sets the AF frame, if the main subject is a person, the AF frame is set using the output of the face detection circuit 36. The CPU 15 receives the size (for example, the number of pixels in the horizontal and vertical directions) and the position (for example, center coordinates) detected from the face detection circuit 36, and sets the AF frame so as to include the area. On the other hand, when the main subject is other than a person (when no face is detected by the face detection circuit), the CPU 15 detects the main subject by extracting feature points from the color information and luminance information of the image, and uses the information. To set the AF frame. When the CPU 15 extracts a region that can be regarded as having the same color and a region that can be regarded as having the same luminance from the color information and the luminance information, the CPU 15 detects a region where both overlap and detects a subject region candidate. Among them, a subject whose size is equal to or larger than a certain level and whose deviation from the center is equal to or smaller than a certain level is set as a main subject. When there are a plurality of subjects, the subject with the largest size is selected. If neither the face nor the main subject is detected in the above processing, an AF frame having a predetermined size (for example, horizontal 18% × vertical 18% of the screen) is set at one central position.

  Next, the blur amount determination will be described with reference to FIG. FIG. 3 is a flowchart showing an operation for determining the amount of blur. Each step in FIG. 3 is executed by the CPU 15 or the scan AF processing circuit 14.

  First, in step S301, the CPU 15 (scan AF processing circuit 14) acquires an image and obtains an AF evaluation value. The CPU 15 controls the scan AF processing circuit 14 to receive the input digital image signal within the AF frame, and a plurality of (for example, four) bandpass filters (BPFs) having different frequency characteristics. An AF evaluation value signal TesBef [n] is calculated. Then, the calculated evaluation value TesBef [n] is recorded in a predetermined area.

  Subsequently, in step S302, the CPU 15 performs a blurring process for reducing the resolution of the image. In other words, the input digital image signal within the AF frame is input to the scan AF processing circuit 14, and is input via a band pass filter (BPF) that extracts the low frequency band frequency in the scan AF processing circuit 14. Reduce the resolution of the digital image signal. Subsequently, in step S303, the CPU 15 calculates a plurality of AF evaluation value signals TesAft [n] from the image subjected to the blurring process in the same manner as in step S301, and records it in a predetermined area. Subsequently, in step S304, the CPU 15 calculates a blur amount determination index based on the change amount of the evaluation value before and after the blurring process, and determines the blur amount (focus degree).

  First, the CPU 15 obtains a change rate of each of the plurality of AF evaluation values. Four AF evaluation values before and after the blur processing, TesBef [0], TesBef [1], TesBef [2], TesBef [3], TesAft [0], TesAft [1], TesAft [2], TesAft [3] Suppose that is required. At this time, the change rates TesVari [0], TesVari [1], TesVari [2], and TesVari [3] of the AF evaluation value in each BPF are obtained as follows.

TesVari [0] = (TesAft [0] −TesBef [0]) / (TesAft [0] + TesBef [0])
TesVari [1] = (TesAft [1] −TesBef [1]) / (TesAft [1] + TesBef [1])
TesVari [2] = (TesAft [2] −TesBef [2]) / (TesAft [2] + TesBef [2])
TesVari [3] = (TesAft [3] −TesBef [3]) / (TesAft [3] + TesBef [3])
Then, the CPU 15 takes the sum of the absolute values of the rate of change (Σabs (TesVari [i])) and uses it as a blur amount determination index.

  Subsequently, in step S305, the CPU 15 evaluates the blur amount determination index. If the blur amount determination index is greater than or equal to the first predetermined value (for example, greater than or equal to 0.17), it is determined that there is no blur amount (in-focus), and the process proceeds to step S306. On the other hand, if the blur amount determination index is within a predetermined range between the first predetermined value and the second predetermined value (for example, not less than 0.11 and less than 0.17), it is determined that the amount of blur is small, and the process proceeds to step S307. . If the blur amount determination index is less than the second predetermined value (for example, less than 0.11), it is determined that the blur amount is large, and the process proceeds to step S308.

  In steps S306, S307, and S308, the CPU 15 sets a scan range of the scan AF process performed in step S204 based on the determined blur amount. Since it is determined in step S306 that there is no blur amount, the scan AF process performed in step S204 only needs to confirm in-focus. For this reason, the scan range is set to an extremely narrow range. The vicinity of the current focus lens group 3 is scanned around the current position. For example, 5 points in the vicinity, and the scan interval is set to an interval equivalent to 2.5 depth, which is half of the normal scan interval. Since it is determined in step SS307 that the blur amount is small, the scan range in the scan AF process performed in step S204 is set according to the degree of the blur amount.

  In this embodiment, when the blur amount determination index is the upper limit of the predetermined range (it can be considered that there is almost no blur amount), a narrow range centering on the current position of the focus lens group 3 is set as the scan range. On the other hand, when the blur amount determination index is the lower limit of the predetermined range (the amount of blur can be considered to be large to some extent), a wide range centering on the current position of the focus lens group 3 is set as the scan range. Actually, the number of scan points when the value of the exponent is the upper limit is MinP, and the number of scan points when the value of the exponent is the lower limit is MaxP.

MaxP- (MaxP-MinP) · (index value-lower limit value) / (upper limit value-lower limit value)
The scan range is obtained by the product of the scan interval and (number of scan points-1). The scan interval is the same as the conventional interval. For example, MaxP is 15 points, MinP is 5 points, the upper limit value is 0.17, the lower limit value is about 0.11, and the scan interval may be an interval corresponding to 5 depths. The positions obtained by shifting half of the obtained scan range from the position of the current focus lens group 3 to the infinity side and the close side are the opposite ends of the scan range. The scan interval is the same as the normal scan interval (for example, an interval corresponding to 5 depths). If the end of the scan range exceeds the position corresponding to infinity of the focus lens group 3 with an allowance, and the position equivalent to the closest end plus the allowance, the position with the respective allowance is added to the scan range. End position.

  In step S308, the same range as when the blur amount determination is not performed is set as the scan range. For example, scanning is started from a position where a margin is added to the position corresponding to infinity of the focus lens group 3, and the maximum value of the AF evaluation value is detected, or the position where the margin is added to the position corresponding to the closest end is reached. Scan until.

  Next, the scan AF process for detecting the in-focus position performed in step S204 will be described with reference to FIG. FIG. 4 is an explanatory diagram of the scan AF operation. As described above, the following processing is performed in the scan range set from the result of the blur amount determination in step S204.

  The CPU 15 changes the scan speed according to the scan interval. The CPU 15 also controls the focus lens group 3 to move to the next scan point within the sensor readout time. When the sensor reading rate is 100 fps, the sensor is driven at a speed at which the focus lens group 3 reaches the next scan point after 10 msec. Therefore, when the scan range is narrowed and the scan interval is narrowed accordingly, the scan speed is slowed down. As a result, when the position of the focus lens group 3 is close to the in-focus position before the scan AF, the scan interval is narrow and the scan speed is slow, so that the change in the display image on the LCD 10 is also small, and the display quality is improved.

  The scan AF processing circuit 14 receives the input digital image signal within the AF frame, and calculates a plurality of AF evaluation value signals via a plurality of (for example, four) band pass filters (BPF) having different frequency characteristics. To do. Among these, the first AF evaluation value signal calculated through the bandpass filter (BPF) having the highest frequency characteristic is a graph indicated by a broken line (TES3) in FIG. The horizontal axis in FIG. 4 is the position of the focus lens group 3, and the left side represents the position of the focus lens group 3 that focuses on a far object and the right side focuses on a near object. In scan AF, the focus lens group 3 is driven from position A to position B in FIG. 4 set based on the result of blur amount determination in step S204, a digital image signal is acquired, and an AF evaluation value signal is calculated. To do. Since the value becomes the maximum at the in-focus position, when the position C in FIG. 4 is set as the in-focus position, the result shown in FIG. 4 is obtained. The positions A and B are set based on the result of the blur amount determination in step S204.

  In order to increase the speed of scan AF, the stop positions of all the focus lens groups 3 are not performed, and an output is acquired at every predetermined scan interval (for example, an interval corresponding to 5 depths). In this case, an AF evaluation value signal may be acquired at three points a1, a2, and a3 in FIG. In such a case, the in-focus position C (the position where the AF evaluation value signal is maximized) is obtained by calculation from the point where the acquired AF evaluation value signal has the maximum value and the points before and after that point.

  The AF evaluation value indicates the maximum value Y1 (a2 in FIG. 4) when the focus lens group 3 is at the position X1, and the AF evaluation values are Y2 and Y3 (a1 in FIG. 4) at the positions X2 and X3 before and after that. , A3). At this time, the position X0 of the focus lens group 3 at the in-focus position C is obtained as in the following formula (1).

However, in the formula (1), Y1> Y3 and Y1 ≧ Y2.

  Conventionally, the in-focus position is obtained in this way. However, since the in-focus position is obtained after the focus lens group 3 has passed, there is an adverse effect on display quality such as once the image displayed on the LCD 10 is blurred, and time is required to return the focus lens group 3 to the in-focus position. There were problems such as becoming.

  Therefore, in this embodiment, the problem is solved by calculating a ratio of a plurality of AF evaluation value signals to generate a plurality of AF evaluation value ratio signals and obtaining the in-focus position using the ratio signals. For example, a case where four AF evaluation value signals are calculated via four band pass filters will be described. In this case, the first AF evaluation value signal TES0 calculated through the band-pass filter having the lowest frequency characteristic and the three AF evaluation value signals (via the other three band-pass filters) ( The second to fourth AF evaluation value signals TES1, TES2, and TES3) are divided. Thus, three AF evaluation value ratio signals (first AF evaluation value ratio signal TES1 / TES0, second AF evaluation value ratio signal TES2 / TES0, and third AF evaluation value ratio signal TES3 / TES0 are generated. .

  As shown in FIG. 4, this characteristic is minimal at a position away from the in-focus position by a predetermined position. The position at which the ratio signal is minimized is determined by the frequency characteristics of the band-pass filter that outputs the respective signals having the ratio. The higher the frequency characteristics of the bandpass filter as the denominator, the closer the position where the ratio signal becomes minimum is closer to the in-focus position. Therefore, the in-focus position can be obtained by obtaining a position where the ratio signal is minimized and adding a predetermined value determined from the frequency characteristics of the bandpass filter to the position. In FIG. 4, the frequency characteristics are set so as to be high frequency characteristics in the order of TES0, TES1, TES2, and TES3. Therefore, the respective peak positions are detected at positions close to the in-focus position in the order of TES1 / TES0, TES2 / TES0, and TES3 / TES0.

  However, it is assumed that the subject includes a frequency that matches the frequency characteristics of all the bandpass filters from the in-focus position. When there is only a specific frequency or when the contrast is strong only at the specific frequency, the ratio signal using the output of the bandpass filter having frequency characteristics other than the specific frequency cannot accurately detect the minimum position. In addition, a position different from the position to be detected by the bandpass filter having the frequency characteristic may be a minimum position under the influence of a frequency component having a strong contrast. Therefore, the peak position may not be detected in the order according to the set frequency characteristics.

  Therefore, in the present embodiment, a plurality of signals having different frequency characteristics (second AF evaluation value to fourth AF evaluation value, etc.) are converted into signals having a frequency characteristic lower than all other signals (the first AF evaluation value). A plurality of ratio signals divided by the AF evaluation value) are calculated. Then, a plurality of in-focus positions are obtained based on the minimum position of the plurality of ratio signals and the frequency characteristics of the signal when calculating the ratio signal. In addition, the reliability of a plurality of ratio signals is calculated from the detected maximum value of the ratio signal, and among the plurality of in-focus positions obtained from the plurality of ratio signals, only the one having a high ratio signal reliability is used. To calculate the in-focus position. Furthermore, when it is determined that the peak position of the signal TES3 having the highest frequency characteristic obtained until it is detected that the ratio signal has reached a peak is highly reliable, focusing is also performed using this peak position. Calculate the position.

  For example, consider the case in FIG. 4 where the reliability of TES3 is high and the in-focus position is obtained using a ratio signal between TES3 and FES. In this case, from the α2 point where the detected value of the ratio signal (TES3 / TES0) is minimized, the position of the focus lens group 3 at the adjacent points α1, α3, and the ratio signal value of the AF evaluation value, the following equation The minimum position is obtained using (2). The AF evaluation value becomes the minimum value Y1 (α2 in FIG. 4) at the position X1 of the focus lens group 3, and the AF evaluation values Y2 and Y3 (α1, α3 in FIG. 4) acquired at the positions X2 and X3 before and after the AF evaluation value. ). At this time, the position X0 of the focus lens group 3 at the in-focus position C is obtained by the following equation (2).

However, in the formula (2), Y1 <Y3 and Y1 ≦ Y2.

  Thus, the minimum position of the ratio signal and the peak position of the signal having the highest frequency characteristic (first AF evaluation value) are obtained. Then, the in-focus position C is obtained from the predetermined value determined from the frequency characteristic of the ratio signal and the in-focus position obtained from the minimum position of the ratio signal and the signal having the highest frequency characteristic (fourth AF evaluation value), The focus lens group 3 is driven to that position. Although the positions of the two coincide theoretically, in reality, they often do not coincide with each other because they include errors in the calculation process. Therefore, an average value of both is obtained, and the position is set as the in-focus position C. When the CPU 15 stops the focus lens group 3 at the in-focus position, the scan AF processing circuit 14 obtains a ratio of a plurality of AF evaluation values and AF evaluation values at the position, and records them in a predetermined recording area of the CPU 15. .

  Next, the reliability determination method will be described with reference to FIG. Except for special cases, the ratio signal of the AF evaluation signal has a valley shape as shown in FIG. 4 in the vicinity of the minimum position when the horizontal axis indicates the distance and the vertical axis indicates the ratio of the AF evaluation values. Therefore, whether or not the ratio of the AF evaluation values is valley-like in the vicinity of the peak, the difference between the maximum value and the minimum value of the ratio signal, the length of the portion that is inclined at a certain value or more, the portion that is inclined The reliability is judged by judging from the gradient of. However, the range to be calculated is a range of about 5 to 9 points near the minimum position, and the threshold for determining reliability is prepared for a ratio signal, and the band pass for which the value is also set Different for each frequency characteristic of the filter.

  As shown in FIG. 4, a scan point that is recognized as being inclined from the minimum position (point α <b> 2) is performed by examining the difference in the ratio of adjacent AF evaluation values. This check starts from the point α2, and the difference from the adjacent point (α1) on the infinity side is examined. If the point α2 is smaller, it is determined that the vehicle is inclined. Next, the difference in the ratio of the AF evaluation value between the point α1 and the adjacent point compared to the point α2 is examined. If the value of the point α1 is small and the difference is equal to or greater than a predetermined value, it is determined that the vehicle is inclined. This check is continued until the magnitude relationship between the values of adjacent points in the range is reversed or the difference is smaller than a predetermined value. In the same way, examine the near side.

  Then, the width to the point recognized as being inclined is the sum of the difference between the value at the infinity side recognized as being inclined and the difference between the value at the closest point and the point α2 as the width L of the mountain. It is calculated and used as the mountain slope SL. The maximum value and the minimum value are the maximum value and the minimum value of the entire ratio signal of the AF evaluation values within the range. The numerical values for determining the reliability of the ratio of the AF evaluation values thus obtained are compared with respective threshold values, and if all the conditions are satisfied, it is determined that the AF evaluation value is reliable. . This reliability determination is performed on the ratio signals of all AF evaluation values from which the minimum position is detected.

  The reliability of the AF evaluation value (TES3 or the like) is determined by whether or not the shape of the AF evaluation value is a mountain shape. The AF evaluation signal has a mountain shape when the horizontal axis indicates the distance and the vertical axis indicates the AF evaluation value, except for the special case of near-far competition. Therefore, whether or not the AF evaluation signal has a mountain shape is determined from the difference between the maximum value and the minimum value of the AF evaluation signal, the length of the inclined portion with an inclination greater than a certain value, and the gradient of the inclined portion. Thus, the reliability of the AF evaluation signal is determined. This specific method is described in Japanese Patent No. 4235422 and Japanese Patent No. 4185541.

  Next, with reference to FIG. 5, the focus maintenance determination performed in step S206 of FIG. 2 will be described. In steps S501 to S503 in FIG. 5, the same operation as the blur amount determination (steps S301 to S303 in FIG. 3) in step S204 in FIG. 2 is performed. The CPU 15 (scan AF processing circuit 14) calculates a plurality of AF evaluation value signals TesBef [n] before the blurring process and a plurality of AF evaluation value signals TesAft [n] after the blurring process, and records them in a predetermined area.

  Subsequently, in step S504, the CPU 15 compares the evaluation values before and after the blurring process, calculates and records a change amount index Sg_TesVariPi based on the change amount of the evaluation value. Similar to step S304 in FIG. 3, the change rates TesVari [1], TesVari [2], TesVari [3], and TesVari [4] of each of the plurality of AF evaluation values are obtained as follows.

TesVari [1] = (TesAft [1] −TesBef [1]) / (TesAft [1] + TesBef [1])
TesVari [2] = (TesAft [2] −TesBef [2]) / (TesAft [2] + TesBef [2])
TesVari [3] = (TesAft [3] −TesBef [3]) / (TesAft [3] + TesBef [3])
TesVari [4] = (TesAft [4] −TesBef [4]) / (TesAft [4] + TesBef [4])
Here, TesBef [1], TesBef [2], TesBef [3], TesBef [4], TesAft [1], TesAft [2], TesAft [3], and TesAft [4] are AF before and after the blurring process. It is an evaluation value. Then, the CPU 15 takes the sum of the absolute values of the change rates (Σabs (TesVari [i])) and sets it as a change index Sg_TesVariPi and records it in a predetermined area.

  Subsequently, in step S505, the CPU 15 performs a restart determination for determining whether to drive the focus lens group 3 stopped at the in-focus position again. The restart determination is performed using outputs from the shake detection circuit 34 and the AE processing circuit 13 in addition to the AF evaluation value calculated by the scan AF processing circuit 14. When the CPU 15 is in focus and starts this process, the AF evaluation value, the output of the shake detection circuit 34, and the output of the AE processing circuit 13 at that time are recorded in a predetermined area, and the restart determination is made. A new value is obtained and compared with the recorded value. As a result, if any of the values fluctuates by a predetermined value or more, the CPU 15 determines to drive the focus lens group 3 again, and the process proceeds to step S506. On the other hand, if the variation of all the values is less than the predetermined value, the CPU 15 determines not to drive again, and proceeds to step S512. In the case of out of focus, the CPU 15 determines not to restart. Thereby, when SW2 is turned on in an out-of-focus state, the process proceeds to step S209 while the focus lens group 3 is fixed.

  In steps S506 to S509, the CPU 15 obtains an AF evaluation value change index before and after the blurring process, similarly to steps S501 to S504. First, in step S506, the CPU 15 acquires an image within the AF frame, calculates a plurality of AF evaluation value signals TesBef [n] having different frequency characteristics, and records them in a predetermined area. Subsequently, in step S507, the CPU 15 performs a blurring process for reducing the resolution of the image through a band pass filter (BPF) that extracts a frequency in a low frequency band in the scan AF processing circuit 14. Subsequently, in step S508, as in step S506, the CPU 15 calculates a plurality of AF evaluation value signals TesAft [n] having different frequency characteristics from the blurred image and records them in a predetermined area.

Subsequently, in step S509, the CPU 15 calculates the change amount index Sg_TesVariNw based on the change amount of the evaluation value before and after the blurring process. The variation index is obtained by the same calculation formula as in step S504,
It is the sum of absolute values (Σabs (TesVari [i])) of the change rates TesVari [1], TesVari [2], TesVari [3], and TesVari [4] for each of the plurality of AF evaluation values.

  Subsequently, in step S510, the CPU 15 calculates the evaluation value change amount index Sg_TesVariNw calculated in step S509 and the in-focus evaluation value change amount index Sg_TesVariPi calculated in step S504 recorded in a predetermined area. Compare. Thereby, the CPU 15 determines whether or not the in-focus state is maintained. When it can be considered that both values are substantially equal in step S511, the process proceeds to step S512. In step S512, the CPU 15 sets the focus lens group 3 to remain fixed at that position, and ends this process.

  On the other hand, if the two values are significantly different (if the change is large), the process proceeds to step S513. In step S513, the CPU 15 sets to execute the scan AF process in step S204 again in response to SW1, and ends the present process.

  If the two values are different (if the change is small), the process proceeds to step S514. In step S514, the CPU 15 performs setting so as to perform the focus fine adjustment scan, and ends this processing.

  Def_Sg_TesVari is calculated from the two evaluation value change index Sg_TesVariNw and Sg_TesVariPi as shown in the following equation (3).

Def_Sg_TesVari = 2 · abs (Sg_TesVariPi−Sg_TesVariNw) / (Sg_TesVariPi + Sg_TesVariNw) (3)
Said determination is performed by calculating | requiring the coefficient which shows the change from the time of focusing of Def_Sg_TesVari represented by Formula (3), and an evaluation value change amount index | exponent, and comparing the value with a predetermined | prescribed threshold value. For example,
When Def_Sg_TesVari <Thr1 is satisfied using a threshold value Thr1 at which both values can be regarded as substantially equal, both values are regarded as substantially equal. When Def_Sg_TesVari> Thr2 is satisfied using a threshold value Thr2 that is considered to be largely different from each other, it is considered that the two values are greatly different from each other. When Thr1 ≦ Def_Sg_TesVari ≦ Thr2 is satisfied, the two values are considered to be different. For example, these threshold values may be about Thr1 = 0.05 and Thr2 = 0.15. Also, these threshold values can be set to different values during recording of a moving image and other than that. If the focus lens group 3 is driven due to a misjudgment during moving image recording, the frequency of giving discomfort is high when viewing the recorded moving image, and the recorded moving image is more likely to change the focus state smoothly. Often preferred. For this reason, during moving image recording, the value of the threshold Thr1 is increased to increase the frequency of fixing the focus lens group 3 as compared to other cases.

  Note that the processing in steps S501 to S504 may be executed only during focusing. Although not shown in order to avoid complications, when SW1 is turned off, the process returns to step S201, and the photographer's intentions such as release and preparation before photographing can be reflected.

  In the present embodiment, the first image obtained from the image sensor 5 is lower than the first image by passing through a band pass filter (BPF) that extracts the frequency of the low frequency band in the scan AF processing circuit 14. A high-frequency second image is generated. However, the present embodiment is not limited to this, and the low-frequency second image is acquired by other methods such as acquiring the first image and the low-frequency second image from the imaging sensor 5. You may do it. For example, the pixel addition function in the image sensor 5 is used to acquire a first image and a second image having a lower frequency than the first image in the driving mode of the solid-state imaging device having a larger number of added pixels. Thus, the first evaluation value and the second evaluation value are obtained.

  As described above, in this embodiment, the control device includes the image generation unit 15a, the evaluation value generation unit (scan AF processing circuit 14), the change amount calculation unit 15b, and the determination unit 15c. The image generation unit 15a generates a first image including a first frequency component and a second image including a second frequency component lower than the first frequency component. The evaluation value generation means (scan AF processing circuit 14) generates a first evaluation value from the first image, and generates a second evaluation value from the second image. The change amount calculation means 15b calculates the change amount (index indicating change) of each of the first evaluation value and the second evaluation value. The determination unit 15c determines (estimates) the degree of focus of the image (a focus state such as whether or not the focus state is maintained) based on the change amount.

  Preferably, the change amount calculation unit 15b obtains a difference between the first change amount calculated at the first timing (Sg_TesVariPi) and the second change amount calculated at the second timing (Sg_TesVariNw). The determination unit 15c determines the degree of focus based on the difference between the first change amount and the second change amount. More preferably, the first timing is an in-focus state, and the second timing is an in-focus maintenance determination time. Preferably, the determination unit 15c determines that the degree of focus is low when the difference between the first change amount and the second change amount is the first amount (an amount larger than the threshold value Thr1). On the other hand, when the difference between the first change amount and the second change amount is the second amount smaller than the first amount (an amount smaller than the threshold Thr1), the determination unit 15c has a high degree of focus. Is determined. Preferably, the determination unit 15c is configured such that the difference between the first change amount and the second change amount is smaller than a predetermined amount (threshold value Thr1) (the first change amount and the second change amount are substantially equal). ), It is determined that the in-focus state is maintained. On the other hand, when the difference between the first change amount and the second change amount is larger than the predetermined amount, the determination unit 15c determines that the in-focus state is not maintained. More preferably, the determination unit 15c changes a predetermined amount (determination threshold: threshold value Thr1) for determining that the in-focus state is maintained according to the depth of field at the time of shooting. More preferably, the determination unit 15c sets the predetermined amount to the first predetermined amount when the depth of field is the first depth of field. The determining unit 15c sets the predetermined amount to a second predetermined amount larger than the first predetermined amount when the depth of field is the second depth of field deeper than the first depth of field. To do.

  Preferably, the image generation unit 15a generates a plurality of images having different in-focus states for focus adjustment, and changes the number of the plurality of images according to the degree of focus determined by the determination unit 15c. More preferably, when the degree of focus is the first value, the image generation unit 15a sets the number of images acquired for focus adjustment to the first number. On the other hand, when the degree of focus is the second value higher than the first value, the image generation unit 15a sets the number of images acquired for focus adjustment to a second number smaller than the first number. Set to.

  Preferably, the change amount calculation unit 15b calculates the sum of change rates of each of the first evaluation value and the second evaluation value (Σabs (TesVari [i])) as the change amount. Then, the determination unit 15c determines the degree of focus based on the sum of the change rates. More preferably, the change rate is a value (TesVari [i]) obtained by dividing the difference between the first evaluation value and the second evaluation value by the sum of the first evaluation value and the second evaluation value. .

  As a result, it is determined whether or not the focus is maintained from the image obtained without moving the focus lens, and AF control is performed according to the result, thereby realizing both improvement in the quality of the display image and speeding up of the AF operation. be able to. Since the evaluation value change amount index is obtained by dividing the difference between the AF evaluation values obtained from the images before and after processing by the sum, it is affected by contrast, the brightness of the subject, and the detection error of the AE processing circuit 13 for determining the brightness. Hateful.

  Next, a second embodiment of the present invention will be described. The present embodiment is different from the first embodiment in that the imaging sensor 5 has an imaging plane phase difference AF function and the focusing operation is performed alone or in combination with this function.

  First, the configuration of the image sensor 5 of the present invention will be described with reference to FIG. 6A and 6B are configuration diagrams of the image sensor 5, in which FIG. 6A shows a cross-sectional view of the pixel, and FIG. 6B shows a plan view of the pixel. The two-dimensionally arranged pixels have a structure as shown in FIG. 6 in order to provide functions as normal imaging pixels and phase difference AF pixels. Two rectangular photodiodes are arranged for one microlens. One photodiode is a reference pixel pixel, and the other photodiode is a reference pixel pixel.

  Pixel rows for performing phase difference AF (phase difference AF pixel rows) are intermittently arranged, and the reference image pixel signals can be independently read out only in this row. The other pixel rows are read by adding the outputs of the pair of photodiodes. In addition, a signal read from the pixel can be amplified. Actually, reading is started for each row from the first row, and in the row for performing the phase difference AF, the signal obtained by adding the outputs of the pair of photodiodes and the signal of the reference image pixel are read. In this row, a signal obtained by adding the outputs of the pair of photodiodes is read out. When the phase difference calculation is performed, the difference between the added signal and the reference image pixel signal is taken to generate a reference image pixel signal. This processing is performed by the phase difference AF processing circuit 30. The added signal is used for recording in a storage memory and displaying on an image display device.

  Next, with reference to FIG. 7, the photographing operation of the imaging apparatus 1 will be described. FIG. 7 is a flowchart showing the shooting operation of the imaging apparatus 1. Each step in FIG. 7 is executed by each unit of the imaging apparatus 1 based on a command from the CPU 15. Steps S701 to S703 and S709 to S715 in FIG. 7 are the same as steps S201 to S203, S205, S208, S206, S207, S210, S211 and S209 in FIG.

  First, after performing the same processing as in the first embodiment in steps S701 to S703, the CPU 15 sets an AF frame in step S704. The specific method is the same as that of the first embodiment, and the result of detection of the main subject is performed by extracting feature points from the photographer's instruction, the output of the face detection circuit 36, and the color information and luminance information of the image performed by the CPU 15. Use to set AF frame. Subsequently, in step S705, the CPU 15 determines whether or not the imaging surface phase AF is possible. If the imaging plane phase difference AF is possible, the process proceeds to step S706. On the other hand, if the imaging plane phase difference AF is not possible, the process proceeds to step S708. This determination is made on the basis of the aperture value at the time of execution of the imaging surface phase difference AF and the relationship between the intermittently arranged pixel row and the AF frame for performing the phase difference AF.

  When the aperture value (F value) at the time of executing the imaging surface phase difference AF is smaller than a predetermined aperture (for example, F11), the CPU 15 determines that the imaging surface phase difference AF is impossible. When the aperture value is larger than the predetermined aperture, the number of pixel rows included in the AF frame for performing the phase difference AF, the contrast value of the signal for performing the phase difference AF (the maximum signal when performing the phase difference AF) The difference between the value and the minimum value) is determined in consideration of the amplification degree.

  The reliability Rel of the phase difference AF process can be obtained by the following equation.

Rel = Log2 (√ (number of rows in AF frame)) + Log2 (contrast value) + Log2 (1 / amplification degree)
If this value is greater than or equal to a predetermined value, the CPU 15 determines that imaging plane phase difference AF is possible. Log2 is a logarithm with a base of 2. For example, when it is determined that the number of lines in the AF frame is 4 or more, the PB value is 1.2 or more, and the amplification degree is 8 times or less, the predetermined value is −1.74.

  If imaging surface phase AF is possible, the CPU 15 performs imaging surface phase AF processing using the phase difference AF processing circuit 30 in step S706. This process will be described later. If it is determined that focusing is possible as a result of the imaging surface phase AF process, the process advances to step S709. On the other hand, if it is not determined that focusing is possible, the process proceeds to step S708. In step S <b> 708, the CPU 15 performs scan AF processing using the scan AF processing circuit 14. Since the scan AF process performs the same process as in the first embodiment except that the information set in step S704 is used as the AF frame, the description thereof is omitted.

  In step S709, the same AFOK or AFNG display processing as that in the first embodiment is performed, and then the process proceeds to step S710. In step S710, the CPU 15 confirms SW2 (second stroke of the release switch). If SW2 is on, the process proceeds to step S715 to execute actual exposure processing. On the other hand, if SW2 is not turned on, in step S711, the CPU 15 determines whether the in-focus state is maintained. In accordance with the result, the CPU 15 performs any of the lens fixing, focus fine adjustment, and re-AF processing as in the first embodiment.

  Next, with reference to FIG. 8, the focus maintenance determination performed in step S711 in FIG. 7 will be described. In addition, the description regarding the part similar to Example 1 demonstrated with reference to FIG. 5 is abbreviate | omitted.

  Steps S801 to S805 are the same as steps S501 to S505 in FIG. Subsequently, in step S806, the CPU 15 determines whether or not an in-focus state is finally obtained by the imaging plane phase difference AF in step S707. When the focused state is finally obtained by the imaging surface phase difference AF, the process proceeds to step S807. On the other hand, if the focus state is not finally obtained by the imaging plane phase difference AF, that is, if the focus state is finally obtained by the scan AF processing in step S708, the process proceeds to step S810.

  In step S807, the CPU 15 determines whether or not the imaging surface phase difference AF is possible. If the imaging surface phase difference AF is possible, the process proceeds to step S808. On the other hand, if the imaging surface phase difference AF is impossible, the process proceeds to step S810. This determination is the same processing as in step S705, and is performed based on the aperture value at the time of execution of the imaging plane phase difference AF and the relationship between the pixel rows and the AF frames that are intermittently arranged to perform the phase difference AF.

  In step S808, the CPU 15 performs imaging surface phase AF processing in the same manner as in step S706, and calculates the defocus amount. This process will be described later. In step S809, if the detected defocus amount is small and can be regarded as in-focus, the process proceeds to step S816. In step S816, the CPU 15 determines that the in-focus state is maintained. On the other hand, if it cannot be regarded as in-focus, the process proceeds to step S810.

  Steps S810 to S818 are the same as steps S506 to S514 in FIG. That is, the CPU 15 calculates an evaluation value change index from a plurality of AF evaluation value signals before and after the blurring process for reducing the resolution of the image, and compares the change index to determine whether or not the in-focus state is maintained. judge. Based on the determination result, the CPU 15 may set the focus lens group 3 to be fixed at that position, or to execute the scan AF process in step S204 again, or to perform a fine focus adjustment scan.

  Here, with reference to FIG. 9, the imaging plane phase AF process performed in step S706 will be described. First, in step S <b> 901, the CPU 15 records the phase difference AF image signal output from the A / D conversion circuit 7 in a predetermined recording area of the phase difference AF processing circuit 30. Subsequently, in step S902, the CPU 15 creates a reference image signal by taking the difference between the signal output by adding the standard image and the reference image and the standard image signal. Then, the pixels for phase difference AF are rearranged to generate a standard image and a reference image.

  In the imaging sensor 5 of the present embodiment, two rectangular photodiodes are arranged for one microlens, one photodiode for the reference pixel and the other for the reference pixel. Therefore, the reading order is “reference pixel + reference pixel” → “reference pixel” → “reference pixel + reference pixel” → “reference pixel” →... → “reference pixel + reference pixel” → “reference pixel” Is recorded in a predetermined recording area of the phase difference AF processing circuit 30. Therefore, only the pixels constituting the reference image are extracted, arranged in the extracted order as a reference image signal, and recorded in a predetermined recording area of the phase difference AF processing circuit 30. Further, the data obtained by adding the standard image and the reference image and the data of the standard pixel are extracted and the difference is obtained, arranged in the obtained order and used as a reference image signal, and recorded in a predetermined recording area of the phase difference AF processing circuit 30. To do.

  In step S903, the CPU 15 corrects the recorded image. In this embodiment, pixels for performing phase difference AF are arranged on the imaging surface. For this reason, as in the case of re-imaging with the secondary imaging optical system, the pupil position of the light beam incident on the field lens for correcting the image distortion caused by the difference in the light beam due to the image height or the phase difference AF sensor is limited. Therefore, it is impossible to dispose the aperture and the mask for blocking unnecessary light flux between the imaging surface and the sensor. As a result, the signal for phase difference AF needs to be corrected because the shading offset differs for each pixel. This shading varies depending on the pixel position (image height) from the optical axis center, the exit pupil position of the photographing lens, and the stop. For this reason, the image correction amount corresponding to each factor is used to correct the image for each pixel for phase difference AF according to each factor.

  The offset differs depending on the amplification factor of the phase difference AF image. Therefore, the image correction amount corresponding to each factor is used to correct the image for each pixel for phase difference AF according to each factor. Details of the image correction method are known, for example, in Japanese Patent Application Laid-Open No. 2012-252280, and thus description thereof is omitted.

Subsequently, in step S904, the CPU 15 sets an initial value for performing the correlation calculation. In step S905, the phase difference AF processing circuit 30 set with the initial value performs a correlation calculation according to the following equation (4) to obtain a correlation value between the reference image a _i and the reference image b _i .

U _k = Σmax (a _{j + 1} , b _{j + k} ) −Σmax (a _j , b _{j + k + 1} ) (4)
In equation (4), max (a, b) means the larger of a and b, k is the image shift amount for performing correlation calculation, j is the number of pixels for performing correlation calculation, and step S904. It has been initialized with.

  Subsequently, in step S906, the CPU 15 acquires and records the correlation value between the standard image and the reference image from the phase difference AF processing circuit 30. If there is a correlation value that has already been temporarily recorded, it is determined whether or not the sign of the correlation value is equal. If there is no temporarily recorded correlation value, the acquired correlation value is temporarily recorded.

  As a result, if the sign is inverted in step S907, or if it is determined that the acquired correlation amount is zero and the signal used for the phase difference AF process is reliable, the process proceeds to step S908. For this determination of reliability, the PB value (ratio between the maximum value and the minimum value of the signal), the similarity between the standard image and the reference image, and the like are used.

  The PB value is obtained by performing both image correction and low-pass processing when performing low-pass filter processing when performing correlation calculation on the signal before image correction of the reference image and reference image used for phase difference AF and the signal after image correction. Then, the average value of the ratio obtained from the six signals of the subsequent signals or the minimum value of the ratio is used. Also, the similarity between the standard image and the reference image is obtained by calculating the difference between the standard image and the reference image at the shift amount K at which the correlation value is minimized by the following equation (5) and using it as the similarity evaluation value.

Evaluation value of similarity = Σabs (a _j −b _{j + k} ) (5)
The CPU 15 determines that the reliability is reliable when the PB value is equal to or greater than a predetermined value and the similarity evaluation value is equal to or less than the predetermined value. The similarity evaluation value may be an expression other than the expression (5) described above, as long as it evaluates the similarity between both images. For example, the similarity evaluation value may be the ratio of the overlapping portion of both images to the standard image or reference image at the shift amount K at which the correlation value is minimized.

In step S908, the CPU 15 calculates an image shift amount that causes the correlation amount to be zero. Since the correlation value is calculated by shifting one pixel at a time, the correlation amount calculated by the phase difference AF processing circuit 30 is rarely zero. Therefore, an image shift amount for which the correlation amount becomes zero is obtained based on two correlation amounts having different signs and a shift amount that gives the correlation amount. As a result of calculating the correlation amount by the equation (4), if the sign of the correlation amount U _k is inverted between K = 1 and K = 1 + 1, the image shift amount δ at which the correlation amount becomes zero by linear interpolation is It is calculated | required like Formula (6).

δ = l + | U _l | / [| U _l | + CU _{l + 1} |] (6)
In equation (6), | z | means the absolute value of z.

  Subsequently, in step S909, the CPU 15 obtains a prediction amount P based on the image shift amount δ using the following equation (7).

P = δ−Δ (7)
In Expression (7), Δ is an image shift amount at the time of focusing.

  Then, by using the base line length determined according to the characteristics of the lens barrel 31, the defocus amount d (of the focus lens group) is used by using the predictive amount P as expressed by the following equation (8). (Movement amount and direction).

d = K · P (8)
In Expression (8), K is a coefficient for converting the prediction amount into the defocus amount. This value depends on the focal length of the lens barrel 31, the value of the diaphragm 4 (aperture value), the image height, and the like. For this reason, a table having these as parameters is prepared in the EEPROM 25, and values are obtained by referring to the table.

  If the sign is not inverted in step S907, the process proceeds to step S911. In step S907, the CPU 15 determines whether or not the shift amount has been reached in the end value of the calculation for obtaining the correlation value (the image shift position is the end). If the image shift position is not the end, the process proceeds to step S913. In step S913, the CPU 15 replaces the acquired correlation value with the temporarily recorded correlation value, and then updates the image shift amount k to k + 1. On the other hand, if the image shift position is the end, the process proceeds to step S911. In step S911, the CPU 15 determines that the phase difference AF is NG, and ends this process.

  In the present embodiment, when the imaging surface phase difference AF pixels of the imaging sensor 5 are intermittently arranged, it is possible to determine whether the in-focus state is more accurately maintained by using the function together. it can. Further, the image sensor in which two rectangular photodiodes are arranged for one microlens has been described as an example, but the present invention can also be applied to an image sensor having another structure capable of phase difference AF. For example, the present invention can also be applied to an image pickup device having a structure in which a light-shielding curtain is provided in the vicinity of a microlens so that only a part of a light beam of an exit pupil enters a photoelectric conversion unit. In each embodiment, a compact digital camera has been described as an example. However, each embodiment can also be applied to AF during a live view of a digital video camera or a digital single-lens reflex camera.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  According to each embodiment, by determining the degree of focus from the difference in AF evaluation values of images having different frequency characteristics, it is determined whether the focus is maintained based on the image obtained without moving the focus lens. Then, AF control is performed according to the determination result. Therefore, according to each embodiment, it is possible to provide a control device, an imaging device, a control method, a program, and a storage medium capable of realizing a high-speed AF operation while suppressing deterioration in display image quality. Can do.

  As mentioned above, although the preferable Example of this invention was described, this invention is not limited to these Examples, A various deformation | transformation and change are possible within the range of the summary.

14 Scan AF processing circuit (evaluation value generating means)
15a Image generation means 15b Change amount calculation means 15c Determination means

Claims (16)

Image generating means for generating a first image including a first frequency component and a second image including a second frequency component lower than the first frequency component;
Evaluation value generating means for generating a first evaluation value from the first image and generating a second evaluation value from the second image;
A change amount calculating means for calculating a change amount of the evaluation value from the first evaluation value to the second evaluation value;
Determining means for determining the degree of focus of the image based on the amount of change. The calculating means obtains a difference between the first change amount calculated as the change amount at a first timing and the second change amount calculated as the change amount at a second timing;
The control device according to claim 1, wherein the determination unit determines the focus degree based on the difference between the first change amount and the second change amount.   The control device according to claim 2, wherein the first timing is an in-focus time, and the second timing is an in-focus maintenance determination time. The determination means includes
When the difference between the first change amount and the second change amount is a first amount, it is determined that the degree of focus is low;
The focus degree is determined to be high when the difference between the first change amount and the second change amount is a second amount smaller than the first amount. The control device according to 2 or 3. The determination means includes
When the difference between the first change amount and the second change amount is smaller than a predetermined amount, it is determined that the in-focus state is maintained,
The in-focus state is determined not to be maintained when the difference between the first change amount and the second change amount is greater than the predetermined amount. Control device.   The control device according to claim 5, wherein the determination unit changes the predetermined amount that determines that the in-focus state is maintained in accordance with a depth of field at the time of shooting. The determination means includes
If the depth of field is a first depth of field, the predetermined amount is set to a first predetermined amount;
When the depth of field is a second depth of field that is deeper than the first depth of field, the predetermined amount is set to a second predetermined amount that is greater than the first predetermined amount. The control device according to claim 6. The image generating means includes
Generate multiple images with different focus states for focus adjustment,
The control device according to claim 1, wherein the number of the plurality of images is changed according to the degree of focus determined by the determination unit. The image generating means includes
When the degree of focus is the first value, the number of the images to be acquired for the focus adjustment is set to the first number,
When the degree of focus is a second value higher than the first value, the number of images to be acquired for the focus adjustment is set to a second number smaller than the first number. The control device according to claim 8. The change amount calculating means calculates a sum of change rates of the first evaluation value and the second evaluation value as the change amount,
The control device according to claim 1, wherein the determination unit determines the degree of focus based on a sum of the change rates.   The rate of change is a value obtained by dividing the difference between the first evaluation value and the second evaluation value by the sum of the first evaluation value and the second evaluation value. The control device according to claim 10.   The control device according to claim 1, wherein the image generation unit generates the second image using a filter that extracts a signal in a specific frequency band. An image sensor that photoelectrically converts an optical image formed via the imaging optical system;
Based on the image data output from the image sensor, a first image including a first frequency component and a second image including a second frequency component lower than the first frequency component are generated. Image generating means;
Evaluation value generating means for generating a first evaluation value from the first image and generating a second evaluation value from the second image;
A change amount calculating means for calculating a change amount of the evaluation value from the first evaluation value to the second evaluation value;
An image pickup apparatus comprising: a determination unit that determines an in-focus degree of an image based on the change amount. Generating a first image including a first frequency component and a second image including a second frequency component lower than the first frequency component;
Generating a first evaluation value from the first image and generating a second evaluation value from the second image;
Calculating a change amount of the evaluation value from the first evaluation value to the second evaluation value;
Determining the degree of focus of the image based on the amount of change. Generating a first image including a first frequency component and a second image including a second frequency component lower than the first frequency component;
Generating a first evaluation value from the first image and generating a second evaluation value from the second image;
Calculating a change amount of the evaluation value from the first evaluation value to the second evaluation value;
And causing the computer to execute a step of determining the degree of focus of the image based on the amount of change. A storage medium storing the program according to claim 15.