JP6935286B2 - Imaging device and its control method - Google Patents

Description

The present invention relates to an imaging device having an autofocus function.

In recent years, imaging devices such as cameras and videos have a high pixel count, and even a slight defocusing becomes conspicuous, and more accurate focusing is required.

This request is the same for stars in the night sky, and a technique is known in which a star is regarded as a minute point light source and focusing is performed so that the area of a high-luminance signal is strictly minimized. When shooting a starry sky, the stars to be shot are limited to subjects located at almost infinity, and the exposure is also set unique to the starry sky, so there may be modes that are independent of other scene modes (below). , Called starry sky shooting mode).

Normally, the focus position in which the subject located at almost infinity is in focus is a position uniquely determined in the infinite focus adjustment performed for each individual image pickup device.

However, the focus may shift during shooting due to the temperature difference between the temperature at the time of adjustment and the temperature difference of the imaging device or the posture difference when actually shooting the night sky. Therefore, it is necessary to focus frequently even during shooting of a star whose distance to the image pickup device does not change during shooting (Fig. 2).

Also, in shooting the starry sky, the city light can be regarded as a point light source in the same way as the stars, and the city light is finite with respect to the stars located at almost infinity. The focus position is slightly different.

As mentioned earlier, the starry sky shooting mode is also required to focus with higher precision, so it is required to correct even a slight focus difference such as the light of a star and the city, which have almost the same appearance.

Here, as typical means for focusing, there are a contrast AF method and a phase difference AF method.

In the contrast AF method, automatic focus adjustment is performed using an evaluation value obtained by filtering a specific frequency component from the luminance signal obtained from the image sensor.

In the phase-difference AF method, light beams from subjects that have passed through different exit pupil regions in the imaging optical system are imaged on a pair of line sensors, and an image is taken from the phase difference of a pair of image signals obtained by the pair of line sensors. Automatic focus adjustment is performed by calculating the amount of defocus of the optical system.

In general, the phase-difference AF method has the advantage that the time required for automatic focus adjustment is shorter than that of the contrast AF method.

However, since the area of a minute point light source such as a star becomes smaller as it comes into focus, it becomes difficult to obtain a clear phase difference, and the focus accuracy may deteriorate.

Therefore, a scene with a minute point light source is counted as one of the scenes that the phase difference AF method is not good at. Therefore, when there is a point light source and other subjects within the same ranging range, a method has been proposed in which the detection accuracy of the subject on the screen is improved by dividing into a plurality of areas and determining the brightness level (Patent Documents). 1).

Japanese Unexamined Patent Publication No. 2010-2438999

However, in the prior art, when there is a point light source and other subjects in the screen, even if the detection accuracy of the subject other than the point light source can be improved by minimizing the influence of the point light source, the point light source It does not improve the detection accuracy of itself.

As described above, it is difficult for a minute point light source to produce a clear phase difference. Therefore, in the phase difference AF method, the detection variation of the defocus amount (hereinafter referred to as reliability) becomes large (reliability) at the focused position in focus. It tends to be less sexual (Fig. 3).

The present invention has been made in view of the above points, and an object of the present invention is to provide an image pickup apparatus capable of focusing on a minute point light source with high accuracy in a phase difference AF method. It is in.

As a technical feature of the present invention, imaging having an imaging means having a plurality of pixels capable of photoelectrically converting light beams that have passed through different pupil regions of an imaging optical system including a focus lens and outputting a pair of image signals. It is a control method of the device, that is, a calculation step of acquiring the image signal and performing a focus detection calculation of the phase difference AF method to calculate the defocus amount, and driving the focus lens based on the calculation result of the calculation step. A control step for controlling the focus to adjust the focus and a setting step for setting a plurality of focus detection areas are provided. In the calculation step, the focus lens serves as a reference focus for a predetermined subject stored in advance. In a state where the subject is located at a predetermined depth away from the position, it is determined whether or not the subject exists according to the brightness information of the focus detection region, and the defocus amount of the focus detection region where the subject exists is used to determine whether or not the subject exists. It is characterized in that the defocus amount is calculated.

According to the present invention, in the phase difference AF method, it is possible to focus with high accuracy even in a scene where a minute point light source such as a starry sky exists.

It is a block diagram of a digital camera. It is a figure which shows the focus detection area of 1st Embodiment. It is a figure which showed the peak bottom of the luminance value of A image and B image of a small frame, and the defocus amount of the large frame. It is a figure which shows the setting example of the focal point detection area. It is a figure which shows the relationship between the posture of an image pickup apparatus, and a scene. It is a figure which shows the image signal obtained from a focal point detection area. It is a figure which shows the correlation amount waveform. It is a figure which shows the correlation change amount waveform. It is a flowchart which shows the focusing process. It is a flowchart which shows the focus detection area setting process at the time of always subdividing the focus detection area. It is a flowchart which shows the focus detection area setting at the time of subdividing the focus detection area when the metering value of a scene is more than a predetermined value. It is a flowchart which shows the focus detection area setting at the time of subdividing the focus detection area when the focal length of a zoom lens is less than a predetermined value. It is a flowchart which shows the focus detection area setting at the time of subdividing the focus detection area when the posture of the image pickup apparatus is near the horizontal direction. It is a flowchart which shows the focus detection area setting at the time of changing the subdivision number in the central part and the peripheral part of the angle of view of a focal point detection area. It is a flowchart which shows the focus detection area setting at the time of subdividing the focus detection area when the F value which is one of the optical information of an interchangeable lens is more than a predetermined value. It is a flowchart which shows the defocus amount calculation process. It is a flowchart which shows the reliability determination process. It is a figure (the 1) which shows the focal point detection area of the 2nd Embodiment. It is a figure (the 2) which shows the focal point detection area of the 2nd Embodiment. It is a flowchart which shows the focusing process at the time of always moving a focus detection area in the starry sky photography mode in 2nd Embodiment. It is a flowchart which shows the focusing process at the time of switching whether or not to perform the 2nd lap according to the metering value of the scene in 2nd Embodiment. It is a flowchart which shows the focusing process at the time of changing the direction which shifts a focus detection area according to the posture of the image pickup apparatus in 2nd Embodiment. It is the luminance information and the deviation rate (No. 1) of the focal point detection region of the third embodiment. It is the luminance information and the deviation rate (No. 2) of the focal point detection region of the third embodiment. It is the luminance information and the deviation rate (No. 3) of the focal point detection region of the third embodiment. It is a flowchart which shows the focus detection area setting of 3rd Embodiment. It is a flowchart which shows the reliability determination of 3rd Embodiment. It is a flowchart which shows the modification of the reliability determination of FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(First Embodiment)
FIG. 1 is a block diagram showing a configuration example of a digital camera. The lens barrel 101 holds a lens group inside the lens barrel 101 to drive the lens. Here, although the lens barrel 101 is an interchangeable lens unit in the present embodiment, it may be composed of a binding lens unit. The zoom lens 102 optically changes the angle of view by adjusting the focal length. The aperture and shutter 103 are used for exposure control to adjust the amount of light. The focus lens 104 adjusts the focus position. The zoom lens 102, the aperture and shutter 103, and the focus lens are imaging optical systems.

The light that has passed through the lens barrel 101 is received by an imaging device 105 that uses a CCD (charge-coupled device), CMOS (complementary metal oxide semiconductor), or the like, and is photoelectrically converted from an optical signal to an electric signal. As will be described later, in the image sensor 105, the CDS / AGC / AD converter 106 performs noise removal processing, gain adjustment, and digitization on the image pickup signal read from the image sensor 105. The CDS / AGC / AD converter 106 outputs the imaging signal to the AF pixel correction unit 109 and the imaging surface phase difference AF signal to the phase difference AF signal processing unit 110.

Since the phase difference AF signal processing unit 110 performs focus detection processing by the phase difference method, the phase difference AF signal processing unit 110 performs a correlation calculation based on two image signals for imaging surface phase difference AF obtained from light flux passing through different pupil regions of the imaging optical system. And calculate the amount of image shift. The details of the image shift amount calculation process in the phase difference AF signal processing unit 110 will be described later.

The timing generator 108 controls the conversion timing of the image sensor 105 into an electric signal and the output timing of the CDS / AGC / AD converter 106 according to the command of the camera control unit 140. The image processing circuit 111 performs pixel interpolation processing, color conversion processing, and the like on the output from the AF pixel correction unit 109, and then sends the output as image data to the internal memory 112.

The display unit 113 displays shooting information and the like together with the image data stored in the internal memory 112. The compression / decompression processing unit 114 compresses / decompresses the data stored in the internal memory 112 according to the image format. The storage memory 115 stores various data such as parameters. The operation unit 116 is a user interface for performing various menu operations and mode switching operations. The posture detection unit 117 detects the posture of the imaging device. The temperature detection unit 118 detects the current temperature of the imaging device.

The camera control unit 140 is composed of an arithmetic unit such as a CPU (central processing unit), and executes various control programs stored in the internal memory 112 in response to a user operation by the operation unit 116. Various control programs are programs for performing, for example, automatic exposure control, zoom control, automatic focusing control, and the like.

The aperture shutter drive unit 121 drives the aperture and the shutter 103. The luminance signal calculation unit 125 calculates the signal that has passed through the CDS / AGC / AD converter 106 and the AF pixel correction unit 109 after being output from the image sensor 105 as the luminance of the subject and the scene.

The exposure control unit 124 controls the exposure value (aperture value and shutter speed) based on the luminance information obtained by the luminance signal calculation unit 125, and notifies the calculation result to the aperture shutter drive unit 121. As a result, automatic exposure control (AE control) is performed.

The zoom lens driving unit 123 drives the zoom lens 102. The zoom control unit 132 controls the zoom lens position according to the zoom operation instruction by the operation unit 116.

The focus lens driving unit 122 drives the focus lens 104. The defocus amount calculation unit 129 calculates the defocus amount based on the image shift amount calculated by the phase difference AF signal processing unit 110. The focus control unit 127 realizes focus adjustment by controlling the drive direction and drive amount of the focus lens from the output result of the defocus amount calculation unit 129. The evaluation value calculation unit 126 extracts a specific frequency component from the luminance information obtained by the luminance signal calculation unit 125, and then calculates it as a contrast evaluation value. The scanning control unit 128 issues a command to the focus control unit 127 to drive a predetermined range with a predetermined drive amount, and at the same time, acquires an evaluation value which is a calculation result of the evaluation value calculation unit 126 at a predetermined focus position to obtain contrast. Calculate the shape of.

The focus position at which the contrast shape calculated by the scanning control unit 128 is the apex is the focusing position. Automatic focusing control (AF) in which the luminous flux is focused on the image sensor 105 surface by driving to the focus position calculated by the defocus amount calculation unit 129 or driving to the focusing position calculated by the scanning control unit 128. Control) is performed.

The interchangeable lens information acquisition unit 134 acquires information such as a focal length and an F value, which are optical characteristic information for each lens unit.

The starting point holding unit 130 holds the focus lens position that focuses on the infinite subject. In general, the focus lens position for an infinite subject varies in each imaging device, so the focus lens position for an infinite subject is adjusted for each individual. This adjusted position will be referred to as the starting point. However, the adjusted focus lens position may shift due to temperature changes, posture changes, and changes over time. Since the member forming the lens barrel 101 may shrink due to a temperature change, the unit portion including the focus lens 104 inside is also affected by the temperature change. Further, when the posture of the image pickup apparatus changes, the unit portion including the focus lens 104 may move in a direction in which the fitting backlash is clogged due to its own weight. Furthermore, as the years pass, the grease in the moving part may change over time, or it may wear due to repeated drive of the focus lens.

Due to these effects, a phenomenon occurs in which the starting point becomes unfocused with respect to an infinite subject.

The departure point calculation unit 131 calculates the amount (referred to as the separation point) that the focus position shifts due to the influence of temperature change, posture change, and aging change with respect to the starting point held by the starting point holding unit 130. The defocus amount calculation unit 129 calculates the defocus amount by executing the calculation of the correlation amount at the starting point or the defocus point calculated by the defocus calculation unit 131. In this way, starting from the reference focal position such as infinite adjustment, focus detection is performed at points (separation points) separated by a predetermined depth in the far side direction and the near side direction, and the correlation amount is calculated. As a result, a highly reliable (less variation) defocus amount can be calculated, so that it is possible to focus on a minute point light source with high accuracy.

The focus detection area setting unit 133 sets the number and size of areas for calculating the amount of image shift (hereinafter referred to as focus detection areas).

The luminance deviation rate calculation unit 135 calculates the luminance rate indicating the degree of luminance separation between the A image and the B image with respect to the peak bottom of the luminance of the image signal (A image, B image).

FIG. 2A shows an example in which the focus detection area is set to 25 frames of 5 × 5. The upper left is one frame, and the lower right is set up to 25 frames. There is a non-detection area (dark black part) between each frame, and the amount of defocus of the subject in the non-detection area cannot be calculated. A pair of image signals (referred to as A image and B image) for focus detection are acquired from the image sensor 105 for each of the arbitrary outlines. The acquired pair of signals are row-added and averaged in the vertical direction in order to reduce the influence of signal noise. Next, a filter process is performed in which a signal component in a predetermined frequency band is extracted from the signal obtained by vertical row addition averaging. Subsequently, the calculation of the amount of correlation between the image signals is performed on the filtered signal (also called a correlation calculation).

FIG. 2B shows a case where there is a large bright star in the outline. For example, if a star with a low magnitude exists within the outline, the brightness value of the image signal with the star becomes high, and even after row addition averaging in the vertical direction, it is not buried in a dark part other than the star. As a result, the amount of correlation of the star image can be calculated accurately.

FIG. 2C shows the case where there is a small dark star in the outline. For example, if a star with a high magnitude is present in the outline, the brightness value of the image signal with the star becomes low, and after row addition averaging in the vertical direction, it may be buried in a dark part other than the star. As a result, the amount of correlation of the star image cannot be calculated accurately.

Therefore, as shown in FIG. 2D, the outline is subdivided into strips in the vertical direction. For example, the large frame is subdivided into seven small frames and called 1 small frame to 7 small frames, respectively. At this time, it is determined from the brightness information of each subframe that the dark small stars exist in the 3 subframes and the 6 subframes, and each subframe in which the stars exist is subdivided even after row addition averaging in the vertical direction. It will not be buried in the dark part due to the effect of conversion.

Next, an example in which the focus detection calculation of a small dark star can be performed by the effect of subdivision will be described with specific numerical values. FIG. 3A shows a specific peak bottom of the brightness values of the A image and the B image of the outline corresponding to FIG. 2C, and the defocus amount of the outline. FIG. 3B shows the specific peak bottoms of the brightness values of the A image and the B image of the seven subdivided small frames corresponding to FIG. 2D, and the defocus amount of the large frame. .. Focusing on the peak bottom of the brightness values of the A image and the B image of each subframe in FIG. 3B, the numerical values of the 3rd and 6th subframes with dark small stars are higher than those of the other subframes. high.

Therefore, for example, by setting the threshold value to 500, a small frame with a star can be detected, and 1.0, which is the average of the defocus amounts of 1.02 and 0.98 for each of the 3 small frames and the 6 small frames, is 1.0. It is the amount of defocus of the stars existing in this outline. However, in FIG. 3A, the row with stars is buried in the dark part due to the influence of the averaging, and the defocus amount cannot be calculated accurately.

Next, the contents of changing the number of subdivisions according to the outline will be described.

FIG. 4A shows an example in which the focus detection region is set to 25 frames of 5 × 5 as in FIG. 2A. The frame visible to the photographer is the outer frame of 401, and the boundary of each large frame and the undetected area are not visible. Usually, the photographer often arranges the star as the subject in 402 in the center of the screen.

Therefore, by increasing the number of subdivisions in the central part of the screen as shown in FIG. 4 (b), the accuracy of star detection is improved, and the number of subdivisions is small in the peripheral part of the screen as shown in FIG. 4 (c). By doing so, the correlation calculation load can be reduced.

FIG. 5A is a composition taken with the image pickup device arranged in the vicinity in the horizontal direction. If it is placed near the horizontal direction, the city light often enters the frame, and bright stars become relatively dark, which may reduce the detection accuracy. In order to deal with such a case, it is possible to suppress a decrease in detection accuracy by increasing the number of subdivisions in the outline in the vicinity of the horizontal direction.

FIG. 5B is a composition taken with the image pickup device arranged in the vicinity of the zenith direction. If it is arranged in the direction of the zenith, it is less likely that city lights will enter the frame, and star lights can be detected accurately. Therefore, in the case of the vicinity of the zenith direction, it is possible to reduce the number of subdivisions in the outline and reduce the load of correlation calculation.

The luminance deviation rate calculation unit 135 calculates the luminance rate indicating the degree of luminance separation between the A image and the B image with respect to the peak bottom of the luminance of the image signal (A image, B image).

P, q, s, and t in FIGS. 6A to 6C represent coordinates in the horizontal direction (x-axis direction), respectively, and p and q are the start point and end point of the pixel region, and s and t, respectively. Are the start and end points of the focus detection region, respectively. The solid line 601 is one image signal A for focusing detection that has been filtered, and the broken line 602 is the other image signal B.

6A shows the image signals A and B before the shift, FIG. 6B shows the image signals A and B shifted in the positive direction, and FIG. 6C shows the image signals A and B shifted in the negative direction. ing. When calculating the correlation amount of the pair of image signals A601 and B602, both the image signals A601 and B602 are shifted by arbitrary constant bits in the direction of the arrow. The sum of the absolute values of the differences between the image signals A601 and B602 after shifting is calculated. Hereinafter, the bit width to be shifted is set to 1 for the sake of simplification of the description.

When the shift amount is i, the maximum shift amount in the minus direction is ps, the maximum shift amount in the plus direction is qt, the start coordinate of the focus detection area 602 is x, and the end coordinate is y, the correlation amount (hereinafter referred to as COR). Can be calculated by the following equation (1).

Here, the range of the shift amount i is ps <i <qt.

FIG. 7 is an example of the relationship between the shift amount and COR. The horizontal axis represents the shift amount, and the vertical axis represents the COR. In FIG. 7A, extreme values 702 and 703 exist in the COR waveform 701 that changes with respect to the shift amount. The degree of coincidence between the pair of image signals A and B is the highest in the shift amount corresponding to the smaller COR among those extreme values.

In FIG. 7B, the difference in the amount of correlation at every other shift in the extreme value 703 of the COR waveform 701 is calculated as the amount of correlation change. Assuming that the shift amount is i, the maximum shift amount in the minus direction is ps, and the maximum shift amount in the plus direction is qt, the correlation change amount ΔCOR can be calculated by the following equation (2). At this time, the relationship of p−s + 1 <q−t-1 is established.

Here, the range of the shift amount i is ps <i <qt.

FIG. 8 is an example of the relationship between the shift amount and the correlation change amount ΔCOR. The horizontal axis represents the shift amount, and the vertical axis represents the correlation change amount ΔCOR. In FIG. 8A, in the ΔCOR waveform 801 that changes with respect to the shift amount, the portion of 802,803 changes from plus to minus. The state in which the amount of correlation change is 0 is called zero cross, and the degree of coincidence between the pair of image signals A and B is the highest. Therefore, the shift amount that gives zero cross is the image shift amount.

In FIG. 8B, 802 in FIG. 8A is enlarged and displayed. 804 is a part of the correlation change amount 801. The shift amount (k-1 + α) that gives a zero cross is divided into an integer part β (= k-1) and a decimal part α. The fractional part α can be calculated by the following equation (3) from the similarity relationship between the triangle ABC and the triangle ADE in the figure.

The integer part β can be calculated by the following equation (4).

The amount of image shift can be calculated from the sum of these α and β.

When there are a plurality of zero crosses having a correlation change amount ΔCOR as shown in FIG. 8A, the one having a larger maxder in the vicinity thereof is designated as the first zero cross. Here, maxder is an index indicating the ease of performing focus detection, and the larger the value, the easier it is to perform more accurate focus detection. The maxder can be calculated by the following formula (5).

In the following embodiment, when there are a plurality of zero crosses of the correlation change amount ΔCOR, the first zero cross is determined by the maxder, and the shift amount that gives the first zero cross is defined as the image shift amount.

FIG. 9 is a flowchart illustrating the focusing process. Step S901 is the start of the focusing process. Step S902 is an initialization process, and performs general initialization processing such as initialization of variables used in the image pickup apparatus. Step S903 determines whether or not the starry sky shooting mode is set. When the user selects the starry sky shooting mode as the shooting mode by the operation unit 116, the process proceeds to the next step S904 to determine whether or not the user has executed the focusing in the starry sky shooting mode by the operation unit 116. As described above, the starting point adjusted for each individual may shift due to temperature change, posture change, and aging change.

Therefore, the user often focuses on the star even during the shooting of the star whose distance from the image pickup device is almost the same. If focusing is not performed, monitoring continues until focusing is performed. In step S905, the brightness of the subject and the scene detected by the brightness signal calculation unit 125 is acquired as a photometric value.

Step S906 acquires the focal length from the current zoom lens position controlled by the zoom control unit 132. In step S907, the posture of the image pickup apparatus is acquired by the posture detection unit 117. As a specific example, by detecting the tilt angle of the image pickup device with an acceleration sensor, it is possible to acquire whether the image pickup device is facing the horizontal direction or the upward zenith direction.

In step S908, the interchangeable lens information acquisition unit 134 acquires information such as the focal length and F value, which are optical characteristic information for each lens unit. Step S909 sets the exposure for the photometric value. The exposure setting here is specialized for focusing in the starry sky shooting mode, and unlike the exposure during normal shooting, the exposure suitable for calculating the defocus amount is set.

Therefore, even if the subject has a brightness that does not cause whiteout (pixel saturation) in normal shooting, whiteout (pixel saturation) may occur during focusing in the starry sky shooting mode.

Step S910 is a process of setting the focus detection area, and the details will be described with reference to the flowchart of FIG.

First, a case where the focus detection region is always subdivided in the starry sky photography mode of FIG. 10-1 will be described. Step S1001 is the start of setting the focus detection area. Step S1002 is the start of the loop processing for each frame, and step S1004 is the end of the loop processing.

In the case of the focus detection region of FIG. 2A, the number of loops is 25, which is the same as the number of large frames, but the number of large frames may be any number.

In step S1003, the large frame is vertically subdivided into seven small frames. Here, the number of subdivisions is set to 7 as in the example of FIG. 2D, but any number may be used as long as the number of vertical pixels is not exceeded. Step S1005 is the end of the focus detection area setting.

Next, a case where the focus detection region is subdivided when the photometric value of the scene of FIG. 10-2 is equal to or higher than a predetermined value will be described. Here, when the metering value is equal to or more than a predetermined value, it indicates that the scene is bright, and when the influence of the city light is large, the detection accuracy of dark and small stars tends to decrease. In step S1006, it is determined whether or not the metering value acquired in advance is equal to or greater than a predetermined value. When it is equal to or more than a predetermined value, it is determined in step S1007 whether or not the current number of subdivisions is less than 7.

If it is less than 7, it is subdivided into 7 by step S1003, and if it is 7 or more, it is changed so as to increase the number of subdivisions by step S1008. The value of + α when increasing the number of subdivisions is a predetermined value as a parameter.

Next, a case where the focal length detection region is subdivided when the focal length of the zoom lens of FIG. 10-3 is less than a predetermined value will be described. Here, when the focal length is less than a predetermined value, it means that the angle of view is on the wide-angle side, and at that time, the size of the star looks small, so the detection accuracy tends to decrease.

In step S1009, it is determined whether or not the focal length acquired in advance is less than a predetermined value. Judgment The following is the same as the case of the above-mentioned metering value.

Next, a case where the focus detection region is subdivided when the posture of the image pickup apparatus shown in FIG. 10-4 is near the horizontal direction will be described. Here, when the attitude of the image pickup device is near the horizontal direction, there is a high possibility that there is a city light at the lower part of the screen as compared with the vicinity of the zenith direction, so that the detection accuracy of dark and small stars tends to decrease.

In step S1010, it is determined whether or not the posture of the image pickup apparatus acquired in advance is near the horizontal direction. Judgment The following is the same as the case of the above-mentioned metering value.

Next, a case where the number of subdivisions is changed in the central portion and the peripheral portion of the angle of view of the focus detection region of FIG. 10-5 will be described.

The photographer generally places the star, which is the subject, in the center of the angle of view. Therefore, in step S1011, when the outline is 7, 8, 9, 12, 13, 14, 17, 18, and 19, the subdivision is performed into seven. The other outline is subdivided into two according to step S1012. Here, the two subdivisions may be any number as long as they are less than seven. Further, the above is an example of the number indicating the central outline, and for example, only 13 may be used. Judgment The following is the same as the case of the above-mentioned metering value.

Next, a case where the focus detection region is subdivided when the F value, which is one of the optical information of the interchangeable lens of FIG. 10-6, is equal to or higher than a predetermined value, will be described. Here, when the F value is equal to or more than a predetermined value, it indicates that the lens is darker, and the amount of light that can be captured by the lens is reduced, so that the star detection accuracy tends to decrease. In step S1013, it is determined whether or not the F value of the interchangeable lens acquired in advance is equal to or greater than a predetermined value. Judgment The following is the same as the case of the above-mentioned metering value.

The details of the defocus amount calculation process in step S911 will be described with reference to the flowchart of FIG.

Step S1101 is the start of the defocus amount calculation process. Steps S1102 and S1117 are the start and end of the loop processing for the number of small frames.

The number of loops is a value determined by the number of large frames and small frames. When the number of large frames is 25 and each large frame is subdivided into 7 small frames, a total of 175 small frames are looped. Step S1103 acquires a pair of A image and B image data for focus detection from the image sensor 105 with respect to the frame set in the focus detection area setting unit. In step S1104, the A image and B image data acquired in step S1103 are subjected to row addition averaging processing in the vertical direction to reduce the influence of signal noise.

Step S1105 is a filter process for extracting components in a predetermined frequency band from the A image and B image data averaged in step S1104. Step S1106 executes a correlation calculation between image signals on the data filtered in step S1105. Step S1107 adds the correlation amount COR, which is the result of the correlation calculation in step S1106.

In step S1108, the difference of the correlation amount COR calculated in step S1107 every other shift is calculated as the correlation change amount ΔCOR. In step S1109, the zero cross in which the sign of the correlation change amount ΔCOR calculated in step S1108 changes is calculated, and the shift amount giving this zero cross, that is, the image shift amount is calculated. Step S1110 determines whether or not at least one zero cross calculated in step S1109 exists.

If there is no zero cross, step S1114 sets NULL, which indicates that there is no defocus amount. When at least one zero cross exists, it is determined in step S1111 whether or not there are a plurality of zero crosses. When there are two or more zero crosses, the defocus amount at which maxder, which is an index indicating the ease of performing focus detection, is maximized is calculated in step S1112.

If there is only one zero cross, the defocus amount for that zero cross is calculated in step S1113. Step S1116 is the end of the defocus amount calculation process.

The details of the reliability determination in step S912 will be described with reference to the flowchart of FIG.

Step S1201 is the start of reliability determination. Steps S1202 and S1206 are the start and end of the loop. The number of loops is the same as the number of defocus amount calculation processes described above. In step S1203, the peak value and the bottom value of the brightness values of the A image and B image data are acquired and the peak bottom is calculated.

Step S1204 compares the peak bottom of the luminance value calculated in step S1203 with a predetermined value. When the peak bottom is equal to or more than a predetermined value, it means that a star can be detected in the frame. The defocus amount calculated in advance for this frame is acquired in step S1205. A comparison is performed for all the frames, and it is determined in step S1207 whether or not at least one of them has acquired a defocus amount that is not NULL.

If the defocus amount can be obtained, the average value of the defocus amounts is calculated in step S1208, and this is used as the final defocus amount. If the defocus amount cannot be obtained, the final defocus amount is set to NULL in step S1209. Step S1212 is the end of the reliability determination.

In step S913, the focus lens 104 is moved to the final defocus amount, that is, in-focus. Step S914 is the end of the focusing process.

According to this embodiment, in the phase difference AF method, it is possible to focus on a minute point light source with high accuracy.

(Second Embodiment)
In the present embodiment, when there is an undetected region between the plurality of focal points detection regions with respect to the predetermined mode, the focus detection region is moved after performing the first defocus amount calculation. Then, after performing the second defocus amount calculation, the value obtained by adding and averaging the first and second defocus amounts is used as the subject defocus amount. Since the configuration of the digital camera and the method of calculating the defocus amount of the phase difference AF method are the same as those of the first embodiment, the description thereof will be omitted.

FIG. 13A shows an example in which the focus detection area is set to 25 frames of 5 × 5. The upper left is one frame, and the lower right is set up to 25 frames. There is a non-detection area (dark black part) between each frame, and the amount of defocus of the subject in the non-detection area cannot be calculated. The frame visible to the photographer is the outer frame of the outline, and the boundary of each outline and the undetected area shall not be visible. In the figure, the non-detection area is arranged in the horizontal direction, but it may be in the vertical direction.

If it is a normal subject (person, building, vehicle, etc.), there is no influence of the undetected area between each frame, but if it is a minute subject and the absolute number of the subject is small, it may not be possible depending on the composition. It may enter the detection area. As an example, if the stars seen in the night sky are only dark stars and the number is small, it may not be possible to focus.

Therefore, FIG. 13B shows an example in which the focus detection region is shifted downward. By shifting by the undetected area, the stars that have entered the undetected area will be rearranged in the detected area. As a result, stars that could not be detected by the first focusing can be detected after the shift (the first is called the first lap detection, and the next is called the second lap detection).

In the second lap detection, a pair of image signals (referred to as A image and B image) for focus detection are acquired from the image sensor 105 for each of the arbitrary outlines. The acquired pair of signals are row-added and averaged in the vertical direction in order to reduce the influence of signal noise.

Next, a filter process is performed in which a signal component in a predetermined frequency band is extracted from the signal obtained by vertical row addition averaging. Subsequently, the calculation of the amount of correlation between the image signals is performed on the filtered signal (also called a correlation calculation).

Next, the case where the focal length is on the telephoto side will be described.

FIG. 14A shows an example in which the focus detection region is set to 25 frames of 5 × 5 as in FIG. 13A, and there is a non-detection region between them. It differs from FIG. 13 (a) in that it zooms to the telephoto side.

Since it is on the telephoto side, even a small star is enlarged and displayed on the wide-angle side as shown in the figure. Therefore, in this case, the non-detection region is not entered, and even if the focus detection region is shifted as shown in FIG. 14B, the situation does not change significantly from that before the shift. Therefore, it is possible to omit the second lap detection in order to reduce the correlation calculation load when the telephoto side has a predetermined magnification or more.

FIG. 15 is a flowchart illustrating the focusing process in the present embodiment.

FIG. 15-1 is a description of the case where the focus detection region is always moved in the starry sky photography mode.

Step S1501 is the start of the focusing process. Step S1502 is an initialization process, and performs general initialization processing such as initialization of variables used in the image pickup apparatus. Step S1503 determines whether or not the starry sky shooting mode is set.

When the user selects the starry sky shooting mode as the shooting mode by the operation unit 116, the process proceeds to the next step S1504 to determine whether or not the user has executed the focusing in the starry sky shooting mode by the operation unit 116. As described above, the starting point adjusted for each individual may shift due to temperature change, posture change, and aging change. Therefore, the user often focuses on the star even during the shooting of the star whose distance from the image pickup device is almost the same.

If focusing is not performed, monitoring continues until focusing is performed. In step S1505, the brightness of the subject and the scene detected by the brightness signal calculation unit 125 is acquired as a photometric value. Step S1506 acquires the focal length from the current zoom lens position controlled by the zoom control unit 132.

In step S1507, the posture of the image pickup apparatus is acquired by the posture detection unit 117. As a specific example, by detecting the tilt angle of the image pickup device with an acceleration sensor, it is possible to acquire whether the image pickup device is facing the horizontal direction or the upward zenith direction.

Step S1508 sets the exposure for the photometric value. The exposure setting here is specialized for focusing in the starry sky shooting mode, and unlike the exposure during normal shooting, the exposure suitable for calculating the defocus amount is set. Therefore, even if the subject has a brightness that does not cause whiteout (pixel saturation) in normal shooting, whiteout (pixel saturation) may occur during focusing in the starry sky shooting mode.

Step S1509 sets the focus detection area for the first lap. Step S1510 is a defocus amount calculation process (on the first lap). The details of the defocus amount calculation process will be described later.

Step S1511 sets the focus detection area for the second lap. In the second lap, the focus detection area is shifted in the direction covering the non-detection area. Step S1512 is a defocus amount calculation process (on the second lap). The details of the defocus amount calculation process will be described later.

Step S1513 is a reliability determination. The details of the reliability determination will be described later. In step S1514, the focus lens 104 is moved to the final defocus amount, that is, in-focus. Step S1515 is the end of the focusing process.

Next, a process of switching whether or not to perform the second lap according to the photometric value of the scene of FIG. 15-2 will be described. Since steps up to step S1508 and steps S1513 and subsequent steps are the same as in FIG. 15-1, they are omitted. In step S1516, it is determined whether or not the metering value acquired in advance is equal to or greater than a predetermined value. Here, when the metering value is equal to or more than a predetermined value, it indicates that the scene is bright. For example, when the influence of the city light is large, the detection accuracy of dark and small stars tends to decrease. Therefore, the focus detection area is moved when the metering value of the current scene is equal to or greater than a predetermined value.

In step S1517, the focus detection area for the first lap is set. Step S1518 is a defocus amount calculation process (on the first lap). Next, in step S1519, the focus detection area for the second lap is set. At this time, the non-detection region in the focus detection region set in the first lap shifts in the detection direction in the second lap. Step S1520 is a defocus amount calculation process (on the second lap).

When the metering value is less than a predetermined value, the entire scene is dark, and the number of stars on the screen tends to increase. Therefore, it is extremely rare that a star exists only in the undetected area. In such a scene, only the first lap is calculated in order to reduce the calculation load. In step S1521, the focus detection area for the first lap is set. Step S1522 is a defocus amount calculation process (on the first lap). Although FIG. 15-2 describes the process of switching whether or not to perform the second lap according to the photometric value of the scene, instead, the process of switching whether or not to perform the second lap according to the focal length is executed. You may. Since the only difference from FIG. 15-2 is step S1516, this determination will be described. It is determined whether or not the focal length acquired in advance is less than a predetermined value. Here, when the focal length is less than a predetermined value, it means that the angle of view is on the wide-angle side, and the detection accuracy tends to decrease because the size of the star is small. Therefore, the focal length detection region is moved when the focal length is less than a predetermined value. Since steps 1517 and subsequent steps are the same as above, they are omitted.

Next, a process of changing the direction of shifting the focus detection region according to the posture of the image pickup apparatus shown in FIG. 15-3 will be described. Since steps up to step S1508 and steps S1513 and subsequent steps are the same as in FIG. 15-1, they are omitted.

In step S1525, it is determined whether or not the posture of the image pickup apparatus acquired in advance is near the horizontal direction. When it is near the horizontal direction, the focus detection area of the first lap is set in step S1517. Step S1518 is a defocus amount calculation process (on the first lap).

Step S1519 is the setting of the focus detection area in the second lap, and shifts upward so that the non-detection area is covered with respect to the focus detection area set in the first lap. If it is placed near the horizontal, the city light will often enter the frame, which will lead to a decrease in the accuracy of star detection. Therefore, in the case of a horizontal vicinity, it is desirable to shift upward and then detect the second lap in order to keep away from the influence of city lights as much as possible. Step S1520 is a defocus amount calculation process (on the second lap). When the posture is not near the horizontal direction, the focus detection region of the first lap is set in step S1521.

Step S1522 is a defocus amount calculation process (on the first lap). Step S1523 is the setting of the focus detection area in the second lap, and shifts downward so that the non-detection area is covered with respect to the focus detection area set in the first lap. If it is placed near the zenith, it is less likely that city lights will enter the frame, but the more it is directed toward the zenith, the more the airplane will cross, and it may not be possible to accurately detect the starlight. Therefore, in the case of the vicinity of the zenith, it is desirable to shift downward and then detect the second lap in order to keep away from the influence of the airplane as much as possible.

Step S1524 is a defocus amount calculation process (on the second lap).

According to this embodiment, in the phase difference AF method, it is possible to focus on a minute point light source with high accuracy.

(Third Embodiment)
In the present embodiment, the threshold value of the deviation rate is set for a predetermined mode, and the value obtained by adding and averaging the defocus amounts of all the focus detection regions whose deviation rate is less than the threshold value is defined as the defocus amount of the subject. Since the configuration of the digital camera and the method of calculating the defocus amount of the phase difference AF method are the same as those of the first embodiment, the description thereof will be omitted.

FIG. 16A shows an example in which the focus detection area is set to 25 frames of 5 × 5. The upper left is one frame, and the lower right is set up to 25 frames. There is a non-detection area (dark black part) between each frame, and the amount of defocus of the subject in the non-detection area cannot be calculated. A pair of image signals (referred to as A image and B image) for focus detection are acquired from the image sensor 105 for each of the arbitrary outlines. The acquired pair of signals are row-added and averaged in the vertical direction in order to reduce the influence of signal noise.

Next, a filter process is performed in which a signal component in a predetermined frequency band is extracted from the signal obtained by vertical row addition averaging. Subsequently, the calculation of the amount of correlation between the image signals is performed on the filtered signal (also called a correlation calculation).

The luminance deviation rate calculation unit 135 calculates the luminance rate indicating the degree of luminance separation between the A image and the B image with respect to the peak bottom of the luminance of the image signal (A image, B image).

In the formula (6), the average value AvePB of the peak bottom PB (A) of the brightness of the A image and the peak bottom PB (B) of the brightness of the B image is calculated. In the formula (7), the deviation rate indicating how much the brightness values of the A image and the B image are separated from each other is calculated by using the average value AvePB calculated by the formula (6).

FIG. 16B shows an enlarged view of the 12 outlines of FIG. 16A. In the 12 outlines, there is one very bright star at the top and one moderately bright star at the bottom. FIG. 16 (c) shows the relationship between the brightness value of the star of FIG. 16 (b) and the deviation rate. As an example, for a very bright star in the upper part, the peak bottom of the brightness of the A image is 2264, and the peak bottom of the brightness of the B image is 4015.

At this time, the divergence rate is 27.9% from the equations (6) and (7). On the other hand, for a moderately bright star, the peak bottom of the brightness of the A image is 1536, and the peak bottom of the brightness of the B image is 1794.

At this time, the divergence rate is 7.7% from the equations (6) and (7). In the case of a very bright star, the brightness may be partially saturated, and the phase difference may not be accurately obtained due to the influence. Set the threshold of the deviation rate that does not affect the phase difference by the parameter. For example, if it is judged that the defocus amount is reliable if it is less than 15%, use the upper defocus amount in the 12 outlines. Instead, the lower defocus amount will be used.

FIG. 17A is a diagram showing the relationship between the peak bottom of the brightness of the A image and the B image of each small frame and the deviation rate when the large frame is subdivided into 7 small frames.

Since there are stars in the 3rd and 6th subframes, the peak bottom of the brightness of the A image and the B image is larger than that of the other subframes in the dark part. In the example, the divergence rates of the 3rd and 6th subframes are 15.8% and 12.2%. On the other hand, FIG. 17 (b) shows the relationship between the peak bottom and the deviation rate of the brightness of the upper and lower frames A and B when the large frame of FIG. 17 (a) is subdivided into two. It is a figure.

At this time, as compared with FIG. 17A, the peak bottoms of the brightness of each of the A image and the B image are averaged due to the influence of the dark part, which is a small value. Therefore, the divergence rate decreases to 9.0% and 7.5%, respectively.

FIG. 18A is a diagram showing the relationship between the peak bottom of the brightness of the A image and the B image of each small frame and the deviation rate when the large frame is subdivided into 7 small frames.

FIG. 18B is a diagram showing the relationship between the peak bottom and the deviation rate of the brightness of the A image and the B image of each small frame when FIG. 18A is zoomed to the telephoto side. Since the stars in FIG. 18B look larger than those in FIG. 18A, very bright stars and the like may be partially saturated. In the example, there are stars in the 3rd frame, the 4th frame, and the 5th frame, but the deviation rate of the 4th frame is particularly large at 31.8%.

Since very bright stars tend to be partially saturated on the telephoto side, it is necessary to set the threshold value of the deviation rate of the brightness value to a larger value than that on the wide-angle side. However, since it is considered that the reliability of the defocus amount is low for the 4th small frame due to the influence of saturation, about 20% is appropriate as the threshold value in order to exclude the 4th small frame.

The process of setting the focus detection area (details of step S910) in the present embodiment will be described with reference to FIG. In the following, the focus detection area is always subdivided in the starry sky photography mode. Step S1901 is the start of the focus detection area setting. Step S1902 is the start of the loop processing for each frame, and step S1904 is the end of the loop processing.

In the case of the focus detection region of FIG. 16A, the number of loops is 25, which is the same as the number of large frames, but the number of large frames may be any number.

Step S1903 subdivides the large frame vertically into seven small frames. Here, the number of subdivisions is set to 7, but any number may be used as long as the number of subdivisions does not exceed the number of vertical pixels. In step S1905, the number of subdivisions is stored in the memory in order to be used in the reliability determination described later. Step S1906 is the end of the focus detection area setting.

Next, the process of performing the reliability determination (details of step S912) in the present embodiment will be described with reference to FIG.

Step S2001 is the start of reliability determination. Steps S2002 and S2008 are the start and end of the luminance value deviation rate determination loop.

In step S2003, the peak value and the bottom value of the brightness values of the A image and B image data are acquired and the peak bottom is calculated. Step S2004 compares the peak bottom of the luminance value calculated in step S2003 with a predetermined value. If the peak bottom is equal to or greater than a predetermined value, it means that a star has been detected in the frame. Therefore, in step S2005, the difference rate between the brightness of the A image and the B image is calculated.

When the peak bottom of the luminance value is less than a predetermined value, it means that there is only a dark part for the frame, so it is not necessary to calculate the deviation rate.

Step S2006 determines whether or not the deviation rate of the luminance value calculated in step S2005 is less than the threshold value. When it is less than the threshold value, the star extracted in the frame has a reliable brightness value, so the defocus amount calculated in advance in step S2007 is acquired.

The deviation rate of the luminance values of all the frames is determined, and it is determined in step S2009 whether or not at least one of them has acquired a defocus amount that is not NULL. If the defocus amount can be obtained, the average value of the defocus amounts is calculated in step S2010, and this is used as the final defocus amount. If the defocus amount cannot be obtained, the final defocus amount is set to NULL in step S2011. Step S2012 is the end of the reliability determination.

FIG. 21 is a flowchart in which the content of changing the threshold value according to a predetermined condition is added to the reliability determination of FIG. 20.

Step 2013 determines whether or not the number of subdivisions in the outline is equal to or greater than a predetermined value. When it is equal to or more than a predetermined value, the threshold value used for determining the deviation rate of the luminance value is changed to a larger value in step S2014. When the number of subdivisions is less than the predetermined value, the threshold value is not changed. Since steps S2002 and subsequent steps are the same as those described above, they will be omitted.

In the above description, when the number of subdivisions of the focal length detection region is equal to or greater than a predetermined number, the threshold value is changed in the direction of increasing the threshold value. However, when the focal length is equal to or greater than the predetermined value, the threshold value is increased. You may make changes. In this case, step 2013 is replaced with determination of whether or not the focal length is equal to or greater than a predetermined value. When the focal length is equal to or greater than a predetermined value, that is, when the focal length is on the telephoto side, bright stars may be partially saturated. Therefore, in step S2014, the threshold value used for determining the deviation rate of the luminance value is changed to a larger value than now. When the focal length is less than the predetermined value, the threshold value is not changed.

Further, when the posture of the image pickup apparatus is near the zenith direction, the threshold value may be increased. In this case, step 2013 is replaced with a determination as to whether or not the posture is near the zenith direction. When the attitude of the image pickup device is near the zenith, it is not easily affected by the city light, and there are many cases where bright stars are partially saturated. Change to a larger value. When the posture of the image pickup device is near the horizontal direction, the threshold value is not changed.

Further, when the F value of the interchangeable lens is less than a predetermined value, the threshold value may be increased. Replace step 2013 with determining whether the F value is less than a predetermined value. Here, when the F value is less than a predetermined value, it means that the lens is brighter. As the amount of light that can be captured by the lens increases, bright stars are often partially saturated. Therefore, in step S2014, the threshold value used for determining the deviation rate of the luminance value is changed to a larger value than now. When the F value is equal to or greater than a predetermined value, the threshold value is not changed.

According to this embodiment, it is possible to focus on a minute point light source with high accuracy in the phase difference AF method.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Claims (6)

An imaging means having a plurality of pixels capable of photoelectrically converting a luminous flux passing through different pupil regions of an imaging optical system including a focus lens and outputting a pair of image signals, and an imaging means having a plurality of pixels.
A calculation means for calculating the defocus amount by acquiring the image signal and performing a focus detection operation of the phase difference AF method.
A control means that controls the drive of the focus lens to adjust the focus based on the calculation result of the calculation means.
A storage means for storing a reference focal position for a predetermined subject in advance,
It is equipped with a setting means for setting a plurality of focus detection areas.
In the calculation means, whether or not the subject exists according to the luminance information of the focus detection region in a state where the focus lens is located at a position separated by a predetermined depth from the reference focus position stored in the storage means. The imaging apparatus is characterized in that the defocus amount is calculated by using the defocus amount of the focus detection region in which the subject is present. The imaging device according to claim 1, wherein the calculation means calculates the defocus amount in the starry sky photographing mode. The imaging device according to claim 1, wherein the luminance information is the peak bottom of the luminance value of the image signal. An imaging means having a plurality of pixels capable of photoelectrically converting a luminous flux passing through different pupil regions of an imaging optical system including a focus lens and outputting a pair of image signals, and an imaging means having a plurality of pixels.
A calculation means for calculating the defocus amount by acquiring the image signal and performing a focus detection operation of the phase difference AF method.
A control means that controls the drive of the focus lens to adjust the focus based on the calculation result of the calculation means.
A storage means for storing a reference focal position for a predetermined subject in advance,
It is equipped with a setting means for setting a plurality of focus detection areas.
When the focus lens is located at a position separated by a predetermined depth from the reference focal position stored in the storage means and there are undetectable regions in the plurality of focus detection regions, the calculation means is the first. After performing the calculation of the defocus amount of 1, the focus detection region is moved, the calculation of the second defocus amount is performed, and the defocus amount is calculated based on the first and second defocus amounts. An imaging device characterized by calculating. An imaging means having a plurality of pixels capable of photoelectrically converting a luminous flux passing through different pupil regions of an imaging optical system including a focus lens and outputting a pair of image signals, and an imaging means having a plurality of pixels.
A calculation means for calculating the defocus amount by acquiring the image signal and performing a focus detection operation of the phase difference AF method.
A control means that controls the drive of the focus lens to adjust the focus based on the calculation result of the calculation means.
A storage means for storing a reference focal position for a predetermined subject in advance,
A measuring means for measuring light and
Setting means for setting multiple focus detection areas and
A means for calculating the deviation rate of the luminance values of a pair of image signals from the luminance information is provided.
The calculation means sets the threshold value of the deviation rate in a state where the focus lens is located at a position separated by a predetermined depth from the reference focal position stored in the storage means.
An imaging device characterized in that the defocus amount is calculated based on the defocus amount in a focus detection region where the deviation rate is less than a threshold value. A control method for an image pickup apparatus having an image pickup means having a plurality of pixels capable of photoelectrically converting light flux passing through different pupil regions of an image pickup optical system including a focus lens and outputting a pair of image signals.
A calculation step of acquiring the image signal, performing a phase difference AF method focus detection calculation, and calculating the defocus amount, and
Based on the calculation result of the calculation step, the control step of controlling the drive of the focus lens to adjust the focus, and the control step.
With setting steps to set multiple focus detection areas,
In the calculation step, the subject exists according to the luminance information of the focus detection region in a state where the focus lens is located at a position separated from the reference focal position by a predetermined depth with respect to the predetermined subject stored in advance. A control method characterized in that it is determined whether or not the subject is present, and the defocus amount is calculated using the defocus amount in the focus detection region where the subject is present.

Images