JP2021071656A - Imaging apparatus and method for controlling the same - Google Patents

Abstract

To make it possible to appropriately follow a subject under various conditions.SOLUTION: An imaging apparatus has: focus detection means that detects a defocus amount on the basis of a signal obtained by photoelectrically converting light that is incident through an imaging optical system; storage means that stores the defocus amount that is repeatedly detected by the focus detection means; determination means that, when a difference between a first defocus amount stored in the storage means and a second defocus amount newly detected by the focus detection means is equal to or more than a threshold, does not determine that the first and second defocus amounts correspond to the same subject, and when the difference is smaller than the threshold, determines that the first and second defocus amounts correspond to the same subject; and setting means that sets the threshold. The setting means sets the threshold on the basis of a distance to the subject corresponding to the first defocus amount and difficulty degree information indicating a degree of difficulty in following the subject.SELECTED DRAWING: Figure 7

Description

The present invention relates to an image pickup apparatus and a control method thereof.

Conventionally, in a camera system such as an interchangeable lens type single-lens reflex camera, an automatic focus adjustment (AF) technique is known, and among them, a technique called phase difference AF is widely used. In this method, a light beam from a subject that has passed through different exit pupil regions of a photographing lens is imaged on a pair of line sensors. Then, the defocus amount of the photographing lens, that is, the focus detection is performed by obtaining the image shift amount, which is the displacement amount of the relative positions of the pair of image signals obtained by photoelectric conversion of the subject image. Automatic focus adjustment is performed by driving the focus lens included in the photographing lens based on the focus detection result.

Most of them have a servo shooting mode in which the focus lens is driven so as to focus not only on a stationary subject but also on a moving subject. The servo shooting mode has a function of predicting how the subject is moving from the past movement history. However, the amount of defocus detected may not necessarily be that of the subject to be photographed because an obstacle passes in front of the subject or the focus is erroneously detected with respect to the background.

As described above, a method of prohibiting the driving of the focus lens by the detected defocus amount has been proposed for the problem of accidentally focusing on an unintended subject or background. For example, Patent Document 1 discloses a method of switching a threshold value of a defocus amount that prohibits driving of a focus adjusting lens according to an image plane velocity. By using the method of Patent Document 1, if it is observed that the subject moves at high speed with respect to the subject moving at high speed, the focus detection lens can be driven even if a larger amount of defocus is detected. Therefore, it is possible to appropriately follow the acceleration and deceleration of the drive of the focus detection lens according to the detected defocus amount.

Further, Patent Document 2 discloses a method of changing a threshold value of a defocus amount used for determining whether or not a desired subject can be tracked according to a camera shake state. According to Patent Document 2, it is determined that it is difficult to follow a desired subject when the camera shake is large, and it is easy to follow a desired subject when the camera shake is small. If the camera is in a state where it is easy to follow the desired subject, it is considered that the detected defocus amount is the result of following the desired subject. On the contrary, if it is difficult for the camera to follow the desired subject, it is not possible to follow the desired subject, and it can be suspected that the detected defocus amount is the result of following the background or another subject.

Japanese Unexamined Patent Publication No. 2009-210815 Japanese Unexamined Patent Publication No. 11-326743

However, the technique described in Patent Document 1 can respond to a change in the movement of a subject only for a subject that continuously moves rapidly. For example, when the subject suddenly makes a large movement that cannot be predicted by the tendency of the movement so far, the threshold value of the defocus amount cannot be switched, and the focus drive is prohibited. Furthermore, the closer the distance between the photographer (camera) and the subject is, the greater the change in the amount of defocus detected with respect to the movement of the subject. The change is detected. Therefore, the following inconvenience occurs when the movement of the subject to be photographed keeps approaching the photographer (camera) or keeps away from the photographer.

In the servo shooting mode described above, the focus detection process is periodically performed, and based on the detection defocus amount obtained by the focus detection process, the focus adjustment lens is driven at any time to keep focusing on the subject. However, according to Patent Document 1, the drive of the focus adjusting lens is stopped according to the result of the focus detection process obtained at the timing when the subject suddenly starts to move. Therefore, the amount of defocus of the subject to be photographed obtained by the focus detection process at the next timing, which is periodically performed, is detected even larger. This operation will be repeated, and as a result, it will not be possible to focus on the desired subject.

Further, according to Patent Document 2, the ease of following the subject is evaluated only by the amount of camera shake regardless of the condition of the subject, but it is easy to follow if the area occupied by the subject within the angle of view is large. On the contrary, if the area of the subject is small, it is difficult to follow. Therefore, determining the ease of tracking a subject based only on the amount of camera shake may erroneously determine the ease of tracking.

The present invention has been made in view of the above problems, and an object of the present invention is to enable the subject to be tracked more appropriately under various conditions.

In order to achieve the above object, the image pickup apparatus of the present invention includes a focus detection means for detecting a defocus amount based on a signal obtained by photoelectric conversion of light incident on an imaging optical system, and the focus. A storage means for storing the defocus amount repeatedly detected by the detection means, a first defocus amount stored in the storage means, and a second defocus amount newly detected by the focus detection means. When the difference is equal to or greater than the threshold value, it is not determined that the first and second defocus amounts correspond to the same subject, and when the difference is smaller than the threshold value, the first and second defocus amounts are obtained. It has a determination means for determining that the focus amount corresponds to the same subject and a setting means for setting the threshold value, and the setting means has a distance to a subject corresponding to the first defocus amount. The threshold value is set based on the difficulty level information indicating the ease of capturing the subject.

According to the present invention, the subject can be tracked more appropriately under various conditions.

The block diagram which shows the schematic structure of the image pickup apparatus in the 1st Embodiment of this invention. The flowchart which showed the shooting process in the servo shooting mode in 1st Embodiment. The flowchart of the focus adjustment process in 1st Embodiment. The flowchart of the focus detection process in 1st Embodiment. The flowchart of continuity determination processing in 1st Embodiment. The flowchart of the moving body prediction processing in 1st Embodiment. The flowchart of the continuity determination threshold value setting process in 1st Embodiment. The figure which shows an example of the defocus map. The figure which shows an example of the past focus detection history and the prediction curve. The figure which shows an example of the change of the image plane position when the subject which was stationary relatively close to a camera suddenly accelerates. The figure which shows an example of the situation which it is difficult to keep catching a subject. The flowchart of the continuity determination threshold value setting process in 2nd Embodiment. The flowchart of the focus detection process in 3rd Embodiment. The figure which shows an example of the user setting screen of the motion characteristic of a subject. The figure explaining the pixel arrangement of the image sensor in the modification.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate explanations are omitted.

<First Embodiment>
FIG. 1 is a block diagram showing a configuration of a digital single-lens reflex camera as an example of an imaging device according to the first embodiment.

The photographing lens 101 (photographing optical system) is simply represented by one lens, but is actually composed of a plurality of lenses including a focus lens for performing focus adjustment. The lens drive circuit 102 has, for example, a DC motor or a stepping motor, and drives and controls the photographing lens 101. The drive control is instructed by the in-lens microprocessor 141, and the drive request for the drive control is propagated from the microcomputer 124 using the lens communication circuit 103.

The lens communication circuit 103 propagates drive requests for various actuators from the microcomputer 124 to the in-lens microcomputer 141, and propagates state information of the various actuators to the microcomputer 124.

The diaphragm 104 is driven by the diaphragm drive circuit 105 and is used to control the luminous flux incident on the camera. The aperture drive circuit 105 changes the optical aperture value of the aperture 104 by being controlled by the in-lens microcomputer 141 based on the drive amount calculated by the microcomputer 124.

The distance measuring circuit 142 calculates the distance from the camera to the subject based on the position of the focus lens of the photographing lens 101. The distance measurement method using the position of the focus lens is an example, and a method such as infrared irradiation may be used. Further, in this configuration, the distance measuring circuit 142 is configured on the lens side and controlled by the in-lens microcomputer 141, but may be configured on the camera body side and controlled by the microcomputer 124.

The main mirror 106 is always arranged in the optical path so as to guide the light flux to the finder (not shown) so as to guide the light flux to the image sensor 113 when photographing is performed. It jumps upward and escapes from the optical path. That is, it is switched whether the light flux incident from the photographing lens 101 is guided to the finder side or the image sensor 113 side. Further, the main mirror 106 is a half mirror so that the central portion thereof can transmit a part of light. The sub-mirror 107 reflects the light flux transmitted through the main mirror 106 and guides it to a focus detection sensor arranged in the focus detection circuit 110 for performing focus detection.

The pentaprism 108 guides the luminous flux reflected by the main mirror 106 to the finder section (not shown). The finder unit also has a focus plate, an eyepiece lens (not shown), and the like. The photometric circuit 109 converts the color and brightness of the subject image imaged on the focus plate (not shown) into an electric signal by a photometric sensor provided in the photometric circuit 109 and provided with a color filter.

The focus detection circuit 110 has a built-in focus detection sensor composed of a plurality of area sensors, and the light beam reflected by the sub mirror 107 is incident on the focus detection sensor by passing through the main mirror 106 and thereby being inside the focus detection sensor. Obtain a pair of dispersed image signals. Then, the defocus amount of the photographing lens 101 is detected by obtaining the image shift amount, which is the displacement amount of the relative positions of the obtained pair of image signals. The defocus amount obtained by the focus detection circuit 110 is propagated to the microcomputer 124, converted into a control amount of the photographing lens 101, and then propagated to the in-lens microcomputer 141 via the lens communication circuit 103. ..

The focal plane shutter 111 is driven by the shutter drive circuit 112 according to the shutter speed set in the camera.

A CCD, CMOS sensor, or the like is used as the image sensor 113, and the subject image formed by the photographing lens 101 is converted into an electric signal.

The clamp circuit 114 and the AGC circuit 115 are circuits that perform basic analog signal processing before A / D conversion is performed on the electric signal output from the image sensor 113, and the clamp level and the AGC reference level are measured by the microcomputer 124. Changes are made. The A / D converter 116 converts the analog output signal of the image sensor 113 processed by the clamp circuit 114 and the AGC circuit 115 into a digital signal (image data).

The video signal processing circuit 117 is realized by a logic device such as a gate array, performs filter processing, color conversion processing, and gamma processing on the digitized image data, and also performs compression processing such as JPEG to the memory controller 120. Output. Further, the video signal processing circuit 117 can output information such as exposure information and white balance of the signal of the image sensor 113 to the microcomputer 124, if necessary. Based on this information, the microcomputer 124 gives instructions for white balance and gain adjustment.

In the case of continuous shooting operation, the shooting data is temporarily stored in the buffer memory 123 without being processed, the unprocessed shooting data is read out through the memory controller 120, and the image processing and compression processing are performed by the video signal processing circuit 117. By doing so, continuous shooting is performed. The number of continuous shots depends on the size of the buffer memory 123. At the time of continuous shooting, the memory controller 120 stores the unprocessed digital image data input from the video signal processing circuit 117 in the buffer memory 123, and stores the processed digital image data in the memory 121. Further, the memory controller 120 outputs image data from the buffer memory 123 and the memory 121 to the video signal processing circuit 117. The memory 121 may be removable. Further, the memory controller 120 can output an image stored in the memory 121 via an external interface 122 that can be connected to a computer or the like.

The operation unit 125 transmits the state of various operation members connected to the operation unit 125 to the microcomputer 124, and the microcomputer 124 controls each unit according to the change of the operation member. The operating members connected to the operating unit 125 include switch 1 (SW1) 126 and switch 2 (SW2) 127. The operation unit 125 interprets the operation states of SW1 (126) and SW2 (127), and if only the operation of SW1 (126) is valid (ON), it is treated as an autofocus (AF) start request event. If the operation of SW2 (127) is valid (ON), it is treated as a shooting request event. Further, when the state (ON) in which the SW2 (127) is operated is maintained, the continuous shooting operation is performed. In addition, operation members (not shown) such as an ISO setting button, an image size setting button, an image quality setting button, and an information display button are connected to the operation unit 125, and the state of each operation member is detected.

The liquid crystal drive circuit 128 drives the external liquid crystal display member 129 and the liquid crystal display member 130 in the finder according to the display command of the microcomputer 124. A backlight such as an LED (not shown) is arranged on the liquid crystal display member 130 in the finder, and the LED is also driven by the liquid crystal drive circuit 128. The microcomputer 124 confirms the memory capacity through the memory controller 120 based on the predicted value data of the image size according to the ISO sensitivity, the image size, and the image quality set before shooting, and then the remaining possible shooting. You can calculate numbers. Then, if necessary, the remaining number of images that can be photographed is displayed on the external liquid crystal display member 129 or the liquid crystal display member 130 in the viewfinder.

The non-volatile memory 131 (EEPROM) can store data even when the camera is not turned on. The power supply unit 132 supplies necessary power to each IC and drive system.

The motion detection circuit 140 is equipped with a detection unit that detects the motion of the digital single-lens reflex camera. The detection unit is, for example, an acceleration sensor, which detects the acceleration when the horizontal axis of the camera body is the rotation axis and the acceleration when the vertical axis of the camera body is the rotation axis.

In the present embodiment, the motion detection circuit 140 is configured on the camera body side and controlled by the microcomputer 124, but may be configured on the lens side and controlled by the in-lens microcomputer 141. Further, the lens may be detachable from the camera body or may be integrally configured with the camera body.

Next, the operation of the photographing process according to the first embodiment of the present invention will be described. Generally, the AF mode of a camera is a mode in which the focusing operation is performed once and then terminated (one-shot shooting mode), and the focusing operation is repeated while predicting the image plane of the subject at a time later than the current time. There are two modes (servo shooting mode) for driving the lens. In the first embodiment, the processing when the camera is set to the servo shooting mode will be described.

FIG. 2 is a flowchart showing a shooting process in the servo shooting mode. In S201, the microcomputer 124 confirms the shooting mode. If it is determined that the current shooting mode is not the servo shooting mode, the process ends. On the other hand, when it is determined that the shooting mode is the servo shooting mode, the process proceeds to S202.

In S202, the microcomputer 124 confirms the state of SW1 obtained by the operation unit 125. When SW1 is OFF, the transition to S201 is performed, and when SW1 is ON, the transition to S203 is performed.

In S203, the focus detection circuit 110 performs the focus adjustment process. The details of the processing in S203 will be described later with reference to FIG.

Next, in S204, the microcomputer 124 confirms the state of SW2 obtained from the operation unit 125. When SW2 is OFF, the transition to S201 is performed, and when SW2 is ON, the transition to S205 is performed.

In S205, the microprocessor 124 drives the focal plane shutter 111 via the shutter drive circuit 112 to perform photographing. After the shooting is completed, the process returns to S201.

As described above, when SW1 is ON in the servo shooting mode, the focus adjustment process of S203 is repeatedly performed unless the mode is changed.

Next, the outline of the focus adjustment process in S203 will be described with reference to FIG.

In S301, the microcomputer 124 performs focus detection and a series of processes associated therewith. The details of the processing in S301 will be described later with reference to FIG.

In S302, the microcomputer 124 can continue to focus-follow (hereinafter, referred to as "subject capture") the target subject that has been subjected to the focus-following operation in the servo shooting mode from the focus detection result obtained by the processing of S301. Performs a continuity determination process that determines whether or not it is. Details will be described later with reference to FIG. 5, but if the subject can be captured continuously, the flag FlagCatchFail = 0, and if the subject capture fails, the flag FlagCatchFail> 0.

In S303, the microcomputer 124 performs a moving object prediction process that predicts what kind of movement the subject being captured is moving from the focus detection history. The details of the processing in S303 will be described later with reference to FIG. In the process of S303, the microcomputer 124 calculates the focus lens drive amount for driving the focus lens constituting the photographing lens 101.

Next, in S304, the microprocessor 124 confirms the value of the flag FlagCatchFail set in S302. If the FlagCatchFail is larger than 0, that is, if the subject capture fails, the transition to S305 is performed, and if not, that is, if the subject capture is completed, the transition to S306 is performed.

In S305, as a process when it is not recognized that the subject can be continuously captured by the current focus detection result, the lens drive is prohibited and the focus detection process is terminated. There are two types of lens drive prohibition processing, and any of them may be selected. The first is a method in which the microcomputer 124 requests the in-lens microcomputer 141 to stop the focus lens included in the photographing lens 101 via the lens communication circuit 103. The second is a method of not executing the drive request based on the defocus amount obtained in the focus detection process in S301 immediately before.

On the other hand, in S306, processing is performed when it is recognized that the subject can be continuously captured based on the current focus detection result. Here, the microcomputer 124 requests the in-lens microcomputer 141 to drive the focus lens included in the photographing lens 101 via the lens communication circuit 103, and ends the focus adjustment process.

FIG. 4 shows an example of the operation of the focus detection process performed in S301. In S401, the microcomputer 124 sets a continuity determination threshold value for determining whether or not the current focus detection result can continue to capture the subject based on the past focus detection history. The details of the processing in S401 will be described later with reference to FIG.

Next, in S402, the microprocessor 124 drives the focus detection circuit 110 to perform focus detection.

In S403, the microcomputer 124 causes the defocus amount (hereinafter, referred to as “detection defocus amount”), which is the focus detection result in S402, to correspond to the focus detection focusing point as shown in FIG. Create a defocus map described in the matrix.

Here, the defocus map created in the present embodiment will be described with reference to FIGS. 8 (a) to 8 (d).

801 corresponds to the image region obtained by the image sensor 113, and 802 indicates the sensor region of the focus detection sensor unit included in the focus detection circuit 110. It is assumed that the focus detection sensor unit of the present embodiment is composed of a total of 49 AF points divided into 7 in the vertical direction and 7 in the horizontal direction. Reference numerals 803 to 805 indicate a part of the divided AF points. When creating the defocus map, the distance measurement result can be obtained for each distance measurement point.

The composition shown in FIG. 8A shows a case in which the subject 806 existing on the near view side and the subject 807 existing on the distant view side exist in the same screen with respect to the photographer (camera), and the distant view side The focus is on person 807. As shown in FIG. 8B, the defocus map at this time is set at the detection defocus amount of 0 and the position of the person 806 on the front side at the AF point 809 set at the position of the person 807 on the distant view side. At the AF point 808, the detected defocus amount is a negative value. Further, the detection defocus amount is a positive value at the AF point set at the background position.

Similarly, in FIG. 8C, the person 810 on the front side is in focus, and the subject 811 on the distant view side is out of focus. As shown in FIG. 8D, the defocus map at this time was set at the detection defocus amount of 0 and the position of the person 811 on the distant view side at the AF point 812 set at the position of the person 810 on the front side. At the AF point 813, the detected defocus amount is a positive value. Further, at the AF point set at the background position, the detected defocus amount is a larger positive value.

In S404, the microcomputer 124 selects the main focusing point from the defocus map created in S403. Specifically, based on the focus detection results up to the previous time, whether the subject is approaching or moving away from the photographer (camera) at the time of this focus detection, that is, how much it is moving in the depth direction. Predict, and let the focusing point closest to this result be the main focusing point. If the mode set by the user is not a multipoint method in which a plurality of AF points are set, the AF point specified by the user is defined as the main AF point.

In S405, the microcomputer 124 extracts the subject area using the defocus map created in S403. Specifically, a region around the main focusing point selected in S404 in which the difference in defocus amount from the main focusing point is within a certain value is set as the subject region (subject detection).

Next, in S406, the microcomputer 124 extracts obstacles using the defocus map created in S403. Specifically, the area in which the defocus amount indicating the in-focus position closer to the subject area extracted in S405 is detected is extracted from the defocus map and regarded as an obstacle.

By the above processing, the focus detection process of S301 is completed, the threshold value for the continuity determination based on the past focus detection history is determined, the focus detection result is acquired by the focus detection process, and the defocus is based on the detected defocus amount. We are creating a map. Then, from the defocus map, the area of the main subject can be extracted, and if there is an obstacle, the area on the defocus map occupied by the obstacle can be specified, and the above information is obtained after the continuity determination process of S302. Referenced to.

FIG. 5 shows an example of the operation of the continuity determination process performed in S302. In S501, the microcomputer 124 compares the detection defocus amount of the main focus measurement point detected in S405 with the magnitude of the continuity determination threshold value set in S401. If the detected defocus amount is smaller than the continuity determination threshold value determined in S401, the transition to S502 is performed, and if the detection defocus amount is equal to or greater than the continuity determination threshold value, the transition to S503 occurs.

In S502, the microcomputer 124 determines that the subject can be continuously captured from the current focus detection result, sets the flag FlagCatchFail to 0, and ends the continuity determination process.

In S503, the microcomputer 124 determines from the current focus detection result that the subject cannot be captured, and increments the flag FlagCatchFail. In addition to the processing in S502, the flag FlagCatchFail is set to an initial value of 0 at a predetermined timing such as when the camera is started or when the servo shooting mode is set.

In S504, the microcomputer 124 determines whether or not the flag FlagCatchFail is 1, that is, whether or not it is determined that the subject capture has failed for the first time this time. If the flag FlagCatchFail is 1, the transition to S506 is performed, and if not, the transition to S505 is performed.

In S506, the microprocessor 124 sets the follow-up timer and then ends the continuity determination process. The follow-up timer indicates the maximum time to wait until it is determined that the subject has been captured. By setting the follow-up timer, if the follow-up timer is counting, the tracking can be resumed immediately when the subject that failed to capture the subject can be detected again.

On the other hand, in S505, the microprocessor 124 determines whether or not the follow-up timer has expired. If the follow-up timer has expired, the transition to S507 is performed, otherwise the continuity determination process is terminated.

In S507, the microcomputer 124 erases data such as the past focus detection history and tracking, clears the flag FlagCatchFail, and ends the continuity determination process. In this way, when the user intentionally changes the subject, the follow-up to the new subject can be started as soon as the follow-up timer expires.

FIG. 6 shows an example of the operation of the moving object prediction process performed in S303. In S601, the current focus detection result in S301 is stored in the non-volatile memory 131 or the storage area in the microcomputer 124 in addition to the past focus detection history. The information recorded in the focus detection history is the current focus detection time and the image plane position. The image plane position is the position of the focus lens that focuses on the subject, which is calculated from the detected defocus amount and the current position of the focus lens.

In S602, the microcomputer 124 determines whether or not the conditions for performing motion prediction are met. The condition includes, for example, determination of whether or not the image plane position has moved in the same direction a plurality of times in a row. If the conditions for motion prediction are met, the transition to S603 is performed, and if not, the transition to S606 is performed.

In S603, the microcomputer 124 calculates the variation between the predicted drive amount and the focus detection result by drawing a prediction curve as shown in FIG. 9 from the past focus detection history. The vertical axis of FIG. 9 indicates the image plane position, the horizontal axis indicates time, and the larger the image plane position of the vertical axis, the farther the distance is. The history of the figure follows the subject approaching the photographer (camera). It shows how it is doing. As described above, the focus adjustment process in S203 is periodically executed in the servo shooting mode, and T _{1 to} T ₅ each represent the execution time of the focus adjustment process performed in S203. For the predicted drive amount, for example, a prediction curve is derived by the collective minimum square method using the past image plane position and each focus detection time, and the image plane position at the prediction destination time is calculated based on this curve. can get. The predicted destination time indicates the time at the time of shooting if SW2 is ON, and indicates the current focus detection time otherwise.

In S604, the microcomputer 124 finally determines whether or not to perform the predictive focus drive based on the variation in the focus detection result calculated in S603. Specifically, if the variation is more than a certain value, it is judged that the prediction cannot be trusted and the prediction focus drive is not performed. If the variation is less than a certain value, it is judged that the prediction can be trusted and the prediction focus drive is performed. .. If it is determined that the predictive focus drive is to be performed, the transition to S605 is performed, otherwise the transition to S606 is performed.

The predictive focus drive is a control for driving the lens from the current image plane position L3 to the image plane position at the time when the future focus is desired based on the prediction curve described with reference to FIG. The black circles in FIG. 9 indicate the image plane position obtained from the detected defocus amount, and the white circles indicate the predicted image plane position. For example defocus amount detected at the time of T ₄ in FIG. 9 is assumed to be x. Here, when the lens drive to aiming adjust the focus at the time of T ₅ is a request the lens drive corresponding to the defocus amount y. To calculate the defocus amount y is to draw approximate curve from the focus detection result in the past, this as prediction curve, it is necessary to predict the image plane position at T _5. In this way, in order to detect the focus at _{T 4 and aim for focusing at T 5} , the lens drive when the focus drive is performed by the defocus amount y is called the predictive focus drive. On the other hand, when the predictive focus drive is not performed, that is, when the prediction is not performed from the past focus detection history or the prediction result is not used, the focus drive amount is the defocus amount x, that is, the detected defocus amount.

In S605, the microcomputer 124 sets the focus drive amount to the predicted drive amount and ends the moving object prediction process. On the other hand, in S606, the microcomputer 124 sets the focus drive amount to the detection defocus amount and ends the moving object prediction process.

Next, with reference to FIG. 7, an example of the operation of the continuity determination threshold setting process performed in S401 is shown. The continuity determination threshold value is a threshold value for determining whether or not the subject can be continuously captured, and is used in the comparison process in S501 described above. As described above, if the detected defocus amount is larger than the continuity determination threshold value, it is determined that the subject capture has failed, and if not, it is determined that the subject capture is successful.

In S701, the microcomputer 124 receives the subject distance information measured by the distance measuring circuit 142 at the time of the previous focus detection via the in-lens microcomputer 141 and the lens communication circuit 103, so that the subject currently being captured is a camera. It is determined whether or not the distance is equal to or greater than a certain distance. If the subject is at a certain distance or more from the camera, the transition to S702 is performed, and if not, the transition to S703 is performed.

In S702, the microcomputer 124 sets the continuity determination threshold value to the first threshold value and ends the continuity determination threshold value setting process.

Ｏ_Ｄは、検出した被写体領域よりも至近側の合焦位置を示すデフォーカス量が検出された測距点の数を示す。Ｏ_Ａは、Ｏ_Ｄの算出のために参照する点数を示し、この最大値は画角全体の測距点の数である。Ｊ_Ｄは、動き検出回路１４０で検出したカメラ振れ量の絶対値を示す。Ｊ_Ｐは、カメラ振れ量Ｊ_Ｄの許容最大値を示す。Ｓ_Ｃは、主測距点と近いデフォーカス量が検出された測距点の数を示す。Ｓ_Ａは、Ｓ_Ｃの算出のために参照する点数を示し、この最大値は画角全体の測距点の数である。Ｓ_Ｐは、被写体サイズ許容最小量を示し、測距点数で表わされる。 On the other hand, in S703, the microcomputer 124 calculates the subject capture difficulty level D, which indicates how easy it is to capture the subject. In the present embodiment, as information indicating the difficulty level of capturing the subject (difficulty level information), the size within the angle of view of the subject area obtained in S405 at the time of the previous focus detection, the presence / absence of an obstacle obtained in S406, and the motion detection. Three types of camera shake information obtained from the angular velocity information detected by the circuit 140 are used. Based on this information, the subject capture difficulty level D is derived by the following equation (1).

O _D is a defocus amount indicating an in-focus position of the near side than the detected object area indicates the number of detected distance measuring point. O _A indicates the number reference for the calculation of O _D, the maximum value is the number of distance measuring points for the entire field angle. _JD indicates the absolute value of the camera shake amount detected by the motion detection circuit 140. J _P denotes an allowable maximum value of the camera shake amount _{J D.} S _C denotes the number of the main detecting point is close defocus amount is detected distance measuring point. S _A denotes the number reference for the calculation of S _C, this maximum value is the number of distance measuring points for the entire field angle. S _P denotes the subject size allowable minimum amount, represented by the distance-measuring points.

The subject capture difficulty level D is a value of 0 or more and 1 or less, and the larger the value, the easier it is to capture the subject, and the smaller the value, the more difficult it is to capture the subject.

Based on the subject capture difficulty calculated in S703, the microcomputer 124 determines whether or not it is easy to capture the subject in S704. As an example, if D ≧ 0.5, it is determined that the subject is easy to capture, and if not, it is determined that the subject is difficult to capture. If D ≧ 0.5, that is, if it is determined that the subject is easy to capture, the transition to S705 is performed, otherwise the transition to S706 is performed.

In S705, the microcomputer 124 sets the continuity determination threshold value to a second threshold value larger than the first threshold value, and ends the continuity determination threshold value setting process. This second threshold value is variable depending on the subject distance, and becomes larger as it gets closer.

In S706, the microcomputer 124 sets the continuity determination threshold value to a third threshold value smaller than the second threshold value. In S704, it is determined that it is difficult to capture the subject in the current shooting situation. Therefore, if the continuity determination threshold is set to the second threshold, the subject is currently captured even if the subject cannot be captured continuously. There is a possibility that it will be judged as a change in the movement of the subject inside. Therefore, in S706, a third threshold value is adopted, which is smaller than the second threshold value and can take a value stepwise according to the subject capture difficulty level D. As an example, if the second threshold value is D ₂ and the first threshold value is D ₁ at the same subject distance, the third threshold value D ₃ is D ₃ = (D _2- D ₁ ), 2D + D ₁ ... (2).
Can be expressed as. By doing so, the third threshold value D ₃ can take a value between _{the first threshold value D 1} and the second threshold value D ₂ step by step according to the difficulty level of capturing the subject. As described above, the continuity determination threshold value is set as the third threshold value, and the continuity determination threshold value setting process is completed.

Next, the effect obtained in the first embodiment will be described with reference to FIGS. 10 and 11. FIG. 10 shows when a subject stationary at a place relatively close to the camera suddenly accelerates in a direction away from the camera. The change in the image plane position is shown. The horizontal axis shows the time and the vertical axis shows the image plane position. The larger the vertical axis, the larger the subject distance. As shown at points t ₁ , t ₂ , and t ₃ , the image plane position does not change while the subject is stationary. However, when the object is rapidly accelerated, a large defocus amount vagaries than from the tendency of the past focus detection history which has been still far as the point t ₅ is observed. As described above, it is difficult to predict that the tracking of a subject that rapidly starts moving from a stationary state will make a large movement at the next moment because the characteristics of the movement of the subject are significantly different from the focus detection history so far. Further, the closer the photographer (camera) is to the subject, the larger the amount of image plane movement at the moment when the subject moves is observed. Therefore, it becomes difficult to observe the continuity of the distance measurement history, especially when following a subject that rapidly moves from a stationary state in a near view.

Therefore, in the present embodiment, by increasing the continuity determination threshold value when the subject is relatively close to the camera, it becomes possible to cope with such a sudden change in the image plane position.

On the other hand, as shown in FIG. 11, it is difficult to keep capturing the subject when there is an obstacle in front of the subject (FIG. 11 (a)), when the subject is small (FIG. 11 (b)), or when the camera is shaking. In a situation, if measures are simply taken according to the distance to the subject, there is a possibility that the subject that is not the tracking target will be captured. Therefore, in the present invention, it is determined that it is difficult to keep capturing the subject, and the continuity determination threshold value is suppressed so that a subject that is not the tracking target is not erroneously detected as a capturing target.

As described above, in the present embodiment, the threshold value for determining whether or not the subject currently being captured can be continuously captured by the current focus detection result is set to a large value in the near view. This is because the closer the photographer (camera) is to the subject, the larger the change in the amount of defocus detected with respect to the movement of the subject. Further, the difficulty of capturing the subject was evaluated, and the threshold value for determining whether or not the current focus detection result could continue to capture the subject currently being captured was dynamically changed according to the evaluation result. Specifically, the evaluation was made based on three factors: the size within the angle of view of the target subject obtained from the defocus information at the time of the previous focus detection, the presence / absence of obstacles, and the camera shake information obtained from the angular velocity information. This makes it possible to follow a subject that suddenly starts moving in a near view. Further, in an environment in which an undesired subject is targeted for erroneous detection and erroneous detection is likely to occur, it is possible to suppress erroneous detection while maintaining subject tracking in a near view.

<Second embodiment>
Next, a second embodiment of the present invention will be described. In the first embodiment, an example in which the third threshold value is set as a numerical value that changes depending on the ease of following the subject has been described. On the other hand, in the second embodiment, under the condition that the third threshold value matches the first threshold value, that is, it is difficult to follow the subject, the difference in the amount of defocus that the subject is recognized as the same as the previous time is large. An example of not doing so will be described.

Hereinafter, the continuity determination threshold setting process according to the second embodiment of the present invention will be described with reference to FIG. In the second embodiment, the process in S401 of FIG. 4 is different from that of the first embodiment, and the process shown in FIG. 12 is executed instead of the process shown in FIG. 7. Other than that, it is the same as that of the first embodiment, so the description thereof will be omitted.

FIG. 12 is a diagram showing an example of the operation of the continuity determination threshold setting process performed in S401 in the second embodiment.

In S1201, the microcomputer 124 receives the subject distance information measured by the distance measuring circuit 142 via the in-lens microcomputer 141 and the lens communication circuit 103, so that the subject currently being captured is above a certain level with respect to the camera. Determine if the distance is. If the subject is at a certain distance or more, the transition to S1202 is performed, and if not, the transition to S1204 is performed.

In S1202, the microcomputer 124 sets the continuity determination threshold value to the first threshold value and ends the continuity determination threshold value setting process.

On the other hand, in S1204, the microcomputer 124 determines whether or not it is difficult to capture the subject. Here, it is determined whether or not any of the following three conditions is met.
-The size within the angle of view of the target subject obtained from the defocus information is smaller than a certain value.-There is an obstacle (there is a subject in front of the desired subject within the angle of view).
-If any of the above conditions is met in which the amount of camera shake obtained from the angular velocity information is greater than a certain value, it is judged that it is difficult to capture the subject, and if not, it is easy to capture the subject. If it is determined that the subject is easy to capture, the transition to S1206 is performed, otherwise the transition to S1202 is performed.

In S1206, the microcomputer 124 sets the continuity determination threshold value to a second threshold value larger than the first threshold value, and ends the continuity determination threshold value setting process. This second threshold value is variable according to the subject distance, and becomes larger as it gets closer.

As described above, in the second embodiment, when there is a concern that the subject cannot be followed, it is difficult to recognize the same subject when the difference in the defocus amount is large. As a result, the continuity determination threshold value can be set more difficult to expand than in the first embodiment, and is particularly suitable when it is desired to avoid capturing an erroneous subject. Moreover, since the amount of calculation can be reduced, the overall performance is not significantly affected.

<Third embodiment>
Next, a third embodiment of the present invention will be described. In the first embodiment, as conditions for evaluating the difficulty of following the subject, the size within the angle of view of the target subject obtained from the defocus information, the presence / absence of obstacles, and the camera shake information obtained from the angular velocity information 3 Was used.

In the third embodiment, pan / tilt information or xy plane motion vector obtained from camera posture and camera shake information, depth of field, user-set subject motion characteristics, subject size acquired by image processing, face, and the like. The difficulty of capturing the subject is expressed by using the information (difficulty level information) of the number of subjects such as animals.

Hereinafter, the difference in the method of calculating the difficulty of capturing a subject in the third embodiment of the present invention will be described by referring to FIG. 13 and the method of calculating the difficulty of capturing a subject. In the third embodiment, the process shown in FIG. 13 is performed instead of the process of FIG. 4 described in the first embodiment, but other than that, the process is the same as that of the first embodiment. Omit. Further, in FIG. 13, the same processing as that in FIG. 4 is assigned the same reference number, and the description thereof will be omitted.

In the focus detection process shown in FIG. 13, in S1304, the microcomputer 124 extracts the position and number of subjects in the image, such as a human face / body or an animal, from the image acquired by the video signal processing circuit 117.

In S1305, the microcomputer 124 selects the main subject from the subjects extracted in S1304 and calculates the subject size in the screen. As the main subject, a subject to be prioritized preset by the user or a subject appearing larger is preferentially selected.

In S1306, the microcomputer 124 determines the main focusing point using the area of the main subject extracted in S1305 and the defocus map created in S1303.

Next, the method of calculating the subject capture difficulty level performed in S703 of FIG. 7 will be described as being different from the first embodiment. Although the method of calculating the subject capture difficulty is different from that of the first embodiment, the other processes are the same as those of FIG. 7, and thus the description thereof will be omitted.

In the present embodiment, as described above, the xy plane motion vector or pan / tilt information, depth of field, subject motion characteristics, subject size, and number of subjects are used to calculate the subject capture difficulty. The xy plane motion vector expresses the difference in the position of the main subject detected in S1305 at the time of the previous focus detection on the screen at the time of the previous focus detection. The pan / tilt information is calculated from the camera posture and camera shake information obtained from the motion detection circuit 140. The depth of field is calculated from the amount of aperture driven by the drive circuit 105, and the movement characteristics of the subject are set in advance by the user on the selection screen shown in FIG. The subject size is acquired by the image processing in S1305, and the number of subjects such as faces and animals used is those detected in S1304. When the movement characteristic of the subject is set to "fast", the continuity determination threshold value can be set large, and when it is set to "slow", the continuity determination threshold value can be set small. By doing so, it is possible to easily respond to an increase or decrease in the defocus amount for a subject that is expected to move fast, and to make it less susceptible to distance measurement noise for a subject that is expected to move slowly.

式（３）において、Ｖ_Ｄは、観測したｘｙ平面動きベクトル（画面上での被写体の動きベクトル）を示す。Ｍ_Ｖは、被写体を安定的に追えていると考えられる画面上での被写体の動き量を示す。Ｅ_ｍは、許容できる最小の被写界深度を示す。Ｅ_Ｄは、今回のカメラ設定での被写界深度を示す。Ｅ_Ｍは、Ｅ_Ｄが取りうる最大値を示す。Ｉ_ｍは、被写体を安定的に追える限界の最小被写体サイズを示す。Ｉ_Ｄは、画像処理によって算出した被写体のサイズを示す。Ｉ_Ａは、画角のサイズを示す。Ｆ_Ｎは、被写体の動き特性設定値の全段数を示す。Ｆ_Ｕは、ユーザの設定した被写体の動き特性設定値を示す。Ｎは顔や動物などの被写体の個数を示す。なお、被写体を検出できなかったときであっても、Ｎの最小値は１とする。 The subject capture difficulty level D in this embodiment can be expressed by the following equation (3).

In the equation (3), V _D indicates the observed xy plane motion vector (motion vector of the subject on the screen). M _V indicates the amount of motion of the object on the screen that are considered to Chase the subject stably. E _m is the minimum depth of field acceptable. E _D indicates the depth of field at the current camera settings. E _M indicates the maximum _{value E D} can take. I _m represents the minimum object size limit able to track an object in a stable manner. _ID indicates the size of the subject calculated by image processing. I _A indicates the size of the angle of view. F _N indicates the total number of steps of the motion characteristic set value of the subject. F _U represents a motion characteristic setting value of the subject set by the user. N indicates the number of subjects such as faces and animals. Even when the subject cannot be detected, the minimum value of N is set to 1.

In addition, the xy plane motion vector may not be observable because it is difficult to capture the subject, such as when the subject has a similar color to the background. In this case, the equation (3), instead of the xy plane motion vector V _D, using the pan-tilt vector P _D of the observed camera. Further, instead of the motion amount M _V of the object on the screen that are considered to Chase the subject stably, using the maximum movement amount M _P of the camera is considered able to track an object in a stable manner.

As described above, according to the third embodiment, the difficulty of capturing the subject is expressed from a viewpoint different from that of the first embodiment, and the current focus detection result continues to capture the subject currently being captured according to the evaluation result. The threshold value for determining whether or not the image can be captured can be dynamically changed. Specifically, if it can be estimated that the subject can follow even if the camera is shaking, the subject continuity determination threshold value may not be reduced. In addition, the threshold value can be set while considering how much the image is affected. Further, it is possible to set the subject continuity determination threshold value in consideration of the ease of shifting the focus to another subject visible on the screen and the movement characteristics of the subject set by the user.

<Modification example>
In the first to third embodiments described above, the case where the camera has a focus detection circuit 110 dedicated to focus detection has been described.

On the other hand, in recent years, a pupil division signal is acquired by an image sensor provided with a plurality of photoelectric conversion units for one microlens, and focus detection is performed based on the phase difference of the obtained plurality of pupil division signals. So-called image plane phase difference AF is performed.

FIG. 15 is a diagram showing a pixel arrangement constituting an image sensor capable of performing image plane phase difference AF, and can be used as the image sensor 113.

As shown in FIG. 15, a plurality of photoelectric conversion units 201A and 201B are arranged so as to correspond to each microlens 201 forming the microlens array. Each region indicated by (0,0), (1,0), (0,1), etc. in the figure indicates a unit pixel 202 in the image sensor, and each unit pixel 202 includes one microlens 201 and a plurality of units. It is composed of photoelectric conversion units 201A and 201B. In the present embodiment, the unit pixel 202 shows a case where two photoelectric conversion units 201A and 201B are arranged in the X-axis direction, but even if they are arranged in the Y-axis direction, three or more photoelectric conversion units can be arranged. May be configured to correspond to each microlens 201.

Of the plurality of unit pixels 202 arranged in the X-axis direction, the subject image composed of the focus detection signal group of the pupil-divided photoelectric conversion unit 201A is the A image, and the pupil-divided photoelectric conversion unit is The subject image composed of the focus detection signal group of 201B is defined as the B image. Then, the amount of image shift (phase difference) is detected by performing a correlation calculation on the A image and the B image. Further, by multiplying the amount of image shift by a conversion coefficient determined according to the focal position and the characteristics of the optical system, the focal position corresponding to an arbitrary subject position on the screen can be calculated. By controlling the focus lens included in the photographing lens 101 based on the focus position information calculated here, the imaging surface phase difference AF for detecting the focal state becomes possible.

Further, an image signal (A + B image) can be obtained by adding and outputting the signal of the photoelectric conversion unit 201A and the signal of the photoelectric conversion unit 201B for each unit pixel 202.

Even when the above-mentioned image sensor is used, the processes described in the first to third embodiments can be performed, and the focus detection circuit 110 can be omitted.

<Other Embodiments>
The present invention also supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device implement the program. It can also be realized by the process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

101: photographic lens, 102: lens drive circuit, 104: aperture unit, 105: aperture drive circuit, 109: photometric circuit, 110: focus detection circuit, 113: image pickup element, 117: video signal processing circuit, 124: microcomputer, 125: Operation member, 140: Motion detection circuit, 141: In-lens microcomputer, 142: Distance measurement circuit

Claims (13)

Focus detection means that detects the amount of defocus based on the signal obtained by photoelectric conversion of the light incident through the photographing optical system.
A storage means for storing the defocus amount repeatedly detected by the focus detection means, and a storage means.
When the difference between the first defocus amount stored in the storage means and the second defocus amount newly detected by the focus detection means is equal to or greater than the threshold value, the first and second defocus amounts Is not determined to correspond to the same subject, and when the difference is smaller than the threshold value, the determination means for determining that the first and second defocus amounts correspond to the same subject, and
It has a setting means for setting the threshold value and
The setting means is an imaging device that sets the threshold value based on the distance to the subject corresponding to the first defocus amount and the difficulty level information indicating the ease of capturing the subject. The imaging apparatus according to claim 1, wherein when the distance is longer than a predetermined distance, the threshold value is made smaller than when the distance is not long. The setting means is characterized in that when the difficulty level information indicates a second difficulty level higher than the first difficulty level, a threshold value smaller than that of the first difficulty level is set. The imaging apparatus according to claim 1 or 2. Further, it has a subject detecting means for detecting a subject within the angle of view based on the defocus amount detected by the focus detecting means.
The difficulty level information includes at least one of the size of the subject, the presence / absence of an obstacle near the subject, and the camera shake information.
The imaging device according to claim 3, wherein the smaller the size of the subject, the higher the difficulty level, the higher the difficulty level when there is an obstacle, and the larger the camera shake, the higher the difficulty level. Further, it has a subject detecting means for detecting a subject within the angle of view based on the defocus amount detected by the focus detecting means.
The difficulty level information is at least one of the movement amount of the subject, pan / tilt information, depth of field, movement characteristics of the subject, the size of the subject, and the number of subjects detected by the subject detecting means. Including
The larger the movement amount of the subject, the higher the difficulty level, the larger the pan / tilt size, the higher the difficulty level, the larger the depth of field, the higher the difficulty level, and the larger the movement characteristic of the subject, the higher the difficulty level. The imaging apparatus according to claim 3, wherein the higher the subject, the higher the difficulty level, and the smaller the number of detected subjects, the higher the difficulty level. Claim 1 or 2 is characterized in that, when the difficulty level information does not correspond to a predetermined condition, the setting means sets a threshold value larger than that when the condition is satisfied. The imaging apparatus according to the above. Further, it has a subject detecting means for detecting a subject within the angle of view based on the defocus amount detected by the focus detecting means.
The conditions are that the size of the subject is smaller than a predetermined value, that there is an obstacle on the side closer to the subject, and that the amount of runout of the focus detection device is larger than the amount of runout determined in advance. The imaging apparatus according to claim 6, further comprising at least one of the above. Further, it has a control means for controlling to drive the focus lens included in the photographing optical system based on the defocus amount.
When the determination means does not determine that the first and second defocus amounts correspond to the same subject, the control means sets the defocus amount newly detected by the focus detection means. The imaging apparatus according to any one of claims 1 to 7, wherein the focus lens is controlled so as not to be driven. The imaging device according to any one of claims 1 to 8, further comprising the photographing optical system. The imaging device according to any one of claims 1 to 8, wherein the photographing optical system is removable. A focus detection step in which the focus detection means detects the amount of defocus based on a signal obtained by photoelectrically converting the light incident through the photographing optical system.
A storage step in which the storage means stores the defocus amount repeatedly detected by the focus detection means.
When the difference between the first defocus amount stored in the storage means and the second defocus amount newly detected in the focus detection step is equal to or greater than the threshold value, the determination means first and second. Defocus amount is not determined to correspond to the same subject, and when the difference is smaller than the threshold value, it is determined that the first and second defocus amounts correspond to the same subject. Process and
The setting means has a setting step of setting the threshold value and the like.
In the setting step, the image pickup apparatus is characterized in that the threshold value is set based on the distance to the subject corresponding to the first defocus amount and the difficulty level information indicating the difficulty of capturing the subject. Control method. A program for causing a computer to execute each step of the control method according to claim 11. A computer-readable storage medium that stores the program according to claim 12.

Images