JP7431555B2 - Imaging device and its control method - Google Patents

Description

The present invention relates to an imaging device and a control method thereof.

Conventionally, automatic focus adjustment (AF) technology has been known in camera systems such as interchangeable lens single-lens reflex cameras, and among them, a method called phase difference AF is widely used. In this method, light beams from a subject that have passed through different exit pupil areas of a photographic lens are imaged onto a pair of line sensors. Then, by determining the amount of image shift, which is the amount of displacement in the relative position of a pair of image signals obtained by photoelectrically converting the subject image, the amount of defocus of the photographic lens is detected, that is, the focus is detected. Automatic focus adjustment is performed by driving a focus lens included in the photographic lens based on the focus detection result.

Many of them also have a servo shooting mode that drives the focus lens so that not only stationary subjects but also moving subjects are in focus. Servo shooting mode has a function that predicts how the subject is moving based on past movement history. However, if an obstacle passes in front of the subject or if the focus is mistakenly detected against the background, the detected defocus amount may not necessarily be that of the subject to be photographed.

To solve this problem of accidentally focusing on an unintended subject or background, a method has been proposed in which driving of the focus lens is prohibited based on the detected amount of defocus. For example, Patent Document 1 discloses a method of switching a defocus amount threshold that prohibits driving of a focusing lens according to an image plane speed. By using the method of Patent Document 1, if it is observed that the object is moving at high speed, the focus detection lens can be driven even if a larger defocus amount is detected. Therefore, it is possible to appropriately follow acceleration or deceleration of the drive of the focus detection lens according to the detected defocus amount.

Further, Patent Document 2 discloses a method in which a threshold value of the amount of defocus used to determine whether a desired subject is being tracked is changed depending on the shake state of the camera. According to Patent Document 2, it is determined that if the camera shake is large, it is difficult to follow the desired subject, and if the camera shake is small, it is determined that it is easy to follow the desired subject. If the camera is in a state where it is easy to follow the desired subject, the detected defocus amount is considered to be the result of tracking the desired subject. Conversely, if the camera is in a state where it is difficult to track the desired subject, it can be suspected that the desired subject is not being tracked and the detected defocus amount is the result of tracking the background or another subject.

Japanese Patent Application Publication No. 2009-210815 Japanese Patent Application Publication No. 11-326743

However, the technique described in Patent Document 1 can only respond to changes in the movement of a subject that continuously moves quickly. For example, if the subject suddenly makes a large movement that cannot be predicted based on past movement trends, the defocus amount threshold cannot be switched, and focus driving is prohibited. Furthermore, the closer the distance between the photographer (camera) and the subject, the greater the change in the amount of defocus detected in response to the movement of the subject. Changes will be detected. Therefore, when the subject to be photographed continues to move toward or away from the photographer (camera), the following inconvenience occurs.

In the servo shooting mode described above, focus detection processing is performed periodically, and the focus adjustment lens is driven at any time based on the detected defocus amount obtained by the focus detection processing to keep the subject in focus. However, according to Patent Document 1, the driving of the focusing lens is stopped in accordance with the result of focus detection processing obtained at the timing when the subject suddenly starts moving. Therefore, the amount of defocus of the subject to be photographed obtained by the focus detection process at the next timing that is periodically performed is detected to be even larger. This operation is repeated, and as a result, it becomes impossible to focus on the desired subject.

Furthermore, according to Patent Document 2, the ease of tracking a subject is evaluated based only on the amount of camera shake, regardless of the situation of the subject; however, if the area occupied by the subject within the angle of view is large, it is easier to track the subject. Conversely, if the subject area is small, it will be difficult to track. Therefore, determining the ease of tracking a subject based only on the amount of camera shake may lead to incorrect determination of the ease of tracking.

The present invention has been made in view of the above problems, and an object of the present invention is to enable tracking of a subject more appropriately under various conditions.

In order to achieve the above object, an imaging device of the present invention includes a focus detection means for detecting a defocus amount based on a signal obtained by photoelectrically converting light incident through a photographing optical system; a storage means for storing the defocus amount repeatedly detected by the detection means; and a storage means for storing the defocus amount which is the same based on the past focus detection history stored in the storage means and the defocus amount newly detected by the focus detection means. a setting means for setting a threshold value for determining whether or not a subject can be continuously captured ; and a subject detection means for detecting a subject within an angle of view based on the amount of defocus detected by the focus detection means. , the setting means sets the first difficulty level as the threshold value when difficulty level information indicating ease of capturing the subject indicates a second difficulty level higher than the first difficulty level. The difficulty level information includes the size of the subject , and the smaller the size of the subject, the higher the difficulty level. It is characterized by

According to the present invention, a subject can be tracked more appropriately under various conditions.

FIG. 1 is a block diagram showing a schematic configuration of an imaging device according to a first embodiment of the present invention. 5 is a flowchart showing photographing processing in servo photographing mode in the first embodiment. 5 is a flowchart of focus adjustment processing in the first embodiment. 5 is a flowchart of focus detection processing in the first embodiment. 5 is a flowchart of continuity determination processing in the first embodiment. 5 is a flowchart of moving object prediction processing in the first embodiment. 5 is a flowchart of continuity determination threshold setting processing in the first embodiment. The figure which shows an example of a defocus map. The figure which shows an example of a past focus detection history and a prediction curve. FIG. 3 is a diagram showing an example of a change in the image plane position when a stationary subject relatively close to the camera suddenly accelerates. A diagram illustrating an example of a situation in which it is difficult to continue capturing a subject. 7 is a flowchart of continuity determination threshold setting processing in the second embodiment. 7 is a flowchart of focus detection processing in the third embodiment. FIG. 3 is a diagram illustrating an example of a user setting screen for motion characteristics of a subject. The figure explaining the pixel arrangement of the image sensor in a modification.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the claimed invention. Although a plurality of features are described in the embodiments, not all of these features are essential to the invention, and the plurality of features may be arbitrarily combined. Furthermore, in the accompanying drawings, the same or similar components are designated by the same reference numerals, and redundant description will be omitted.

<First embodiment>
FIG. 1 is a block diagram showing the configuration of a digital single-lens reflex camera as an example of an imaging device according to the first embodiment.

Although the photographing lens 101 (photographing optical system) is simply represented by one lens, it is actually composed of a plurality of lenses including a focus lens for performing focus adjustment. The lens drive circuit 102 includes, for example, a DC motor or a stepping motor, and drives and controls the photographing lens 101. The drive control is instructed by the in-lens microcomputer 141, and a drive request for the drive control is propagated from the microcomputer 124 using the lens communication circuit 103.

The lens communication circuit 103 transmits requests for driving various actuators from the microcomputer 124 to the in-lens microcomputer 141, and also transmits status information of the various actuators to the microcomputer 124.

The aperture 104 is driven by an aperture drive circuit 105 and is used to control the light beam incident on the camera. The aperture drive circuit 105 changes the optical aperture value of the aperture 104 based on the drive amount calculated by the microcomputer 124 and is controlled by the in-lens microcomputer 141 .

The distance measurement circuit 142 calculates the distance from the camera to the subject based on the position of the focus lens included in the photographic lens 101. Note that the distance measuring method using the position of the focus lens is just one example, and a method such as infrared irradiation may also be used. Further, in this configuration, the distance measuring circuit 142 is configured on the lens side and controlled by the in-lens microcomputer 141, but it may also be configured on the camera body side and controlled by the microcomputer 124.

The main mirror 106 is normally arranged in the optical path so as to reflect the light flux to guide it to the finder section (not shown), but when shooting, it guides the light flux to the image sensor 113. Jump upwards and evacuate from the light path. That is, it is switched whether to guide the light flux incident from the photographing lens 101 to the finder side or to the image sensor 113 side. Further, the main mirror 106 is a half mirror so that a portion of the light can pass through the central portion thereof. The sub-mirror 107 reflects the light beam that has passed through the main mirror 106 and guides it to a focus detection sensor disposed within a focus detection circuit 110 for performing focus detection.

The pentaprism 108 guides the light beam reflected by the main mirror 106 to a finder section (not shown). The finder section also includes a focusing plate, an eyepiece lens (not shown), and the like. The photometric circuit 109 converts the color and brightness of a subject image formed on a focusing plate (not shown) into electrical signals using a photometric sensor provided with a color filter disposed within the photometric circuit 109 .

The focus detection circuit 110 has a built-in focus detection sensor made up of a plurality of area sensors, and when the light beam transmitted through the main mirror 106 and reflected by the sub-mirror 107 enters the focus detection sensor, a light beam is detected within the focus detection sensor. A pair of spectroscopic image signals is obtained. Then, the amount of defocus of the photographing lens 101 is detected by determining the amount of image shift, which is the amount of displacement in the relative positions of the pair of obtained image signals. The defocus amount obtained by the focus detection circuit 110 is transmitted to the microcomputer 124, converted into a control amount for the photographing lens 101, and then transmitted to the in-lens microcomputer 141 via the lens communication circuit 103. .

The focal plane shutter 111 is driven by a shutter drive circuit 112 according to the shutter speed set in the camera.

The image sensor 113 uses a CCD, a CMOS sensor, or the like, and converts a subject image formed by the photographic lens 101 into an electrical signal.

The clamp circuit 114 and the AGC circuit 115 are circuits that perform basic analog signal processing on the electric signal output from the image sensor 113 before A/D conversion. changes will be made. The A/D converter 116 converts the analog output signal of the image sensor 113 processed by the clamp circuit 114 and the AGC circuit 115 into a digital signal (image data).

The video signal processing circuit 117 is realized by a logic device such as a gate array, and performs filter processing, color conversion processing, gamma processing, and compression processing such as JPEG on the digitized image data, and sends the data to the memory controller 120. Output. Further, the video signal processing circuit 117 can output information such as exposure information and white balance of the signal from the image sensor 113 to the microcomputer 124 as necessary. Based on this information, the microcomputer 124 issues instructions for white balance and gain adjustment.

In the case of continuous shooting operation, the unprocessed shooting data is temporarily stored in the buffer memory 123, the unprocessed shooting data is read out through the memory controller 120, and the video signal processing circuit 117 performs image processing and compression processing. By doing this, you can shoot continuously. The number of continuous shots depends on the size of the buffer memory 123. During continuous shooting, the memory controller 120 stores unprocessed digital image data input from the video signal processing circuit 117 in the buffer memory 123 and stores processed digital image data in the memory 121. Further, the memory controller 120 outputs image data from the buffer memory 123 and the memory 121 to the video signal processing circuit 117. Note that the memory 121 may be removable. Further, the memory controller 120 can output images stored in the memory 121 via an external interface 122 that can be connected to a computer or the like.

The operating section 125 informs the microcomputer 124 of the status of various operating members connected to the operating section 125, and the microcomputer 124 controls each section according to changes in the operating members. The operation members connected to the operation section 125 include a switch 1 (SW1) 126 and a switch 2 (SW2) 127. The operation unit 125 interprets the operation states of SW1 (126) and SW2 (127), and if only the operation of SW1 (126) is valid (ON), it is treated as an autofocus (AF) start request event. Further, if the operation of SW2 (127) is in a valid state (ON), it is treated as a photographing request event. Further, when the SW2 (127) is maintained in an operated state (ON), a continuous shooting operation is performed. Other operation members (not shown) such as an ISO setting button, an image size setting button, an image quality setting button, and an information display button are connected to the operation unit 125, and the state of each operation member is detected.

The liquid crystal drive circuit 128 drives the external liquid crystal display member 129 and the finder liquid crystal display member 130 according to display commands from the microcomputer 124 . A backlight such as an LED (not shown) is arranged in the finder liquid crystal display member 130, and the LED is also driven by the liquid crystal drive circuit 128. The microcomputer 124 checks the memory capacity through the memory controller 120 based on the predicted image size data according to the ISO sensitivity, image size, and image quality set before shooting, and determines the remaining capacity for shooting. Able to calculate numbers. Then, if necessary, the remaining number of shots that can be taken is displayed on the external liquid crystal display member 129 or the finder internal liquid crystal display member 130.

Non-volatile memory 131 (EEPROM) can store data even when the camera is not powered on. The power supply section 132 supplies necessary power to each IC and drive system.

The motion detection circuit 140 is equipped with a detection section that detects motion of a digital single-lens reflex camera. The detection unit is, for example, an acceleration sensor, and detects acceleration when the horizontal axis of the camera body is the rotation axis, and acceleration when the vertical axis of the camera body is the rotation axis.

In this embodiment, the motion detection circuit 140 is configured on the camera body side and controlled by the microcomputer 124, but it may also be configured on the lens side and controlled by the in-lens microcomputer 141. Furthermore, the lens may be removably attached to the camera body, or may be configured integrally with the camera body.

Next, the operation of the photographing process in the first embodiment of the present invention will be explained. In general, camera AF modes include two modes: a mode in which the focusing operation is performed once and the end (one-shot shooting mode), and a mode in which the focusing operation is repeated while predicting the image plane of the subject at a time later than the current time. There are two types of modes (servo shooting mode) in which the lens is driven by the camera. In the first embodiment, processing when the camera is set to servo photography mode will be described.

FIG. 2 is a flowchart showing photographing processing in the servo photographing mode. In S201, the microcomputer 124 confirms the shooting mode. If it is determined that the current photographing mode is not the servo photographing mode, the process ends. On the other hand, if it is determined that the photographing mode is the servo photographing mode, the process moves to S202.

In S202, the microcomputer 124 checks the state of SW1 obtained by the operation unit 125. If SW1 is OFF, the process moves to S201, and if SW1 is ON, the process moves to S203.

In S203, the focus detection circuit 110 performs focus adjustment processing. Note that details of the process in S203 will be described later with reference to FIG.

Next, in S204, the microcomputer 124 checks the state of SW2 obtained from the operation unit 125. If SW2 is OFF, the process moves to S201, and if SW2 is ON, the process moves to S205.

In S205, the microcomputer 124 drives the focal plane shutter 111 via the shutter drive circuit 112 to take an image. After photographing ends, the process returns to S201.

As explained above, when SW1 is ON in the servo shooting mode, the focus adjustment process of S203 is repeatedly performed unless the mode is changed.

Next, an overview of the focus adjustment process in S203 will be described with reference to FIG. 3.

In S301, the microcomputer 124 performs focus detection and a series of processes associated therewith. Note that details of the process in S301 will be described later with reference to FIG.

In S302, the microcomputer 124 determines, based on the focus detection result obtained through the processing in S301, that the focus tracking of the target subject in the servo shooting mode can be continued (hereinafter referred to as "subject capture"). Continuity determination processing is performed to determine whether or not the The details will be described later with reference to FIG. 5, but if the subject can be captured continuously, the flag FlagCatchFail=0, and if the subject fails, the flag FlagCatchFail>0.

In S303, the microcomputer 124 performs a moving object prediction process to predict the movement of the object being captured based on the focus detection history. Note that details of the process in S303 will be described later with reference to FIG. In the process of S303, the microcomputer 124 calculates a focus lens drive amount for driving the focus lens that constitutes the photographic lens 101.

Next, in S304, the microcomputer 124 checks the value of the flag FlagCatchFail set in S302. If FlagCatchFail is greater than 0, that is, if subject capture has failed, the process moves to S305; otherwise, if the subject has been successfully captured, the process moves to S306.

In S305, as a process when the current focus detection result does not indicate that the subject has been successfully captured, lens driving is prohibited and the focus detection process is ended. Note that there are two types of lens drive prohibition processing, and either one may be selected. The first is a method in which the microcomputer 124 requests the in-lens microcomputer 141 to stop the focus lens included in the photographing lens 101 via the lens communication circuit 103. The second method is to avoid executing a drive request based on the defocus amount obtained in the focus detection process in the previous step S301.

On the other hand, in S306, processing is performed when it is determined that the current focus detection result indicates that the subject has been successfully captured. Here, the microcomputer 124 requests the in-lens microcomputer 141 to drive the focus lens included in the photographic lens 101 via the lens communication circuit 103, and the focus adjustment process ends.

FIG. 4 shows an example of the operation of the focus detection process performed in S301. In S401, the microcomputer 124 sets a continuity determination threshold for determining whether the current focus detection result continues to capture the subject, based on the past focus detection history. Note that details of the process in S401 will be described later with reference to FIG.

Next, in S402, the microcomputer 124 drives the focus detection circuit 110 to perform focus detection.

In S403, the microcomputer 124 sets the defocus amount (hereinafter referred to as "detected defocus amount"), which is the focus detection result in S402, so that it corresponds to the focus detection point, as shown in FIG. Create a defocus map written in the matrix.

Here, the defocus map created in this embodiment will be explained using FIGS. 8(a) to 8(d).

Reference numeral 801 corresponds to an image area obtained by the image sensor 113, and reference numeral 802 indicates a sensor area of a focus detection sensor section included in the focus detection circuit 110. It is assumed that the focus detection sensor section of this embodiment is configured with a total of 49 distance measuring points divided into seven vertically and horizontally seven. 803 to 805 indicate some of the divided ranging points. When creating a defocus map, the configuration is such that distance measurement results are obtained for each distance measurement point.

The composition shown in FIG. 8(a) shows a case where a subject 806 on the foreground side and a subject 807 on the far side with respect to the photographer (camera) exist in the same screen, and The person 807 is in focus. In the defocus map at this time, as shown in FIG. 8(b), the detected defocus amount is 0 at the distance measurement point 809 set at the position of the person 807 on the far side, and the amount of detected defocus is set at the position of the person 806 on the near side. At the distance measuring point 808, the detected defocus amount has a negative value. Furthermore, the detected defocus amount is a positive value at the distance measurement point set at the background position.

Similarly, in FIG. 8C, a person 810 in the foreground is in focus, and a subject 811 in the distance is out of focus. In the defocus map at this time, as shown in FIG. 8(d), the detected defocus amount is 0 at the distance measuring point 812 set at the position of the person 810 on the foreground side, and it is set at the position of the person 811 on the far side. At the distance measuring point 813, the detected defocus amount has a positive value. Furthermore, the detected defocus amount has an even larger positive value at the distance measurement point set at the background position.

In S404, the microcomputer 124 selects the main distance measurement point from the defocus map created in S403. Specifically, based on the previous focus detection results, the current focus detection determines whether the subject is approaching or moving away from the photographer (camera), in other words, how far the subject has moved in the depth direction. The distance measurement point closest to this result is set as the main distance measurement point. Note that if the mode set by the user is not a multi-point method in which a plurality of distance measurement points are set, the distance measurement point specified by the user is determined as the main distance measurement point.

In S405, the microcomputer 124 extracts the subject area using the defocus map created in S403. Specifically, an area around the main focus point selected in S404 where the difference in defocus amount from the main focus point is within a certain value is determined as a subject area (subject detection).

Next, in S406, the microcomputer 124 extracts obstacles using the defocus map created in S403. Specifically, an area in which a defocus amount indicating a focus position closer to the subject area extracted in S405 has been detected is extracted from the defocus map, and is regarded as an obstacle.

The above process completes the focus detection process in S301, determines the threshold value for continuity determination based on the past focus detection history, obtains the focus detection result by the focus detection process, and defocuses based on the detected defocus amount. A map is being created. Then, from the defocus map, it is possible to extract the area of the main subject and, if there is an obstacle, specify the area on the defocus map that the obstacle occupies. Referenced.

FIG. 5 shows an example of the operation of the continuity determination process performed in S302. In S501, the microcomputer 124 compares the detected defocus amount of the main focus point detected in S405 with the continuity determination threshold set in S401. If the detected defocus amount is smaller than the continuity determination threshold determined in S401, the process proceeds to S502, and if it is equal to or greater than the continuity determination threshold, the process proceeds to S503.

In S502, the microcomputer 124 determines that the subject has been successfully captured based on the current focus detection result, sets the flag FlagCatchFail to 0, and ends the continuity determination process.

In S503, the microcomputer 124 determines that the subject has not been captured based on the current focus detection result, and increments the flag FlagCatchFail. Note that the flag FlagCatchFail is set to an initial value of 0 at a predetermined timing, such as when starting the camera or setting the servo shooting mode, in addition to the process in S502.

In S504, the microcomputer 124 determines whether the flag FlagCatchFail is 1, that is, whether it is determined that object capture has failed for the first time. If the flag FlagCatchFail is 1, the process moves to S506; otherwise, the process moves to S505.

In S506, the microcomputer 124 sets a follow-up observation timer and then ends the continuity determination process. The follow-up timer indicates the maximum waiting time until it is determined that the subject has been captured. By setting the follow-up timer, if the follow-up timer is counting, it is possible to immediately resume tracking when the subject for which subject capture has failed is detected again.

On the other hand, in S505, the microcomputer 124 determines whether the follow-up timer has expired. If the follow-up observation timer has expired, the process moves to S507; otherwise, the continuity determination process ends.

In S507, the microcomputer 124 erases data such as past focus detection history and tracking, clears the flag FlagCatchFail, and ends the continuity determination process. In this way, when the user intentionally changes the subject, tracking of the new subject can be started immediately after the follow-up timer expires.

FIG. 6 shows an example of the operation of the moving object prediction process performed in S303. In S601, the current focus detection result in S301 is added to the past focus detection history and stored in the storage area of the nonvolatile memory 131 or the microcomputer 124. The information recorded in the focus detection history is the current focus detection time and image plane position. Note that the image plane position is the position of the focus lens where the subject is in focus, which is calculated from the detected defocus amount and the current position of the focus lens.

In S602, the microcomputer 124 determines whether conditions for performing moving object prediction are met. The conditions include, for example, determining whether the image plane position has moved in the same direction multiple times in succession. If the conditions for performing moving object prediction are met, the process transitions to S603; otherwise, the process transitions to S606.

In S603, the microcomputer 124 calculates the predicted drive amount and the variation in focus detection results by drawing a prediction curve as shown in FIG. 9 from the past focus detection history. The vertical axis in Figure 9 shows the image plane position, and the horizontal axis shows time. The larger the image plane position on the vertical axis, the farther the distance is. The history in the figure follows the subject as it approaches the photographer (camera). It shows how it is done. As described above, the focus adjustment process in S203 is performed periodically in the servo shooting mode, and T ₁ to T ₅ each represent the execution time of the focus adjustment process in S203. The predicted drive amount can be calculated by, for example, deriving a prediction curve using the collective least squares method using the past image plane position and each focus detection time, and calculating the image plane position at the predicted destination time based on this curve. can get. The predicted destination time indicates the time at the time of photographing if SW2 is ON, and otherwise indicates the current focus detection time.

In S604, the microcomputer 124 determines whether to finally perform predictive focus driving based on the variation in the focus detection results calculated in S603. Specifically, if the dispersion is above a certain value, it is determined that the prediction cannot be trusted and the predictive focus drive is not performed, and if the dispersion is less than a certain value, it is determined that the prediction can be trusted and the predictive focus drive is performed. . If it is determined that predictive focus driving is to be performed, the process moves to S605; otherwise, the process moves to S606.

Predictive focus driving is control for driving the lens from the current image plane position L3 to the image plane position at a future time when focusing is desired, based on the prediction curve explained with reference to FIG. The black circles in FIG. 9 indicate image plane positions determined from the detected defocus amount, and the white circles indicate predicted image plane positions. For example, assume that the defocus amount detected at time _T4 in FIG. 9 is x. If the objective is to drive the lens and focus at time _T5 , then the lens drive corresponding to the defocus amount y is required. In order to calculate this defocus amount y, it is necessary to draw an approximate curve from past focus detection results and use this as a predicted curve to predict the image plane position at _T5 . In this way, in order to perform focus detection at _T4 and aim for focusing at _T5 , lens drive in which focus drive is performed by the defocus amount y is called predictive focus drive. On the other hand, when predictive focus driving is not performed, that is, when prediction is not performed from past focus detection history or when prediction results are not used, the focus driving amount becomes the defocus amount x, that is, the detected defocus amount.

In S605, the microcomputer 124 sets the focus drive amount to the predicted drive amount, and ends the moving object prediction process. On the other hand, in S606, the microcomputer 124 sets the focus drive amount to the detected defocus amount, and ends the moving object prediction process.

Next, with reference to FIG. 7, an example of the operation of the continuity determination threshold setting process performed in S401 will be described. The continuity determination threshold is a threshold for determining whether the subject can be captured continuously, and is used in the comparison process in S501 described above. As described above, if the detected defocus amount is larger than the continuity determination threshold value, it is determined that the subject capture has failed, and otherwise, it is determined that the subject capture has been successful.

In S701, the microcomputer 124 receives object distance information measured by the distance measurement circuit 142 during the previous focus detection via the in-lens microcomputer 141 and the lens communication circuit 103, so that the object currently being captured is detected by the camera. Determine whether the distance is greater than a certain distance. If the subject is at a certain distance or more from the camera, the process transitions to S702; otherwise, the process transitions to S703.

In S702, the microcomputer 124 sets the continuity determination threshold to the first threshold, and ends the continuity determination threshold setting process.

Ｏ_Ｄは、検出した被写体領域よりも至近側の合焦位置を示すデフォーカス量が検出された測距点の数を示す。Ｏ_Ａは、Ｏ_Ｄの算出のために参照する点数を示し、この最大値は画角全体の測距点の数である。Ｊ_Ｄは、動き検出回路１４０で検出したカメラ振れ量の絶対値を示す。Ｊ_Ｐは、カメラ振れ量Ｊ_Ｄの許容最大値を示す。Ｓ_Ｃは、主測距点と近いデフォーカス量が検出された測距点の数を示す。Ｓ_Ａは、Ｓ_Ｃの算出のために参照する点数を示し、この最大値は画角全体の測距点の数である。Ｓ_Ｐは、被写体サイズ許容最小量を示し、測距点数で表わされる。 On the other hand, in S703, the microcomputer 124 calculates the subject capture difficulty level D, which indicates how easy the conditions are for capturing the subject. In this embodiment, the information representing the object capture difficulty level (difficulty information) includes the size within the field of view of the object area obtained in S405 during the previous focus detection, the presence or absence of an obstacle obtained in S406, and the motion detection. Three pieces of camera shake information obtained from the angular velocity information detected by the circuit 140 are used. Based on this information, the subject capture difficulty level D is derived using the following equation (1).

_OD indicates the number of ranging points where a defocus amount indicating a focus position closer to the detected subject area was detected. _OA indicates the number of points referred to for calculating _OD , and its maximum value is the number of distance measurement points over the entire angle of view. _JD indicates the absolute value of the amount of camera shake detected by the motion detection circuit 140. _JP indicates the maximum allowable value of camera shake amount _JD . _SC indicates the number of distance measuring points where a defocus amount close to that of the main focusing point is detected. S _A indicates the number of points referred to for calculating S _C , and its maximum value is the number of distance measurement points over the entire angle of view. S _P indicates the minimum allowable object size and is expressed by the number of distance measurement points.

The subject capturing difficulty level D is a value between 0 and 1, and the larger the value, the easier it is to capture the subject, and the smaller the value, the more difficult it is to capture the subject.

Based on the subject capture difficulty level calculated in S703, the microcomputer 124 determines whether the subject is easy to capture in S704. As an example, if D≧0.5, it is determined that the subject is easy to capture; otherwise, it is determined that the subject is difficult to capture. If D≧0.5, that is, if it is determined that the subject is easy to capture, the process moves to S705; otherwise, the process moves to S706.

In S705, the microcomputer 124 sets the continuity determination threshold to a second threshold that is larger than the first threshold, and ends the continuity determination threshold setting process. This second threshold value is variable depending on the distance to the subject, and becomes larger as the subject gets closer.

In S706, the microcomputer 124 sets the continuity determination threshold to a third threshold that is smaller than the second threshold. In step S704, it is determined that it is difficult to capture the subject under the current shooting situation, so if the continuity determination threshold is set to the second threshold, even if the subject cannot be captured continuously, it will be difficult to capture the subject. There is a possibility that it will be interpreted as a change in the movement of the subject inside. Therefore, in S706, a third threshold value that is smaller than the second threshold value and that takes a stepwise value according to the subject capture difficulty level D is adopted. As an example, if the second threshold is D ₂ and the first threshold is D ₁ at the same subject distance, the third threshold D ₃ is D ₃ = (D ₂ - D ₁ )・2D+D ₁ (2)
It can be expressed as By doing so, the third threshold D ₃ can take a value between the first threshold D ₁ and the second threshold D ₂ in stages according to the subject capture difficulty level. As described above, the continuity determination threshold is set as the third threshold, and the continuity determination threshold setting process is completed.

Next, the effects obtained in the first embodiment will be explained using FIGS. 10 and 11. FIG. 10 shows a case where a stationary subject relatively close to the camera suddenly accelerates in a direction away from the camera. shows the change in the image plane position. The horizontal axis indicates time, and the vertical axis indicates image plane position. The larger the vertical axis, the larger the subject distance. As at points t ₁ , t ₂ , and t ₃ , the image plane position does not change while the subject is stationary. However, when the subject suddenly accelerates, a large change in defocus amount is observed, such as at point _t5 , which cannot be predicted from the trend of the past focus detection history where the subject has been stationary. In this way, when tracking a subject that starts moving rapidly from a stationary state, the characteristics of the subject's movement are significantly different from the previous focus detection history, so it is difficult to predict that the subject will move significantly at the next moment. Furthermore, the closer the distance between the photographer (camera) and the subject, the greater the amount of image plane movement observed at the moment the subject moves. Therefore, it becomes difficult to observe the continuity of the distance measurement history, especially when tracking a subject that rapidly moves from a stationary state in the foreground.

Therefore, in this embodiment, by increasing the continuity determination threshold when the subject is relatively close to the camera, it is possible to cope with such sudden changes in the image plane position.

On the other hand, as shown in Figure 11, it is difficult to continue capturing the subject when there is an obstacle in front of the subject (Figure 11(a)), when the subject is small (Figure 11(b)), or when the camera is shaking. If you simply take measures based on the distance to the subject, you may end up capturing a subject that you are not tracking. Therefore, in the present invention, it is determined when the situation is such that it is difficult to continue capturing the subject, and the continuity determination threshold value is suppressed to prevent erroneously detecting a subject that is not a tracking target as a capture target.

As described above, in the present embodiment, the threshold value for determining whether the current focus detection result continues to capture the currently captured subject is set to a large value in the foreground. This is because the closer the distance between the photographer (camera) and the subject, the greater the change in defocus amount detected with respect to the movement of the subject. Furthermore, the difficulty of capturing the subject was evaluated, and in accordance with this evaluation result, the threshold value used to determine whether the current focus detection result could continue to capture the currently captured subject was dynamically changed. Specifically, evaluation was made using three factors: the size of the target object within the angle of view obtained from defocus information at the time of previous focus detection, the presence or absence of obstacles, and camera shake information obtained from angular velocity information. This makes it possible to track objects that suddenly start moving in the foreground. In addition, in environments where it is easy to falsely detect undesired objects as capture targets, it is possible to suppress false detections while maintaining close subject tracking performance.

<Second embodiment>
Next, a second embodiment of the present invention will be described. In the first embodiment, an example has been described in which the third threshold value is set as a numerical value that changes depending on the ease of tracking the subject. In contrast, in the second embodiment, when the third threshold matches the first threshold, that is, under conditions where it is difficult to track the subject, the difference in defocus amount is increased to determine that the subject is the same as the previous one. An example of not doing so will be explained below.

The continuity determination threshold setting process in the second embodiment of the present invention will be described below with reference to FIG. 12. Note that the second embodiment differs from the first embodiment in the process in S401 of FIG. 4, and the process shown in FIG. 12 is executed instead of the process shown in FIG. 7. The rest is the same as the first embodiment, so the explanation will be omitted.

FIG. 12 is a diagram illustrating an example of the operation of the continuity determination threshold setting process performed in S401 in the second embodiment.

In S1201, the microcomputer 124 receives object distance information measured by the distance measuring circuit 142 via the in-lens microcomputer 141 and the lens communication circuit 103, so that the currently captured object is closer to the camera than a certain level. Determine whether the distance is . If the distance of the subject is a certain distance or more, the process transitions to S1202, otherwise the process transitions to S1204.

In S1202, the microcomputer 124 sets the continuity determination threshold to the first threshold, and ends the continuity determination threshold setting process.

On the other hand, in S1204, the microcomputer 124 determines whether the subject is difficult to capture. Here, it is determined whether any of the following three conditions apply.
- The size of the target subject within the angle of view obtained from defocus information is smaller than a certain value - There is an obstacle (there is an object in front of the desired subject within the angle of view)
- If any of the above-mentioned conditions in which the amount of camera shake obtained from the angular velocity information is greater than a certain value is met, it is determined that it is difficult to capture the subject; otherwise, it is determined that it is easy to capture the subject. If it is determined that the subject is easy to capture, the process moves to S1206; otherwise, the process moves to S1202.

In S1206, the microcomputer 124 sets the continuity determination threshold to a second threshold that is larger than the first threshold, and ends the continuity determination threshold setting process. This second threshold value is variable depending on the distance to the subject, and becomes larger as the subject gets closer.

As explained above, in the second embodiment, if there is a fear that the subject cannot be continued to be tracked, it is made difficult to recognize that the subject is the same subject when the difference in defocus amount becomes large. This makes it possible to set the continuity determination threshold more difficult to expand than in the first embodiment, which is particularly suitable when it is desired to avoid capturing the wrong subject. Furthermore, since the amount of calculation can be reduced, overall performance is not significantly affected.

<Third embodiment>
Next, a third embodiment of the present invention will be described. In the first embodiment, the following three conditions are used to evaluate the difficulty of tracking a subject: the size of the target subject within the angle of view obtained from defocus information, the presence or absence of obstacles, and camera shake information obtained from angular velocity information. One was used.

In the third embodiment, pan/tilt information or xy plane motion vectors obtained from camera posture and camera shake information, depth of field, motion characteristics of the subject set by the user, subject size obtained by image processing, face and Information on the number of objects such as animals (difficulty level information) is used to express the difficulty of capturing the objects.

Hereinafter, with reference to FIG. 13 and an explanation of the method for calculating the difficulty level of capturing a subject, the difference in the method of calculating the difficulty level of capturing the subject in the third embodiment of the present invention will be explained. Note that in the third embodiment, the process shown in FIG. 13 is performed instead of the process shown in FIG. Omitted. Further, in FIG. 13, the same processes as those in FIG. 4 are given the same reference numerals, and the description thereof will be omitted.

In the focus detection process shown in FIG. 13, in S1304, the microcomputer 124 extracts from the image acquired by the video signal processing circuit 117 the position and number of objects in the image, such as a human face/body or an animal.

In S1305, the microcomputer 124 selects a main subject from among the subjects extracted in S1304, and calculates the size of the subject within the screen. As the main subject, a subject that should be prioritized in advance by the user or a subject that appears larger is preferentially selected.

In S1306, the microcomputer 124 determines the main focus point using the main subject area extracted in S1305 and the defocus map created in S1303.

Next, differences from the first embodiment regarding the method of calculating the subject capture difficulty level performed in S703 of FIG. 7 will be explained. Note that although the method for calculating the subject capture difficulty level is different from that in the first embodiment, other processing is the same as the processing in FIG. 7, so a description thereof will be omitted.

In this embodiment, as described above, the xy plane motion vector or pan/tilt information, the depth of field, the motion characteristics of the subject, the size of the subject, and the number of subjects are used to calculate the subject capture difficulty level. The xy plane motion vector expresses the difference in the position of the main subject detected in S1305 during the previous focus detection on the screen at the previous focus detection. The pan/tilt information is calculated from the camera posture and camera shake information obtained from the motion detection circuit 140. The depth of field is calculated from the aperture amount driven by the drive circuit 105, and the movement characteristics of the subject are set in advance by the user on the selection screen shown in FIG. The object size is obtained by image processing in S1305, and the number of objects such as faces and animals detected in S1304 is used. Note that when the motion characteristic of the subject is set to "fast", the continuity determination threshold can be set to a large value, and when the motion characteristic of the subject is set to "slow", the continuity determination threshold can be set to a low value. By doing so, it is possible to easily respond to an increase or decrease in the defocus amount for a subject that is assumed to be moving quickly, and to be less affected by distance measurement noise for a subject that is assumed to be moving slowly.

式（３）において、Ｖ_Ｄは、観測したｘｙ平面動きベクトル（画面上での被写体の動きベクトル）を示す。Ｍ_Ｖは、被写体を安定的に追えていると考えられる画面上での被写体の動き量を示す。Ｅ_ｍは、許容できる最小の被写界深度を示す。Ｅ_Ｄは、今回のカメラ設定での被写界深度を示す。Ｅ_Ｍは、Ｅ_Ｄが取りうる最大値を示す。Ｉ_ｍは、被写体を安定的に追える限界の最小被写体サイズを示す。Ｉ_Ｄは、画像処理によって算出した被写体のサイズを示す。Ｉ_Ａは、画角のサイズを示す。Ｆ_Ｎは、被写体の動き特性設定値の全段数を示す。Ｆ_Ｕは、ユーザの設定した被写体の動き特性設定値を示す。Ｎは顔や動物などの被写体の個数を示す。なお、被写体を検出できなかったときであっても、Ｎの最小値は１とする。 The subject capture difficulty level D in this embodiment can be expressed by the following equation (3).

In equation (3), _VD represents the observed xy plane motion vector (the motion vector of the subject on the screen). _MV indicates the amount of movement of the subject on the screen when it is considered that the subject can be stably tracked. E _m indicates the minimum allowable depth of field. _ED indicates the depth of field with the current camera settings. E _M indicates the maximum value that E _D can take. I _m indicates the minimum object size at which the object can be stably tracked. _ID indicates the size of the subject calculated by image processing. _IA indicates the size of the angle of view. _FN indicates the total number of stages of the motion characteristic setting values of the subject. _FU indicates the motion characteristic setting value of the subject set by the user. N indicates the number of subjects such as faces and animals. Note that even when the subject cannot be detected, the minimum value of N is set to 1.

Furthermore, the xy plane motion vector may not be observed because it is difficult to capture the subject, such as when the subject has a similar color to the background. At this time, in equation (3), the pan/tilt vector _PD of the observed camera is used instead of the xy plane motion vector _VD . Further, instead of the amount of movement M _V of the subject on the screen that is considered to be able to stably track the subject, the maximum movement amount M _P of the camera that is considered to be able to stably track the subject is used.

As explained above, according to the third embodiment, the difficulty of capturing a subject is expressed from a different perspective than the first embodiment, and depending on the evaluation result, the current focus detection result continues to reflect the subject currently being captured. It is possible to dynamically change the threshold value for determining whether the image is captured. Specifically, if the situation is such that it can be assumed that the subject can be tracked even if the camera is shaking, the subject continuity determination threshold need not be made small. Further, the threshold value can be set while considering how much influence it will have on the image. Furthermore, the subject continuity determination threshold can also be set in consideration of the ease with which the focus shifts to another subject visible on the screen and the motion characteristics of the subject set by the user.

<Modified example>
In the first to third embodiments described above, a case has been described in which the camera has the focus detection circuit 110 dedicated to focus detection.

On the other hand, in recent years, pupil division signals are acquired by an image sensor in which a plurality of photoelectric conversion units are provided for one microlens, and focus detection is performed based on the phase difference of the obtained plurality of pupil division signals. So-called image plane phase difference AF is performed.

FIG. 15 is a diagram showing a pixel array constituting an image sensor capable of performing image plane phase difference AF, and can be used as the image sensor 113.

As shown in FIG. 15, a plurality of photoelectric conversion units 201A and 201B are arranged to correspond to each microlens 201 forming a microlens array. In the figure, each region indicated by (0,0), (1,0), (0,1), etc. indicates a unit pixel 202 in the image sensor, and each unit pixel 202 includes one microlens 201 and a plurality of microlenses 201. It is composed of photoelectric conversion units 201A and 201B. Note that although this embodiment shows a case in which two photoelectric conversion units 201A and 201B are arranged in the X-axis direction in the unit pixel 202, it is also possible to arrange three or more photoelectric conversion units in the Y-axis direction. may be configured to correspond to each microlens 201.

Among the plurality of unit pixels 202 arranged in the X-axis direction configured in this way, a subject image composed of a focus detection signal group of the pupil-divided photoelectric conversion unit 201A is referred to as an A image, and the pupil-divided photoelectric conversion unit The subject image composed of the focus detection signal group 201B is referred to as a B image. Then, by performing a correlation calculation on the A image and the B image, the amount of image shift (phase difference) is detected. Furthermore, by multiplying the image shift amount by a conversion coefficient determined according to the focus position and the characteristics of the optical system, it is possible to calculate the focus position corresponding to an arbitrary subject position within the screen. By controlling the focus lens included in the photographing lens 101 based on the focal position information calculated here, imaging plane phase difference AF for detecting the focal state becomes possible.

Further, by adding and outputting the signal of the photoelectric conversion unit 201A and the signal of the photoelectric conversion unit 201B for each unit pixel 202, an image signal (A+B image) can be obtained.

Even when the above-described image sensor is used, the processes described in the first to third embodiments can be performed, and the focus detection circuit 110 can be omitted.

<Other embodiments>
Further, the present invention provides a system or device with a program that implements one or more functions of the above-described embodiments via a network or a storage medium, and one or more processors in a computer of the system or device executes the program. This can also be realized by reading and executing processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the embodiments described above, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, the following claims are hereby appended to disclose the scope of the invention.

101: Photographing lens, 102: Lens drive circuit, 104: Aperture unit, 105: Aperture drive circuit, 109: Photometry circuit, 110: Focus detection circuit, 113: Image sensor, 117: Video signal processing circuit, 124: Microcomputer, 125: Operation member, 140: Movement detection circuit, 141: In-lens microcomputer, 142: Distance measurement circuit

Claims (12)

a focus detection means for detecting a defocus amount based on a signal obtained by photoelectrically converting light incident through the photographing optical system;
storage means for storing the defocus amount repeatedly detected by the focus detection means;
Setting a threshold value for determining whether the same subject can be continuously captured based on the past focus detection history stored in the storage means and the amount of defocus newly detected by the focus detection means. a setting means for
a subject detection means for detecting a subject within the angle of view based on the defocus amount detected by the focus detection means,
The setting means sets the threshold value to a second difficulty level that is higher than the first difficulty level, when the difficulty level information indicating the ease of capturing the subject indicates a second difficulty level that is higher than the first difficulty level. We set a threshold that makes it difficult to determine that the same subject is being captured continuously .
The imaging device is characterized in that the difficulty level information includes the size of the subject, and the smaller the size of the subject, the higher the difficulty level . The setting means further sets a threshold based on a distance corresponding to a past focus detection history stored in the storage means,
Imaging according to claim 1, characterized in that when the distance is longer than a predetermined distance, a threshold value is set that makes it more difficult to determine that the same subject has been continuously captured than when the distance is not longer. Device. The difficulty level information further includes at least one of the presence or absence of an obstacle closer to the subject and camera shake information,
The imaging device according to claim 1 or 2, wherein the difficulty level is higher when the obstacle is present , and the difficulty level is higher as the camera shake is larger. The difficulty level information includes at least one of the following : amount of movement of the subject, pan/tilt information, depth of field, motion characteristics of the subject, size of the subject, and number of subjects detected by the subject detection means. including
The larger the amount of movement of the object , the higher the difficulty, the larger the pan/tilt, the higher the difficulty, the larger the depth of field, the higher the difficulty, and the movement characteristics of the object. 3. The difficulty level is higher as the number of detected objects is larger, the difficulty level is higher as the object is larger, and the difficulty level is higher as the number of detected objects is smaller. Imaging device. The setting means sets a threshold value that makes it easier to determine that the same subject has been continuously captured when the difficulty level information does not fall under a predetermined condition than when the condition falls under the condition. The imaging device according to claim 1 or 2. The conditions include that the size of the same subject is smaller than a predetermined value, that there is an obstacle closer to the subject, and that the amount of shake of the focus detection device is greater than the predetermined amount of shake. The imaging device according to claim 5 , comprising at least one of the following. further comprising a control means for controlling to drive a focus lens included in the photographing optical system based on the amount of defocus,
If it is not determined that the same subject can be continuously captured based on the past focus detection history and the defocus amount newly detected by the focus detection means, the control means 7. The imaging apparatus according to claim 1, wherein control is performed so that the focus lens is not driven based on the defocus amount newly detected by the focus detection means. The imaging device according to any one of claims 1 to 7 , comprising the imaging optical system. The imaging device according to any one of claims 1 to 7 , wherein the imaging optical system is removable. a focus detection step of detecting a defocus amount based on a signal obtained by photoelectrically converting the light incident through the photographing optical system;
a storage step of storing the defocus amount repeatedly detected in the focus detection step;
Setting a threshold value for determining whether the same subject can be continuously captured based on the past focus detection history stored in the storage step and the defocus amount newly detected in the focus detection step. a setting process to
a subject detection step of detecting a subject within the angle of view based on the defocus amount detected in the focus detection step,
In the setting step, when the difficulty level information indicating the difficulty of capturing the subject indicates a second difficulty level that is higher than the first difficulty level, the threshold value is set to be the same as the first difficulty level. We set a threshold that makes it difficult to determine that the subject is being continuously captured .
The method for controlling an imaging device , wherein the difficulty level information includes the size of the subject, and the smaller the subject size, the higher the difficulty level . A program for causing a computer to execute each step of the control method according to claim 10 . A computer readable storage medium storing the program according to claim 11 .

Images