JP5403111B2 - Image tracking device - Google Patents

Description

  The present invention relates to an image tracking device, an image tracking method, and an imaging device.

The image to be focused is stored as a reference image (template image), an image matching the reference image is detected from the captured images by template matching processing, and the focus detection area is moved to the detection position. There is known an image tracking device that tracks the image (for example, see Patent Document 1).
In this apparatus, each time the focus detection area is moved to the detection position by the pattern matching process, the reference image is updated with the target image at the detection position, and the original target can be accurately tracked even when the target image changes. I am doing so.
Prior art documents related to the invention of this application include the following.

JP 2006-058431 A

  By the way, in sports photography such as soccer, there is a case where it is desired to keep following a player who always holds the ball. However, in the conventional image tracking device described above, every time the player holding the ball changes, an operation for specifying the player to be tracked must be performed, and the operation is complicated and the player who always holds the ball is tracked. There is a problem that it is not suitable for such shooting.

An image tracking device according to a first aspect of the present invention includes an imaging unit that repeatedly acquires image information within a screen by an optical system, a focus detection unit that detects a plurality of defocus amounts for a plurality of focus detection areas, and the focus detection unit. A focus adjustment control unit that performs focus adjustment control using the defocus amount detected by the determination unit, and a determination unit that determines a determination focus detection area used for focus adjustment control by the focus adjustment control unit from the plurality of focus detection areas A calculation unit that calculates a predicted defocus amount based on information related to past focus adjustment control by the focus adjustment control unit, and a control unit that controls to store reference information including tracking target information in the storage unit A tracking calculation unit that calculates the position of the tracking target using the reference information and the image information repeatedly acquired by the imaging unit, and a half-press Determining whether a difference between the defocus amount of the first focus detection area corresponding to the position of the focus detection area designated when the operation is performed and the predicted defocus amount is smaller than a first threshold value; When the difference between the first determination unit and the first determination unit is not determined to be smaller than the first threshold, the defocus of the second focus detection area corresponding to the position of the tracking target calculated by the tracking calculation unit And a second determination unit that determines whether or not a difference between the amount and the predicted defocus amount is smaller than a second threshold , and the determination unit causes the difference to be less than the first threshold by the first determination unit. Is determined to be smaller, the first focus detection area is determined as the determined focus detection area, and the determined focus is determined to be smaller than the second threshold by the second determination unit. Determining the second focus detection area as out area, wherein, when the difference by the first determination unit is determined to be smaller than the first threshold value, the difference by the first determination unit is the The image information of the region corresponding to the position of the first focus detection area acquired after being determined to be smaller than the first threshold is replaced with the reference information stored in the storage unit, and the first determination unit When it is not determined that the difference is smaller than the first threshold, the reference information stored in the storage unit is stored in the storage unit with image information of an area corresponding to the position of the second focus detection area. The reference information is replaced with reference information generated using the reference information .
An image tracking device according to a sixth aspect of the invention includes an imaging unit that repeatedly acquires image information within a screen by an optical system, a focus detection unit that detects a plurality of defocus amounts for a plurality of focus detection areas, and the focus detection unit. A focus adjustment control unit that performs focus adjustment control using the defocus amount detected by the determination unit, and a determination unit that determines a determination focus detection area used for focus adjustment control by the focus adjustment control unit from the plurality of focus detection areas A calculation unit that calculates a predicted defocus amount based on information related to past focus adjustment control by the focus adjustment control unit, and a control unit that controls to store reference information including tracking target information in the storage unit A tracking calculation unit that calculates the position of the tracking target using the reference information and the image information repeatedly acquired by the imaging unit, and a half-press Determining whether a difference between the defocus amount of the first focus detection area corresponding to the position of the focus detection area designated when the operation is performed and the predicted defocus amount is smaller than a first threshold value; When the difference between the first determination unit and the first determination unit is not determined to be smaller than the first threshold, the defocus of the second focus detection area corresponding to the position of the tracking target calculated by the tracking calculation unit And a second determination unit that determines whether or not a difference between the amount and the predicted defocus amount is smaller than a second threshold, and the determination unit causes the difference to be less than the first threshold by the first determination unit. Is determined to be smaller, the first focus detection area is determined as the determined focus detection area, and the determined focus is determined to be smaller than the second threshold by the second determination unit. And determining the second focus detection area as out area.
An image tracking device according to a seventh aspect of the invention includes an imaging unit that repeatedly acquires image information within a screen by an optical system, a focus detection unit that detects a plurality of defocus amounts for a plurality of focus detection areas, and the focus detection unit. A focus adjustment control unit that performs focus adjustment control using the defocus amount detected by the calculation, a calculation unit that calculates a predicted defocus amount based on information on past focus adjustment control by the focus adjustment control unit, and tracking The position of the tracking target is calculated using a control unit that controls to store reference information including target information in a storage unit, the reference information, and the image information that is repeatedly acquired by the imaging unit. The tracking calculation unit, the defocus amount of the first focus detection area corresponding to the position of the focus detection area designated when the half-press operation is performed, and the predicted default And a first determination unit that determines whether or not the difference from the first amount is smaller than a first threshold, and the control unit is determined by the first determination unit that the difference is smaller than the first threshold. In this case, image information of a region corresponding to the position of the first focus detection area acquired after the first determination unit determines that the difference is smaller than the first threshold is stored in the storage unit. The tracking information calculated by the tracking calculation unit is replaced with the reference information stored in the storage unit when the difference is not determined to be smaller than the first threshold by the first determination unit. It is characterized by replacing the reference information generated using the image information of the area corresponding to the position of the second focus detection area corresponding to the position of the target and the reference information stored in the storage unit.

  According to the present invention, in a shooting scene in which the tracking target needs to be switched during tracking shooting, it is possible to automatically switch to the intended target smoothly without performing an operation of switching the tracking target.

The figure which shows the structure of the imaging device provided with the image tracking device of one embodiment The figure which shows the detailed structure of a body drive control apparatus The figure which shows the detailed structure of the image pick-up element for image tracking The figure which shows the pixel structure of the image pick-up element for image tracking Diagram showing the focus detection area set in the shooting screen The figure for demonstrating the to-be-photographed object tracking method of one embodiment. The figure for demonstrating the to-be-photographed object tracking method of one embodiment. The figure for demonstrating the to-be-photographed object tracking method of one embodiment. The figure for demonstrating the to-be-photographed object tracking method of one embodiment. Flowchart showing subject tracking processing of an embodiment Flowchart showing tracking control initial processing of one embodiment The flowchart which shows the focusing area determination processing of one embodiment The flowchart which shows the tracking calculation processing of one embodiment

The focus adjustment state of the photographic lens (in this embodiment, the defocus amount) is detected in a plurality of focus detection areas set in the photographic screen, and the photographic lens is focused based on the defocus amount in one of the areas. The automatic focus adjustment (AF) function to be driven and the image of the subject to be tracked in the photographed image are stored as a template image (reference image). An image pickup apparatus (single-lens) having an image tracking function for tracking a subject to be tracked while searching for a position (template matching), and tracking an object while driving a photographing lens with an AF function and an image tracking function An embodiment of a reflex digital still camera will be described.

  FIG. 1 shows a configuration of an imaging apparatus (single-lens reflex digital still camera) 1 including an image tracking apparatus according to an embodiment. In FIG. 1, illustration and description of camera devices and circuits not directly related to the present invention are omitted. In a camera 1 according to an embodiment, an interchangeable lens 3 is attached to a camera body 2 in a replaceable manner. The camera body 2 is provided with a first image sensor 4 for capturing a subject image and recording the image. The first image sensor 4 can be constituted by a CCD, a CMOS, or the like. At the time of photographing, the quick return mirror 5 and the sub mirror 6 are retracted to a position outside the photographing optical path indicated by a solid line, the shutter 7 is opened, and a subject image is formed on the light receiving surface of the first image sensor 4 by the photographing lens 8.

  At the bottom of the camera body 2, a focus detection optical system 9 and a distance measuring element 10 for detecting the focus adjustment state of the photographing lens 8 are provided. In this embodiment, an example in which a focus detection method based on a pupil division phase difference detection method is adopted is shown. The focus detection optical system 9 guides the pair of focus detection light fluxes that have passed through the photographing lens 8 to the light receiving surface of the distance measuring element 10 and forms a pair of optical images. The distance measuring element 10 includes, for example, a pair of CCD line sensors, and outputs a focus detection signal corresponding to the pair of optical images. Before shooting, the quick return mirror 5 and the sub mirror 6 are set at positions in the shooting optical path as indicated by broken lines, and the pair of focus detection light beams from the shooting lens 8 are transmitted through the half mirror portion of the quick return mirror 5. Then, the light is reflected by the sub mirror 6 and guided to the focus detection optical system 9 and the distance measuring element 10.

  A finder optical system is provided on the upper part of the camera body 2. Before shooting, the quick return mirror 5 and the sub mirror 6 are in positions indicated by broken lines, and subject light from the shooting lens 8 is reflected by the quick return mirror 5 and guided to the focusing screen 11, and the subject image is displayed on the focusing screen 11. Form an image. The liquid crystal display element 12 superimposes and displays information such as a focus detection area mark on the subject image formed on the focusing screen 11 and displays various photographing information such as an exposure value at a position outside the subject image. The subject image on the focusing screen 11 is guided to the eyepiece window 15 via the penta roof prism 13 and the eyepiece lens 14 so that the photographer can visually recognize the subject image.

  The finder optical system at the top of the camera body 2 is provided with a second image sensor 16 that captures a subject image for subject tracking and photometry. Details of the second image sensor 16 will be described later. The subject image formed on the focusing screen 11 is re-imaged on the light receiving surface of the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18. The second image sensor 16 outputs an image signal corresponding to the subject image. The subject image formed on the focusing screen 11 before photographing is guided to the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18, and the subject image is formed on the light receiving surface of the second image sensor 16. Is re-imaged. As will be described in detail later, tracking control and exposure calculation are performed based on the subject image captured by the second image sensor 16.

  The camera body 2 is also provided with a body drive control device 19 and an operation member 20. The body drive control device 19 is constituted by peripheral components such as a microcomputer, a memory, and an A / D converter, the details of which will be described later, and performs various controls and calculations of the camera 1. The operation member 20 includes a switch and a selector for operating the camera 1 such as a shutter button, a focus detection area selection switch, and a shooting mode selection switch.

The interchangeable lens 3 is provided with a zooming lens 8a, a focusing lens 8b, a diaphragm 21, a lens drive control device 22, and the like. In this embodiment, the photographic lens 8 is represented by a zoom lens 8a, a focusing lens 8b, and a diaphragm 21, but the configuration of the photographic lens 8 is not limited to the configuration shown in FIG. The lens drive control device 22 includes a microcomputer (not shown) and peripheral components such as a memory, a drive circuit, and an actuator, and performs drive control of the lenses 8a and 8b and the diaphragm 21 and detection of their set positions. Information such as a focal length and an open aperture value of the interchangeable lens 3 is stored in a memory built in the lens drive control device 22.

  The body drive control device 19 and the lens drive control device 22 communicate via the contact 23 of the lens mount, and transmit information such as a lens drive amount and an aperture value from the body drive control device 19 to the lens drive control device 22. Lens information and aperture information are transmitted from the lens drive control device 22 to the body drive control device 19.

  FIG. 2 shows a detailed configuration of the body drive control device 19. Note that illustration and description of control functions not directly related to the present invention are omitted. The body drive control device 19 includes an element control circuit 19a, an A / D converter 19b, a microcomputer 19c, a memory 19d, and the like. The element control circuit 19 a controls charge accumulation and reading of the second image sensor 16. The A / D converter 19b converts the analog image signal output from the second image sensor 16 into a digital image signal. The microcomputer 19c constitutes a tracking control unit 19e, an exposure control unit 19f, a focus detection calculation unit 19g, and a lens drive amount calculation unit 19h according to a software form. The memory 19d stores information such as a template image for image tracking and a defocus amount, or lens information such as a focal length, an open F value, an aperture value, and a conversion coefficient from an image shift amount to a defocus amount. To do.

  The tracking control unit 19e stores, as a template image, an image corresponding to the tracking target position manually specified by the photographer or the tracking target position automatically set by the camera 1 in the subject image captured by the second image sensor 16. The target position is recognized by searching for an image area that matches or is similar to the template image from among the images that are stored in 19d and repeatedly photographed thereafter. The exposure calculation unit 19f calculates an exposure value based on the image signal captured by the second image sensor 16.

  The focus detection calculation unit 19g detects the focus adjustment state of the photographing lens 8, here the defocus amount, based on the focus detection signal corresponding to the pair of optical images output from the distance measuring element 10. Although details will be described later, a plurality of focus detection areas are set in the shooting screen of the shooting lens 8, and the distance measuring element 10 outputs a focus detection signal corresponding to a pair of optical images for each focus detection area. Then, the focus detection calculation unit 19g detects the defocus amount based on the focus detection signal corresponding to the pair of optical images for each focus detection area. The lens driving amount calculation unit 19h converts the detected defocus amount into a lens driving amount.

  FIG. 3 is a front view showing a detailed configuration of the second image sensor 16. The second imaging element 16 includes a plurality of pixels (photoelectric conversion elements) 26 (here, horizontal 16 × vertical 12 = 192) arranged in a matrix. As shown in FIG. 4, each pixel 26 is divided into three portions 26a, 26b, and 26c, and primary color filters of red R, green G, and blue B are provided in these portions 26a, 26b, and 26c, respectively. . Thereby, the RGB signal of the subject image can be output for each pixel 26.

  Next, a subject tracking operation according to an embodiment will be described. 5 to 9 are diagrams for explaining the subject tracking method according to the embodiment, and FIGS. 10 to 12 are flowcharts illustrating the subject tracking processing according to the embodiment. The body drive control device 19 selects the tracking target area manually from the subject image captured by the second imaging element 16 or automatically sets the tracking target area after the camera automatically sets the tracking target area. When the shutter button is pressed halfway, the subject tracking process is started.

  The quick return mirror 5 is set in the photographing optical path shown by the broken line in FIG. 1 except when the shutter button is fully pressed to shoot, and the subject light incident from the photographing lens 8 is placed on the focusing screen 11. Imaged. The subject image on the focusing screen 11 is guided to the second image sensor 16 via the penta roof prism 13, the prism 17 and the imaging lens 18, and the subject image signal is repeatedly output from the second image sensor 16.

  A plurality of focus detection areas are set on the shooting screen of the shooting lens 8, and an area mark is superimposed on the subject image on the focusing screen 11 by the liquid crystal display element 12 to display the position of each focus detection area. In this embodiment, as shown in FIG. 5, an example is shown in which focus detection areas 45a to 45k are set at 11 locations in the shooting screen. When an arbitrary area is selected by the focus detection area selection switch of the operation member 20, the mark of the area is lit up.

  In step 1 of FIG. 10, when the focus detection area 45g is selected by the focus detection area selection switch of the operation member 20 as shown in FIG. 6A, and the shutter button of the operation member 20 is half-pressed in this state, The focus detection area 45g is stored in the memory 19d as the first AF area, and a tracking symmetrical subject is designated. Here, an example is shown in which the photographer manually specifies the first AF area and the subject to be tracked. However, for example, in a camera having a function of automatically recognizing a subject, the first AF area and A tracking target subject may be set.

  In the subsequent step 2, the tracking initial image (the image acquired first after starting the image tracking process) is acquired by the second imaging element 16, and the pair of corresponding focus detection areas 45a to 45k is acquired by the distance measuring element 10. Obtain a light image. In step 3, the defocus amount (focus adjustment state of the photographing lens 8) of each focus detection area is detected based on the pair of optical images corresponding to the focus detection areas 45 a to 45 k acquired by the distance measuring element 10. In step 4, the defocus amount detected for the selected focus detection area 45 g is converted into a lens drive amount, and the lens drive amount is transmitted to the lens drive control device 22. The lens drive control device 22 drives the focusing lens 8b of the photographing lens 8 according to the lens drive amount, and performs focus adjustment.

  In step 5, the tracking control initial processing subroutine shown in FIG. 11 is executed. In step 101 of FIG. 11, the subject color information of the image in the area corresponding to the focus detection area 45g (first AF area) in the tracking initial image is stored. In step 102, as shown in FIG. 6A, the same color information area indicating the same color information as the subject color information is detected in the periphery of the focus detection area 45g in the tracking initial image. The information area is an initial tracking subject area 47. Here, an example in which the tracking subject area 47 is determined based on the subject color information is shown, but in order to simplify the processing, the size of the tracking subject area is uniformly set to 3 × 3 pixels, The size of the subject area may be determined according to the distance information of the photographing lens 8.

  In step 104, the image of the tracking subject area 47 in the tracking initial image is stored in the memory 19d as a template image 48 (see FIG. 6B) used for the subsequent tracking process, and in step 105, the tracking subject area 47 is moved back and forth. An area enlarged by predetermined pixels (here, 2 pixels) on the left and right is set as the search area 49 and stored in the memory 19d. Thereafter, the process returns to step 6 in FIG.

When the initial processing of the tracking control is completed, the process proceeds to step 6 in FIG. 10 to check whether or not the shutter button of the operation member 20 has been fully pressed, that is, whether or not the shutter release operation has been performed. If the shutter release operation is not performed, the process proceeds to step 7, where a tracking next image is acquired from the second image sensor 16 and a pair of optical images for focus detection is acquired for each focus detection area 45 a to 45 k by the distance measuring element 10. To do. In step 8, the defocus amount (the focus adjustment state of the photographing lens 8) of all the focus detection areas is detected based on the pair of optical images corresponding to the focus detection areas 45 a to 45 k acquired by the distance measuring element 19. Instead of detecting the defocus amounts of all the focus detection areas, the focus detection areas around the first AF area (here 45 g) (here 45 b, 45 c, 45 f, 45 h 45 j, 45 k around the first AF area 45 g). ) May be detected.

  In step 9, a focus area determination processing subroutine shown in FIG. 12 is executed to determine a focus detection area for focus adjustment. In this embodiment, the dynamic AF mode is selected as the AF mode by the shooting mode selection switch of the operation member 20, and the focus detection area (initial AF area) selected first is given priority over the image tracking calculation result. And The dynamic AF mode is an AF mode that preferentially selects the first AF area.

In step 301 of FIG. 12, it is determined whether or not the focus area can be adopted as a focus adjustment area based on the defocus amount detected in the first AF area (here, 45 g). Specifically, the current defocus amount (hereinafter referred to as the defocus amount history predicted based on the history of the defocus amount used for focus adjustment in the past, the drive amount and position history of the focusing lens 8b at the time of focus adjustment, and the like). The “predicted defocus amount”) is compared with the defocus amount of the first AF area (45 g here) detected in step 8 of FIG. 10 this time, and the difference D (= (detected defocus amount) − (predicted). It is determined whether or not the defocus amount)) is within the adoption determination threshold value. Here, the adoption determination threshold value in step 301 is appropriately set according to the distance between the camera and the tracking target subject, the movement speed of the tracking target subject, and the like, and is stored in advance in the memory 19d.

  In addition, by continuing tracking, it is possible to predict the amount of movement of the tracking target subject based on the history of defocus amount, lens drive amount, lens position, etc. during past focus adjustment. The amount of defocus in the near future can be predicted.

  Whether to predict the defocus amount based on the history of defocus amount, lens drive amount, lens position, etc. during past focus adjustment, and whether to adopt the first AF area as the in-focus area based on this predicted defocus amount Instead of determining, the defocus amount detected this time in the first AF area may be compared with a predetermined threshold value to determine whether or not the first AF area is adopted as the in-focus area. That is, when the defocus amount detected this time in the first AF area is less than or equal to a predetermined threshold value, the first AF area is adopted as the in-focus area, and the defocus amount detected this time in the first AF area is larger than the predetermined threshold value. If it is larger, the first AF area is not adopted as the focusing area. In particular, immediately after the tracking process is started, since there is no past defocus amount, lens drive amount, and lens position history, an in-focus area is determined by this determination method.

  If the difference D is within the threshold, the current predicted position of the target subject that has been tracked up to the previous time is substantially equal to the position of the subject detected this time in the first AF area, and the first AF area is in focus. Adopt as an area. There are cases where the target subject that has been tracked up to the previous time and the subject detected this time in the first AF area are the same subject or different subjects. In the former case, as shown in FIG. 6A, the target subject that has been tracked up to the previous time is captured in the first AF area and is continuously tracked. On the other hand, as shown in FIG. 8A, the latter case is different from the subject A that has been tracked until the previous time. However, as a result of the photographer changing the composition of the camera 1 or the subject moving, Another subject B at the same position is captured and tracked in the first AF area.

If the difference D is within the threshold value, the process proceeds to step 302, and the initial AF area is determined as the focus area. In this case, since the determination is made based only on the defocus amount of the first AF area (here, 45 g) without using the defocus amount of other focus detection areas or the image tracking calculation result, the processing required for image tracking is simple. become. Thereafter, the process returns to step 10 in FIG. 10, and the defocus amount detected this time in the first AF area is converted into a lens drive amount and transmitted to the lens drive control device 22. The lens drive control device 22 controls focusing of the focusing lens 8b according to the lens drive amount.

  As described above, in one embodiment, the current defocus amount in the first AF area (here, 45 g), and the current time predicted based on the history of the defocus amount, lens drive amount, lens position, etc. during past focus adjustment. If the difference D from the defocus amount is within the adoption determination threshold value, the first AF area is adopted as the in-focus area regardless of whether the subject is the same in the previous time and the current time. Image tracking can be continued by automatically switching the subject to be tracked without redoing tracking control from the setting operation.

  On the other hand, if the difference D between the current predicted defocus amount and the defocus amount detected this time in the first AF area is larger than the adoption determination threshold, tracking is performed up to the previous time as shown in FIGS. This is a case where the target subject has moved to a position other than the first AF area. In the example shown in FIG. 7, it moves to the position corresponding to the focus detection areas 45b, 45f, 45j, and in the example shown in FIG. 9, it moves to the position corresponding to the focus detection areas 45f, 45j.

When the difference D is larger than the adoption determination threshold value, the process proceeds to step 303, and the tracking calculation processing subroutine shown in FIG. 13 is executed. In step 201 of FIG. 13, the following tracking image (FIG. 6 (
a) in the same manner as the template image 48 (see FIG. 6B) from the search area 49 in FIG. 6B), the cut-out image and the template image 48 are compared, and the hue B / G for each pixel. , R / G difference is calculated. When obtaining the difference between the hues B / G and R / G for each pixel, the calculation can be performed using the raw image signal output from the second image sensor 16, so that the white balance or filter for the raw image signal can be obtained. Preprocessing such as processing is not required, and the tracking processing can be simplified. Note that a color difference for each pixel may be calculated instead of the hue difference.

  When the difference calculation with the template image 48 is completed in the search area 49 of the tracking next image, the process proceeds to step 202, where an area having the smallest hue difference is searched, and that area is determined as a new tracking subject area 47. Here, the image information of the template image 48 may be updated using the image information of the new tracking subject region 47. In that case, for example, by adding 20% of the image information of the new tracking subject region 47 to 80% of the image information of the previous template image 48 to generate a new template image 48, the image information of the template image 48 is gradually changed. It is updated and it becomes easy to follow the change of the tracking subject.

  In subsequent step 203, an area enlarged by predetermined pixels (here, 2 pixels) on the front, rear, left and right sides of the new tracking subject area 47 is set as a new search area 49. Thereafter, the process returns to step 304 in FIG.

In step 304 of FIG. 12, the defocus amount detected this time in the focus detection area (45f in the example shown in FIG. 7) corresponding to the new tracking subject region 47, the defocus amount at the time of past focus adjustment, and the lens drive amount. The current predicted defocus amount predicted based on the history such as the lens position is compared, and it is determined whether or not the difference D ′ is within the adoption determination threshold value. That is, it is determined whether or not the focus detection area at the image tracking position is adopted as the focusing area. If the difference D ′ is within the adoption determination threshold value, the process proceeds to step 305, and the focus detection area corresponding to the new tracking subject region 47, that is, the focus detection area at the image tracking position (45f in the example shown in FIG. 7) is focused. The area is determined, and the process returns to step 10 in FIG. Note that the adoption determination threshold value in step 304 may be the same value as the adoption determination threshold value in step 301.

  In step 10 in FIG. 10, the defocus amount detected this time in the focus detection area at the image tracking position is converted into a lens drive amount and transmitted to the lens drive control device 22. The lens drive control device 22 controls focusing of the focusing lens 8b according to the lens drive amount.

  On the other hand, if the difference D ′ between the defocus amount detected this time in the focus detection area of the new tracking subject region 47 and the “predicted defocus amount” is larger than the adoption determination threshold value, the process proceeds to step 306. In step 306, the difference D ”between the defocus amount detected in the focus detection area around this time and the above“ predicted defocus amount ”from the focus detection areas around the first AF area (here 45 g) is the smallest. The focus detection area (for example, 45h shown in FIG. 9) is detected, and the defocus amount detected this time in the focus detection area with the smallest difference D ″ is compared with the predicted defocus amount. It is determined whether it is within the adoption determination threshold value.

  When the difference D ″ is within the adoption determination threshold value, the process proceeds to step 307, and the focus detection area (for example, 45h shown in FIG. 9) is determined as the focus area. The defocus amount detected this time in the focus detection area determined as the focus area around the AF area is converted into a lens drive amount and transmitted to the lens drive control device 22. The lens drive control device 22 is the lens drive amount. Accordingly, focusing control of the focusing lens 8b is performed.

  On the other hand, if the difference D ″ is larger than the adoption determination threshold value, the process proceeds to step 308 to determine that there is no focus area. Note that the adoption determination threshold value in step 306 is the same value as the adoption determination threshold value in step 301. In addition, the focus detection area to be adopted in step 306 may be all focus detection areas other than the initial AF area (here 45 g), and in step 304, focus detection of the image tracking position is performed. If it is determined that the area is not adopted as the focused area, the focused area determination process may be terminated without performing the focused area adoption determination in step 306 without assuming the focused area.

  In step 10 of FIG. 10 after the focus area determination process is completed, focus control is performed as described above. When it is determined that there is no in-focus area, lens driving is not performed. In step 11, it is determined whether or not the determined focus area matches the first AF area (here 45 g). If the focus area does not match the first AF area, the process returns to step 6 to repeat the above-described processing. .

  If the in-focus area matches the first AF area, the process proceeds to step 12, and the template image 48 is reacquired. In other words, the focus control is performed according to the defocus amount of the first AF area (here, 45 g) (step 10 in FIG. 10), and the image captured by the second image sensor 16 is used. The subject color information of the image in the area corresponding to is stored. Further, a same color information area showing color information similar to the subject color information is detected in the periphery of the first AF area in this image, and the same color information area is set as a tracking subject area 47. Then, the image of the tracking subject area 47 in the image is stored again in the memory 19d as the template image 48 used for the subsequent tracking processing, and the template image 48 is changed. Accordingly, in the subsequent image tracking processing, image tracking is performed using the new template image 48. In the example shown in FIG. 8, the subject B is tracked instead of the target subject A that has been tracked until the previous time.

As described above, according to the embodiment, the second image sensor 16 obtains the subject image in the photographing screen by the photographing lens 8, and the image at the tracking target position in the subject image is the template image (reference image). ) And is repeatedly detected by the second image sensor 16 based on the subject image and the template image (image tracking function). Further, the defocus amount (focus adjustment state) of the photographing lens 8 is detected in the focus detection area set in the photographing screen (AF function), and at least one of the detection result by the image tracking function and the detection result by the AF function is detected. Based on this, a focus detection area (focusing area) for adjusting the focus of the photographing lens 8 is determined, and the focus of the photographing lens 8 is adjusted according to the defocus amount of the determined focusing area. When the in-focus area is the preset initial AF area, the image of the initial AF area in the subject image is used as a new template image, and the template image in the memory 19d is changed.
This makes it possible to keep track of a player who always holds the ball in sports shooting such as soccer, for example, without specifying the player to be tracked every time the player holding the ball changes. It is possible to acquire an image of the selected player as a template image, and to keep track of the player who always holds the ball by changing the template image. Needless to say, the present invention is not limited to various sports shooting, and in a shooting scene where it is necessary to switch the tracking target during tracking shooting, it is possible to automatically switch to the intended target smoothly without performing an operation of switching the tracking target.

As an in-focus area determination method, in one embodiment, a defocus amount is predicted based on past focus adjustment information, that is, a defocus amount, a lens drive amount, a lens position, etc. When the difference between the defocus amount and the detected defocus amount is within the threshold value, the initial AF area is determined as the in-focus area.
This makes it possible to easily capture the player holding the ball and the players around the player when it is desired to keep following the player who always holds the ball in sports photography such as soccer. Even for players, the tracking target can be switched smoothly and accurately to the player holding the new ball.

As another method for determining the focus area, in one embodiment, when the defocus amount detected in the first AF area is within a threshold value, the first AF area is determined as the focus area. .
If the newly detected defocus amount in the first AF area is a small value within the threshold value, there is a high possibility that the tracking target exists in the first AF area, and the tracking target is subsequently captured and tracked in the first AF area. be able to. In addition, this method for determining the focus area does not require a defocus amount prediction process based on information related to past focus adjustments, and thus is effective as a method for determining the focus area immediately after the start of image tracking.

According to one embodiment, when the initial AF area is not the focus area, the focus detection area is adjusted based on the defocus amount of the focus detection area corresponding to the target position detected by the image tracking function. Whether to adjust the focus of the photographic lens 8 as a focal area is determined.
Thus, for example, when shooting a player who always holds the ball in sports shooting such as soccer, the player holding the ball and the surrounding players move fast, and the photographer moves those players from the first AF area. Even when it is removed, those players can be continuously captured and tracked by the image tracking function.

Furthermore, according to one embodiment, when the focus detection area corresponding to the target position detected by the first AF area and the image tracking function is not the focus area, the focus around the first AF area excluding these areas. The in-focus area is determined based on the defocus amount detected in the detection area.
As a result, when a player who always holds the ball in sport shooting such as soccer, for example, wants to keep following the player, the player holding the ball and the surrounding players move fast, and the photographer moves the player from the image tracking position. You can continue to capture and track those players even when you have removed them.

  According to one embodiment, the template image is changed using the subject image acquired by the second image sensor 16 after the focus adjustment of the photographing lens 8 is performed according to the defocus amount of the first AF area. As a result, a clear image of the new tracking target can be used as the template image when the tracking target is switched, and the tracking of the new target thereafter can be reliably performed.

  In the above-described embodiment, a single-lens reflex digital still camera has been described as an example. However, the present invention can be applied to all imaging devices capable of acquiring images in time series. For example, a consumer digital camera or a video It can also be applied to cameras.

In the focusing area determination process shown in FIG. 12 described above, if the initial AF area is not adopted as the focusing area in step 301, the tracking calculation process is immediately performed in step 303 to determine the focusing area based on the image tracking result. Although an example of starting the process has been shown, if it is determined in step 301 that the first AF area is not adopted as the focus area, the focus area determination process in step 306 may be performed. That is, when the first AF area is not adopted as the focusing area, the defocus amount detected this time in the focus detection area around the first AF area from the focus detection areas around the first AF area, Search for the focus detection area where the difference between the defocus amount used for focus adjustment and the predicted defocus amount predicted based on the lens position and drive amount history during focus adjustment is the smallest, and the focus detection area Determine the focus area.
There is a high possibility that the target subject that has been tracked up to the previous time or another subject that is different from the target subject but in the vicinity thereof is captured in the first AF area or the surrounding focus detection area.

  In addition, when the first AF area is not adopted as the in-focus area, the difference between the defocus amount detected this time in all focus detection areas and the above “predicted defocus amount” is the smallest among all focus detection areas. The focus detection area may be searched and the focus detection area may be determined as the focus area.

DESCRIPTION OF SYMBOLS 1 Single-lens reflex digital still camera 2 Camera body 3 Interchangeable lens 8 Shooting lens 9 Focus detection optical system 10 Distance measuring element 16 2nd image pick-up element 19 Body drive control apparatus 19a Element control circuit 19b A / D converter 19c Microcomputer 19d Memory 20 Operation member 22 Lens drive control device

Claims (7)

An imaging unit that repeatedly acquires image information in the screen by the optical system;
A focus detection unit that detects a plurality of defocus amounts for a plurality of focus detection areas;
A focus adjustment control unit that performs focus adjustment control using the defocus amount detected by the focus detection unit;
A determination unit that determines a determination focus detection area used for focus adjustment control by the focus adjustment control unit from the plurality of focus detection areas ;
A calculation unit for calculating a predicted defocus amount based on information on past focus adjustment control by the focus adjustment control unit;
A control unit that controls to store the reference information including the tracking target information in the storage unit;
A tracking calculation unit that calculates the position of the tracking target using the reference information and the image information repeatedly acquired by the imaging unit;
It is determined whether the difference between the defocus amount of the first focus detection area corresponding to the position of the focus detection area designated when the half-press operation is performed and the predicted defocus amount is smaller than a first threshold value. A first determination unit that
When the difference is not determined to be smaller than the first threshold by the first determination unit, the defocus amount of the second focus detection area corresponding to the position of the tracking target calculated by the tracking calculation unit and the predicted data A second determination unit that determines whether or not the difference from the focus amount is smaller than a second threshold,
The determining unit determines the first focus detection area as the determined focus detection area when the first determination unit determines that the difference is smaller than the first threshold;
When the second determination unit determines that the difference is smaller than the second threshold, the second focus detection area is determined as the determined focus detection area;
The control unit is acquired after the first determination unit determines that the difference is smaller than the first threshold when the first determination unit determines that the difference is smaller than the first threshold. The image information of the area corresponding to the position of the first focus detection area is replaced with the reference information stored in the storage unit,
When the difference is not determined to be smaller than the first threshold by the first determination unit, the reference information stored in the storage unit is image information of a region corresponding to the position of the second focus detection area; An image tracking apparatus that replaces the reference information generated using the reference information stored in the storage unit . An image tracking device according to claim 1,
When the second determination unit determines that the difference between the defocus amount of the second focus detection area and the predicted defocus amount is greater than a second threshold, the determination unit selects the determination focus detection area. In addition, the focus adjustment control unit does not perform focus adjustment control . An image tracking device according to claim 1 ,
When the second determination unit determines that the difference between the defocus amount of the second focus detection area and the predicted defocus amount is greater than a second threshold value,
A third determination unit that determines whether or not a difference between a defocus amount of a third focus detection area located around the first focus detection area and the predicted defocus amount is smaller than a third threshold;
When the third determination unit determines that the difference between the defocus amount of the third focus detection area and the predicted defocus amount is smaller than a third threshold value, the determination unit determines the third focus detection area as the third focus detection area. An image tracking device characterized in that it is determined as a determined focus detection area . An image tracking device according to claim 3 ,
When the third determination unit determines that the difference between the defocus amount of the third focus detection area and the predicted defocus amount is larger than a third threshold, the determination unit determines the determination focus detection area. In addition, the focus adjustment control unit does not perform focus adjustment control . An image tracking device according to any one of claims 1 to 4,
  The first focus corresponding to the position of the determined focus detection area specified when the half-press operation is performed, rather than the second focus detection area corresponding to the position of the tracking target calculated by the tracking calculation unit. An image tracking apparatus having a photographing mode for preferentially determining a detection area. An imaging unit that repeatedly acquires image information in the screen by the optical system;
  A focus detection unit that detects a plurality of defocus amounts for a plurality of focus detection areas;
  A focus adjustment control unit that performs focus adjustment control using the defocus amount detected by the focus detection unit;
  A determination unit that determines a determination focus detection area used for focus adjustment control by the focus adjustment control unit from the plurality of focus detection areas;
  A calculation unit for calculating a predicted defocus amount based on information on past focus adjustment control by the focus adjustment control unit;
  A control unit that controls to store the reference information including the tracking target information in the storage unit;
  A tracking calculation unit that calculates the position of the tracking target using the reference information and the image information repeatedly acquired by the imaging unit;
  It is determined whether the difference between the defocus amount of the first focus detection area corresponding to the position of the focus detection area designated when the half-press operation is performed and the predicted defocus amount is smaller than a first threshold value. A first determination unit that
  When the difference is not determined to be smaller than the first threshold by the first determination unit, the defocus amount of the second focus detection area corresponding to the position of the tracking target calculated by the tracking calculation unit and the predicted data A second determination unit that determines whether or not the difference from the focus amount is smaller than a second threshold,
  The determining unit determines the first focus detection area as the determined focus detection area when the first determination unit determines that the difference is smaller than the first threshold;
  The image tracking device according to claim 1, wherein the second focus detection area is determined as the determined focus detection area when the second determination unit determines that the difference is smaller than the second threshold. An imaging unit that repeatedly acquires image information in the screen by the optical system;
  A focus detection unit that detects a plurality of defocus amounts for a plurality of focus detection areas;
  A focus adjustment control unit that performs focus adjustment control using the defocus amount detected by the focus detection unit;
  A calculation unit for calculating a predicted defocus amount based on information on past focus adjustment control by the focus adjustment control unit;
  A control unit that controls to store the reference information including the tracking target information in the storage unit;
  A tracking calculation unit that calculates the position of the tracking target using the reference information and the image information repeatedly acquired by the imaging unit;
  It is determined whether the difference between the defocus amount of the first focus detection area corresponding to the position of the focus detection area designated when the half-press operation is performed and the predicted defocus amount is smaller than a first threshold value. A first determination unit that
  The control unit is acquired after the first determination unit determines that the difference is smaller than the first threshold when the first determination unit determines that the difference is smaller than the first threshold. The image information of the area corresponding to the position of the first focus detection area is replaced with the reference information stored in the storage unit,
  When the first determination unit does not determine that the difference is smaller than the first threshold value, the reference information stored in the storage unit corresponds to the tracking target position calculated by the tracking calculation unit. An image tracking device that replaces with reference information generated using image information of a region corresponding to a position of a bifocal detection area and the reference information stored in the storage unit.