JP5167750B2 - TRACKING DEVICE, IMAGING DEVICE, AND TRACKING METHOD - Google Patents

Description

  The present invention relates to a tracking device, an imaging device, and a tracking method.

  In the captured image, an image of the area specified by the photographer is used as a template image, and an image that matches the template image is searched by a pattern matching method from images repeatedly captured thereafter, and the area that matches the template image Is known as a main subject region intended by the photographer, and a tracking device that tracks the main subject is known (see Patent Document 1).

JP 2006-058431 A

  However, since the above-described conventional tracking device performs subject tracking by pattern matching using a template image, for example, if another object crosses in front of the subject to be tracked, the subject to be tracked May be lost, or another subject may be mistaken for tracking.

The invention according to claim 1 is a tracking device that tracks an object in the screen by comparing an image sensor that acquires image information in the screen by an optical system with image information in the screen and target image information. Based on the tracking means, the focus detection means for detecting the focus adjustment state of the optical system related to at least one focus detection area set in the screen, and the focus adjustment state in the focus detection area corresponding to the position in the target screen A focus adjustment unit that performs focus adjustment of the optical system, and a previous focus adjustment state and a current focus adjustment state in the focus detection region corresponding to the target position based on the focus adjustment state detected by the focus detection unit. Control for prohibiting focus adjustment control by the focus adjustment means when it is determined by the first determination means that the difference between the previous focus adjustment state and the current focus adjustment state is large. means , Based on a result of verification by the tracking means, a second determination unit that determines whether or not track the target, the control means is determined not to be tracked target by the second determining means, and first determination When the means determines that the difference between the previous focus adjustment state and the current focus adjustment state is large, the focus adjustment control by the focus adjustment means is prohibited .
The invention according to claim 7 is a tracking method for acquiring image information in a screen by an optical system, tracking image information in the screen and target image information to track a target in the screen, Detecting the focus adjustment state of the optical system with respect to at least one focus detection area set to, and performing the focus adjustment of the optical system based on the focus adjustment state in the focus detection area corresponding to the position in the target screen, Based on the detected focus adjustment state, a first determination is made to determine a difference between the previous focus adjustment state and the current focus adjustment state in the focus detection region corresponding to the target position. When it is determined that the difference between the adjustment state and the current focus adjustment state is large, the focus adjustment is prohibited, and a second determination is made as to whether or not the object has been tracked based on the result of the verification by tracking, It is determined that the target cannot be tracked by 2 determination Is, and if it is determined that the large difference between the focus adjustment state of the current and the focus adjustment state of the last in the first determination, and inhibits the focus adjustment control.

  According to the present invention, it is possible to improve the tracking performance for a subject to be tracked.

  FIG. 1 shows a configuration of an image pickup apparatus (single-lens reflex digital still camera) including a subject tracking device according to an embodiment. An interchangeable lens 11 is attached to the camera body 10 in a replaceable manner. The camera body 10 is provided with a first image sensor 12 for capturing a subject image and recording the image. The first image sensor 12 can be composed of a CCD, a CMOS, or the like. At the time of shooting, the quick return mirror 13 is retracted to a position outside the shooting optical path indicated by a solid line, the shutter 14 is opened, and a subject image is formed on the light receiving surface of the first image sensor 12 by the shooting lens 15.

  At the bottom of the camera body 10, a focus detection optical system 16 and a distance measuring element 17 for detecting the focus adjustment state of the photographing lens 15 are provided. In this embodiment, an example in which a focus detection method based on a pupil division phase difference detection method is adopted is shown. The focus detection optical system 16 guides the pair of focus detection light beams that have passed through the photographing lens 15 to the light receiving surface of the distance measuring element 17 and forms a pair of optical images. The distance measuring element 17 includes a pair of CCD line sensors, for example, and outputs a focus detection signal corresponding to the pair of optical images. Prior to photographing, the quick return mirror 13 is set at a position in the photographing optical path as indicated by a broken line, and the focus detection light beam from the photographing lens 15 passes through the half mirror portion of the quick return mirror 13 and is subtracted by the sub mirror 18. The light is reflected and guided to the focus detection optical system 16 and the distance measuring element 17.

  A finder optical system is provided on the upper part of the camera body 10. Prior to shooting, the quick return mirror 13 is in a position indicated by a broken line, and subject light from the shooting lens 15 is guided to the focusing screen 20, and a subject image is formed on the focusing screen 20. The liquid crystal display element 21 superimposes and displays various information such as a focus detection position and an exposure value on the subject image formed on the focusing screen 20. The subject image on the focusing screen 20 is guided to the eyepiece window 24 via the penta roof prism 22 and the eyepiece lens 23 so that the photographer can visually recognize the subject image.

  The finder optical system at the top of the camera body is provided with a second image sensor 25 that captures a subject image for subject tracking and photometry. Details of the second image sensor 25 will be described later. The subject image formed on the focusing screen 20 is re-imaged on the light receiving surface of the second image sensor 25 via the penta roof prism 22, the prism 26 and the imaging lens 27. The second image sensor 25 outputs an image signal corresponding to the subject image.

  The camera body 10 is also provided with an operation member 28, a control device 29, and a lens driving device 30. The operation member 28 includes a switch and a selector for operating the camera such as a shutter button and a focus detection area selection switch. The control device 29 is composed of a CPU and its peripheral components, and performs various controls and calculations of the camera. The lens driving device 30 includes a motor and a driving circuit, and adjusts the focus of the photographing lens 15.

  FIG. 2 shows a detailed configuration of the control device 29. Note that illustration and description of control functions not directly related to the present invention are omitted. The control device 29 includes various control units configured by CPU software. The CCD control unit 31 controls charge accumulation and readout of the second image sensor 25. The A / D converter 32 converts the analog image signal output from the second image sensor 25 into a digital image signal. The exposure calculation unit 33 calculates an exposure value based on the image signal captured by the second image sensor 25. The timer 38 has a clock function and is used for measuring elapsed time.

  The focus detection calculation unit 34 detects the focus adjustment state of the taking lens 15, here, the defocus amount, based on the focus detection signal corresponding to the pair of optical images output from the distance measuring element 17. Although details will be described later, a plurality of focus detection areas are set in the shooting screen of the shooting lens 15, and the distance measuring element 17 outputs a focus detection signal corresponding to a pair of optical images for each focus detection area. The focus detection calculation unit 34 detects the defocus amount based on the focus detection signal corresponding to the pair of light images for each focus detection area. The lens driving amount calculation unit 35 converts the detected defocus amount into a lens driving amount. The lens driving device 30 drives a focusing lens (not shown) of the photographic lens 15 according to the lens driving amount to adjust the focus.

  The tracking control unit 36 stores, as a template image, an image of the tracking target area manually specified by the photographer or the tracking target area automatically set by the camera among the subject images captured by the second image sensor 25 as a template image. At the same time, the focus detection calculation unit 34 is controlled to detect the defocus amount of the photographic lens 15 in the tracking target area. Then, an image area that matches the template image is searched from images that are repeatedly picked up thereafter, and the defocus amount of the photographic lens 15 at a position corresponding to the image area that matches the template image is detected, and the previous tracking is performed. If there is no significant change in the defocus amount between the target area and the current image area, the current image area is set as a new tracking target area. Such a procedure is repeated to track a specific subject. The storage unit 37 includes information such as the template image and the defocus amount during the tracking operation by the tracking control unit 36, or the focal length, open F value, aperture value, image shift amount / defocus amount conversion coefficient of the photographing lens 15, and the like. Lens information is stored.

  FIG. 3 is a front view showing a detailed configuration of the second image sensor 25. The second imaging element 25 includes a plurality of pixels (photoelectric conversion elements) 40 (here, horizontal 16 × vertical 12 = 192) arranged in a matrix. As shown in FIG. 4, each pixel 40 is divided into three portions 40a, 40b, and 40c, and primary color filters of red R, green G, and blue B are provided in these portions 40a, 40b, and 40c, respectively. . Thereby, the RGB signal of the subject image can be output for each pixel 40.

  Next, a subject tracking operation according to an embodiment will be described. 5 to 6 are diagrams for explaining a subject tracking method according to the embodiment, and FIGS. 7 to 10 are flowcharts illustrating subject tracking processing according to the embodiment. The control device 29 is configured such that the photographer manually designates the tracking target area in the subject image captured by the second image sensor 25 or the camera automatically sets the tracking target area, and then the shutter button of the operation member 28. When is pressed halfway, the subject tracking process is started.

  The quick return mirror 13 is set in the photographing optical path shown by the broken line in FIG. 1 except when the shutter button is fully pressed to shoot, and the subject light incident from the photographing lens 15 is placed on the focusing screen 20. Imaged. Then, the subject image on the focusing screen 20 is guided to the second image sensor 25 via the penta roof prism 22, the prism 26 and the imaging lens 27, and the subject image is repeatedly output from the second image sensor 25.

  A plurality of focus detection areas are set on the shooting screen of the shooting lens 15, and an area mark is superimposed on the subject image on the focusing screen 20 by the liquid crystal display element 21 to display the position of each focus detection area. In this embodiment, as shown in FIG. 5, an example is shown in which focus detection areas 45a to 45k are set at 11 locations in the shooting screen. Further, when an arbitrary area is selected by the focus detection area selection switch of the operation member 28, the mark of the area is lit up.

  By selecting the focus detection area 45f corresponding to the subject image by the focus detection area selection switch of the operation member 28 and pressing the shutter button of the operation member halfway in this state, it is possible to designate a subject that is symmetrical to the tracking. In response to this half-press operation, the control device 29 starts subject tracking processing.

  In Step 1 of FIG. 7, an initial tracking image (an image acquired first after starting the subject tracking process) is acquired by the second image sensor 25, and a selected focus detection area (here, 45 f) is acquired by the distance measuring element 17. A pair of optical images corresponding to is acquired. In the subsequent step 2, the defocus amount (focus adjustment state of the photographing lens 15) of the focus detection area 45f is detected based on a pair of optical images corresponding to the focus detection area 45f acquired by the distance measuring element 17. In step 3, the defocus amount detected for the selected focus detection area 45f is converted into a lens drive amount, and the photographing lens 15 is driven by the lens drive device 30 to perform focus adjustment.

  In step 4, an initial process of tracking control shown in FIG. 8 is executed. In step 101 of FIG. 8, the subject color information of the image in the area corresponding to the focus detection area 45f in the tracking initial image is stored. In step 102, as shown in FIG. 5A, the same color information area indicating the color information similar to the subject color information is detected in the periphery of the focus detection area 45f in the tracking initial image. The information area is an initial tracking subject area 47. In step 104, the image of the tracking subject region 47 in the tracking initial image is stored in the storage unit 37 as a template image 48 (see FIG. 5B) used for the subsequent tracking processing. In step 105, the tracking subject region 47 is stored. A search area 49 is an area enlarged by predetermined pixels (here, 2 pixels) in the front, rear, left, and right directions.

  When the initial processing of the tracking control is completed, the process proceeds to step 5 in FIG. 7 to check whether or not the shutter button of the operation member 28 has been fully pressed, that is, whether or not the shutter release operation has been performed. If the shutter release operation is not performed, the process proceeds to step 6 to acquire a tracking next image (see FIG. 6A) from the second image sensor 25, and focus detection is performed for each focus detection area 45a to 45k by the distance measuring element 17. Acquire a pair of light images. Then, based on the pair of light images, the focus detection calculation unit 34 detects the defocus amount (the focus adjustment state of the photographing lens 15) for each of the focus detection areas 45a to 45k.

  In step 7, the tracking calculation process shown in FIG. 9 is performed. In step 201 in FIG. 9, an area having the same size as the template image 48 is sequentially cut out from the search area 49 in the tracking next image (see FIG. 6A), the cut-out image and the template image 48 are compared, and the pixel Every time, the difference between hues B / G and R / G is calculated. Then, the hue difference for each pixel is summed for each hue, and the summed hue difference for each hue is summed for each cut image. When obtaining the difference between the hues B / G and R / G for each pixel, the calculation can be performed using the raw image signal output from the second image sensor 25. Therefore, a white balance or filter for the raw image signal can be obtained. Preprocessing such as processing is not required, and the tracking processing can be simplified. Note that a color difference for each pixel may be calculated instead of the hue difference. As described above, since the hue difference or the color difference is used in the process of FIG.

  When the difference calculation with the template image 48 is completed in the search area 49 for the tracking next image, the process proceeds to step 202, where the area where the total value of the hue differences is the smallest (hereinafter referred to as the minimum hue difference) is searched. It is determined whether or not the hue difference is equal to or less than a preset threshold value. This threshold value is a reference value for determining whether or not the subject in the minimum hue difference area is the tracking target subject. If the minimum hue difference is less than or equal to the threshold value, the tracking initial image is displayed in the search area 49. It can be predicted that the specified tracking subject exists.

  An appropriate value for this threshold is determined by various experiments and simulations. The threshold value may be a fixed value or a dynamically changing value. For example, the threshold value is changed according to the amount of movement of the tracking subject. The value may be changed depending on the shooting mode. For example, the threshold value is increased in a mode in which a sports scene in which the moving amount of the tracking subject is large is shot. Further, the template image 48 may be changed based on the pattern, that is, the RGB value. Further, it may be changed according to the history of the minimum hue difference.

  Note that a small hue difference between the cut-out image and the template image 48 means that the ratio of the cut-out image and the template image 48 is large. On the other hand, a large hue difference means that the proportion of the cut-out image and the template image 48 is small, in other words, the proportion of mismatch is large. In this way, it is determined whether or not the tracking subject has been tracked based on the ratio at which the cut-out image and the template image 48 match.

  When the minimum hue difference is equal to or smaller than the threshold value, the process proceeds to step 203, and the area of the minimum hue difference in the next tracking image is determined as a new tracking subject area 47 as shown in FIG. In the subsequent step 204, the template image 48 is updated. In this embodiment, a new template image 48 is generated by adding 20% of the image information of the new tracking subject region 47 to, for example, 80% of the image information such as color information and luminance information of the original template image 48. When the tracking subject is lost, the original template image 48 may be used as it is, or the image of the new tracking subject region 47 may be used as the template image 48. In step 206, an area obtained by enlarging the new tracking subject area 47 by a predetermined number of pixels (here, 2 pixels) in the front, rear, left, and right directions is set as a new search area 49.

  On the other hand, when the minimum hue value is larger than the threshold value in step 202, the process proceeds to step 205, where the tracking subject is hidden by another subject in the search area 49 as shown in FIG. It is determined that the subject has been lost, and the “subject disappearance flag” is set. In step 206, since the tracking subject is lost, the search area 49 may be an area enlarged by two pixels as described above, or may be further enlarged. For example, an area enlarged by predetermined pixels (here, 3 to 4 pixels) in front, rear, left, and right of the tracking subject area 47 is set as a new search area 49. By expanding the search area 49, there is a high possibility that the tracking subject can be captured again.

  The search area 49 may be enlarged in a similar shape as described above (proportional enlargement) (FIG. 11A), or the movement direction of the tracking subject is predicted from the tracking history or the like. You may make it expand only to a moving direction (direction of an arrow) (FIG.11 (b)). Further, the search area 49 may be shifted as it is in the moving direction (in the direction of the arrow) of the tracking subject predicted (FIG. 11C). Further, the search area 49 may be changed to the full screen (FIG. 11 (d)). That is, the search area 49 can be changed to a new search area 49 ′ by enlarging, shifting, changing to the full screen, or the like. FIGS. 11A to 11D are diagrams showing how the search area 49 is changed to a new search area 49 ′.

  When the tracking calculation process shown in FIG. 9 is completed, the process returns to step 8 in FIG. 7 to execute the focus adjustment control process shown in FIG. In step 301, it is confirmed whether or not the “subject disappearance flag” is set. If it is set, the timer 38 is activated and the routine proceeds to step 306. If the timer 38 has already been started, the process proceeds to step 306 as it is.

  If it is determined in step 301 that the tracking subject has not been lost, the process proceeds to step 303. If the timer 38 has been activated because the tracking subject has been lost in step 301 before, the timer 38 is stopped and reset. If the timer 38 is started in step 303 described later, assuming that the difference in defocus amount exceeds the threshold value, the measurement is continued as it is. Here, resetting the timer 38 and proceeding to step 303 means that it is determined that the tracking subject has been able to be tracked again after the focus adjustment is prohibited, and the prohibition of the focus adjustment due to the loss of the tracking subject is canceled. .

  In step 303, the defocus amount detected in the focus detection area at the previous position of the subject region 47 is compared with the defocus amount detected in the focus detection area at the current position of the subject region 47. It is determined whether or not it is below the set threshold value. This threshold value is a reference value for determining that the tracking subject in the previous subject region and the tracking subject in the current subject region are the same subject.

  If it is determined that the same tracking subject as the previous time is captured with the difference in the defocus amount being equal to or smaller than the threshold value, the process proceeds to step 304. In this case, if the timer 38 is activated, the timer 38 is stopped and reset. In step 304, the lens drive amount is calculated based on the current defocus amount in the subject region 47 detected in step 6 (FIG. 7), and the lens drive device 30 is controlled to perform focus adjustment control of the taking lens 15. On the other hand, if the difference in the defocus amount exceeds the threshold value and it is determined that the subject supplemented this time is not the same as the previous subject, the timer 38 is activated and the routine proceeds to step 306. If the timer 38 has already been started, the process proceeds to step 306 as it is.

  In step 306, it is determined whether a predetermined time has elapsed since the timer 38 was activated. If it is determined that the predetermined time has elapsed, the process proceeds to step 307, and if it is determined that the predetermined time has not yet elapsed, the process proceeds to step 305. In step 305, the focus adjustment control of the taking lens 15 is prohibited (AF lock), and the process returns to the process of FIG.

  On the other hand, if it is determined in step 306 that the timer 38 has elapsed, the process proceeds to step 307. In step 307, the timer 38 is reset, focus detection is performed using the focus detection area used in the previous focus adjustment control process, and the defocus amount is obtained. In step 308, the lens driving amount is calculated based on the defocus amount obtained in step 307, and the lens driving device 30 is controlled to perform focus adjustment control of the photographing lens 15. Thereafter, the processing returns to FIG.

  The predetermined time may be set in advance unique to the camera, or may be set by the photographer. For example, the setting may be made using the operation member 28 or may be made by a combination of the operation member 28 and a menu screen (displayed on a liquid crystal display device not shown). The predetermined time may be set by the photographer, for example, between 0.2 seconds and 1 second, or may be set unique to the camera between 0.5 seconds and 1 second.

  As described above, in this embodiment, when it is determined that the tracking subject has been lost, or the difference between the defocus amount for the previous subject and the defocus amount for the current subject exceeds a preset threshold value. When the determination is made, the focus adjustment control is prohibited (AF lock) for a predetermined time. When a predetermined time has elapsed since the focus adjustment control was prohibited (AF lock), the focus adjustment is resumed (the AF lock is released) using the focus detection area used in the previous focus adjustment control process. .

  As a result, the tracking process does not stop completely. Even when the tracking subject cannot be tracked because it is temporarily hidden by another subject, the tracking subject is likely to appear again in the vicinity of the same location. In such a case, tracking of the tracking subject that has been tracked immediately and accurately can be resumed.

  If the shutter button of the operation member 28 is fully pressed in step 5 of FIG. 7, the process proceeds to step 9 to set the aperture (not shown) of the interchangeable lens 11 and retract the interchangeable lens 11 of the quick return mirror 13 from the photographing optical path. Then, a series of photographing control such as opening / closing control of the shutter 14, charge accumulation and charge readout control of the first image pickup device 12, picked-up image processing, image recording, and the like are performed. Note that the shooting control is not directly related to the present invention, and thus detailed description thereof is omitted.

  As described above, according to the embodiment, the focus adjustment control of the photographing lens is prohibited based on the defocus amount of the focus detection area corresponding to the position of the tracking subject area searched using the template image of the tracking subject. For example, even if another object crosses in front of the subject to be tracked, the subject to be tracked is inadvertently adjusted with respect to the other object. It is possible to prevent a situation in which another subject having a hue similar to tracking symmetry is misidentified as a tracking target and focus adjustment is performed, and the tracking performance for the tracking target subject can be improved.

  FIGS. 12A and 12B are diagrams illustrating this state. In FIG. 12A, it is assumed that the subject A is being tracked using the tracking subject region 47. It is assumed that the subject A moves to the right in the figure and the subject B moves to the left in the figure, and both the subject A and the subject B are dressed in the same color, and the subject B is closer to the camera than the subject A. It is assumed that it is located in As shown in FIG. 12B, it is assumed that the subject A is hidden behind the subject B when the subject B crosses the subject A in front of it.

  In such a case, in step 202 of FIG. 9, since subject B is dressed in the same color as subject A, it is determined that the minimum hue difference is equal to or less than the threshold value, and the search area 49 In some cases, it is determined that there is a tracking subject.

  However, in the present embodiment, in step 303 of FIG. 10, the defocus amount detected in the focus detection area 45i (FIG. 12A) at the previous position of the subject area 47 and the current position of the subject area 47 are determined. The defocus amount detected in the focus detection area 45j (FIG. 12B) is compared, and it is determined whether or not the difference between the two is equal to or less than a preset threshold value. As a result, the defocus amount for the previous subject A is clearly different from the defocus amount for the current subject B, and therefore the process proceeds to step 306 in step 303 of FIG. Then, until a predetermined time elapses after the timer 38 is activated, the focus adjustment control is prohibited in step 305, and the subject B is not inadvertently focused.

  Further, according to the embodiment, the image information on the screen by the photographing lens and the image information of the template image of the tracking subject are collated, and it is determined whether the tracking subject has deviated from the tracking position based on the collation result. Thus, it is possible to accurately determine whether or not the tracking subject has disappeared.

  Further, according to one embodiment, a search area for searching for a tracking subject is set in the screen by the photographing lens, and the tracking subject is searched in the search area. If the tracking subject is out of the search area, the search is performed. Since the search for the tracking subject is performed by enlarging the area, there is a high possibility that the tracking subject can be captured again.

  In the above-described embodiment, the example in which the tracking target subject is manually specified by selecting the focus detection area corresponding to the tracking target subject using the focus detection area selection switch has been described. You may make it designate automatically. For example, a subject captured in the area of the closest defocus amount among the defocus amounts detected in a plurality of focus detection areas may be set as the tracking target subject, or the tracking target subject using the human face recognition technology may be used. A subject may be designated.

  In the above-described embodiment, an example in which the pupil division phase difference detection method is used as a method of detecting the focus adjustment state of the photographing lens 15 is shown. However, the focus indicating the focus adjustment state of the photographing lens 15 by the contrast method. An evaluation value may be detected.

  Furthermore, in the above-described embodiment, an example in which the same color information area is set as the tracking subject area based on the color information of the subject in the image area corresponding to the focus detection area designated as the tracking target subject has been described. For example, a 4 × 4 pixel area corresponding to the focus detection area regardless of the color information may be used as the tracking subject area. Alternatively, the size of the tracking subject region may be changed according to the image magnification corresponding to the focal length of the photographing lens, such as a wide region when the focal length of the photographing lens is short and a narrow region when the focal length is long. .

  In the embodiment described above, the tracking subject area is determined based on the subject color information. However, the tracking subject area is determined based on information obtained by adding luminance information to the subject color information. Also good.

  In the embodiment described above, as shown in FIG. 10, when the tracking subject is lost in the tracking process by color, the focus adjustment control is prohibited (AF lock) for a predetermined time. However, even when the tracking subject is lost in the tracking processing by color, the defocus amount detected in the focus detection area at the previous position of the subject region 47 and the defocus amount detected in a predetermined focus detection area (described later) If the difference is equal to or less than a preset threshold value, focus adjustment control may be performed using the defocus amount of the predetermined focus detection area, assuming that the tracking subject is not lost.

  FIG. 13 is a flowchart showing a modification of the focus adjustment control of FIG. Steps similar to those in FIG. 10 are given the same step numbers. In step 301 in FIG. 13, it is confirmed whether or not the “subject disappearance flag” is set. If it is set, the process proceeds to step 309. In step 309, the defocus amount is detected in a predetermined focus detection area, and the difference between the defocus amount detected in the focus detection area at the previous position of the subject region 47 and the defocus amount detected in the predetermined focus detection area. Is less than or equal to a preset second threshold value. The second threshold value is preferably smaller than the threshold value in step 303.

  If it is determined in step 309 that the defocus amount difference is equal to or smaller than the second threshold value, the process proceeds to step S304. In step 304, the lens drive amount is calculated based on the defocus amount detected in the predetermined focus detection area, and the lens drive device 30 is controlled to perform focus adjustment control of the photographing lens 15.

  On the other hand, if it is determined in step 309 that the difference in defocus amount is not equal to or smaller than the second threshold value, the timer 38 is activated and the process proceeds to step 306 as in FIG. If the timer 38 has already been started, the process proceeds to step 306 as it is. The processing after step 306 is the same as in FIG. 10, but at step 307, focus detection is performed using a predetermined focus detection area. In this case, the template image is basically not updated.

  The process of FIG. 13 is set as a mode for determining whether or not the tracking subject is lost by giving priority to the comparison processing based on the defocus amount even when the tracking subject is lost in the tracking process by color.

  The predetermined focus detection area is one focus detection area (for example, near the center of the shooting screen) out of a plurality of focus detection areas set in the shooting screen regardless of the tracking area. The focus detection area f) is an area that is preset as a priority area. Note that the predetermined focus detection area may be a focus detection area corresponding to a region having the smallest difference value from the template image. In this case, in step 307, focus detection may be performed using the focus detection area corresponding to the region having the smallest difference value. Alternatively, focus detection may be performed using the focus detection area used in the previous focus adjustment control process as the predetermined focus detection area.

  Here, if the direction of the tracking subject changes with respect to the camera, the rate of matching with the template image 48 decreases, and it may be determined that the tracking subject has been lost. However, if the focus detection area used in the previous focus adjustment control process is set as the predetermined focus detection area and the process of FIG. 13 is performed, even if it is determined that the tracking subject has been lost in the color tracking process, the predetermined focus detection area Can be used to keep track of the tracking subject.

  If the threshold value in step 202 is too small, the probability that it is determined that the tracking subject has been lost in the tracking processing by color increases. However, even if it is determined that the tracking subject is lost in the tracking process by color by performing the process of FIG. 13 with the focus detection area corresponding to the region having the smallest difference value from the template image as the predetermined focus detection area, the predetermined detection is performed. It is possible to continue tracking the tracking subject using the focus detection area.

  In the above-described embodiment, an example in which the tracking process is performed by calculating the difference between the hues B / G and R / G for each pixel has been described. Instead of the hue, the tracking process may be performed using luminance information. For example, the G value may be regarded as luminance information and the tracking process may be performed using only the G value data. Luminance information obtained by adding RGB values to a predetermined coefficient may be used.

  In the above-described embodiment, an example in which the present invention is applied to a single-lens reflex digital still camera has been described. However, the present invention is not limited to a single-lens reflex digital still camera such as a compact digital camera or a video camera. In addition, the present invention can be applied to any imaging device capable of acquiring an image, and the same effects as those described above can be obtained.

  Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired.

The figure which shows the structure of the camera (imaging device) of one embodiment The figure which shows the detailed structure of the control apparatus of the camera of one Embodiment. Front view of the second image sensor The figure which shows the filter of the pixel of a 2nd image pick-up element. The figure for demonstrating the subject tracking operation | movement of one Embodiment. The figure for demonstrating the subject tracking operation | movement of one Embodiment. The flowchart which shows the tracking control of one embodiment Flowchart showing tracking control initial processing of one embodiment The flowchart which shows the tracking calculation processing of one embodiment The flowchart which shows the focus adjustment control of one Embodiment The figure which shows a mode that the search area 49 is changed into the new search area 49 '. The figure for demonstrating the subject tracking operation | movement of one Embodiment. The flowchart which shows the modification of the focus adjustment control of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 15 Shooting lens 16 Focus detection optical system 17 Distance measuring element 25 2nd imaging element 29 Control apparatus

Claims (11)

A tracking device,
An image sensor for acquiring image information in the screen by an optical system;
Tracking means for comparing the image information in the screen with the image information of the target to track the target in the screen;
Focus detection means for detecting a focus adjustment state of the optical system with respect to at least one focus detection area set in the screen;
Focus adjusting means for adjusting the focus of the optical system based on the focus adjustment state in the focus detection region corresponding to the position of the object in the screen;
First determination for determining a difference between the previous focus adjustment state and the current focus adjustment state in the focus detection region corresponding to the target position based on the focus adjustment state detected by the focus detection unit. Means,
Control means for prohibiting the focus adjustment control by the focus adjustment means when it is determined by the first determination means that the difference between the previous focus adjustment state and the current focus adjustment state is large ;
Second determination means for determining whether or not the object has been tracked based on the result of the collation by the tracking means;
When the control unit determines that the target cannot be tracked by the second determination unit, and the first determination unit determines that the difference between the previous focus adjustment state and the current focus adjustment state is large. And the focus adjustment control by the focus adjustment means is prohibited . The tracking device according to claim 1 ,
When the control unit determines that the target cannot be tracked by the second determination unit, and the first determination unit determines that the difference between the previous focus adjustment state and the current focus adjustment state is small In the tracking device, the focus adjustment control by the focus adjustment means is performed. In the tracking device according to claim 1 or 2 ,
The tracking means compares the image information in the screen with the target image information,
The tracking device according to claim 2, wherein the second determination unit determines whether or not the target has been tracked based on a rate at which the image information in the screen matches the image information of the target. The tracking device according to any one of claims 1 to 3 ,
The tracking device expands a search area set in the screen when it is determined by the second determination unit that the target cannot be tracked. An imaging device comprising:
Imaging apparatus characterized by comprising a tracking apparatus according to any one of claims 1-4. The imaging apparatus according to claim 5 ,
The imaging device includes a first imaging device for acquiring the image information and recording an image in the screen, and a second imaging device for acquiring image information for the tracking means to track the position of the target. An imaging apparatus comprising: A tracking method,
Obtain image information on the screen by the optical system,
The image information in the screen is compared with the target image information to track the target in the screen,
Detecting a focus adjustment state of the optical system with respect to at least one focus detection region set in the screen;
Based on the focus adjustment state in the focus detection area corresponding to the position of the target in the screen, focus adjustment of the optical system,
Based on the detected focus adjustment state, a first determination is made to determine a difference between the previous focus adjustment state and the current focus adjustment state in the focus detection region corresponding to the target position,
If it is determined in the first determination that the difference between the previous focus adjustment state and the current focus adjustment state is large, the focus adjustment is prohibited .
Based on the result of the verification by the tracking, a second determination is made as to whether or not the object has been tracked,
When it is determined in the second determination that the object cannot be tracked and the first determination determines that the difference between the previous focus adjustment state and the current focus adjustment state is large, the focus adjustment control is performed. A tracking method characterized by prohibition . The tracking method according to claim 7 ,
If it is determined in the second determination that the target cannot be tracked, and the first determination determines that the difference between the previous focus adjustment state and the current focus adjustment state is small, the previous focus adjustment control The tracking method characterized by performing. In the tracking method of any one of Claims 7-8 ,
A tracking method characterized by performing the tracking by enlarging a search area set in the screen when it is determined by the second determination that the target cannot be tracked. In the tracking method of any one of Claims 7-9 ,
A tracking method, wherein, after the focus adjustment is prohibited, it is determined that the object has been tracked, the focus adjustment prohibition is canceled. In the tracking method of any one of Claims 7-10 ,
A tracking method, wherein the prohibition of the focus adjustment is canceled when the state in which the target cannot be tracked continues for a predetermined time after the focus adjustment is prohibited.

Images