JP2012138013A - Tracking device and control method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a tracking device and method capable of highly accurately detecting occurrence of erroneous tracking when performing tracking based on a feature of a tracking target.SOLUTION: A tracking device for tracking the position of a tracking target in a moving image executes processing for tracking a subject using feature information of the tracking target for a current processing target frame image to thereby detect a tracking target position at this time, extracts an image between the tracking target position at this time and a tracking target position at the last time from the processing target frame image, and determines whether a tracking target position detected based on similarity between the extracted image and the feature information is outside the tracking target. The tracking device, when it is determined that the detected tracking target position is not outside the tracking target, holds the tracking target position at this time as the tracking target position at the last time to be used in the determination.

Description

  The present invention relates to an apparatus and a method for tracking a specified tracking target.

As a method of tracking any object,
A pattern matching method that searches for a region having a high degree of coincidence with a stored template image or a template image that is sequentially updated during tracking,
-Relative difference method to find the target position from the image difference between the current frame and the previous frame,
A color / luminance matching method for extracting a single or a plurality of colors, luminances, or histograms from an object and searching for a region having a high degree of coincidence with them is known.

  The pattern matching method can achieve high accuracy when searching for a stationary object, but there are movements such as a change in the distance between the object and the tracking device, a rotation of the object, and a posture change in the vertical and horizontal directions of the tracking device itself. Tracking performance in the situation was not enough. The relative difference method has a drawback that tracking cannot be performed after another object crosses in front of the object or the object is completely out of frame. The color / brightness matching method can cope with the movement of the tracking target to some extent, but the ability to identify the target is low only by the distribution of the color and luminance, and the performance is sufficient in situations where many similar colors and luminance are distributed in the background It wasn't.

  In addition, there are limits to the tracking performance of various tracking methods, and there are cases where tracking becomes impossible, but whether the tracking target has disappeared from within the angle of view, or whether tracking has been erroneously performed to another object due to the tracking performance limit, etc. In some cases, it was difficult to correctly grasp the tracking situation.

  Various proposals for improving the tracking performance have been made for such problems. Regarding the disappearance of a tracking target due to an intersection with an object other than the tracking target, Patent Document 1 describes a configuration that stores the characteristics immediately before the tracking target disappears and enables tracking when the tracking target reappears using the characteristics. ing. Moreover, in patent document 1, state determinations, such as the presence or absence of a contact with another object, a crossing, are performed by the time change of the total area of a some mobile body.

Japanese Patent Application No. 4-320254

  However, when tracking the characteristics of the tracking target and performing tracking based on the characteristics, the tracking of other objects by the approach, contact, crossing, etc. with other objects that have similar characteristics to the tracking target has started. There was a case. Further, since it is not possible to recognize that the tracking is incorrect, another object is tracked as it is, and even after the tracking target and the position of the other object are separated from each other, it is not possible to return to the tracking target normally. That is, although there is a proposal for improving the tracking performance, it is a fact that the tracking performance is still limited. Therefore, based on the limitations of such tracking performance, this is properly detected when erroneous tracking occurs, and tracking is automatically interrupted, or a warning is displayed that there is a high possibility that erroneous tracking has occurred. Treatment is sought.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a tracking device and method capable of detecting the occurrence of erroneous tracking with high accuracy when tracking is performed based on the characteristics of a tracking target. To do.

In order to achieve the above object, a tracking device according to the present invention comprises the following arrangement. That is,
A tracking device that tracks a position of a tracking target based on feature information of the tracking target by performing a tracking process on a frame image of the moving image,
A degree-of-similarity calculation means for dividing the frame image to be tracked this time into a plurality of blocks, and calculating the degree of similarity with the feature information for each of the plurality of blocks;
Detecting means for detecting a block having the highest similarity calculated by the similarity calculating means as a current tracking target position;
The block through which a line segment connecting the current tracking target position detected by the detection means and the previous tracking target position obtained as a result of the previous tracking process passes is obtained from the target frame image. Extracting means for extracting from a plurality of blocks;
Determination means for determining that the tracking target position detected by the detection means is outside the tracking target based on the similarity calculated by the similarity calculation means of the block extracted by the extraction means.

  When tracking is performed based on the characteristics of the tracking target, occurrence of erroneous tracking is detected with high accuracy and a simple configuration. By making it possible to detect erroneous tracking, it is possible to automatically stop tracking and display a warning to the operator.

1 is an overall block diagram of a digital camera 100 according to an embodiment. 1 is an external view of a digital camera 100. FIG. The figure explaining the tracking method of the object by an embodiment. The figure explaining the tracking method of the object by an embodiment. The figure which shows an example of similarity distribution. The figure which shows another example of a tracking object. The figure explaining a weight table. The figure explaining the change of similarity distribution by the effect | action of a weight table. The figure which showed the similarity distribution of FIG.8 (c) by the numerical value. The figure explaining the similarity distribution which a false tracking generate | occur | produces. The figure which showed the similarity distribution of FIG.10 (c) by the numerical value. The figure explaining the detection method of a false tracking. The figure which shows the other example of the similarity distribution which a false tracking generate | occur | produces. The figure which showed the similarity distribution of FIG.13 (c) by the numerical value. The figure explaining the detection method of a false tracking. The flowchart explaining the tracking process by embodiment. The flowchart explaining the tracking process by embodiment.

  Hereinafter, a preferred embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating a configuration example of the entire system of a digital camera 100 (hereinafter, camera 100) according to an embodiment.

  The lens 10 collects external light. The lens driving circuit 42 moves the position of the lens 10 back and forth along the optical axis to adjust the focal point and adjust the angle of view. The camera shake correction circuit 40 drives the lens 10 based on the camera shake amount measured by the camera shake detection circuit 44 and changes the position of the lens 10 along the optical axis so as to cancel the camera shake. It is the composition which performs a typical camera shake correction. The shake amount detection circuit 44 includes a gyro sensor. In FIG. 1, camera shake correction is realized by driving a lens. However, camera shake can be similarly corrected by driving the image sensor 16. The lens barrel portion including the lens 10 may be extended and retracted, and the body volume when the camera is not used may be reduced to improve portability. The amount of light that has passed through the lens 10 is adjusted by the diaphragm 14. The system control unit 60 controls the diaphragm 14 by transmitting the diaphragm control information to the diaphragm driving circuit 26. The diaphragm 14 may be any known diaphragm, such as an iris diaphragm composed of a plurality of wings, or a round diaphragm in which a plate is previously punched with various diameters. The system control unit 60 uses the diaphragm 14 and the diaphragm drive circuit 26 to control the diaphragm so as to reduce the amount of light when the subject brightness is high. When the subject brightness is low, the system controller 60 opens the diaphragm and captures a large amount of light. To control. In addition, the amount of light can be adjusted in the same manner as the stop as a configuration in which an ND filter can be inserted and removed from the optical path. The system control unit 60 can control the mechanical shutter 12 by transmitting mechanical shutter control information to the mechanical shutter drive circuit 28. The exposure time during still image shooting is determined by the opening / closing time of the mechanical shutter 12, and this time is determined by the system control unit 60 and an instruction is given to the mechanical shutter drive circuit 28.

  Light that has passed through the lens 10, the mechanical shutter 12, and the aperture 14 is received by the image sensor 16. The system control unit 60 can control the image sensor 16 by transmitting an image sensor control signal to a timing generator (hereinafter, TG (Timing Generator)) 24. The TG 24 drives the image sensor 16 based on the control information received from the system control unit 60. The image sensor 16 periodically performs exposure to the element and reading of the exposed data, and this work is performed based on the drive signal from the TG 24. It is possible to read out only a specific line or a specific area from the data exposed by the image sensor 16. This can be realized by changing the reading method by the read control pulse output from the TG 24. The system control unit 60 determines an optimum reading method according to the situation and instructs the TG 24. For example, since high resolution is required during still image shooting, all data of the image sensor 16 is read out, and when a high frame rate such as 30 fps / 60 fps is required during electronic viewfinder or movie shooting, only specific lines are thinned out. And read them out. The electronic viewfinder is also referred to as EVF (Electrical Viewfinder). The TG 24 can control the exposure time of the image sensor 16. This is made possible by outputting a drive signal from the TG 24 to the image pickup device 16 so as to release the charge charged by the device at an arbitrary timing.

  Image data read from the image sensor 16 passes through a CDS (Correlated Double Sampler) circuit 18. The CDS circuit 18 has a main role of removing noise components of image data by a correlated double sampling method. Thereafter, the image data is input to a PGA (Programmable Gain Amplifier) 20. The PGA 20 can attenuate / amplify the image data level of the input image data. The system control unit 60 can control the amplification amount by transmitting the amplification level to the PGA 20. Normally, the exposure of the image sensor 16 is appropriately achieved by appropriately setting the exposure amount to the image sensor 16 with the aperture 14 and appropriately setting the exposure time with the shutter. In addition to this, the digital camera 100 can play a role of changing exposure of image data in a pseudo manner by attenuating / amplifying the image data signal with the PGA 20. This is provided as a function to the user based on the concept of sensitivity as one of the exposure conditions at the time of shooting along with the aperture and shutter speed.

  Image data is converted from an analog signal to a digital signal by an A / D (Analog / Digital Converter) circuit 22. Depending on the device, the bit width of the digital signal may be 10 bits, 12 bits, 14 bits, etc., and the image processing circuit 50 in the subsequent stage can support a plurality of types of bit widths. In the figure, the CDS circuit 18, the PGA 20, the A / D 22, and the TG 24 are expressed as separate blocks, but it is also possible to adopt a circuit in which these functions are mounted in one IC package.

  The digitized image data from the A / D 22 is input to the image processing circuit 50. The image processing circuit 50 includes a plurality of blocks and realizes various functions.

  The image sensor 16 generally extracts a specific color component for each pixel through a color filter. The image signal from the A / D 22 is in a data format corresponding to the pixel and color filter arrangement of the image sensor 16, and is used for automatic exposure control (AE) that performs exposure control by evaluating only the luminance component. It is a format that is not suitable for use. The image processing circuit 50 has a function of removing color information from an image signal and extracting only luminance information. Conversely, it also has a function of extracting only color information, and can be used for white balance processing that identifies the light source color of the subject and appropriately adjusts the color.

  Further, the image processing circuit 50 has a function of extracting only the frequency component of the signal read from the image sensor 16, and the extracted frequency component signal can be used during auto focus control (AF). it can. Further, the image processing circuit 50 has a function of setting which region of the frequency component of the data read from the image sensor 16 is to be extracted, and setting the division within the region. Further, at the time of AF processing, it is possible to drive the image sensor 16 to be suitable for distance measurement. In the case of television AF using the image sensor 16, it is necessary to drive the focus lens in synchronization with the drive rate (frame rate) of the image sensor 16. Therefore, the higher the drive rate of the image sensor 16, the faster the focus lens is moved. be able to. For this reason, the image sensor 16 can be driven so as to have a fast drive rate only during AF. Conversely, by lowering the frame rate and reading more pixel data from the image sensor 16, data that can be analyzed by the image processing circuit 50 increases, and more accurate distance measurement is possible. is there. Such use of the image sensor 16 can be determined adaptively according to the shooting mode of the camera 100 and the brightness of the subject.

Further, the image processing circuit 50 has a function of operating the increase / decrease of the level of the image signal digitized by the A / D 22 and the color effect of the image, and also plays a role of adjusting the image quality of the captured image. Regarding the level of the image signal,
-A function to increase or decrease the level with a uniform amplification factor for the entire image,
-Tone curve (gamma) function that converts the signal level according to the magnitude of the original signal level,
・ Function to increase / decrease the level with an amplification factor according to the frequency component of each area in the screen,
Various image signal level adjustments are possible.

  The image data digitized from the A / D 22 can be input to the image processing circuit 50 and simultaneously stored in the temporary storage memory 30. Once the image data stored in the temporary storage memory 30 can be read again, it is possible to refer to the image data from the system control unit 60 or to input the read image data to the image processing circuit 50. Further, the image data processed by the image processing circuit 50 can be written back to the temporary storage memory 30, or arbitrary data can be written from the system control unit 60 to the temporary storage memory 30.

  Image data appropriately processed by the image processing circuit 50 is input to the image recognition circuit 38. The image recognition circuit 38 is capable of recognizing a person's face, its facial expression, and character information, if any, in addition to recognizing the brightness, focus, and color of the input image. is there. It is possible to input a plurality of images to the image recognition circuit 38. For example, it is possible to determine whether the images are the same by inputting two images and comparing the characteristics of the two images. It is possible. In addition to the method of recognizing an image by the image recognition circuit 38, the system control unit 60 can also perform image recognition processing. The system control unit 60 can execute a computer program coded in advance on the CPU, but reads the image data stored in the temporary storage memory 30 from the system control unit 60 and analyzes the image data to analyze the situation of the scene. Can be recognized.

  When outputting to an image display device 108 such as an LCD, the image data subjected to image processing by the image processing circuit 50 is developed on the VRAM 34, converted into analog data by the D / A 36, and converted into analog data. To display. The electronic viewfinder is realized by sequentially updating the image display device 108 to display continuous images read from the image sensor 16. When the image data is expanded on the VRAM 34, the VRAM 34 is adapted to correspond to various display forms such that one image data is maximized on the image display device 108, or a plurality of images are displayed on a multi-screen. Can be deployed on top.

  The image display device 108 can display not only images but also arbitrary information alone or together with images. Camera status display, character information such as shutter speed, aperture value, sensitivity information selected by the user or determined by the camera, graphs such as luminance distribution measured by the image processing circuit 50, face recognition results, scene recognition results Etc. can also be displayed. The display position and display color of information can be arbitrarily selected. By displaying these various information, a user interface can be taken. The image display device 108 can also display image data stored in the image storage medium 82. If the image data is compressed, the image data is decompressed by the compression / decompression block 32 and the data is expanded in the VRAM 34. This data is converted into analog data by the D / A 34 and output.

  The image storage medium 82 is non-volatile, and can mainly store captured image data. Regarding the storage of image data, it is possible to have a folder hierarchy or to name folders and file names in ascending order in the order of shooting. Shooting information such as aperture value, shutter speed, ISO sensitivity, shooting time, and the like can be added to each image data, and these can be stored together with the image data. Furthermore, the stored image data can be read out, and the image can be copied, moved, and erased. The strobe light emission circuit 46 drives the strobe unit 90 to emit light according to an instruction from the system control unit 60. The operation unit 70 includes various operation switches described later with reference to FIG.

  FIG. 2 is an external view of the camera 100 according to the embodiment. A lens 10 is disposed on the front surface of the camera 100 so that a subject image can be captured. A strobe unit 90 is disposed on the same surface of the lens 10. When the main subject is dark, a sufficient amount of light can be obtained by causing the flash unit 90 to emit light, and a fast shutter speed can be maintained even in the dark and a suitable image can be obtained. In FIG. 2, the lens 10 and the strobe unit 90 are arranged on the same plane, but this is not a limitation, and it is also possible to arrange the strobe so that it faces the top of the camera in order to avoid the strobe light directly hitting the main subject. is there. A red-eye phenomenon is known in which a person's eyes appear red when a person is photographed with a flash in a dark environment. This is a phenomenon in which the pupil of the eye tends to be large in a dark environment, and when the strobe is emitted in this state, the strobe light is reflected on the retina and appears red as a photograph. In order to reduce this red-eye phenomenon, it is possible to close the pupil by pre-flashing the strobe unit 90 or another light projecting device before photographing and making it mistaken that this is a bright environment.

  An image display device 108 is disposed on the back of the camera. As described above, not only images but also character information, graphs, icons, and the like can be displayed on the image display device 108, which is an important member that interfaces with the user. In recent years, electronic viewfinders have become mainstream in digital cameras, and a subject is captured by referring to a continuous image output to the image display device 108 and used as a viewfinder. At this time, the photometry area information and distance measurement area information in AE and AF can be displayed superimposed on the live image. Furthermore, the recognition status of the subject can be displayed with the result of recognizing the person's face superimposed on a frame on the person's face, or the status of the background scene such as blue sky, sunset, or backlight can be recognized and displayed as an icon. Is possible. In addition, a conventional optical viewfinder 106 can be provided. The electronic viewfinder has advantages such as easy realization of a high field of view, easy viewing of a subject depending on the size of the image display device 108, and no difference in angle of view (parallax) between a captured image and a viewfinder image. On the other hand, power is required to operate the image sensor 16 and the image display device 108, and there is a concern about battery consumption. For this reason, when a large number of shots are desired while avoiding battery consumption, the electronic viewfinder function can be turned off and the optical viewfinder 106 can be used.

  The power on button 102 is a switch for performing a power on / off operation of the camera 100. The mode switch 110 can switch camera operation modes such as a still image shooting mode, a moving image shooting mode, and a playback mode. In the figure, it is expressed as a member that can switch several modes, but it is possible to have many still image modes such as a landscape shooting mode and a person shooting mode optimized for a specific scene to be shot.

  The parameter selection switches 151, 153, 155, 157, and 159 allow the user to select shooting conditions such as a distance measurement area and a photometry mode, to feed a page when playing back a captured image, and to set camera operation in general. You can choose. Furthermore, On / Off of the above-described electronic viewfinder can be selected. Further, the image display device 108 can display an image and can be configured as an input device as a touch panel.

  A shooting preparation & start button 104 is arranged above the camera. The shooting preparation & start button 104 is a single operation member, but it is possible to realize a two-step pressing operation when the button is pressed lightly (SW1 operation) and when the button is pressed deeply (SW2 operation). In the case of a camera that performs automatic exposure control and automatic focus control with the camera, pressing the shooting preparation & start button 104 shallowly performs automatic exposure control and focus control as shooting preparation, and pressing it deeply performs still image shooting and image recognition. An operation for giving an instruction can be realized.

  The automatic exposure control operates so as to obtain a suitable exposure in the shooting mode selected by the mode switch 110. The shooting mode includes a specific mode such as a portrait mode, a landscape mode, a night view mode, and a general mode such as an auto mode. There are also modes in which the user specifies the shutter speed and aperture value at the time of shooting, such as the shutter speed priority mode and the aperture priority mode. In these modes, the shooting sensitivity set by the PGA 20 can be automatically selected and set appropriately, or the user can specify the sensitivity in advance. When the user designates the sensitivity in advance, the S / N of the image signal decreases as the shooting sensitivity is increased. Therefore, it is assumed that the user who wants to give priority to the image quality selects the low sensitivity. The AF control can be switched so that a suitable pin can be aligned in each shooting mode.

  A method for realizing tracking of the tracking target designated as the main subject in order to capture the focus and brightness of the main subject to be photographed under suitable conditions in the camera 100 having the above configuration will be described. Hereinafter, the system control unit 60 will be described assuming that a computer (not shown) executes a tracking process by executing a computer program stored in a memory (not shown). The tracking process may be executed for all frames acquired as a moving image, or may be executed for selected frames such as every other frame.

FIG. 3A shows an example of a scene to be photographed from now on. A person 303 is located at the center of the screen 301 and the person is playing with the ball 305. To shoot so that the focus and brightness of such scenes are suitable, for example,
-Using the face detection result detected by the human face detection function, set the distance measurement area and photometry area near the face according to the face position and size,
AF and AE are performed using distance information and luminance information obtained from the set distance measurement area and photometry area,
Such a form is known. In addition, when the face detection function is not installed, a plurality of distance measurement areas are set in the screen, distance information obtained from each is comprehensively determined, for example, AF control is performed with priority on the near distance side. The form is known.

  On the other hand, when it is desired to continue focusing on a specific target or a certain part of the target, it is desirable to set the position in a form that can be designated and to keep track of the position. FIG. 3A shows a state in which a tracking target designation frame (hereinafter, designated frame 311) is displayed at the center of the screen 301. When the user presses down the operation member 153 and the shooting preparation & start button 104 at an arbitrary timing, the system control unit 60 (tracking process) interprets the specification frame 311 as an instruction for specifying the tracking target. Then, the system control unit 60 stores an object existing inside or in the vicinity of the designated frame 311 as a tracking target, and thereafter continues to track the tracking target. Further, the image display device 108 is used as a touch panel, the timing when the screen 301 is pressed is interpreted as a tracking target designation instruction, and an object existing in the vicinity of the pressed position 313 is stored as a tracking target and is continuously tracked. You can also. When the touch panel is mounted, there is an advantage that it is not necessary to display the designated frame 311 in advance or align the tracking target within the designated frame 311.

  FIG. 3B shows a state in which the captured image 301 is subjected to image processing and the inside of the screen is evenly divided into a plurality of small regions (32 × 24). Hereinafter, such division into small areas is referred to as block division, and each of these small areas is referred to as a block. In the present embodiment, a method for realizing tracking using a low-resolution image divided into blocks as shown in FIG.

  It is assumed that the operator has designated the pressing position 313 in FIG. 3A and designated the ball 305 as a tracking target. This designated position corresponds to the block 315 in the intra-screen block division image shown in FIG. 3B (coordinates are (13, 19)). The system control unit 60 extracts feature information from the block 315 or its peripheral blocks and stores it in a memory (for example, the temporary storage memory 30). In the present embodiment, image information is used as the feature information. For example, there are luminance information, color information, contrast information, edge information, and the like.

  As an example, FIG. 4B shows how feature information is extracted and stored in RGB format. A plurality of feature colors can be stored (in FIG. 4B, n feature colors are stored). For example, in the block division image of FIG. 3B, a plurality of colors can be stored, such as RGB at coordinates (13, 19), RGB at coordinates (14, 19), RGB at coordinates (13, 20),. . When extracting feature information, it is possible to check the color distribution around the block 315 specified in advance, and perform processing such as excluding from the tracking target feature color that there is a high possibility of a background color for a color with a large distribution. preferable. Further, when there are a plurality of colors that can become features, it is possible to prioritize which feature color is to be tracked and store them in the storage area of FIG. 4B in order of priority. One method of prioritizing a plurality of colors is to give priority to a thing with a small area among colors distributed in the vicinity of a designated block 315. This is based on the idea that among the colors of the tracking target, a color with a smaller area can be an important color that characterizes the tracking target. In addition, the camera 100 does not automatically extract a plurality of feature colors, but allows the operator to specify a plurality of positions in the tracking target, and allows a plurality of tracking target feature colors to be stored together with the priority order. Also good.

  When the system control unit 60 extracts the tracking target feature information as described above, the system control unit 60 enters the tracking operation. The frame images constituting the moving image of the subject are continuously generated in a predetermined cycle, and the system control unit 60 searches the frame images for areas similar to the characteristics of the tracking target, and selects areas with high similarity. Perform the work to be tracked. First, as shown in FIG. 3B, the system control unit 60 divides the screen into blocks, and calculates the similarity with the stored feature for each block. (Expression 1), (Expression 2), and (Expression 3) show expressions used by the tracking process of this embodiment when calculating the similarity.

In (Expression 1), the stored characteristic colors R _(N) , G _(N) , B _(N) and the colors R _{(X, Y)} , G _{(X, Y)} , B _{( X,} the difference value [Delta] R _(X of _{Y), Y), ΔG (} X, Y), calculates the .DELTA.B _{(X, Y).}
Here, N = feature color number (1 to n), X = number of horizontal blocks, and Y = number of vertical blocks.

  In this embodiment, since the characteristic color is expressed using the RGB format, a difference value is obtained for each of the three elements. When n characteristic colors to be tracked are stored, N is calculated for each of 1 to n in the equation. In this way, a difference value with one or a plurality of (n) feature colors is calculated for the same block. There are various methods for determining which feature color difference value to use when storing a plurality of feature colors. For example, a difference value is calculated according to the priority order shown in FIG. 4B, and when a difference value less than a predetermined value is found, the feature color is adopted, or a difference value with all feature colors is obtained. There is also a method of adopting a feature color having the smallest difference value after all are calculated.

Subsequently, whether the difference value calculated in (Expression 1) is less than a predetermined value ThRGB is checked in (Expression 2) below.
Here, ThRGB = predetermined value.

  This is to determine that the colors are similar when the color difference value between the block color and the characteristic color is at least less than ThRGB. In this embodiment, ThRGB is a predetermined value, but ThRGB can be adaptively changed according to the situation. For example, when many colors similar to the characteristic color of the tracking target are distributed in the screen, it is difficult to distinguish the tracking target from the background. Therefore, in such a case, if ThRGB is set smaller and the color difference condition is set more strictly, the occurrence of erroneous tracking can be reduced.

For the block in which the color difference value satisfies the condition of (Expression 2), the color similarity of each block is calculated using (Expression 3) below.

  The color similarity is a mixture of how similar each color difference value ΔR, ΔG, ΔB is within the range of ThRGB. When the minimum value of ΔR, ΔG, and ΔB is represented by 0 and the maximum value is represented by 255, for example, when all three colors of ThRGB = 40, ΔR, ΔG, and ΔB are 20, the color similarity of the block is 50. %. Similarly, when ΔR, ΔG, and ΔB are all zero, the feature color is completely matched, and the color similarity of the block is 100%. On the other hand, when ΔR, ΔG, and ΔB are all 40, they are not similar in the ThRGB range, and the color similarity is 0%. By the way, even if the condition of (Equation 2) is not satisfied, it is not similar at all, and the color similarity of the corresponding block is zero.

  As a result of calculating the color similarity for each block in the screen in this way, for example, the similarity distribution shown in FIG. 5B is obtained, and the similarity is obtained only in the region of the ball that is the tracking target specified in FIG. You can see how it goes up high. From the detailed numerical example of the similarity shown in FIG. 5A, it can be seen that the block 321 having the similarity of 100% indicates the highest similarity in the screen, and this block position is the tracking target position. It is judged that. When the tracking target position is indicated to the user as the detection result, the highest similarity block position may be displayed as a point as it is, but a continuous area composed of blocks with a certain degree of similarity is surrounded by a rectangular frame as a tracking target area. It may be displayed superimposed on the image. At that time, it is necessary to calculate which block is included in the tracking target area. In the present embodiment, the determination is made based on whether or not the maximum similarity in the screen exceeds the effective color similarity obtained by multiplying the predetermined coefficient ThRatio.

Here, ThRatio = predetermined value.
According to (Expression 4), when the maximum evaluation value is 100% and the ThRatio is 40%, the effective color similarity is 40%, and a continuous region having a similarity of 40% or more is a tracking target region. In the example of FIG. 5A, the continuous area indicated by shading corresponds to this, and the area surrounding this continuous area (in this embodiment, a rectangle containing the continuous area) is the tracking target area 317 to be displayed. . An image obtained by superimposing the tracking target area on the image is shown in FIG.

  As described above, tracking is realized by searching for a region with high similarity to the characteristics of the tracked target, but if other objects with high similarity appear in the screen other than the tracked target, May occur. In this embodiment, the method using the color similarity is described. However, for example, even if the similarity is calculated based on the contrast information or the pixel pattern, the tracking performance is still limited, and any tracking method is used. There are concerns about mistracking other similar objects. Therefore, it is desired to detect that erroneous tracking has occurred.

  Hereinafter, an example of occurrence of erroneous tracking will be described with reference to FIGS. It is assumed that a pure white ball position 411 is designated as a tracking target on the screen 401 in FIG. 6A, the tracking target area 421 is detected, and tracking is started normally. However, after that, as shown in FIGS. 6B and 6C, when the ball rolls to a position 413 and a position 415 and the distance from the pure white wall 417 having a color similar to the ball approaches, There is a case where the processing is misunderstood that the wall 417 is a tracking target and erroneous tracking occurs. In other words, the tracking target area is recognized as 421 → 423 → 425 → 427, and the ball at the position 419 may not be tracked as shown in FIG.

  In order to reduce the occurrence of such erroneous tracking, there is a method of applying a weight to the calculated color similarity of each block based on the currently detected tracking target position. For example, a higher weight is applied to the vicinity of the tracking target position that can be detected at present, and the weight is reduced as the distance increases toward the screen edge. By doing this, the similarity of other objects with high similarity appearing at a distant position is lowered, and the tracking target area is not suddenly transferred to other similar objects appearing on the screen. Realize.

  FIG. 7A shows an example of calculating the weight. TargetPos 501 in the figure is a tracking target position that can be detected at present. Using this TargetPos 501 as a base point, two function curves WeightRatioIn 523 and WeightRatioOut 533 are connected to form a curve whose weight decreases toward the screen edge.

X = number of horizontal blocks, Y = number of vertical blocks.

  WeightRatioIn 523 forms a curve from TargetPos 501 to a position separated by a distance DistDown 511. At this time, the coefficient WeightDownRatio 503 arbitrarily sets how much weight is reduced in DistDown 511 which is the end point of WeightRatioIn 523. For example, it can be varied according to the color distribution status in the screen including the background. More specifically, when many colors of the same color as the tracking target are distributed in the screen, the coefficient WeightDownRatio 503 is set so that the weight on the screen edge side is lighter than DistDown, thereby reducing the occurrence of erroneous tracking to the background. It is possible to set to. Subsequently, the area from DistDown 511 to DistZero 513 draws a curve in which the weight gradually decreases due to WeightRatioOut 533. Based on these curves, the weight value of each block position (X, Y) is calculated.

  FIG. 7B shows another example of calculating a simpler weight. FIG. 7B shows a method in which the weight of each position is calculated by connecting a straight line from the TargetPos 501 to the weight reduction position DistZero 513. However, in the case of such a simple line segment, when the tracking target moves with respect to the current tracking target position TargetPos 501, the degree of similarity decreases even if a high degree of similarity appears even if the distance from the TargetPos 501 is a little away. Resulting in. For this reason, there is a concern that the tracking performance is lowered, and attention must be paid to the slope of the line segment to be set.

  For example, how the weighting process shown in FIG. 7A works effectively with respect to the erroneous tracking example shown in FIG.

  FIG. 8A shows the distribution of similarity calculated when the ball is at the position 411 in FIG. The white wall of the house has a large area with high similarity. On the other hand, although the ball is a tracking target, the similarity is lowered by rolling away from the tracking start time. In this way, a phenomenon in which the degree of similarity often decreases due to movement of the tracking target, rotation, change in ambient light, etc. after the start of tracking, and when other objects with high similarity appear in such a state, There is a higher possibility of mistracking there. Therefore, the weight table shown in FIG. 8B is obtained from the position 411 that is the current tracking target position by the weight calculation shown in FIG. From this weight distribution, it can be seen that the weight near the ball position is strong and no weight is applied toward the edge of the screen. As a result of multiplying the similarity distribution of FIG. 8A and the weight table of FIG. 8B, the distribution image of FIG. 8C is obtained. Before weighting, the wall of the house was the region with the highest similarity as shown in FIG. 8A. However, the similarity of the wall of the house was reduced by weighting, and the ball as shown in FIG. It can be seen that the similarity in the vicinity (near position 411) is the highest region in the screen. FIG. 9 shows a detailed numerical distribution of similarity corresponding to FIG. 8C, and the block with the highest similarity in the screen is the ball position where the position 601 is located. Furthermore, if (Equation 4) is applied with ThRatio = 40%, the effective color similarity is 28% = 70 * 40/100, and a shaded area in the figure with a similarity of 28% or more is detected as a tracking target area. As a result, it is shown that the tracking to the wall 417 of the house could be prevented.

  However, even if weighting is performed, if the distance between the tracking target and another similar object is too close, erroneous tracking may occur. For example, FIG. 10A shows the distribution of similarity calculated when the ball is at the position 413 as shown in FIG. It can be seen that similarities are distributed over the white wall 417 of the house and the ball area at position 413. On the other hand, as shown in FIG. 10B, a weight with the position 413 as a base point is calculated, multiplied by the similarity distribution, and the distribution of FIG. 10C is generated so as to prevent erroneous tracking. Referring to FIG. 10C, the area having the highest similarity in the screen although weighted is the house wall 417, and erroneous tracking occurs at this point. FIG. 11 shows a detailed weighted similarity distribution (numerical distribution of similarity corresponding to FIG. 10C). The block with the highest similarity in the screen is the block 701, and the tracking target position moves from the position of the ball that has been tracked (block 711) (723). Furthermore, if (Equation 4) is applied with ThRatio = 40%, the effective color similarity is 26% = 67 * 40/100, and the shaded area in the figure with a similarity of 26% or more is the tracking target area 703. It will be detected.

  Similarity between movements (line segment 723) when switching from block 711 (movement source of tracking target position) whose tracking target is a ball part to block 701 (movement destination of tracking target position) which is a part of a house wall From the situation of the degree, it can be inferred that a false tracking to another object has occurred. FIG. 12 shows a distribution of similarity of blocks on a line segment 723 connecting the ball position (block 711) and the house wall (block 701) shown in FIG. A line segment 723 connecting the coordinates (18, 14) of the ball position (block 711) to the coordinates (14, 10) of the house wall 701 has coordinates (17, 13), (16, 12), (15, 11). ) 3 blocks exist. The similarity of each of these blocks is, for example, compared with the effective similarity of 26% previously calculated in (Equation 4), the coordinates (18, 14) and the coordinates (16, 12) (15, 11) ( 14 and 10) all exceed the effective similarity. On the other hand, it can be seen that the similarity of the block of coordinates (17, 13) is lower than the effective similarity. From this, it is estimated that there is an area other than the tracking target such as the background between the position of the block 711 and the position of the block 701, and the position of the block 711 and the position of the block 701 are different objects. It is estimated that the possibility is high. As described above, both the movement source and the movement destination of the tracking target position are equal to or higher than the effective similarity, and any of the blocks existing on the line segment connecting the movement source and the movement destination is less than the effective similarity. Thus, it is possible to detect that there is a high possibility that erroneous tracking has occurred.

  Further, the determination method of erroneous tracking based on the similarity distribution between the movement source and the movement destination (similarity distribution between movement of the tracking target position) is limited to the determination using the effective similarity as described above. is not. For example, a predetermined value such that the movement source is not less than the first predetermined value, the movement destination is not less than the second predetermined value, and any of the blocks on the line segment between the movements is less than the third predetermined value. It is also possible to make a similar determination using the value of. In addition, what is necessary is just to set the 1st-3rd predetermined value suitably, and all of them may be the same value, and may not be so. Alternatively, use a determination condition that erroneous tracking occurs when the ratio between the similarity between the moving base and the moving destination and the lowest similarity among the similarities of blocks existing during the movement exceeds a predetermined value. Is also possible.

  The above-described method works effectively when the approach speed between the tracking target and another similar object is slow to some extent. However, when the approach speed is fast, there is a case where erroneous tracking cannot be detected by this method. This is because the above-described method must be able to detect the presence of a background or the like during the movement of the tracking target position. For this reason, if a tracking target and another similar object contact or intersect each other soon after quickly approaching another similar object at a fast speed and detecting the background between movements, a false tracking was performed. It becomes difficult to detect this.

  This is shown in FIG. FIG. 13A shows the similarity distribution when the ball is at the position 415 and is in contact with the house wall 417 as shown in FIG. Similarities are distributed between the wall 417 of the house and the ball, but they are in contact with each other, and even if the weighting of FIG. 13B based on the position 415 of the ball is performed, FIG. As shown, the highest similarity is the position of the wall 417 of the house. FIG. 14 shows a detailed numerical distribution of similarity corresponding to FIG. In this case, until now, the block 811 that is the ball position has been tracked, but the tracking target position has moved from the block 811 to the block 801 because the highest evaluation value in the screen is the wall of the house (block 801). End up. Further, when the effective similarity = 34% (= 87 * 40/100) is calculated using (Equation 4) in order to obtain the tracking target area, the area indicated by shading in the figure has an effective similarity. Determined. As a result, the region 803 including the ball (region 813) as well as the house wall 417 is detected as the tracking target region. As described above, when contacting with another similar object, the similarities of the tracking target and the other similar object are integrated, resulting in erroneous tracking.

  In order to catch the moment of occurrence of this erroneous tracking, the movement state of the movement source, the movement destination, and the degree of similarity between the movements are examined as in FIG. FIG. 15 shows the similarity distribution. From FIG. 15, the similarity of the blocks (15, 12) (15, 11) existing on the line segment between the movements is high, and it is difficult to guess that the movement is between different objects as shown in FIG. It is a situation. Therefore, attention is paid to the movement distance of the tracking target position and the change of the tracking target area size.

  The tracking target position has moved 3 blocks from the coordinates (16, 13) to the coordinates (14, 10), and the tracking target area size is an area surrounding both the house and the ball from the size of the area 813 surrounding the ball. It has changed greatly to the size of 803. If the tracking target is moving in the distance direction approaching from far to near, for example, it is assumed that the size does not increase suddenly but gradually increases, and at the same time the tracking target position changes It is usually difficult to imagine a sudden change in size. Therefore, when the movement distance of the tracking target position is equal to or greater than a predetermined value and the change amount of the tracking target area size is equal to or greater than the predetermined value, it can be estimated that an erroneous tracking to another object has occurred.

  At this time, the change in the size of the tracking target area may be the area of the area, or if the area is rectangular, the amount of change in the size of the tracking target area is the amount of change in horizontal size, vertical size, or diagonal length. Also good.

  In this way, even if the tracking target and other objects similar to the tracking target approach rapidly and contact and crossing occur, the tracking error is detected by monitoring the change in the tracking target position and the tracking target area. It is possible to detect the occurrence of.

  The flow of the tracking operation and erroneous tracking detection will be shown below in a flowchart.

  FIG. 16 is an overall flow of the camera 100 equipped with the tracking method of the present embodiment. After the power-on button 102 of the camera 100 is pressed, the CPU is activated in the system control unit 60 and the power is supplied to the memory and the like, and the camera system (system control unit 60) is activated (S101). After that, the lens 10 (lens barrel) necessary for shooting operation with the camera, the various devices such as the mechanical shutter 12 and the mechanical aperture 14, the devices such as the image sensor 16 and the image display device 108 are activated (S103). ). When performing live view display of the subject image on the image display device immediately after the camera is activated, the system control unit 60 performs AE / AWB / AF processing so that the exposure, color, and focus of the image displayed in the live view are appropriate. Is started (S105). Thereafter, a live image starts to be output to the image display device 108 (S107).

The system control unit 60 checks whether the camera operator has instructed whether to activate the subject tracking function of the present embodiment during live view display (S109). As described above with reference to FIG.
-A designated frame 311 is displayed in advance on the image display device, and an object existing in the designated frame when the instruction button is pressed is set as a tracking target.
In a touch panel type image display device, by pressing an arbitrary position on the screen, an object at that position (for example, the pressed position 313) is set as a tracking target.
Etc. If there is a tracking target designation, the system control unit 60 performs tracking target feature extraction processing (S110), and stores the tracking target feature in a memory (for example, the temporary storage memory 30) as shown in FIG. 4B. . During live view display, the system control unit 60 performs AE / AWB / AF processing for continuously adjusting the exposure, color, and focus of the live image to the subject (S113, S115, S117). In parallel with these, if tracking is started (S111, YES), the system control unit 60 executes the tracking target search process of FIG. 17 (S201), and the tracking result is obtained by, for example, AF processing. It is used for focusing processing and AE processing.

  When the system control unit 60 detects that the SW1 button is pressed as a shooting preparation instruction (S121), the aperture value and shutter speed for still image shooting, AE for determining ISO sensitivity, and the main subject are in focus. AF is performed (S123, S125). Normally, it is assumed that preparation for still image shooting is completed when SW1 is pressed, and it is only waited for SW2 to be pressed as a shooting start instruction. However, since the subject tracking is often a moving subject when the subject tracking is activated, the system control unit 60 performs the tracking target search process while holding the SW1 button (S201). Further, the system control unit 60 may be configured to perform AE and AF according to the change of the subject using the tracking result (S133, S135). When the system control unit 60 detects that the SW2 is pressed (S141), the system control unit 60 performs a photographing process (S161). However, in the case of shooting in a low illumination environment or shooting in a backlight scene, if the flash emission is valid, a light control operation for determining the amount of flash emission is performed before shooting (S153). After shooting, the live view process is resumed according to the pressing state of the SW1 button and SW2 button.

  FIG. 17 shows a tracking target search process including erroneous tracking detection according to the present embodiment.

  First, the system control unit 60 acquires the tracking target position PreTargetPosX, Y, the region size PreTargetSizeX, Y, and the previous tracking target position similarity PreTargetEva as tracking target information at the time of execution of the previous tracking process (S203). ). Note that the “previous tracking process” is the most recent tracking process among the tracking processes for images of frames that are past the current processing target frame. Further, the evaluation value described in the flowchart is synonymous with “similarity”. At this time, since there is no previous value for the first tracking after the start of tracking, a predetermined value is used for the tracking target position, the region size, and the similarity as the tracking target position as initial information. .

  Next, the system control unit 60 generates the weight table described in FIG. 7A or 7B using the tracking target position PreTargetPosX, Y as a base point (S205). Subsequently, the system control unit 60 calculates the similarity between the characteristic color of the tracking target stored in S110 of FIG. 16 and each block (S207). At this time, the similarity may be calculated only with respect to a block having a weight value that is somewhat large with reference to the weight table calculated in the previous S205. This is preferably performed in order to eliminate useless similarity calculation processing because blocks whose weighted similarity is below a predetermined value cannot be a tracking target in the first place.

  Next, the system control unit 60 performs weighting on the calculated similarity (S209). This is a process for generating the weighted similarity distribution shown in (c) using the pre-weighted similarity distribution shown in (a) and the weight table shown in (b) in FIGS. Equivalent to. Subsequently, the system control unit 60 extracts the highest evaluation value from the weighted similarity distribution (S211), and confirms whether the highest evaluation value exceeds a predetermined value ThTargetEva1 (S213). When the value is below the predetermined value ThTargetEva1, the system control unit 60 determines that a block similar to the stored characteristic color of the tracking target is not found in the screen, and determines “LOST” as the tracking result (S215).

  On the other hand, when the highest evaluation value exceeds the predetermined value, the system control unit 60 temporarily determines the highest evaluation value position as the current tracking target position TargetPosX, Y (S217). Here, the provisional confirmation is described because there is a possibility that the tracking target position may not be adopted due to the subsequent error tracking detection. Furthermore, the system control unit 60 detects a rectangular area including a block including TargetPosX, Y blocks and including a block having similarities of a predetermined value or more as a tracking target area TargetSizeX, Y (S219). Here, the effective color similarity calculated by Expression 4 may be used as the predetermined value used for the continuous region determination.

  After the new tracking target position TargetPosX, Y is provisionally determined, the system control unit 60 checks whether a false tracking has occurred. First, the transfer from the tracking target to another similar object is checked. For this reason, as explained in FIG. 12, the system control unit 60 takes a similarity distribution of blocks existing between the previous tracking target position PreTargetPosX, Y and the movement of TargetPosX, Y by the current tracking. Then, a minimum evaluation value MinEva between movements is extracted from this similarity distribution S221.

The system control unit 60, as a condition for checking whether or not erroneous tracking to another object having a color similar to the tracking target has occurred,
The similarity MaxEva of the tracking target position this time is greater than the first predetermined value ThTop1,
The previous tracking target position similarity PreTargetEva exceeds the second predetermined value ThTop2,
The minimum evaluation value MinEva during the movement of the tracking target position is less than the third predetermined value ThBottom,
If the following three hold, it is determined that an error tracking to another object has occurred (S223). Note that this determination condition is merely an example, and the occurrence of tracking may be determined only by “the minimum evaluation value MinEva during the movement of the tracking target position being less than the third predetermined value ThBottom”. When it is determined that an erroneous tracking has occurred, that is, it is determined that the tracking target position temporarily determined in step S217 is not a tracking target. When an erroneous tracking occurrence is detected, the tracking process related to the tracking target may be stopped immediately, or a process for providing two options as described below may be performed. That is, the system control unit 60 determines whether or not the similarity of the previous tracking target position is larger than the predetermined value ThTargetEva2, in other words, whether or not the previous tracking position has a certain degree of similarity. . If it is determined that there is a certain degree of similarity even with the previous tracking position, the system control unit 60 retains the previous tracking position without moving to the newly detected tracking target position. That is, the tracking position is held by rewriting the previous tracking target position PreTargetPosX, Y to TargetPosX, Y provisionally determined in S217 (S229, S231). On the other hand, if the similarity is less than the predetermined value ThTargetEva2 in S225, the system control unit 60 determines that there is no similar object already in the previous tracking target position, and “LOST” to interrupt the tracking. "(S227).

  If it is determined in S223 that the transfer is not to a different object, the system control unit 60 checks whether a contact or intersection with the different object has occurred. The system control unit 60 calculates the amount of movement from the tracking target position with respect to the previous tracking target as an error tracking determination material (S233), and calculates the amount of change in the tracking target region size from the previous tracking target region size (S235). . Then, the system control unit 60 checks whether the movement amount ΔPos exceeds the fourth predetermined value ThPos or whether the size change amount ΔSize exceeds the fifth predetermined value ThSize. When both the movement amount and the size change amount exceed the predetermined values, the system control unit 60 determines that an erroneous tracking due to contact with another object as shown in FIG. 14 has occurred (S237). When the occurrence of erroneous tracking is detected in S237, the system control unit 60 determines “LOST” as a tracking result in the sense of interrupting tracking (S239). The system control unit 60 determines that tracking has succeeded when no tracking error is detected in S237, and determines that the tracking target position TargetrPosX, Y and the tracking target area TargetSizeX, Y have been detected as the tracking result “FIND (S241).

  The result of determining the tracking target position and area in S231 and S241 is visually provided to the operator by superimposing and drawing the tracking frame on the live image displayed on the image display device. When the occurrence of erroneous tracking is detected in S227 and S239, the tracking operation is automatically interrupted, the drawing of the tracking frame information is also erased, and further information indicating that the tracking is interrupted is displayed on the image display device or a warning sound. It is good also as a form notified by.

  In the processing of S233 to S237, the amount of change in size and the amount of movement of the tracking target position are used, but only the amount of change in size may be used. This is because the amount of movement of the tracking target position depends on the frame acquisition timing.

  As described above, when the tracking using the similarity to the tracking target feature is sequentially performed, the erroneous tracking detection process is also performed at the same time, and it is possible to correctly determine whether the tracking result is updated / held / suspended.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (12)

A tracking device that tracks the position of a tracking target in a moving image,
Detecting means for detecting the current tracking target position by performing subject tracking processing using the tracking target feature information on the current processing target frame image;
Extracting means for extracting an image between the previous tracking target position and the current tracking target position from the processing target frame image;
A determination unit that determines whether or not the current tracking target position detected by the detection unit is outside the tracking target based on the similarity between the image extracted by the extraction unit and the feature information;
Holding means for holding the current tracking target position as the previous tracking target position used by the determination means when it is determined that the current tracking target position detected by the detection means is not a tracking target; A tracking device comprising: The detection means includes
A similarity calculation unit that divides the current frame image to be processed into a plurality of blocks, and calculates a similarity with the feature information for each of the plurality of blocks;
Means for detecting, as the current tracking target position, a block having the highest similarity calculated by the similarity calculation means;
The extraction means includes
A block through which a line segment connecting the current tracking target position and the previous tracking target position passes is extracted from the plurality of blocks obtained from the processing target frame image;
The determination means includes
Determining that the tracking target position detected by the detection means is outside the tracking target based on the similarity calculated by the similarity calculation means of the block extracted by the extraction means;
The tracking device according to claim 1.   The determination unit determines that the tracking target position detected by the detection unit is not a tracking target when at least one of the similarities calculated by the similarity calculation unit of the extracted block is smaller than a first predetermined value. The tracking device according to claim 2, wherein the tracking device is determined to be. The determination means includes
At least one of the similarities calculated by the similarity calculating means of the extracted block is smaller than a first predetermined value;
Among the plurality of blocks obtained from the target frame image, the similarities calculated by the similarity calculating unit for the blocks corresponding to the current tracking target position and the previous tracking target position are respectively Greater than the second predetermined value and the third predetermined value;
In this case, the tracking device according to claim 2, wherein the tracking target position detected by the detection unit is determined to be outside the tracking target. The detection means further detects a tracking target region composed of one or a plurality of blocks including the current tracking target position based on the similarity of blocks around the current tracking target position,
The determination means further includes the detection means when the amount of change in size between the previous tracking target area detected in the previous tracking process and the current tracking target area is larger than a fourth predetermined value. The tracking device according to claim 3, wherein the detected tracking target position is determined to be outside the tracking target.   The determination means performs the determination based on the size change amount when a movement amount from the previous tracking target position to the current tracking target position exceeds a fifth predetermined value. 5. The tracking device according to 5.   7. The determination unit according to claim 5, wherein the determination unit performs determination based on the size change amount when all the similarities of the extracted blocks are equal to or greater than the first predetermined value. Tracking device.   The tracking device according to claim 1, wherein the feature information is at least one of luminance information, color information, contrast information, and edge information to be tracked.   9. The tracking process related to the tracking target is stopped when the determination unit determines that the tracking target position detected by the detection unit is outside the tracking target. 9. The tracking device described.   10. The previous tracking position is determined as the current tracking position when the determination unit determines that the tracking target position detected by the detection unit is not a tracking target. The tracking device according to any one of the above. A control method of a tracking device that tracks the position of a tracking target in a moving image,
A detecting step for detecting a current tracking target position by executing subject tracking processing using the tracking target feature information on the current processing target frame image;
An extracting step for extracting an image between the previous tracking target position and the current tracking target position from the processing target frame image;
Determination means for determining whether or not the current tracking target position detected in the detection step is outside the tracking target based on the similarity between the image extracted in the extraction step and the feature information Process,
When it is determined that the current tracking target position detected in the detection step is not a tracking target, the updating unit stores the current tracking target position as a previous tracking target position used in the determination step. And a holding step for holding the tracking device.   A computer program that causes a computer to function as each unit of the tracking device according to any one of claims 1 to 9.