JP2011243229A - Object tracking device and object tracking method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object tracking device and an object tracking method that can track an object recorded on a dynamic image at a high speed using a particle method.SOLUTION: A high-speed object tracking device introduces an average value shift method between a prediction model and an observation model in a particle method, searches a position of an image feature amount that is most similar to an image feature amount of an object to be tracked by the average value shift method, and moves a sample to the searched position. Accordingly, an amount of weighting calculation in the observation model is reduced.

Description

  The present invention relates to a technique for an object tracking apparatus and an object tracking method for tracking an object captured in a moving image using a particle method.

  There are a wide variety of application fields related to tracking of cars, people, fish, etc., and examples include a method of detecting the center of gravity of an object to be tracked and tracking the center of gravity using a linear model. However, it is known that in an actual environment, the object being tracked is often lost due to overlap with a building and other objects.

  Therefore, tracking methods using a Kalman filter or a particle method (particle method) using time-series image frames constituting a video (moving image) are used (see Non-Patent Documents 1 and 2). In particular, since the particle method is a technique for modeling that the state variable of the object changes according to a non-Gaussian distribution, it is possible to cope with a steep position change of the object. Hereinafter, the conventional particle method will be described.

  The particle method is a method for estimating the state statistically and efficiently when the state (position and size) of the object to be tracked cannot be directly known. Usually, the state of the object can be estimated only from the change and distribution of the observed image, and it is impossible to directly grasp the true state due to the influence of noise or the like.

  Specifically, as shown in FIG. 7, the state of the object is recursively estimated from the video through two processing systems of a prediction model and an observation model. In the particle method, hundreds to tens of thousands of samples (tracking indices) approximating the state (position, size) of an object are used, but in general, an image frame at the previous time t-1 in a certain recursive process. A predetermined number of samples S are selected from the resampling process.

  In the prediction model, the position of each sample S at time t is dynamically predicted according to a predetermined system model. At this time, the size of the sample S ′ positioned at the prediction destination is uniform and may be diffused from one sample to a plurality of samples according to the size (weight) of the sample S before prediction. As a result, a change occurs in the distribution density of the sample, and the sample apparently diffuses and moves.

  Thereafter, in the observation model, a weight corresponding to the position of the prediction destination is obtained from the image frame at time t, and each sample S 'positioned at the prediction destination is weighted. By displaying a sample having a size corresponding to the weight, the object can be tracked.

Greg Welch, 1 other, “An Introduction to the Kalman Filter”, TR 95-041, Department of Computer Science University of North Carolina at Chapel Hill, NC 27599-3175, July 24, 2006 Genshiro Kitagawa, “Introduction to Time Series Analysis”, Iwanami Shoten, February 24, 2005, p.216-223 “Brown Movement”, [online], [October 27, 2008 search], Internet <URL: http://www1.parkcity.ne.jp/yone/math/mathB03_11.htm>

  However, since the particle method described above requires an extremely large number of samples in order to track an object with high accuracy, the processing time in the prediction model and the observation model increases as the number of samples increases. There was a problem of loss of sex. Specifically, as the number of samples S increases, the predicted sample S ′ also increases, resulting in a problem that it takes time to weight the observation model.

  The present invention has been made in view of the above, and it is an object of the present invention to provide an object tracking device and an object tracking method that can rapidly track an object captured in a moving image using a particle method. To do.

According to a first aspect of the present invention, there is provided an object tracking device for tracking an object photographed in a moving image using a particle method, wherein a moving image composed of a plurality of time-series image frames is accumulated. Storage means for storing, and predicting means for predicting a position at which a tracking index representing an object photographed in the image frame at time t-1 moves at the next time t in accordance with a predetermined system model used in the particle method. An arbitrary search area is specified from the image frame at time t, and the color distribution q = {q _u } _{u = 1 ... m at} time t−1 (note that there is a ∧ symbol on q. , Σ _{u =} 1 to m q _u = 1, and m is the quantization level of the pixel gray value in the color distribution), and the color distribution p (y) = {p of the search area at time t _{u (y)} u = 1} ... m ( however, there ∧ symbols on the p Sigma _{u =} a _{1~m p u} = _1, y computes the a is) position of the search area, the degree of similarity between the color distribution p (y) of the search area and the color distribution q of the object [rho ( y) ≡ρ [p (y) , q] = Σ u = 1~m √ (p u (y) q u) by shifting the search area in the direction in which the larger the similarity [rho (y) is the maximum The search for the position of the search region is calculated by calculating the color distribution at position y ₀ using p (y) = {p _u (y ₀ )} _{u = 1. y 0), q] = Σ} u = 1~m √ a _{(p u (y 0) q} u) _{calculated, y 1 = (Σ i =} 1~n_h x i w i g ( ‖ _(y 0 -x ^{_{i) / h‖ 2)) /}} (Σ i = 1~n_h w i g ( ‖ _{(y 0 -x i) / h‖} 2)) ( where there ∧ symbol on the y, _{w i} is at least sigma _{u = 1 to m√} (q _u / (p _u (y ₀ ))), G (x) is the first derivative of k (x) and g (x) = − ((d + 2) / (nh ³ )) Σ _{i = 1 to n_h} (X _i −x), k (x) = 0.5 (d + 2) (1−x ^T x) (when x ^T x <1), and k (x) = 0 (x ^T x> 1), d is 2, h is a radius of a search region appropriately selected in a range of 2 to 6, and i (= 1 to n_h) is a number of a pixel included in the radius h. the position _{y 1} of the shift destination using certain) _{calculates, p (y) = {p} u (y 1)} u = 1 ... by calculating the color distribution at positions _{y 1} with _m, [rho [P (y ₁ ), q] = Σ _{u = 1 to} m √ (p _u (y ₁ ) q _u ) is calculated, and the calculated similarity ρ [p (y ₁ ), q] is similar degree _{ρ [p (y 0),} q] smaller or than the value of An average value shift method in which the search region is shifted and repeated calculation is repeated (however, if the luminance value of a pixel included in the radius h is equal to or less than a predetermined value, the next search region is not calculated without calculating the similarity). And the predicted position of the tracking index is corrected to the search position, the image feature amount corresponding to the corrected predicted position is calculated using the image frame at time t, and the image The gist of the invention is to include an estimation unit that weights the tracking index moved to the corrected predicted position using the feature amount and estimates the tracking index as a tracking index at time t.

According to a second aspect of the present invention, there is provided an object tracking method for processing by an object tracking apparatus that tracks an object captured in a moving image using a particle method. A tracking index representing an object photographed in the image frame at time t−1 is stored in accordance with a step of accumulating a moving image composed of a plurality of image frames and a predetermined system model used in the particle method. Predicting the position to move at time t, specifying an arbitrary search area from the image frame at time t, and color distribution q = {q _u } _{u = 1 of the} object at time t−1. _.m (where ∧ is above q, Σ _{u =} 1 to m q _u = 1, m is the quantization level of the pixel gray value in the color distribution), and at the time t Search area color distribution p (y) = {p _{u (y)} u = 1} ... m ( however, there ∧ symbol over p, a _{Σ u = 1~m p u = 1} , y is a is a position of the search area) is calculated, said Similarity ρ (y) ≡ρ [p (y), q] = Σ _{u = 1 to} m √ (p _u (y) q between the color distribution q of the object and the color distribution p (y) of the search area p (y) = {p _u (y ₀ )} _{u =} to search the position of the search area where the similarity ρ (y) is maximized by shifting the search area in the direction in which _u ) increases. _{1 ... m} is used to calculate the color distribution at position y ₀ , and ρ [p (y ₀ ), q] = Σ _{u = 1 to m} √ (p _u (y ₀ ) q _u ) , Y ₁ = (Σ _{i = 1 to n_h} x _i w _i g (‖ (y ₀ −x _i ) / h‖ ² )) / (Σ _{i = 1 to n_h} w _i g (‖ (y ₀ −x _i ) / h‖ ²⁾⁾ (However, there ∧ symbol on the y, _{w i} Is a value calculated using at least Σ _{u = 1 to m} √ (q _u / (p _u (y ₀ ))), and g (x) is the first derivative of k (x) and g (x ) = − ((D + 2) / (nh ³ )) Σ _{i = 1 to n_h} (x _i −x), k (x) = 0.5 (d + 2) (1−x ^T x) (x ^T x <1), k (x) = 0 (when x ^T x> 1), d is 2, and h is the radius of the search region appropriately selected in the range of 2-6. , I (= 1 to n_h) is the number of the pixel included in the radius h) to calculate the shift destination position y ₁ , and p (y) = {p _u (y ₁ )} _{u = 1 ... m} is used to calculate the color distribution at position y ₁ to calculate ρ [p (y ₁ ), q] = Σ _{u = 1} to _m √ (p _u (y ₁ ) q _u ), The value of the calculated similarity ρ [p (y ₁ ), q] is An average value shift method in which the search area is shifted and repeated calculation is repeated until the similarity is smaller than the value of ρ [p (y ₀ ), q] (however, the luminance value of the pixels included in the radius h is constant). If the value is less than or equal to the value, the step of shifting to the next search area without calculating the similarity), correcting the predicted position of the tracking index to the search position, and using the image frame at time t Calculating an image feature amount corresponding to the corrected predicted position and weighting the tracking index moved to the corrected predicted position using the image feature amount, and estimating the tracking index at time t The gist is to have.

  ADVANTAGE OF THE INVENTION According to this invention, the target object tracking apparatus and target object tracking method which can track the target object currently image | photographed by the particle method at high speed can be provided.

It is a functional block diagram which shows the functional structure of the target tracking device which concerns on this Embodiment. It is explanatory drawing for demonstrating the state of the sample in this Embodiment. It is a figure which shows the similarity distribution of the grasshopper area coefficient in the average value shift method. It is a flowchart which shows the processing flow of the target object estimation apparatus which concerns on this Embodiment. It is a figure which shows the positional relationship of each sample with respect to the similarity by a grasshopper char coefficient. It is a figure which shows the result of having tracked two fish which swims sharply in an aquarium. It is explanatory drawing for demonstrating the state of the conventional sample.

  FIG. 1 is a functional configuration diagram showing a functional configuration of the object tracking device according to the present embodiment. The object tracking device 100 includes an input unit 11, a prediction unit 12, a search unit 13, an estimation unit 14, a display unit 15, and a storage unit 31.

  The input unit 11 has a function of receiving an input of an image in which a moving object such as a fish swimming in an aquarium or a person crossing an intersection is photographed.

  The accumulation unit 31 has a function of accumulating videos input after being received by the input unit 11 as a plurality of time-series image frames. As such a storage unit 31, for example, a storage device such as a memory or a hard disk is generally used, and not only inside the object tracking device 100 but also an external device that can be electrically connected via a communication network. It is also possible to use a storage device.

  The prediction unit 12 performs the same processing as the processing of the prediction model in the particle method, and specifically, tracking that is captured in the image frame at time t−1 according to a predetermined system model used in the particle method. A sample (tracking index) showing a target object as a target has a function of predicting a position to move at the next time t. Specific processing will be described below.

The sample is obtained by approximating the state (position, size) of the object as described above, and is usually composed of hundreds to tens of thousands of samples. The prediction unit 12 selects N samples S _t−1 ⁽ⁿ⁾ from the resampling process of the image frame Z _{t−1 at} the previous time t−1 in a certain recursive process. Here, the weighting amount for each sample is π _t−1 ^(n), and the conditional probability that the state (position, size) of the sample is X _t−1 when the image frame is Z _t−1 is p. _{Assuming that} (X _t-1 | Z _t-1 ), the state of the sample after selection can be schematically shown as shown in the uppermost part of FIG. The sample shown at the top of FIG. 2 is shown in a size corresponding to the weighting amount. The weighting amount is an image feature amount of an object obtained from an image frame, and specifically, image strength, edge strength, color, texture strength, and the like can be used. Note that the sum of the weights in each recursive process is set to 1.

Thereafter, the prediction unit 12 calculates a predicted position at which the N samples S _t−1 ⁽ⁿ⁾ after selection move to the next time t in accordance with the predetermined system model described above. Here, when the image frame is Z _t−1 , the conditional probability that the state (position, size) of the predicted sample at time t is X _t is p (X _t | Z _t−1 ). The state after the prediction can be schematically shown as shown in the second stage from the top of FIG. Note that the size of each sample positioned at each predicted position is uniform, and is spread from one sample to a plurality of samples according to the size (weight) of the sample before prediction. Moreover, as a predetermined | prescribed system model, a random walk model (refer nonpatent literature 3) and a motion model can be used, for example. As a result, a change occurs in the distribution density of the sample, and the sample apparently diffuses and moves.

  The search unit 13 specifies an arbitrary search region from the image frame at time t, and uses the average value shift method (Mean Shift method) and the image feature amount of the object at time t−1 and the image of the search region at time t. It has a function of calculating the similarity with the feature amount, shifting the search area in a direction in which the similarity increases, and searching for a position where the similarity is maximum. Specific processing will be described below.

  The average value shift method is generally known as a method of performing noise removal at high speed, but is also applied to a method of tracking an object in an image by replacing it with an image feature amount based on a histogram. As the image feature amount based on the histogram, a gradient value relating to color (primary differential value: hereinafter referred to as “color distribution”) or the like can be used. In other words, the average value shift method is a method of searching for similar regions in an image frame at the next time centering on the result of tracking the image frame at the previous time, using the color distribution of the object being tracked as a model feature. It is a method of tracking things from time to time. Here, if a similar region is simply searched, the search range becomes enormous, and the search time may increase or misdetection may occur. Therefore, in the mean value shift method, an isotropic kernel (for example, Gaussian distribution) ), The color distribution is smoothed by performing convolution integration, and it is also possible to stabilize the calculation by suppressing excess noise. Hereinafter, a more specific processing flow will be described.

First, the search unit 13 specifies an arbitrary search region from the image frame at time t, calculates the color distribution of the object at time t−1, and calculates the color distribution of the search region at time t (step). S101). Here, the color distribution of the object is represented by equation (1), and the color distribution of the search region is represented by equation (2). Note that m corresponds to the quantization level of the pixel gray value in the color distribution. In the equation (2), y indicates the position of the search area.

Next, the search unit 13 calculates the similarity between the color distribution of the object and the color distribution of the search area, and moves the search area in the direction in which the similarity is maximized by the hill-climbing calculation (step S102). Specifically, as an index representing the degree of similarity between the color distributions, a grasshopper coefficient shown in Expression (3) is used.

This coefficient becomes larger as the degree of similarity between the two color distributions is higher. Tracking is realized by searching for a position y that maximizes this coefficient. Here, the tracking process will be specifically described. First, the color distribution at the position y ₀ is calculated using the equation (4), and the equation (5) is evaluated (step S102a).

Then, to calculate the position _{y 1} of the destination using equation (6) (step S102b). Note that g (x) is the first derivative of k (x). The value of d is 2, and h is appropriately selected within the range of 2-6. Note that h is the radius of the search area, and i is the number of pixels included in the radius h (i = 1 to n _h ).

Thereafter, the color distribution at the position y ₁ is evaluated using Expression (7), and Expression (8) is calculated (Step S102c).

Until the calculation result of Expression (8) becomes smaller than the calculation result of Expression (5), y ₁ is substituted into y ₀ and the iterative calculation is repeated (step S102d).

  Finally, the search unit 13 sets the position of the search area where the similarity is maximized in step S102 as the search position (step S103).

  As described above, since the Batterchaa coefficient is used for the similarity calculation in the average value shift method, the similarity distribution becomes smooth, and as shown in FIG. It is possible to quickly search for the convergence position (▽ point) that is most similar from the mark (○). That is, in the case of simple similarity calculation, there are a plurality of maximum values, but when calculating the similarity using the Batterchaer coefficient, a unimodal maximum value (maximum value) is obtained. The effect of speeding up based on smoothing can be obtained.

  In the process of step S102, the search unit 13 can also shift to the next search area without calculating the similarity when the luminance value of the pixel included in the area of the radius h is equal to or less than a certain value. It is. For example, in the case of 8-bit gradation (256 gradations), the similarity is not calculated for a search area including a pixel with a luminance value of less than 10, or less than 10 of the pixels included in the search area. It is possible to handle as no. Since there are fewer search areas to be subjected to similarity calculation, it is possible to search for the position of the search area where the similarity is maximized at high speed.

  The estimation unit 14 performs the same processing as the observation model processing in the particle method, and specifically corrects the predicted position predicted by the prediction unit 12 using the search position searched by the search unit 13. Then, the image feature amount corresponding to the corrected predicted position is calculated using the image frame at time t, and the sample moved to the corrected predicted position is weighted using the image feature amount, and the time t It has a function to estimate as a tracking index.

The observation method of the particle method performs weighting using the sample of the position predicted by the prediction model as described above, but the estimation unit 14 according to the present embodiment performs a search as shown in the lower half of FIG. The predicted position is corrected to be the search position searched by the unit 13. Then, the weighting from the image frame Z _t at time t by calculating the weight corresponding to the predicted position. Outermost When _{| (Z} t X _t), the state after weighting in Figure 2 as a result, when the image frame is a Z _t sample states (position, size) of the conditional probability is X _t p It can be modeled as shown in the bottom row.

  The display unit 15 has a function of displaying the weighted sample state.

  Then, the processing flow of the target object estimation apparatus which concerns on this Embodiment is demonstrated. FIG. 4 is a flowchart showing a processing flow of the object estimation apparatus according to the present embodiment. First, the input unit 11 receives an input of a video in which a moving object is photographed, and accumulates it in the accumulation unit 31 as a plurality of time-series image frames (step S201).

  Next, according to a predetermined system model used in the particle method, the prediction unit 12 is a position where a sample indicating an object to be tracked captured in an image frame at time t-1 moves to the next time t. Is predicted (step S202).

  Subsequently, the search unit 13 specifies an arbitrary search region from the image frame at time t, and uses the average value shift method to calculate the color distribution of the object at time t-1 and the color distribution of the search region at time t. The similarity is calculated, the search area is shifted in the direction in which the similarity is increased, and a position where the similarity is maximized is searched (step S203).

  Thereafter, the estimation unit 14 corrects the predicted position using the search position, calculates an image feature amount corresponding to the corrected predicted position using the image frame at time t, and uses the image feature amount to correct the corrected position. The sample moved to the predicted position is weighted and estimated as a sample at time t (step S204).

  Finally, the display unit 15 displays the weighted sample state (step S205).

  By performing the processing of step S201 to step S205 using all the image frames stored in the storage unit 31, it becomes possible to sequentially track the object shown in the video.

  FIG. 5 is a diagram illustrating the positional relationship of each sample with respect to the similarity based on the grasshopper area coefficient. The three-dimensionally displayed distribution indicates the similarity of the color distribution of the object at time t-1 and the color distribution of the image frame at time t, based on the Batterchaer coefficient. In addition, ◯ indicates a sample, and ☆ indicates a position where the similarity is maximum. The process of step S203 and step S204 shows a state where a plurality of samples have moved to a position where the similarity is maximum.

  FIG. 6 is a diagram showing the results of tracking two fish swimming steeply in the aquarium. FIG. 6A shows the tracking result in the conventional method, and FIG. 6B shows the tracking result in the present embodiment. In the case of the conventional method, the circle mark as a tracking result is displayed only for one fish out of two fish, but in this embodiment, it is displayed for two fish, so it is steep. It can be understood that even a moving mobile body can be tracked quickly and stably. In this tracking, the similarity is calculated using the color histogram distribution in the search process in the search unit 13. In particular, in order to enhance robustness, the color conversion model of RGB was applied to the color information of RGB, and similar calculation was performed using H (hue) information. The pixel of the object related to H is made into a histogram in the range of 0 to 360 degrees and divided into 12 parts every 30 degrees. A search is performed between image frames using 12 pieces of divided data. Although there is a method of tracking the contour of the object instead of the color, when the object is a fish, the shape changes greatly depending on the observation direction, so the object may be lost. Therefore, it is possible to track the object more stably by using the color as the image feature amount.

  According to the present embodiment, the average value shift method is introduced between the prediction model processing and the observation model processing in the particle method, and is most similar to the image feature amount of the object to be tracked by the average value shift method. Since the position of the image feature amount to be searched is searched and the sample is moved to the position after the search, the weighting calculation amount in the observation model is reduced as shown in the lowermost stage of FIG. 2, and a high-speed object tracking device is provided. Can do.

  Finally, the object tracking device described in each embodiment is configured by a computer, and each process of each functional block is executed by a program. Further, each processing operation of the object tracking device described in each embodiment is recorded as a program on a recording medium such as a compact disk or a floppy (registered trademark) disk, and this recording medium is incorporated into a computer or recorded. A program recorded on a medium can be downloaded to a computer via an arbitrary communication line, or installed from a recording medium, and the computer can be operated with the program, whereby each processing operation described above functions as an object tracking device. Of course, it can be made.

  It should be noted that the object tracking device described in the present embodiment can be applied particularly in the technical fields of the multimedia field, the coding field, the communication field, and the video surveillance field.

DESCRIPTION OF SYMBOLS 11 ... Input part 12 ... Prediction part 13 ... Search part 14 ... Estimation part 15 ... Display part 31 ... Accumulation part 100 ... Object tracking apparatus S101-S103, S102a-S102d ... Step S201-S205 ... Step

Claims (2)

In an object tracking device that tracks an object captured in a moving image using the particle method,
Storage means for storing moving images composed of a plurality of time-series image frames;
Predicting means for predicting the position at which the tracking index representing the object imaged in the image frame at time t-1 moves at the next time t according to a predetermined system model used in the particle method;
An arbitrary search region is identified from the image frame at time t, and the color distribution q = {q _u } _{u = 1...} M of the object at time t−1 (provided that there is a ∧ symbol above q, Σ _{u =} 1 to m q _u = 1, where m is the quantization level of the pixel gray value in the color distribution) and the color distribution p (y) = {p _{u of the} search area at time t _{(y)} u = 1 ...} m ( however, there ∧ symbol over p, a _{Σ u = 1~m p u = 1} , y is a is a position of the search area) is calculated, said target similarity between the color distribution p (y) of the search area and the color distribution q of the object ρ (y) ≡ρ [p ( y), q] = Σ u = 1~m √ (p u (y) q u ) that is shifted the search area in the direction of increasing by the similarity [rho (y) is to search for the position of the search region to be a maximum, p (y) = {p u (y 0)} u = _... by calculating the color distribution at the position _{y 0} with _{m, ρ [p (y 0} ), q] = Σ u = 1~m √ (p u (y 0) q u) is calculated and y ₁ = (Σ _{i = 1 to n_h} x _i w _i g (‖ (y ₀ −x _i ) / h‖ ² )) / (Σ _{i = 1 to n_h} w _i g (‖ (y ₀ −x _i )) / H ‖ ² )) (where ∧ is above y, and w _i is a value calculated using at least Σ _{u = 1 to m} √ (q _u / (p _u (y ₀ ))) , G (x) is the first derivative of k (x), g (x) = − ((d + 2) / (nh ³ )) Σ _{i = 1 to n_h} (x _i −x), and k (x ) = 0.5 (d + 2) (1−x ^T x) (when x ^T x <1), k (x) = 0 (when x ^T x> 1), and d is 2. , H is a radius of a search region that is appropriately selected within a range of 2 to 6, and i (= 1 to n_h) is the number of the pixel included in the radius h), and the shift destination position y ₁ is calculated, and p (y) = {p _u (y ₁ )} _{u = 1. .m} is used to calculate the color distribution at position y ₁ and ρ [p (y ₁ ), q] = Σ _{u = 1 to} m √ (p _u (y ₁ ) q _u ) Average value shifting method in which the search region is shifted and the iterative calculation is repeated until the similarity ρ [p (y ₁ ), q] is smaller than the similarity ρ [p (y ₀ ), q]. (However, if the luminance value of the pixel included in the radius h is equal to or less than a certain value, the search means performs the following search without shifting the similarity to the next search area);
The predicted position of the tracking index is corrected to the search position, an image feature amount corresponding to the corrected predicted position is calculated using the image frame at time t, and the corrected predicted position is calculated using the image feature amount An estimation means for weighting the tracking index moved to the position and estimating it as a tracking index at time t;
An object tracking device characterized by comprising: In an object tracking method processed by an object tracking device that tracks an object captured in a moving image using a particle method,
By the object tracking device,
Storing a moving image composed of a plurality of time-series image frames;
Predicting the position at which the tracking index representing the object imaged in the image frame at time t-1 moves at the next time t according to a predetermined system model used in the particle method;
An arbitrary search region is identified from the image frame at time t, and the color distribution q = {q _u } _{u = 1...} M of the object at time t−1 (provided that there is a ∧ symbol above q, Σ _{u =} 1 to m q _u = 1, where m is the quantization level of the pixel gray value in the color distribution) and the color distribution p (y) = {p _{u of the} search area at time t _{(y)} u = 1 ...} m ( however, there ∧ symbol over p, a _{Σ u = 1~m p u = 1} , y is a is a position of the search area) is calculated, said target similarity between the color distribution p (y) of the search area and the color distribution q of the object ρ (y) ≡ρ [p ( y), q] = Σ u = 1~m √ (p u (y) q u ) that is shifted the search area in the direction of increasing by the similarity [rho (y) is to search for the position of the search region to be a maximum, p (y) = {p u (y 0)} u = _... by calculating the color distribution at the position _{y 0} with _{m, ρ [p (y 0} ), q] = Σ u = 1~m √ (p u (y 0) q u) is calculated and y ₁ = (Σ _{i = 1 to n_h} x _i w _i g (‖ (y ₀ −x _i ) / h‖ ² )) / (Σ _{i = 1 to n_h} w _i g (‖ (y ₀ −x _i )) / H ‖ ² )) (where ∧ is above y, and w _i is a value calculated using at least Σ _{u = 1 to m} √ (q _u / (p _u (y ₀ ))) , G (x) is the first derivative of k (x), g (x) = − ((d + 2) / (nh ³ )) Σ _{i = 1 to n_h} (x _i −x), and k (x ) = 0.5 (d + 2) (1−x ^T x) (when x ^T x <1), k (x) = 0 (when x ^T x> 1), and d is 2. , H is a radius of a search region that is appropriately selected within a range of 2 to 6, and i (= 1 to n_h) is the number of the pixel included in the radius h), and the shift destination position y ₁ is calculated, and p (y) = {p _u (y ₁ )} _{u = 1. .m} is used to calculate the color distribution at position y ₁ and ρ [p (y ₁ ), q] = Σ _{u = 1 to} m √ (p _u (y ₁ ) q _u ) Average value shifting method in which the search region is shifted and the iterative calculation is repeated until the similarity ρ [p (y ₁ ), q] is smaller than the similarity ρ [p (y ₀ ), q]. (However, if the luminance value of the pixel included in the radius h is equal to or less than a certain value, shift to the next search area without calculating the similarity);
The predicted position of the tracking index is corrected to the search position, an image feature amount corresponding to the corrected predicted position is calculated using the image frame at time t, and the corrected predicted position is calculated using the image feature amount Weighting the tracking index moved to, and estimating it as a tracking index at time t;
An object tracking method characterized by comprising: