JP4382597B2 - Estimation apparatus, estimation method, and estimation program - Google Patents

Description

  The present invention relates to an estimation device, an estimation method, and an estimation program for estimating a true value of a physical quantity of an object based on an observed value of a physical quantity related to the object, and more specifically, an actual image from a camera or the like. Relates to the field of image recognition for visual tracking of objects in an image.

  Tracking a target or a target person with the passage of time in a captured actual moving image is one of the central issues in image recognition.

  Here, in recent years, particle filters have attracted attention from various fields as a time-series analysis method for signals added with non-Gaussian noise (see, for example, Non-Patent Document 1). Even in the real-time vision field, it is possible to statistically estimate and predict object motion on real images soiled with non-Gaussian noise such as noise and obstacles that cannot be done with Kalman filter. Applied research has become popular (see, for example, Non-Patent Document 2 and Non-Patent Document 3).

  The particle filter is an approximate calculation method of the Bayes filter, and expresses a distribution function with finite particles, and uses it to perform time series estimation and prediction.

In other words, the particle filter is a kind of Monte Carlo method based on sampling, and the distribution of the estimated position of the target object is approximated by a collective expression based on the position and weight of the particle, so that even a non-Gaussian distribution can be handled.
N. J. et al. Goldon, J.H. J. et al. Thurmond, A. F. M.M. Smith: “A New Approach to Nonlinear and Non-Gaussian Bayesian State Evaluation”, IEEE Proceedings Radar Signal Processing, Vol. 140, pp. 107-113 (1993) (Gordon, NJ, Salmond, JJ and Smith, AFM: Novel approach to nonlinear non-Gaussian Bayesian state estimation, IEEE Proc Radar Signal Processing, vol.140, pp.107-113 (1993)) M.M. Isard, A.D. Blake: “Condensation-Conditional Density Propagation for Visual Tracking”, International Journal of Computer Vision, Vol. 1, pp. 5-28 (1998) (M. Isard and A. Blake: CONDASATION-Conditional Density Propagation for Visual Tracking, Int'l J. Computer Vision, vol. 28, no. 1, pp. 5-28 (1998)) M.M. J. et al. Black, A. D. Jepson: "Trace recognition on time axis using condensation algorithm", IEEE International Conference Automatic Face Gesture, pp. 16-21 (1998) (Black, MJ and Jepson, AD: Recognizing temporal trajectories using the condensation algorithm, IEEE Int'l Conf Automat Face Gesture, pp. 16-21 (1998))

  However, the number of particles cannot be infinite. Therefore, the nature and accuracy of the particle filter depend greatly on the sampling method. This is an active discussion as a central problem with particle filters.

  In particular, for visual tracking of target objects (target persons), finite computational resources, constraints such as real-time calculations, and visual tracking using particle filters in visual tracking in complex real environments meet the following conditions: However, it is not always clear whether it should be realized.

  That is, firstly, it is necessary to perform robust tracking that is not easily influenced by surrounding noise (such as a similar object) by performing observation only at the particle position based on prior knowledge about the tracking target.

  Furthermore, secondly, it is necessary to perform real-time processing with a finite computational resource while suppressing a calculation amount to be small.

  Therefore, an object of the present invention is to provide an estimation device, an estimation method, and an estimation program for estimating a real value of a physical quantity of an object in real time based on an observed value of the physical quantity related to the object in a real environment. It is.

  Another object of the present invention is to provide a visual tracking device, a visual tracking method, and a visual tracking program capable of performing robust visual tracking on a moving image shot in a real environment while suppressing the amount of calculation. It is to be.

  An estimation apparatus according to an aspect of the present invention estimates a physical quantity of an object in real time by using means for preparing observation data of a physical quantity related to the object and a quantity related to the feature quantity of the object by a condensation method. If the variance of the estimated physical quantity of the object by the condensation method exceeds a predetermined value, the weight is determined by the augmented particle filter method instead of the condensation method. A means for estimating, extracting, and tracking physical quantities of objects in real time, and using a weight by the condensation method according to the variance of the estimated values of physical quantities of objects by the Augmentary Particle Filter method being less than a predetermined value. Means for extracting and tracking the physical quantity of the object.

  According to another aspect of the present invention, there is provided an estimation device that prepares digital data of values of pixels in a target image area including a target object, and a feature amount of the target object by a condensation method in the target image area. A means for estimating and extracting and tracking the position of the object in real time with the amount related to the weight as the weight, and the variance of the estimated value of the position of the object by the condensation method exceeds a predetermined value. In this image area, instead of the condensation method, a means for estimating, extracting and tracking the position of the object in real time using the weight by the augmented particle filter method, and the position of the object by the augmented particle filter method When the variance of the estimated value is less than or equal to a predetermined value, the position of the object is Means for estimating, extracting and tracking between.

  Preferably, the feature amount of the object is the color of the object.

According to still another aspect of the present invention, there is provided an estimation method in which an observation device acquires observation data of a feature amount related to a physical quantity of an object, and an arithmetic device relates to the feature quantity of the object by a condensation method. A step of estimating and extracting and tracking the physical quantity of the object in real time using the quantity as a weight, and a computing device when the variance of the estimated value of the physical quantity of the object by the condensation method exceeds a predetermined value. Instead of the session method, the step of estimating and extracting the physical quantity of the target object in real time using the weight by the augmented particle filter method , and the computing unit is the estimated value of the physical quantity of the target object by the auxiliary particle filter method. The physical quantity of the target object using the weight by the condensation method. And a step of tracking out.

According to still another aspect of the present invention, there is provided an estimation method in which an imaging device acquires digital data of values of each pixel in a target image region including a target object, and an arithmetic device is a target image. in the region, a step of positioning the extracted estimated in real time tracking of the object the quantity associated with the feature amount as the weight of the object by condensation method, computing device, the position of the object by condensation method When the variance of the estimated value exceeds a predetermined value, the position of the target is estimated in real time in the target image area by using the weight by the augmented particle filter method instead of the condensation method. extracting tracks Te, computing device, variance of the estimate of the position of an object by Ogujiriari particle filter method is less than a predetermined value Depending on to become, and a step of tracking extracted by estimating the position of the object in real time using the weight by condensation method.

  Preferably, the feature amount of the object is the color of the object.

  According to still another aspect of the present invention, a program for causing a computer to execute a method of estimating a physical quantity related to an object, the program preparing observation data of the physical quantity related to the object; , A step of estimating and extracting the physical quantity of the target object in real time using the quantity related to the feature quantity of the target object by the condensation method, and a variance of the estimated physical quantity of the target object by the condensation method are predetermined. In response to exceeding the value, a step of estimating and tracking the physical quantity of the target object in real time using the weight by the augmented particle filter method instead of the condensation method, and an object by the augmented particle filter method When the variance of the estimated value of the physical quantity falls below a predetermined value, And extracting and tracking the physical quantity of the object using the weights by the calculation method.

  According to still another aspect of the present invention, means for preparing observation data of feature quantities related to the physical quantity of the target object and physical quantities of the target object are obtained by weighting the quantity related to the feature quantity of the target object by the condensation method. In place of the condensation method according to the means for extracting, tracking, estimating by time, and when at least one component of the covariance matrix of the estimated physical quantity of the object by the condensation method exceeds a corresponding predetermined value , A means for estimating and extracting a physical quantity of an object in real time using weights by the Augerary particle filter method, and at least one component of a covariance matrix of an estimated value of the physical quantity of the object by the Augerary particle filter method The object's physics using weights by the condensation method Means for extracting and tracking the quantity.

  According to still another aspect of the present invention, means for preparing digital data of the value of each pixel in the target image area including the target object, and the feature amount of the target object by the condensation method in the target image area. Means for estimating and extracting and tracking the position of the object in real time with the associated quantity as a weight; and a predetermined value corresponding to at least one component of the covariance matrix of the estimated position of the object by the condensation method. A means for estimating, extracting, and tracking the position of the target object in real time using weights by the augmented particle filter method instead of the condensing method in the target image area. At least one component of the covariance matrix of the estimated value of the position of the object by the particle filter method is equal to or less than the corresponding predetermined value. Depending on the situation, it comprises means for estimating, extracting and tracking the position of the object in real time using weights by means of a condensation method.

According to still another aspect of the present invention , there is provided an estimation method in which an observation device acquires observation data of a feature quantity related to a physical quantity of an object, and an arithmetic device is a feature of the object by a condensation method. a step of physical quantity is extracted by estimating in real time tracking of the object an amount related to the amount as a weight, computing device, at least one component of the covariance matrix of the estimated value of the physical quantity of the object by condensation method There in response to exceeding a corresponding predetermined value, instead of the condensation method, using the weight by Ogujiriari particle filter method, the steps of the physical quantity is extracted by estimating in real time tracking of the object, computing device , at least one component of the physical estimate of the amount of the covariance matrix corresponding place of the object by Ogujiriari particle filter method In response to a value below and a step of extracting to track physical quantity of an object using the weight by condensation method.

According to still another aspect of the present invention , there is provided an estimation method in which an imaging device prepares digital data of values of each pixel in a target image region including a target object, and an arithmetic device is a target image. in the region, a step of positioning the extracted estimated in real time tracking of the object the quantity associated with the feature amount as the weight of the object by condensation method, computing device, the position of the object by condensation method In response to at least one component of the covariance matrix of the estimated value exceeding the corresponding predetermined value, in the target image area, instead of using the condensation method, the weight is determined by the augmented particle filter method. a step of tracking extracted by estimating the position of an object in real time, arithmetic unit, pair by Ogujiriari particle filter method When at least one component of the covariance matrix of the estimated value of the object position falls below the corresponding predetermined value, the object position is estimated and extracted in real time using the weight by the condensation method and tracked And a step of performing.

  According to still another aspect of the present invention, there is provided a program for causing a computer to execute a method for estimating a physical quantity related to an object, wherein the program prepares observation data of a feature quantity related to the physical quantity of the object. A step of estimating, extracting, and tracking the physical quantity of the object in real time using the amount related to the feature quantity of the object as a weight by the condensation method, and an estimate of the physical quantity of the object by the condensation method. When at least one component of the covariance matrix exceeds the corresponding predetermined value, the physical quantity of the object is estimated and extracted in real time using weights by the Augmentary Particle Filter method instead of the condensation method. Co-sharing the tracking step with the estimated physical quantity of the object using the Augmented Particle Filter method A step of extracting and tracking a physical quantity of an object using a weight according to a condensation method in response to at least one component of a scatter matrix being equal to or less than a corresponding predetermined value.

  According to the present invention, even in complex situations in real time and in a real environment, the robustness of condensation that is mainly predictive and the convergence speed of the augmented particle filter method that emphasizes observations are well integrated to achieve stable tracking. Can be performed.

[Embodiment 1]
[Hardware configuration]
The visual tracking device according to the present invention will be described below. This visual tracking device is realized by software executed on a computer such as a personal computer or a workstation, and extracts a target object (or target person) from a target actual moving image and tracks it in real time. Is for.

  However, as will be apparent from the following description, the present invention is not limited to visual tracking described as a specific example, and more generally, based on observation data of a physical quantity related to an object, Applicable when estimating values in real time.

  FIG. 1 shows the appearance of this visual tracking device.

  Referring to FIG. 1, the system 20 includes a computer main body 40 having a CD-ROM (Compact Disc Read-Only Memory) drive 50 and an FD (Flexible Disk) drive 52, and a display device connected to the computer main body 40. , A keyboard 46 and a mouse 48 as input devices also connected to the computer main body 40, and a camera 30 for capturing an image connected to the computer main body 40. In the apparatus of this embodiment, a video camera including a CCD (solid-state imaging device) is used as the camera 30, and a target object or a target person is detected from a moving image captured by the camera 30, It is assumed that processing to be tracked in real time is performed.

  In other words, the camera 30 prepares digital data of the value of each pixel in the target image area, which is a moving image including an object (target person).

  FIG. 2 shows the configuration of the system 20 in the form of a block diagram. As shown in FIG. 2, in addition to the CD-ROM drive 50 and the FD drive 52, the computer main body 40 constituting the system 20 includes a CPU (Central Processing Unit) 56 and a ROM (Read Only Memory (RAM) 58, RAM (Random Access Memory) 60, hard disk 54, and image capturing device 68 for capturing images from camera 30 are included. A CD-ROM 62 is attached to the CD-ROM drive 50. An FD 64 is attached to the FD drive 52.

  As described above, the main part of the visual tracking device is constituted by computer hardware and software executed by the CPU 56. Generally, such software is stored and distributed in a storage medium such as a CD-ROM 62 or FD 64, read from the storage medium by the CD-ROM drive 50 or FD drive 52, and temporarily stored in the hard disk 54. Alternatively, when the device is connected to the network, it is temporarily copied from the server on the network to the hard disk 54. Then, it is further read from the hard disk 54 to the RAM 60 and executed by the CPU 56. In the case of network connection, the program may be directly loaded into the RAM 60 and executed without being stored in the hard disk 54.

  The computer hardware itself and its operating principle shown in FIGS. 1 and 2 are general. Therefore, the most essential part of the present invention is software stored in a storage medium such as the FD 64 and the hard disk 54.

  As a recent general trend, various program modules are prepared as part of a computer operating system, and an application program generally calls a module in a predetermined arrangement to advance processing when necessary. is there. In such a case, the software itself for realizing the visual tracking device does not include such a module, and the visual tracking device is realized only when the computer cooperates with the operating system. However, as long as a general platform is used, it is not necessary to distribute software including such modules, and the software itself not including these modules and the recording medium storing the software (and the software distributes on the network). Data signal) can be considered to constitute the embodiment.

[Basic principles of visual tracking according to the present invention]
First, the outline of the procedure of the present invention is summarized. In the present invention, the condensation method (CONDENSATION) which is one of basic algorithms in visual tracking and an auxiliary particle filter (Auxiliary filter) which has improved some of its problems are described. The processing is performed by switching the Particle Filter (hereinafter referred to as “APF”) method under appropriate conditions under a condition specific to visual tracking. Condensation is robust to shielding, but cannot be tracked with high accuracy. On the other hand, the APF method attaches importance to the observation result and can accurately and accurately track the tracking target. However, if there is a shielding, the APF method often converges to noise. Therefore, by switching between these two conditions under the conditions described below, the advantages of each algorithm are utilized, and robust real-time tracking is performed in more situations in the real environment.

  In the following, first, as a premise for explaining a specific example of visual tracking in a moving image, a configuration for switching between a condensation method and an APF method will be described more generally, and then the visual tracking in a moving image will be described. A configuration for applying to the above will be described.

(Condensation method)
The continuous value state variable vector _xt to be tracked at time t is defined as follows. Here, when the present invention is applied to visual tracking, the continuous value state variable vector x _t indicates, for example, the true position coordinates of the object.

Further, the history of the continuous value state variable vector x _t is as follows.

Similarly, the observation variable vector z _t is defined as follows. Here, the observation variable vector z _t indicates, for example, the position coordinates of the observed object when the present invention is applied to visual tracking.

The history of the observation variable vector z _t is as follows.

Using a set of particles and weights at time t as in the following equation (0), the posterior distribution for the observed value of the state variable vector x _t is approximated as in equation (1).

Here, δ (...) Represents a Dirac delta function. That is, π _t ⁽ⁿ⁾ indicates the weight, and s _t ⁽ⁿ⁾ indicates the position of the particle. And π _t ⁽ⁿ⁾ satisfies the following formula (2).

  At this time, it is assumed that the dynamics to be tracked have a first-order Markov property expressed by the following formula (3) and that the conditional independence of the observation expressed by the formula (4) is satisfied.

At this time, the prior distribution of the state variable vector x _t can be approximated by the following equation (5).

The posterior distribution estimated from the observed variable vector z _t by Bayes' theorem can be written as the following equation (6).

Here, k _t is a constant for normalization. By repeatedly using the equations (5) and (6), it is possible to estimate the state variable vector x _t at an arbitrary time t.

As a problem peculiar to a filter using sampling / importance resampling (hereinafter referred to as “SIR”) such as a condensation method, particles {s _{t −1} ⁽ⁿ⁾ , n = 1,..., N} is resampled, and π _{t is} a weight for obtaining new particles {s _t ^(j) , j = 1,. _{Since −1} ⁽ⁿ⁾ is independent of the observation variable vector z _{t at} each time, there is a point where the state space is searched regardless of the observation. Therefore, SIR filters are greatly affected by outliers and often make tracking inefficient.

(APF method)
As an effective technique for the problem of how to handle outliers in a SIR filter such as the condensation method, there is the above-described APF method. Since this APF method is disclosed in detail in the literature: M. Pitt and N. Shephard: Filtering via Simulations: Auxiliary Particle Filters, Journal of American Statistical Association, (1999), the outline will be described below.

  In this APF method, dynamics are approximated using representative points such as an average value and a mode, the likelihood at the representative points is calculated using the dynamics, and sampling is performed again using that as a weight. By paying attention to the representative point, it is possible to quickly converge to the target point without much influence from the outlier.

  FIG. 3 is a diagram illustrating a difference in particle distribution due to a difference in sampling method between the condensation method and the APF method. In FIG. 3, white circles indicate the positions of particles sampled by condensation, and white diamonds indicate the positions of particles sampled by the APF method.

  FIG. 4 is a diagram showing a difference in convergence speed to the target resulting from a difference in particle distribution. FIG. 4 shows the relationship between the number of frames and the number of particles required for a certain particle to find the target until the estimated value matches the target position.

  Assuming that m = 1,..., M represents the particle index, the following equation (7) is established for each particle from equation (6).

At this time, μ _t ^(m) is a representative point of the following equation.

  At this time, when the above equation is approximated, the following equation (8) is obtained.

This equation (8) becomes the first-stage weight λ _t ^(m) as follows.

As the next step, sampling is performed N times based on the first-stage weight λ _t ^(m) , and a second-stage weight π _t ⁽ⁿ⁾ is newly determined as in the following equation (9).

Here, in Expression (9), the value represented by the following expression is μ _t ^(m) associated with _st ⁽ⁿ⁾ .

(Dynamic switching of resampling method)
Condensation estimates the state of the tracking target with emphasis on prior knowledge, as compared with the APF method, which focuses on observation and estimates the state of the tracking target. The APF method can accurately and accurately track the tracking target, but prior knowledge is more important than observation results in visual tracking in complex situations where there are multiple features similar to the tracking and tracking target. It is considered that more stable tracking can be performed.

  First, let the following equation represent the expected value of x at time t.

At this time, the i component of the estimated value variance σ _est is expressed by Expression (10) as follows. When the present invention is applied to visual tracking, the i component represents a coordinate axis component.

The resampling algorithm is dynamically switched as follows using the variance σ _est of the estimated value as a value indicating the accuracy of the estimation.

  By dynamically switching the algorithm in this way, it is possible to successfully combine the strong property of shielding mainly for prediction of condensation and the speed of convergence to the tracking target of the APF method.

  Here, the threshold value for switching determination is expressed as a threshold vector for each component as follows.

  Although such a threshold vector is not particularly limited, for example, a predetermined value can be determined in advance experimentally.

Further, the estimated value variance σ _est is compared with the threshold value for each component, but the magnitude of the vector of the estimated value variance σ _est itself may be compared with the threshold value γ.

  FIG. 5 is a flowchart for explaining the switching algorithm of the present invention described above. Note that FIG. 5 illustrates a case where the present invention is applied to visual tracking.

  Here, what is adopted as the weight in the case of visual tracking will be described in detail later.

  Referring to FIG. 5, visual tracking is started based on digital data of the value of each pixel in a target image area that is a moving image including a target object (target person) prepared by camera 30. Then (step S100), the system 20 performs visual tracking by the condensation method (step S102).

  Subsequently, the system 20 determines whether or not the variance of the estimated value exceeds a predetermined value on at least one space axis (step S104). If the variance of the estimated values does not exceed the predetermined value in any spatial axis, the process returns to step S102.

  On the other hand, if the variance of the estimated value exceeds a predetermined value on at least one spatial axis, the system 20 performs visual tracking by the APF method (step S106).

  Subsequently, the system 20 determines whether or not the variance of the estimated value exceeds a predetermined value in all the space axes (step S108). When the variance of the estimated value is equal to or smaller than the predetermined value in any spatial axis, the process returns to step S102.

  On the other hand, if the variance of the estimated value exceeds a predetermined value on at least one spatial axis, the system 20 continues visual tracking by the APF method (step S106).

(Learning dynamics by maximum likelihood estimation)
In order to perform robust tracking in more situations, it is necessary to learn the dynamics to be tracked from the image sequence. The following quadratic linear equation (11) having a bias term is assumed as the dynamics to be tracked.

The parameters of Equation (11) are collectively expressed as θ as follows, and w _t is white Gaussian noise.

In this case, the following R _t is zero mean Gaussian distribution of the dispersion C.

  For this reason, the logarithmic likelihood of θ is expressed by the following equation (12).

The maximum likelihood estimation value of the parameter β may be obtained by solving ∂L / ∂θ = 0 for the point that maximizes this. Although it is difficult to directly obtain this, it can be obtained by setting the following smoothing estimated value as the expected value of x _t .

  At this time, the following equations (13) to (16) hold for each parameter.

However, here it is as follows.

  Filtering is repeated again using the parameters updated by the equations (13) to (16). This is repeated until the parameters converge.

  The above is batch learning, which is a procedure for updating model parameters after all observation results are obtained.

  However, in order to perform tracking while learning dynamics in an actual environment, it is necessary to perform dynamics learning online. Therefore, by applying a fixed lag Kalman smoothing to the filtering estimated value at time t and applying the batch learning once within the window frame, the learning coefficient η is used as the following equation (19). And rewrite as (20).

  In the case of visual tracking, for example, the window size may be 60 frames, but stable learning can be performed if the window has a certain size and is dense.

  The system 20 updates the parameters according to the equations (13) to (16). Thereby, learning of dynamics can be performed online.

  Although not particularly limited, for example, in the following description, η = 0.01.

(Experiment using real images)
The position of the tracking target (x, y) on the image plane is taken as the state variable, and the experimental results using the actual image will be described below.

  In the following experiment, for example, the number of particles N is set to 800. At this time, the processing time for one frame is about 8 msec even in a personal computer equipped with a CPU that operates with a 2 GHz clock, and processing can be performed in sufficient real time. In the following experiment, images captured using a digital video camera were captured into a personal computer, and tracking of the obtained images and dynamics online learning were performed on the obtained continuous images.

(Color model)
When the target of visual tracking is a human, estimation may be performed using the skin color as the feature amount of the target object and the skin color likelihood in the image as the weight. However, an example in which redness (“redness”) is used as a weight instead of the skin color itself will be described below.

  Of course, other colors may be used as weights depending on the color of the object. Further, the feature amount is not necessarily limited to a color as long as it is a feature amount that can identify an object (or a target person) in an image. For example, feature amounts such as brightness, shape, pattern, and size are used. You can also.

Each red, green and blue components of the pixel z _t in the image at time _{t, R (z t),} G (z t), and B (z _t). At this time, the redness of the pixel z _t is defined as in the following equation (21).

Using the average value μ _h and variance σ _h ² of the histogram of the redness of the tracking target obtained in advance, it is expressed using a normal distribution as in the following equations (22) and (23).

However, r (x _t ) to μ _{h is} assumed, and r (x _t ) does not depend on the position of the tracking target.

(Experimental result)
(Online learning of dynamics from real images)
The above-described dynamics online learning is applied to track a ball (a diameter of about 16 pixels on the image plane) that a person swings to the left and right.

  FIG. 6 is a diagram illustrating an example of an image in an environment in which such an online learning experiment is performed.

  Condensation was used as the tracking algorithm.

  FIG. 7 is a diagram showing a locus of estimated values obtained in the experiment shown in FIG.

FIG. 7 also shows the results when dynamics are assumed to be x _{t + 2} = 2x _{t + 1} −x _t + Bw _t (no learning).

  When online learning of dynamics is performed, the tracking target may be lost in the initial stage of learning, but stable tracking can be performed even in a portion where the speed direction changes as learning progresses.

FIG. 8 is a diagram illustrating the relationship between the progress of learning and the standard deviation σ _est of the estimated value.

As learning progresses, σ _est becomes smaller and tracking can be performed with certainty. FIG. 8 also shows the result when the switching algorithm (SWITCH in the figure) is applied with the threshold value γ = (88) ^T. Even in the initial stage of learning, the tracking target is stable and can be tracked without losing sight.

  FIG. 9 is a diagram showing the learning progress, condensation, and switching of the APF method at this time. As the learning progresses, the number of frames selected by the APF method decreases, and the tracking that focuses on observation changes to the tracking that prioritizes prior knowledge.

(Visual tracking with occlusion)
Next, the results of tracking a red ball (with a diameter of about 16 pixels in the image) that is simply vibrated will be described. In this experiment, since the ball passes behind the shielding plate (board), the temporary ball cannot be observed.

FIG. 10 is a diagram illustrating an experimental environment, and FIG. 11 is a diagram illustrating a locus of estimated values. The threshold value of the switching algorithm was set to γ = (55) ^T based on the result obtained by applying condensation to the same image sequence in advance. A colored portion in FIG. 11 indicates a range where the tracking target is hidden behind the shielding plate.

  In the condensation, the tracking target moved and disappeared before the particles converged on the tracking target after exceeding the shielding. On the contrary, in the case of the APF method, the shielding is not exceeded and noise is generated. In many cases, it converged by mistake.

  On the other hand, when the switching algorithm of the present invention is used, the properties of both can be well fused and stable tracking can be performed.

  Table 1 shows the number of times such a task was repeated 50 times and the ball could be accurately tracked across the shield.

  As described above, by dynamically switching the resampling method, the convergence speed of the APF method that focuses on the robustness of observation and observation, even in complex situations in real time and real environments, and the convergence speed It is possible to perform a stable tracking by fusing them well. In the present invention, by switching between the two algorithms, an increase in the amount of calculation is suppressed and processing in real time is sufficiently possible. Also, by learning the dynamics to be tracked online, it is possible to perform more robust tracking than assuming a certain dynamic.

[Modification of Embodiment 1]
In FIG. 5 of the first embodiment described above, as a determination criterion in the switching algorithm, “whether or not the variance of the estimated value exceeds a predetermined value on at least one space axis” (step S104) and “all Whether or not the variance of the estimated value exceeds a predetermined value on the space axis (step S108).

However, as a switching algorithm, instead of the i component of the estimated value variance σ _est shown in Equation (10), more generally, an estimated value covariance matrix can be used as a criterion.

That is, the covariance matrix V _est (t) is expressed as follows using σ _{est, i, j} (t) that is the (i, j) component of the estimated value covariance matrix. In this case as well, when the present invention is applied to visual tracking, i and j respectively represent coordinate axes in the (i, j) component.

At this time, as a modification of the first embodiment, the resampling algorithm is dynamically switched according to the following rules. Here, γ _{i, j} is a threshold value corresponding to σ _{est, i, j} (t), and is, for example, a predetermined constant determined by an experiment in advance.

  FIG. 12 is a flowchart for explaining a switching algorithm according to a modification of the first embodiment, and is a diagram to be compared with FIG. FIG. 12 also illustrates the case where the present invention is applied to visual tracking.

  Referring to FIG. 12, visual tracking is started based on digital data of a value of each pixel in a target image area which is a moving image including a target object (target person) prepared by camera 30. Then (step S100), the system 20 performs visual tracking by the condensation method (step S102).

Subsequently, the system 20 determines whether or not at least one component of V _est (t) exceeds a corresponding predetermined value γ _{i, j} (step S104 ′). If any component does not exceed the predetermined value, the process returns to step S102.

  On the other hand, when at least one component exceeds a predetermined value, the system 20 performs visual tracking by the APF method (step S106).

Subsequently, the system 20 determines whether or not all components of V _est (t) exceed the corresponding predetermined value γ _{i, j} (step S108 ′). If any component is equal to or less than the corresponding predetermined value γ _{i, j} , the process returns to step S102.

On the other hand, if at least the component exceeds the corresponding predetermined value γ _{i, j} , the system 20 continues visual tracking by the APF method (step S106).

  Even with the configuration as described above, the same effects as those of the first embodiment can be obtained.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 is an external view of a system according to an embodiment of the present invention. It is a block diagram which shows the hardware constitutions of the system concerning embodiment of this invention. It is a figure which shows the difference in particle distribution by the difference in the sampling method of a condensation method and an APF method. It is a figure which shows the difference in the convergence speed to the target resulting from the difference in particle distribution. It is a flowchart for demonstrating the switching algorithm of this invention. It is a figure which shows the example of the image in the environment where the experiment of online learning was conducted. It is a figure which shows the locus | trajectory of the estimated value obtained by the experiment shown in FIG. It is a figure which shows the relationship between progress of learning, and standard deviation (sigma) _est of an estimated value. It is a figure which shows the mode of a change of learning progress, a condensation method, and APF method. It is a figure which shows an experimental environment. It is a figure which shows the locus | trajectory of an estimated value. It is a flowchart for demonstrating the modification of the switching algorithm of this invention.

Explanation of symbols

  20 system (visual tracking device), 30 camera, 40 computer body, 42 monitor.

Claims (12)

Means for preparing observation data of feature quantities related to physical quantities of the object;
Means for estimating and extracting a physical quantity of the object in real time by using a quantity related to the feature quantity of the object as a weight by a condensation method; and
When the variance of the estimated value of the physical quantity of the object by the condensation method exceeds a predetermined value, the physical quantity of the object is calculated by using the weight by the augmented particle filter method instead of the condensation method. Means to estimate, extract and track in real time;
Means for extracting and tracking the physical quantity of the object using the weight according to the condensation method in response to the variance of the estimated value of the physical quantity of the object by the augmented particle filter method being a predetermined value or less. An estimation device. Means for preparing digital data of the value of each pixel in the target image area including the target;
Means for estimating, extracting, and tracking in real time the position of the object in the image area of interest by estimating the position of the object in real time by using a quantity related to the feature quantity of the object by a condensation method;
When the variance of the estimated value of the position of the object by the condensation method exceeds a predetermined value, the weight is determined by an augmented particle filter method instead of the condensation method in the target image area. Means for estimating and extracting and tracking the position of the object in real time;
When the variance of the estimated value of the position of the object by the augmented particle filter method is equal to or less than a predetermined value, the position of the object is estimated and extracted in real time using the weight by the condensation method. And estimating means.   The estimation apparatus according to claim 2, wherein the feature amount of the object is a color of the object. An observation device acquiring observation data of a feature quantity related to a physical quantity of an object;
A computing device that estimates and extracts the physical quantity of the object in real time by using a quantity related to the feature quantity of the object as a weight by a condensation method; and
In accordance with the fact that the variance of the estimated value of the physical quantity of the object by the condensation method exceeds a predetermined value , the arithmetic unit uses the weight by the augmented particle filter method instead of the condensation method, Estimating, extracting and tracking physical quantities of objects in real time;
The arithmetic unit extracts the physical quantity of the object using the weight by the condensation method when the variance of the estimated value of the physical quantity of the object by the augmented particle filter method is a predetermined value or less. And a step of tracking. The imaging device acquiring digital data of the value of each pixel in the target image area including the target; and
A computing device that estimates, extracts and tracks the position of the target object in real time by using a quantity related to the feature quantity of the target object as a weight by a condensation method in the target image region;
In accordance with the fact that the variance of the estimated value of the position of the object by the condensation method exceeds a predetermined value , the arithmetic device replaces the condensation method in the object image area, and replaces the auxiliary particle filter. Using the weights in accordance with a method to estimate and track the position of the object in real time;
In response to the variance of the estimated position of the object by the augmented particle filter method being equal to or less than a predetermined value , the arithmetic unit determines the position of the object in real time using the weight by the condensation method. An estimation method comprising: extracting, tracking, and tracking.   The estimation method according to claim 5, wherein the feature amount of the object is a color of the object. A program for causing a computer to execute a method for estimating a physical quantity related to an object, the program comprising:
Preparing observation data of feature quantities related to the physical quantity of the object;
Estimating and extracting the physical quantity of the target object in real time by using a quantity related to the feature quantity of the target object as a weight by a condensation method; and
When the variance of the estimated value of the physical quantity of the object by the condensation method exceeds a predetermined value, the physical quantity of the object is calculated by using the weight by the augmented particle filter method instead of the condensation method. Estimating and extracting and tracking in real time;
Extracting and tracking the physical quantity of the object using the weight by the condensation method in response to the variance of the estimated value of the physical quantity of the object by the augmented particle filter method being a predetermined value or less. A program to prepare. Means for preparing observation data of feature quantities related to physical quantities of the object;
Means for estimating and extracting a physical quantity of the object in real time by using a quantity related to the feature quantity of the object as a weight by a condensation method; and
When at least one component of the covariance matrix of the estimated value of the physical quantity of the object by the condensation method exceeds a corresponding predetermined value, the weight is determined by an augmented particle filter method instead of the condensation method. Means for estimating, extracting and tracking the physical quantity of the object in real time;
When at least one component of the covariance matrix of the estimated physical quantity of the object by the augmented particle filter method is equal to or less than the corresponding predetermined value, the weight of the object is determined by the condensation method. An estimation apparatus comprising: means for extracting and tracking a physical quantity. Means for preparing digital data of the value of each pixel in the target image area including the target;
Means for estimating, extracting, and tracking in real time the position of the object in the image area of interest by estimating the position of the object in real time by using a quantity related to the feature quantity of the object by a condensation method;
In response to the fact that at least one component of the covariance matrix of the estimated value of the position of the object by the condensation method exceeds a corresponding predetermined value, in place of the condensation method in the target image area Means for estimating, extracting, and tracking the position of the object in real time using the weights by an augmented particle filter method;
When at least one component of the covariance matrix of the estimated position of the object by the augmented particle filter method is equal to or less than a corresponding predetermined value, the weight of the object is determined by the condensation method. An estimation device comprising means for estimating, extracting, and tracking a position in real time. An observation device acquiring observation data of a feature quantity related to a physical quantity of an object;
A computing device that estimates and extracts the physical quantity of the object in real time by using a quantity related to the feature quantity of the object as a weight by a condensation method; and
In response to the fact that at least one component of the covariance matrix of the estimated value of the physical quantity of the object by the condensation method exceeds the corresponding predetermined value, the arithmetic device replaces the condensation method with an auxiliary particle filter Using the weight according to a method to estimate and track the physical quantity of the object in real time;
The computing device uses the weight by the condensation method when at least one component of the covariance matrix of the estimated physical quantity of the object by the augmented particle filter method is equal to or less than a corresponding predetermined value. Extracting and tracking a physical quantity of the object. The imaging device preparing digital data of the value of each pixel in the target image area including the target;
A computing device that estimates, extracts and tracks the position of the target object in real time by using a quantity related to the feature quantity of the target object as a weight by a condensation method in the target image region;
In response to the fact that at least one component of the covariance matrix of the estimated value of the position of the object by the condensation method exceeds a predetermined value corresponding to the arithmetic unit, In place of the session method, the step of estimating and extracting the position of the target object in real time using the weight by the augmented particle filter method and tracking it;
The computing device uses the weight by the condensation method when at least one component of the covariance matrix of the estimated value of the position of the object by the augmented particle filter method is equal to or less than a corresponding predetermined value. And estimating and extracting and tracking the position of the object in real time. A program for causing a computer to execute a method for estimating a physical quantity related to an object, the program comprising:
Preparing observation data of feature quantities related to the physical quantity of the object;
Estimating and extracting the physical quantity of the target object in real time by using a quantity related to the feature quantity of the target object as a weight by a condensation method; and
When at least one component of the covariance matrix of the estimated value of the physical quantity of the object by the condensation method exceeds a corresponding predetermined value, the weight is determined by an augmented particle filter method instead of the condensation method. Using, extracting and tracking the physical quantity of the object in real time;
When at least one component of the covariance matrix of the estimated physical quantity of the object by the augmented particle filter method is equal to or less than the corresponding predetermined value, the weight of the object is determined by the condensation method. A program comprising: extracting and tracking a physical quantity.

Images