KR100902771B1 - Method for tracking multiple objects using particle filter - Google Patents

Abstract

One aspect of the present invention relates to a multi-object tracking method using a particle filter, and more particularly, to a method for enabling tracking of multiple objects by estimating a posterior distribution of a multiple shape using a particle filter which is a nonlinear estimation method.

In the multi-object tracking method using the particle filter of the present invention, a plurality of particle filters each track a plurality of objects corresponding to each of them in real time, the object is located at one pixel position at the time, A first step of obtaining a probability of the object being seen at the pixel position from a filter; A second step of obtaining a probability that the object is not visible at the pixel position from the filter at that time and the object is covered by another object; And a third step of tracking the object by obtaining a probability that the object is located at the pixel position by adding the probability obtained in the first step and the probability obtained in the second step.

Particle Filters, Multi-Objects, Tracking, Pixel Values, Weights

Description

Method for tracking multiple objects using particle filter

One aspect of the present invention relates to a multi-object tracking method using a particle filter, and more particularly, to a method for enabling tracking of multiple objects by estimating a posterior distribution of a multiple shape using a particle filter which is a nonlinear estimation method.

Embedded systems using robots have become increasingly widespread and complex, from industrial equipment to personal computing devices, and have been required for everyday applications. Robots or automation systems must use visual information available to identify and track objects related to the job. To accomplish these, real-time object tracking has been a significant problem in many computer vision applications such as visual servo operation of robots, monitoring, human-machine interfaces, smart environments, and the like.

One of the major challenges in this is how to track and locate a set of objects in real time, while maintaining accurate object recognition.

In the study of multi-object tracking, Jaco Vermaak, Arnaud Doucet, and Patrick Perez increased the number of objects to find a single object tracking filter to enable multi-object tracking. However, this method suffers from the loss of information about which filter is tracking which object when the objects overlap or are considered.

As a method of determining the state space of the filter, there is a method of continuously estimating the position of an object by applying an indicator to various peaks of the distribution. This method also occurs when the label information given to each other crosses each other when tracking the same object, and when the tracking object increases, the labeling problem of the indicator along with the expansion of the state space is exponentially calculated. There is an increasing problem.

One aspect of the present invention is to provide a method for enabling the tracking of multiple objects that has been impossible in reality because the computational amount of the computer increases exponentially.

According to an aspect of the present invention, in a method in which a plurality of particle filters track a plurality of objects respectively corresponding to them in real time, the objects are located at one pixel position at a corresponding time, and at the pixel positions from the filter. A first step of obtaining a probability of seeing the object; A second step of obtaining a probability that the object is not visible at the pixel position from the filter at that time and the object is covered by another object; And a third step of tracking the object by obtaining a probability that the object is located at the pixel position by adding the probability obtained in the first step and the probability obtained in the second step. Provides a multi-object tracking method using.

The first step may include: a first-first step of obtaining a probability that the object is at one pixel position at a corresponding time; Calculating a probability of the object being seen from the filter at the pixel position at the corresponding time; And a first to third step of multiplying the probability obtained in step 1-1 and the probability obtained in step 1-2.

In addition, the first and second steps may include determining the observability of each object with respect to the filter for pixels corresponding to distance information to each of the objects, geometric characteristics of the object, and surface features of the object. A step 1-2-1 of accumulating the weights of the objects and reflecting them in the image space; 1-2-2 normalizing the weights of the accumulated objects for each pixel; And re-normalizing the weight of the normalized object with respect to the pixel position of the object to reflect the weight of the pixel normalized from the image space to the particle space again. Provides a multi-object tracking method.

Also, the second step may include: a second step of obtaining a probability that the object is covered by another object from the filter at the pixel position at the corresponding time; Calculating a probability of the object not being visible at the pixel position from the filter at the corresponding time; And a second to third multiplication of the probability obtained in step 2-1 and the probability obtained in step 2-2.

In addition, the present invention provides a multi-object tracking method using a particle filter, wherein another filter is generated to track another in real time when another object is generated.

The present invention also provides a multi-object tracking method using a particle filter, wherein when one of the objects disappears, a filter for tracking it in real time is removed.

According to one aspect of the present invention, by multiplying the non-linear probability distribution that can track only one object can be multi-object tracking.

In addition, according to another aspect of the present invention, by overcoming the computational difficulties in tracking multiple objects, it is possible to track multiple objects in real time by improving the complexity of the exponential calculation to linear complexity.

In addition, according to another aspect of the present invention, it is possible not only to track the object but also to recognize the position of the robot.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, embodiments of the present invention may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. Shapes and sizes of the elements in the drawings may be exaggerated for clarity, elements denoted by the same reference numerals in the drawings are the same elements.

Christopher Rasmussen developed the Joint Likelihood Filter (LLF). Using the LLF, the object can be tracked using distance information to the object, geometrical features of the object, and surface features of the object based on the theoretical principle of the Joint Probability Data Association Filter (JPDAF).

In the present invention, the following three items will be described using the object tracking method.

In order to avoid the exponentially increasing computational complexity, we will discuss the MPF (Mixture Particle Filter) rather than the Joint Bayesian Distribution.

1) target filtering distribution

는 현재시간 t에서의 물체의 상태이고,

는 현재의 시간까지 측정된 데이터를 나타낸다. 다중물체 필터링 분포도(multi-target filtering distribution)를 베이시안(Bayesian) 필터 형식으로 나타내면 수학식 1과 같다.The post-distribution of a general particle filter is roughly calculated. here,

Is the state of the object at the current time t,

Represents data measured up to the current time. The multi-target filtering distribution is represented by Equation 1 in the form of a Bayesian filter.

는 측정확률에 대한 값을 계산하고, 각각의 물체에 대한 변환모델(transition model)인

는 전 상태를 기준으로 한 현재상태의 예측모델이다.Observation likelihood function

Computes the value for the probability of measurement and computes the transition model for each object.

Is a prediction model of the current state based on the previous state.

2. In order to overcome the problem of intersecting objects and tracking objects in the MPF, keep the measurement model part as a joint observation likelihood model, and when the objects are visible in the current data. The principle distribution that can be calculated in real time by decomposing the probability distribution and the probability distribution when not shown.

Joint observation likelihood model

Tracking multiple objects in a joint probability distribution is the most accurate method, but it is not possible with computer capabilities. Here we present a more accurate analysis of the intersection of objects by decomposing the basic measurement model, assuming each filter is independent of each other.

에 대한 관측 가능성 모델을 분해하게 된다.

는 수학식 2와 같이 표현할 수 있다.Under the assumption that each filter tracks only one object, the update step cannot be executed independently for each object. To overcome these problems,

Decompose the observability model for.

Can be expressed as in Equation 2.

은 물체수 K에 대한 필터를 의미하고,

는 물체 k가 보인다는 것을 의미하며,

는 물체 k가 보이지 않는다는 것을 의미한다. 수학식 2는 4개의 요소들로 구성된다.In equation (2)

Means filter for the number of objects K,

Means that the object k is visible,

Means that the object k is not visible. Equation 2 consists of four elements.

(A) Observation likelihood:

(B) likelihood of target k being visible:

(C) Observation model for hidden targets:

: 물체 k가 보이지 않을 가능성((likelihood of target k not being visible)(D)

(Likelihood of target k not being visible)

Then, the above four decomposition elements (A, B, C, D) will be described.

은 일반적인 관측모델과 같다.(One)

Is the same as a general observation model.

은 물체가 보일 가능성(visibility likelihood)을 나타낸다. 이 확률분포도는 3단계의 과정을 거쳐 계산된다. 먼저, 필터에 대한 각각의 물체의 관측 가능성을 각각의 물체까지의 거리정보, 물체의 기하학적 특징, 및 물체의 표면특징에 해당하는 화소들에 대한 물체들의 가중치를 축적하여 이미지 공간에 반영한다. 이후에, 각각의 화소에 대하여 축적된 물체의 가중치를 정규화한다. 이후에, 해당 물체의 화소 위치에 대하여 정규화된 물체의 가중치를 재 정규화하여 이미지 공간으로부터 파티클 공간으로 정규화된 화소의 가중치를 다시 반영한다.(2)

The likelihood of an object being visible Indicates. This probability distribution is calculated in three steps. First, the observability of each object with respect to the filter is reflected in the image space by accumulating weights of objects for pixels corresponding to distance information to each object, geometric characteristics of the object, and surface features of the object. Thereafter, the weights of the accumulated objects for each pixel are normalized. Thereafter, the weight of the normalized object is renormalized with respect to the pixel position of the object to reflect the weight of the pixel normalized from the image space to the particle space again.

The probability of the object being shown according to Bayes' law is given by Equation 3.

The first step in Equation 3 is interpreted as Equation 4.

: 물체 k가 있는 위치에서 화소값이 물체 k와 얼마나 유사한지를 나타내는 가중치here,

Is a weight indicating how similar a pixel value is to object k at the location of object k

: 현재의 추적 대상 물체 k의 화소 위치

: Pixel position of current tracking object k

: 추적 대상 물체 k의 모든 화소 위치

: All pixel positions of tracked object k

Equation 4 merely means a weight for the position of the pixel belonging to the object. Here, Equation 4 is established on the assumption that one pixel must belong to only one object.

Equation 3 shows normalization by particle weight on an image when an object is covered by another object.

은 정규화 요소인데, 모든 화소값은 각각의 이미지상에서 같은 위치에 있는 화소값을 합하여 그 합한 값으로 각각의 이미지상에 있는 값들을 정규화해야 한다.Equation 5 indicates which of the objects the current pixels are tracking comes from. here,

Is a normalization factor, and every pixel value must sum the pixel values at the same location on each image and normalize the values on each image to that sum.

After the normalization operation, the normalization operation is performed on Equation 6 on each image. Equation 6 shows how normalized pixel values are applied on all images.

의 X_j는 화소 위치이고, Pixel(x_k,_t)는 추적하고 있는 물체의 위치(

)이다. 모든 물체가 추적되고 있고, 그 물체들의 모든 화소 위치를 나타내고 정규화하기 위해 E라는 함수를 사용한다.here,

Where X _j is the pixel position and Pixel (x _k , _t ) is the position of the object being tracked (

)to be. All objects are being tracked, and we use a function called E to represent and normalize all pixel positions of those objects.

은 숨겨진 물체에 대한 관측 모델(observation model for hidden targets)을 나타낸다. 수학식 2에서

는 물체가 다른 물체에 가려 보이지 않을 때의 확률분포도를 표현한다. 이 확률분포가 중요한 이유는, 가려진 물체에 너무 많은 가중치가 주어지면 가리고 있는 물체의 분포도가 변하게 되고 그 반대로 가리고 있는 물체에 너무 많은 가중치를 두면 가려진 물체의 분포도가 결국 사라지게 되기 때문이다. 이런 문제점들을 극복하기 위해 수학식 7에서 그 값을 구한다.(3)

Represents an observation model for hidden targets. In equation (2)

Represents the probability distribution when the object is hidden from other objects. This probability distribution is important because, if too many weights are placed on an obscured object, the distribution of the obscured object will change, and vice versa, if too much weight is placed on the obscured object, the distribution of the obscured object will eventually disappear. To overcome these problems, the value is obtained from Equation 7.

는 1에서

를 뺀 값인데, 물체 k가 보이지 않을 가능성을 나타낸다.(4)

At 1

Minus, indicating the probability that object k is invisible.

A process of obtaining Equation 2 described above will be described with reference to FIG. 1.

First, in a method in which a plurality of filters track a plurality of objects respectively corresponding to them in real time, a probability of the object being shown at one pixel position at the corresponding time and the object appearing at the pixel position from the filter are obtained. . Specifically, in a method in which a plurality of filters respectively track a plurality of objects corresponding to them in real time, a probability (S100) of the object at one pixel position at a corresponding time and the pixel position at the corresponding time. In step S500, the probability of the object being seen from the filter is multiplied with each other (S500). Here, S200 has three steps, which will be described with reference to FIG. 2.

First, the observability of each object with respect to the filter is reflected in the image space by accumulating weights of objects for pixels corresponding to distance information to each object, geometric characteristics of the object, and surface features of the object (S210). ).

Thereafter, the weights of the accumulated objects for each pixel are normalized (S220).

Subsequently, the weight of the normalized object is renormalized with respect to the pixel position of the object to reflect the weight of the pixel normalized from the image space to the particle space again (S230).

Second, in a method in which a plurality of filters track a plurality of objects respectively corresponding to them in real time, the probability that the object is not visible at the pixel position from the filter at that time and that the object is covered by another object is determined. Obtain Specifically, in a method in which a plurality of filters track a plurality of objects respectively corresponding to them in real time, a probability (S300) of the object being covered by another object from the filter at the pixel position at the corresponding time, At this time, a probability S400 is obtained from the filter at the pixel position and multiplied with each other (S600).

Third, the probability finally obtained in the first and second is added (S700).

3. The principle in which another object is generated or one object disappears during object tracking will be described with reference to FIGS. 3 to 5.

Filter creation means that when another object is created, another filter is created that tracks it in real time, and is defined as a background filter. This background filter works mutually exclusive on all filters, excluding what other filters are tracking. However, this filter is excluded from normalization because it should not affect the distribution of the entire filter.

In FIG. 3, there are three objects of the first object 100, the second object 200, and the third object 300, and the first filter 110 and the second filter corresponding to each of them are tracked in real time. 210, there is a third filter 310. In FIG. 4, since the fourth object 400 is generated, a fourth filter 410 is generated to track the same in real time.

Filter deletion means that if one of the objects disappears, the filter that tracks it in real time is removed. Filter removal does not mean that the object you are tracking disappears from other objects, but disappears from the entire image.

In FIG. 3, there are three objects of the first object 100, the second object 200, and the third object 300, and the first filter 110 and the second filter corresponding to each of them are tracked in real time. 210, there is a third filter 310. In FIG. 5, since the third object 300 disappears, the third filter 310 that tracks this in real time has been removed.

확률분포도가 정의되고, 이것은 필터 k와 관련된 물체가 존재한다는 것을 의미하며, 임계값을 정의하여 추적하고 있는 필터의 분포도를 언제 제거하는지를 결정한다. 수학식 8은 물체가 존재할 때의 계산식이다.As such, the filter is automatically removed, thereby reducing the burden on the entire tracking operation. therefore,

A probability distribution is defined, which means that there is an object associated with filter k, and defines a threshold to determine when to remove the distribution of the filter being tracked. Equation 8 is calculated when an object exists.

A common joint particle filter has a computational difficulty because each object is a Kd dimension. Where K is the number of objects you want to track and d is the dimension of each filter. If each particle filter requires N objects, then N ^K objects are needed to track the co-distribution of the ^K objects. If the object regeneration step is assumed to be O (N) and the size of the pixel with the object is M, the complexity of the joint particle filter is 0 (MN ^K ).

In order to simplify the complexity of the common particle filter, the present invention resolves the measurement model linearly. If the object regeneration step is called O (N), the basic particle filter takes O (MN) and the proposed measurement model takes O (3MNK). The reason why O (3MNK) is required in the proposed measurement model is that O (MNK) is required to reflect the image space in the object space and O (MNK) is required to normalize it. This is because O (MNK) is required to reflect this back into the space where the object is located.

1 is a flowchart of a multi-object tracking method using a particle filter of the present invention.

FIG. 2 is a detailed flowchart of the step of obtaining the probability of seeing an object from a filter at a pixel position at the corresponding time of FIG. 1.

3 is a conceptual diagram illustrating an object of the present invention and a filter tracking the object.

FIG. 4 is a conceptual diagram illustrating an object and a filter tracking the object when another object is generated in FIG. 3.

FIG. 5 is a conceptual diagram illustrating an object when one object disappears from FIG. 3 and a filter tracking the object.

                 <Explanation of symbols for the main parts of the drawings>

100: first object 110: first filter

200: second object 210: second filter

300: third object 310: third filter

400: fourth object 410: fourth filter

Claims (6)

In the method of the plurality of particle filters to track each of a plurality of objects respectively corresponding to them in real time, A first step of obtaining a probability that the object is seen at one pixel position at that time and the object is visible at the pixel position from the filter; A second step of obtaining a probability that the object is not visible at the pixel position from the filter at that time and the object is covered by another object; And A third step of tracking the object by obtaining a probability that the object is located at the pixel position by adding the probability obtained in the first step and the probability obtained in the second step; Multi-object tracking method using a particle filter comprising a. The method of claim 1, The first step, Calculating a probability that the object is at one pixel position at a corresponding time; Calculating a probability of the object being seen from the filter at the pixel position at the corresponding time; And A first to third step of multiplying the probability obtained in the first-first step by the probability obtained in the first-second step; Multi-object tracking method using a particle filter comprising a. The method of claim 2, Step 1-2, A first reflecting the observation possibility of each object with respect to the filter in the image space by accumulating weights of objects with respect to pixels corresponding to distance information to the object, geometric characteristics of the object, and surface features of the object; -2-1 step; 1-2-2 normalizing the weights of the accumulated objects for each pixel; Re-normalizing the weight of the normalized object with respect to the pixel position of the object to reflect the weight of the pixel normalized from the image space to the particle space again; Multi-object tracking method using a particle filter comprising a. The method of claim 1, The second step, Calculating a probability that the object is covered by another object from the filter at the pixel position at the corresponding time; Calculating a probability of the object not being visible at the pixel position from the filter at the corresponding time; And Steps 2 to 3 multiplying the probability obtained in step 2-1 and the probability obtained in step 2-2; Multi-object tracking method using a particle filter comprising a. The method of claim 1, The method of tracking a multi-object using a particle filter, characterized in that another filter is generated to track another in real time when another object is generated. The method of claim 1, When one of the objects disappears, the multi-object tracking method using a particle filter, characterized in that the filter for tracking it in real time is removed.

Images