KR101402206B1 - Multiple target tracking method with kinematics and feature information of targets - Google Patents

Abstract

The present invention relates to a multiple target tracking method, more particularly, to a multiple target tracking method using distance information of a target, feature information-based Kalman filter, and joint probability data-association (JPDA) method. According to the present invention, a JPDA method generates not only normalized distance squared (NDS) distance information but also an effective matrix through dual conditions based on the correlation information of an image. By using this, the identification of other objects, which closely exist, can be possible and the complexity decreases as the number of cases which are taken into account decrease.

Description

[0002] Multiple target tracking methods using kinematics and geometry information are proposed,

The present invention relates to a multi-target tracking technique, and more particularly, to a multi-target tracking method using distance information of a target, a shape information-based Kalman filter, and a JPDA (Joint Probability Data Association) technique .

Multi-target tracking technology is a technology that tracks multiple targets simultaneously based on the results detected from each target. The need for high-precision tracking technology is needed to reduce the trajectory error rate of multiple moving targets and increase the trajectory retention rate to enable real-time accurate multi-target tracking.

The results detected with each target can be obtained from kinematics information such as distance and velocity acceleration of the moving object. Based on the detected results, the trajectory can perform multiple target tracking through a data association technique that defines the relationship between measurements and trajectories, and a filter technique that improves each trajectory over time. The drawing showing this is shown in steps S110 to S140 in Fig.

Therefore, performance of multi-target tracking is different according to the technique of data association technique used. Nearest Neighbor (NN) technique that can perform fast operation and JPDA (Joint Probability Data- Association: association probability data association) technique can be used.

Even if the exact data association technique is used, if several targets move across each other or if several targets exist simultaneously in adjacent areas, distance information alone will cause errors in associating the trajectories corresponding to each target, and the trajectory retention rate will also decrease.

Therefore, it is necessary to utilize not only the distance information but also the shape information of the target detected from the target in order to improve the multi-target tracking performance.

1. Korean Registered Patent No. 10-1280348

1. Jung, Young-Heon et al., "Fast track merging technique for multiple target tracking in complex environments" Journal of the Korean Institute of Military Science and Technology, 15 (4), pp 513 ~ 518,

The present invention has been proposed in order to solve the problem according to the above background art. It is possible to utilize JPDA (Joint Probability Data-Association) by using tracking information and data related information only from the distance measurement information among the detection results of the target, It is an object of the present invention to improve the performance of a data association and thereby provide an accurate multi-target tracking method.

In order to achieve the above-mentioned object, the present invention improves the data associating performance using JPDA (Joint Probability Data-Association) by using tracking information and data related information only from the distance measurement information among the detection results of the target, This provides accurate multi-target tracking.

The multi-target tracking method includes:

A detection step of detecting an image of a target using an image sensor;

An input data generating step of generating position information and image shape information as input data by using the detected image;

Gating to consider only those targets having a distance measure existing within a predefined radius of gates based on the distance information according to the position information among the input data;

Generating effective matrices and association probabilities using targets having distance measurements determined to be valid through gating, image shape information corresponding to the targets, and JPDA (Joint Probability Data Association) technique A data association performing step of performing a data association for the data; And

And an updating step of updating the plurality of temporary trajectories by reflecting the association probability results to the Kalman filter.

In this case, the image shape information may be shape similarity information between the targets, which is generated by calculating a correlation value using the image size value of the detected image.

The updating of the plurality of temporary trajectories may be performed by reflecting the association probability result as a weight on the filter gain of the Kalman filter.

Further, the detecting step may include: initializing the wake; And generating a plurality of latent trajectories using the Kalman filter.

Also, the generation of the effective matrix may be performed using NDS (Normalized Distance square) distance and image correlation information as a double condition.

In addition, the association probability generation may be performed only for an event considered through the generated effective matrix.

In addition, the update step may include a step of updating the temporary trajectory based on the information about the gate value of the temporary trajectory in the case of the same track occurrence considering the number of the related events of the target, Removing or managing the management information.

Also, the effective matrix is expressed by the following equation.

는 번째 측정치가 표적

의 게이트내에 존재하면 1 이고, 그렇지 않으면 0 이고,

는 측정치의 이노베이션으로,

번째 측정치와 표적

로 인해 주어지는 측정 예측치의 차이로 정의되며,

는 측정치 이노베이션의 오차 공분산 행렬,

는 게이트를 정의하는 임계치이고,

는 표적

의 이미지 값과,

번째 검출된 이미지의 2차원 이미지 상관도 함수이며,

값은 이 함수에 대한 임계치 값으로 정의되는 것을 특징으로 할 수 있다.(Here,

Lt; RTI ID = 0.0 >

1 < / RTI > if it is in the gate of the transistor,

With the innovation of measurements,

The second measure and the target

, And is defined as the difference between the measured predicted values given by &

Is the error covariance matrix of the measured innovation,

Is a threshold value defining a gate,

Target

Of the image,

Dimensional image correlation function of the first detected image,

Value is defined as a threshold value for this function.

(여기서,

는

번째 측정치가

번째 트랙에 연관되는 어떤 사건

에 대한 복합확률이며,

는 이때 발생하는 사건행렬,

가중치값으로 곱해지는 상관도 결과값으로 정의되는 것을 특징으로 할 수 있다. Also,

(here,

The

The second measure

Any event associated with the ith track

Lt; RTI ID = 0.0 >

Is an event matrix,

And a correlation value that is multiplied by a weight value is defined as a result value.

According to the present invention, the JPDA (Joint Probability Data-Association) technique generates not only NDS (Normalized Distance Squared) distance information but also an effective matrix based on the correlation information of an image through a double condition, Identification of other objects is possible, and the complexity is reduced by reducing the number of cases to be considered.

In addition, as another effect of the present invention, when the association probability is obtained based on the generated effective matrix, the correlation value of each target corresponding to each trajectory is multiplied by the weight value, thereby adding reliability to the target and distance, And increasing the allocation rate.

In addition, as another effect of the present invention, a trajectory is generated through a Kalman filter based on a newly generated association probability. Unlike the case where tracks are managed only with basic detection / non-detection information, In order to reduce errors (gate extension, parallel track generation) that can occur while applying the data association probability, it is possible to set the center of the various objects as the center of the trajectory by newly setting the trajectory elimination condition, or to maintain the trajectory of the same position The problem can be solved.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a general distance information based multiple target flow diagram.
2 is a block diagram of a multi-target tracking system based on shape information and distance information according to an embodiment of the present invention.
3 is a multi-target tracking flowchart based on shape information and distance information according to an embodiment of the present invention.
FIG. 4 is a flowchart illustrating a data association process in which JPDA (Joint Probability Data Association) technique based on shape information and distance information is generated in step S350 of generating an effective matrix and an association probability shown in FIG. 3 FIG.
5 is a conceptual diagram illustrating trap management obtained through the JPDA technique and Kalman filter based on shape information and distance information according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

Like reference numerals are used for similar elements in describing each drawing.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. The term "and / or" includes any combination of a plurality of related listed items or any of a plurality of related listed items.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Should not.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a multi-target tracking method using kinematics and shape information according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

First, in order to facilitate understanding of the present invention, a JPDA technique based on distance information and a target tracking method using a Kalman filter will be briefly described.

Given the set of measurements up to time, the predicted values for the target state vector and the associated error covariance matrix are:

, 즉, 표적의 위치정보로 표현된다.

는 표적의 동적 운동 모델이 대해 이미 알고 있는 상태변환 행렬이고,

는 표적 운동의 과정 잡음의 공분산 행렬이다.Here, the state vector of the target

That is, the position information of the target.

Is the state transformation matrix already known for the dynamic motion model of the target,

Is the covariance matrix of the process noise of the target motion.

시간의 유효측정 영역 내의 측정치 개수를 이라 할 때 각

번째 측정치

와 표적

에 대한 예측치

의 차이인 이노베이션을 다음식과 같이 구한다.

When the number of measurements in the effective measurement area of time is denoted by

Th measurement

And target

Estimates for

Is the difference between the two.

와

번째 측정치에 관한 이노베이션 값

이 게이트내에 존재하면 유효하다고 판단하는 유효 행렬

을 생성한다. Based on the innovation values thus obtained,

Wow

Innovation value for the second measure

If it is present in the gate, an effective matrix

.

는

번째 측정치가 표적

의 게이트내에 존재하면 1 이고, 그렇지 않으면 0 이다.

는 게이트를 정의하는 임계치이다. 획득한 유효 행렬로부터

번째 측정치가 갖게 되는 연관확률을 구하는 식은 이미 널리 알려져 있으므로, 생략하도록 한다. Herein,

The

Lt; RTI ID = 0.0 >

1 < / RTI >

Is a threshold defining the gate. From the obtained effective matrix

The second equation for obtaining the probability of association of the second measurement is already well known, so it is omitted.

들을 구하게 되며,

는

가 표적

에 연관될 확률은 어떤 사건

을 포함하는 모든 복합 사건

에 대한 복합 확률의 합

, 표적

로부터 아무런 측정치도 발생하지 않을 확률로써 다음식과 같이 표현된다. In summary, all possible join events from the obtained valid matrix

However,

The

A target

The probability of being associated with an event

All complex cases involving

Sum of the combined probabilities for

, Target

Is expressed by the following equation as a probability that no measurement will occur.

에 대한 상태 추정치와 이와 관련된 오차 공분산 행렬은 다음과 같다.Target

And the associated error covariance matrix are as follows.

here,

는 이노베이션의 오차공분산 행렬,

은 필터게인,

은 측정잡음의 공분산 행렬을 나타낸다. 또한,

은 측정치와 상태변수의 관계를 나타내는 상태변환행렬로, 적용 시스템마다 다르게 정의될 수 있으며,

는

의 역행렬을 의미한다.here,

Is the error covariance matrix of innovation,

The filter gain,

Represents the covariance matrix of the measurement noise. Also,

Is a state transformation matrix indicating the relationship between the measured value and the state variable, and can be defined differently for each application system,

The

≪ / RTI >

는 이노베이션의 가중합으로 다음식과 같다.And

Is the weighted sum of innovation.

Since the multi-target tracking technique using the conventional JPDA and Kalman filter performs tracking and data association using only the distance information, there is a problem that if the targets are overlapped with each other or exist at a close distance, the tracking accuracy becomes poor.

2 is a block diagram of a multi-target tracking system based on shape information and distance information according to an embodiment of the present invention. Referring to FIG. 2, an image sensor 210 for detecting an image of a target to generate image data of a position and / or shape of a target as input data, an image sensor 210 for detecting a target based on distance information of the detected image, A gating unit 220 for gating to consider only those targets having distance measurements that are within a predetermined radius, a target having distance measurements determined to be valid through gating, image feature information corresponding to the target, and a JPDA technique A data associating unit 230 for performing a data association to generate an association matrix and an association matrix using the correlation matrix, and a navigation data management unit for updating, maintaining and / 240, and the like.

3 is a multi-target tracking flowchart based on shape information and distance information according to an embodiment of the present invention. Referring to FIG. 3, a data association process (steps S310 to S350) through a JPDA technique (step S340, S360, and S370) and a creation and update process of a wake-up via a Kalman filter are performed in the same manner as a conventional tracking system.

In step S321, based on the image detected through the image sensor, positional information of the target and image information of the shape are used as input data. Of course, prior to this, the wake up is initialized and a temporary wake is generated using the Kalman filter (steps S310 and S320).

Only the targets having distance measurements that are within a predefined radius of the gate based on the distance information of the detected image are considered (step S330). The data association technique is performed with the targets having the distance measurement value determined to be valid through gating and the image information corresponding to the target.

In the present invention, the state vector for the target at time t is expressed as follows.

은 기존과 동일한 표적의 위치 정보이며,

는 표적의 형상 정보를 활용하기 위한 이미지 크기값(intensity)에 대한 정보이다. 본 발명에서는 이미지 크기값을 활용하여, 상관도 값을 구함으로써 표적들간의 형상 유사도 정보를 활용한다. here,

Is the location information of the same target as the existing one,

Is information on an image size value to utilize the shape information of the target. In the present invention, shape similarity information between targets is utilized by obtaining correlation values by utilizing image size values.

In step S350, an embodiment of the present invention utilizes image shape information in a general JPDA technique to newly generate an effective matrix and / or an association probability. The newly obtained association probability result is reflected as a weight on the filter gain of the Kalman filter. The trajectory is updated based on the Kalman filter system constants so generated in step S360. In step S370, a trajectory maintenance and / or removal process for the updated trajectory result is performed.

Of course, it is also possible to generate a new track using the Kalman filter using measurements that are not used in the data association technique in the gatewaying step S330.

FIG. 4 is a flowchart illustrating a data association process in which JPDA (Joint Probability Data Association) technique based on shape information and distance information is generated in step S350 of generating an effective matrix and an association probability shown in FIG. 3 FIG. Referring to FIG. 4, the JPDA technique obtains an effective matrix for all possible events based on the state information of all the gauges existing in all the gates, calculates probable probabilities for each event, Probability is obtained.

In one embodiment of the present invention, new effective matrix and association probabilities are generated using image information as well as distance information of measured values that are determined to be valid through gating.

In steps S410 to S430 of FIG. 4, the predicted distance and the current measured distance to generate the valid matrix satisfy the condition regarding the distance existing in the defined gate, and at the same time, Each element value of the effective matrix is set to 1 only when the shape similarity condition having a correlation of at least a threshold value is satisfied through a two-dimensional normalized cross correlation between the target image and the detected image. Otherwise, if the two conditions are not satisfied in step S420, each element value of the valid matrix is set to 0 (step S421). The effective matrix is as follows.

는

번째 측정치가 표적

의 게이트내에 존재하면 1이고, 그렇지 않으면 0이다. 는 게이트를 정의하는 임계치이다.

는 표적

의 이미지 값과,

번째 검출된 이미지의 2차원 이미지 상관도 함수이며,

값은 이 함수에 대한 임계치 값이다. 이 값이 높을수록 표적

와 측정치

가 유사한 형상을 가지고 있다고 할 수 있어, 이는 가까운 거리에 다른 물체가 동시에 존재하여도, 형상정보를 통하여 표적간의 연관성 높은 표적으로 측정치를 할당함으로써 표적-항적의 연관률을 증가시킬 수 있으며, 오할당률도 줄일 수 있게 된다. Here,

The

Lt; RTI ID = 0.0 >

1 < / RTI > Is a threshold defining the gate.

Target

Of the image,

Dimensional image correlation function of the first detected image,

The value is the threshold value for this function. The higher the value,

And measurement

, It is possible to increase the linkage of the target-wake by allocating the measurement to the target with a high correlation between the targets through the shape information even if other objects exist simultaneously at a close distance, .

In addition, the JPDA technique, which considers the number of events occurring in an effective matrix, reduces the complexity of operations by reducing the number of events that may occur due to the increased number of events.

The association probability calculation is possible based on the generated effective matrix. In the conventional JPDA technique, an association probability corresponding to each measurement value is obtained as shown in Equation (4).

In one embodiment of the present invention, since the similarity of shapes indicates a correlation between a target and a measurement, this value is given as a uniform weight to the probability of association with distance. The probability of association in the present invention is obtained by the following equation.

은 수학식 5 및 수학식 6을 통하여, 칼만 필터의 추정값에 연산 적용된다(단계 S440,S450).Association probability

Is applied to the estimated value of the Kalman filter through equations (5) and (6) (steps S440, S450).

5 is a conceptual diagram illustrating trap management obtained through JPDA technique and Kalman filter based on shape information and distance information according to an embodiment of the present invention. 5 is a block diagram of a trajectory management method for generating, improving, and eliminating trajectories for multiple targets according to a preferred embodiment of the present invention mentioned in step S370 of FIG.

The initial generated arbitrary trajectory through the initialization process is called a Potential Track (510). At this time, if two consecutive traces are not detected, or if the generated trajectories exist at a very short distance and simultaneously the multiple measured values under consideration are regarded as the same trajectory, the same trajectory may occur consecutively three times, If this is greater than the specified gate value, the track is removed. These three conditions are set as track removal conditions.

If an effective target is detected at the next time in the potential wake, it is considered to be a valid tentative track (520). Here too, if one of the above three conditions of track removal is satisfied among the various wakes, the track is removed. When a detected result updated by the Kalman filter is continuously obtained at a valid trajectory, it is regarded as a Confirmed Track (530).

This determined trajectory assumes a valid track and stores the memory trace for a short time (540) even if the detection result does not occur at the next time. This is because the probability that a track will be maintained in the determined trajectory is considered to be still valid even if there is no instantaneous result.

Thereafter, if there is a detection result again at the next time, the confirmation trajectory is continuously maintained. Otherwise, if the detection condition is satisfied or the remaining removal conditions are satisfied, the track is removed (550).

210: image sensor
220: gate gate
230:
240:

Claims (9)

A detection step of detecting an image of a target using an image sensor;
An input data generating step of generating position information and image shape information as input data by using the detected image;
Gating to consider only those targets having a distance measure existing within a predefined radius of gates based on the distance information according to the position information among the input data;
Generating effective matrices and association probabilities using targets having distance measurements determined to be valid through gating, image shape information corresponding to the targets, and JPDA (Joint Probability Data Association) technique A data association performing step of performing a data association for the data; And
An updating step of updating a plurality of temporary trajectories by reflecting an association probability result to a Kalman filter;
And a plurality of target tracking methods using kinematics and shape information.
The method according to claim 1,
Wherein the image shape information is shape similarity information between the targets generated by obtaining a correlation value using an image size value of the detected image.
The method according to claim 1,
Wherein updating of the plurality of temporal trajectories is performed by reflecting the association probability results as a weight to the filter gain of the Kalman filter.
The method according to claim 1,
The detecting step may include: initializing the wake; And generating a plurality of latent trajectories using the Kalman filter. ≪ Desc / Clms Page number 20 >
The method according to claim 1,
Wherein the generation of the valid matrix is performed using NDS (Normalized Distance square) distance and image correlation information as a double condition. The method of claim 3,
Wherein the association probability generation is performed only for events considered through the generated effective matrix.
The method according to claim 1,
In the case of the same track occurrence taking into account the number of the associated targets of the target and the number of the associated targets of the plurality of the temporary wakes, the updating step removes the temporary wakes based on the information of the gate value of the wake-up wakes The method comprising the steps of: (a) controlling a plurality of target tracking points by using the kinematic and shape information.
The method according to claim 1,
The validity matrix is:

(Here,

Lt; RTI ID = 0.0 >

1 < / RTI > if it is in the gate of the transistor,

With the innovation of measurements,

The second measure and the target

, And is defined as the difference between the measured predicted values given by &

Is the error covariance matrix of the measured innovation,

Is a threshold value defining a gate,

Target

Of the image,

Dimensional image correlation function of the first detected image,

Value is defined as a threshold value for this function. ≪ RTI ID = 0.0 > 8. < / RTI >
9. The method of claim 8,
The association probability,

(here,

The

The second measure

Any event associated with the ith track

Lt; RTI ID = 0.0 >

Is an event matrix,

And a correlation value that is multiplied by a weight value is defined as a result value.

Images