KR101404653B1 - Template Reconstruction Method for Correlation Video Tracking - Google Patents

Abstract

The present invention relates to a reference image reconstruction method for correlation video tracking, through which tracking can be stably and accurately performed by removing clutter and noise from a stored reference image, and promptly renewing target information if a target is rotated such that the shape of the target is changed. The method includes the steps of matching templates extracted from a basic reference image and an input image; renewing a reconstruction reference image with an input template produced through the addition of target images extracted from the input image according to the maximum element value of a correlation matrix produced through matching of the templates; and comparing the renewed reconstruction reference image with a basic reference image at a predetermined time point to select a reference image excellently reflecting the current target shape as a basic reference image.

Description

{Template Reconstruction Method for Correlation Video Tracking}

The present invention relates to a reference image restoration method for correlated tracking that can maintain stable and accurate tracking by quickly updating target information when a target is rotated and the shape is changed by removing clutter and noise from the stored reference image.

A correlation tracker (hereinafter referred to as a tracker) calculates the position of a target on an input image based on a target image stored in a memory and extracts information on the target. The target image stored in the tracker is called a reference template. The tracker continuously reflects the target image extracted from the input image to the reference image, and maintains the reference image close to the current target shape.

In general, a method of quickly reflecting a target shape having a variable shape and direction on a reference image is to replace the reference image with the current target shape. However, when this method is used, the target tracking is likely to fail if the current target shape is severely contaminated with clutter and noise. Therefore, when the current target shape is reflected in the reference image, it should be reflected at a low rate and designed to be insensitive to clutter and noise.

A series of processes for reflecting the current target shape on the reference image is called a reference template update method and can be expressed as the following equation (1).

.............식(1)

(1)

)의 범위는

로서 0에 가까운 작은 값이다. t와 b는 이미지 픽셀값의 2차원 행렬로서,

은 n번째(현재) 프레임에서 사용된 참조 영상이고,

은 n번째 입력 영상에서 표적 영역을 추출한 정합 영상(matched template)이고, 상기

은 다음 프레임(n+1)에 사용될 참조 영상을 나타낸다. 또한, 상기

는 n번째(현재) 프레임에서 참조영상(

)과 정합영상(

)의 일치 정도, 즉 유사도를 나타내는 척도로서 템플릿 정합(template matching)으로 생성되는 값이다. 상기 가중치(

)와 유사도(

)는 모두 현재의 표적 형상을 참조 영상에 반영하는 속도를 결정하는데, 자동으로 결정되는

와 달리

는 개발자 혹은 사용자의 주관에 따라 결정되는 설정치(parameter)이다. Here, the weight (

) Ranges from

Which is a small value close to zero. t and b are two-dimensional matrices of image pixel values,

Is the reference image used in the n-th (current) frame,

Is a matched template obtained by extracting a target area from an n-th input image,

Represents a reference image to be used in the next frame (n + 1). In addition,

(Reference frame) in the n-th (current) frame

) And matching image (

), That is, a value indicating the degree of similarity, and is a value generated by template matching. The weights (

) And similarity (

) All determine the rate at which the current target shape is reflected in the reference image,

Unlike

Is a parameter that is determined by the developer or user's subjective opinion.

1 shows a conventional reference image updating method for updating a reference image repeatedly by reflecting a current target shape on a reference image.

)을 표적의 중심을 기준으로 입력 영상에서 추출된 정합영상(

)으로 갱신하여 다음 프레임(n+1)에 사용될 참조영상(

)을 생성한다. 이후 동일한 방법으로 n+1번째 프레임에서 상기 식(1)을 다시 적용하여 메모리에 저장된 참조영상(

)을 표적의 중심을 기준으로 입력 영상에서 추출된 정합영상(b_n+1)으로 갱신하여 다음 프레임(n+2)에 사용될 참조 영상(t_{n +2})을 생성한다. As shown in FIG. 1, the correlation tracker applies Equation (1) in the n-th (current) frame to a reference image

) Is extracted from the input image with reference to the center of the target

) To be used in the next frame (n + 1)

). Thereafter, the above equation (1) is applied again to the (n + 1) -th frame in the same manner and the reference image

) To around the center of the target by updating the matched image (b _{n + 1)} extracted from the input image and generates the next frame (reference used in the n + 2) Image (t _{n +2).}

)를 작게 설정하여 클러터가 참조영상으로 유입되는 것을 최소화하고, 표적이 회전할 때는 가중치(

)를 크게 설정하여 변화한 표적의 형상을 빠르게 반영하여야 한다. In the reference image update method, minimizing the distortion of the reference image and improving the matching performance are important factors for improving the tracking performance of the correlation tracker. In other words, in environments with high noise and clutter, weights

) Is set small so that the clutter is minimized to the reference image, and when the target rotates, the weight

) Should be set large so that the shape of the changed target should be reflected quickly.

)와 같은 설정치의 의존도가 크므로 경험적 지식에 따라 운용환경에 알맞은 설정치 입력도 중요하지만 구조적으로 설정치의 의존도를 낮추는 것이 최적의 성능을 이끌어내는데 유리하다.The correlation tracker uses the weight (

). Therefore, it is important to input the set value appropriate to the operating environment according to the empirical knowledge, but structurally, lowering the dependence of the set value is advantageous in deriving the optimum performance.

)를 낮게 설정하면 표적이 회전할 때 발생하는 형상 변화를 적시에 참조영상에 반영하지 못하여 추적 성능이 떨어지게 되고, 가중치(

)를 높게 설정하면 변화한 표적의 형상은 빠르게 반영할 수 있으나 잡음 및 클러터의 유입이 증가되는 단점이 있다. However, when updating the reference image,

) Is set to a low value, the tracking performance is degraded because the shape change occurring when the target rotates can not be reflected in the reference image in a timely manner, and the weight

) Is set to a high value, the shape of the changed target can be quickly reflected, but there is a disadvantage in that noise and clutter flow are increased.

FIG. 2 is a view showing a problem of the related art by reference images and reference blocks.

)를 크게 설정한 추적기로 클러터에 접근하는 표적을 추적할 때 추적기에 저장된 표적정보를 나타낸다. Fig. 2 is a graph showing the weight

) Indicates a target information stored in the tracker when tracking a target approaching the clutter.

)가 기준 유사도(

)밑으로 빠르게 떨어져 입력 영상(a)에 클러터의 유입이 차단되나, 표적의 이동속도가 느린 경우에는 (b)와 같이 유입된 클러터가 표적대신 정합되어

가

이상으로 유지되어 참조영상에 클러터의 유입이 가속된다. 이때, 상기

는 참조영상의 갱신을 수행할 때 정합의 신뢰성을 판단하는 기준으로 사용되는 기준 유사도(

)이다. As shown in FIG. 2, when the target moving speed is high, the degree of similarity

) Is the reference similarity (

), The input of the clutter is blocked in the input image (a), but when the target is moving slowly, the incoming clutter is matched to the target as shown in (b)

end

And the flow of the clutter is accelerated to the reference image. At this time,

Is used as a criterion for judging the reliability of the matching when performing the update of the reference image

)to be.

)와 기준 유사도(

)를 조정해야 하는 부담을 가지게 되고 설정치들 간의 고질적인 트레이드-오프로 인해 최적의 설정치 도출이 난항을 겪게 된다. If the weight of the clutter flowing into the reference image (b) is higher than the target, the tracer will recognize the clutter as a target, and the tracking will fail. If the weight of the incoming clutter is low, The size of the target is also distorted, leading to the possibility of tracking failure. As a result, developers and users will endlessly weight

) And reference similarity (

), And there is a difficulty in deriving the optimum set value due to the inherent trade-off between the set values.

Accordingly, an object of the present invention is to provide a reference image restoration method for correlated tracking that can reliably and accurately maintain tracking by rapidly updating target information when the target is rotated and the shape is changed by removing clutter and noise from the stored reference image .

Another object of the present invention is to provide a reference image restoration method for correlation tracking that can minimize the loss of tracking performance due to the accumulation of cluttering successors by adding a robust restoration reference image to the persistent clutter to the correlation tracker .

Also, according to the present invention, by comparing a restoration reference image with a previous reference reference image and selecting a reference image that reflects the current target shape as a reference reference image, a robust restoration reference image is added to the existing correlation tracker Therefore, it is possible to minimize the loss of tracking performance due to the accumulation of clutter that continues.

According to an aspect of the present invention, there is provided a method for restoring a reference image for correlation tracking, the method comprising: matching a template extracted from a reference reference image with an input image; Updating the restored reference image with the input template generated by cumulative addition of the target image extracted from the input image according to the maximum element value of the correlation matrix generated by the template matching; And comparing the updated restored reference image with a previous reference reference image at a predetermined point of time and selecting a reference image that reflects the current target shape well as a reference reference image.

The predetermined point is a point of time when the maximum element value increases and a maximum point is formed, or a point when the target is obstructed by obstacles and then reacquired.

Wherein the step of generating the restoration reference image comprises: measuring a maximum element value of the correlation matrix generated by template matching to determine increase / decrease of the maximum element value; Updating the input template by cumulatively adding the target images matched in the input image to the input template when the maximum element value increases; And stopping the update of the input template when the increase of the maximum element value stops, and updating the restored reference image with the last updated input template.

The selecting step compares the updated restored reference image with a maximum element value of the basic reference image, and selects a reference image having a larger maximum element value as a basic reference image.

)를 높게 설정하는 것을 특징으로 한다.In order to remove the clutter from the restored reference image in preparation for rotation of the target when updating the restored reference image, a fixed value weight value

) Is set to be high.

When the restoration reference image is updated, the residual rate of the past reference image is reduced to reflect the rotated target image.

According to another aspect of the present invention, there is provided a method for restoring a reference image for correlation tracking, comprising: measuring a maximum element value of a correlation matrix generated by template matching, ; Updating the input template by cumulatively adding the target images matched in the input image to the input template when the maximum element value increases; Stopping the update of the input template when the increase of the maximum element value stops, and updating the restored reference image with the last updated input template; And comparing the updated restored reference image with a basic reference image at a predetermined point of time and selecting a reference image that reflects the current target shape well as a basic reference image.

The predetermined point is a point of time when the maximum element value increases and a maximum point is formed, or a point when the target is obstructed by obstacles and then reacquired.

Wherein the selecting of the reference reference image is performed by comparing the updated restored reference image with a maximum element value of a previous reference reference image and selecting a reference image having a largest maximum value as a reference reference image.

The present invention has the effect of improving the target tracking performance of the image tracker by removing the clutter and noise from the stored reference image and updating the target information quickly by maintaining the stable and accurate tracking when the target rotates to change the shape .

)의 변화에 따른 입력 탬플릿의 생성 및 갱신과 복원 참조 영상의 갱신을 나타낸 그래프.
도 4는 기본 참조영상의 대체 발생시 상관도가 도약함을 나타낸 그래프.
도 5는 본 발명에 따른 상관추적을 위한 참조 영상 복원 방법의 흐름도.
도 6은 등속으로 전진하는 표적이 야지와 도심지에서 발생할 수 있는 장애물을 통과할 때 상관 추적기가 산출하는

의 변화를 나타낸 도면.
도 7은 참조영상 복원 방법이 실제로 적용되는 상황을 나타낸 도면.
도 8은 종래 방법 및 종래 방식의 응용 방법이 본 발명에 따른 참조영상 복원 방법과 기능 및 성능 면에서 명확한 차이가 있음을 나타낸 도면.BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram illustrating a reference image updating process according to a related art; FIG.
2 is a view showing a problem of the prior art as a reference image and a reference block.
FIG. 3 shows the similarity (

) And the update of the restoration reference image.
FIG. 4 is a graph showing that the correlation is leaped when replacement of the basic reference image occurs. FIG.
5 is a flowchart of a reference image restoration method for correlation tracking according to the present invention.
FIG. 6 is a graph showing the relationship between a target moving at constant velocity and an obstacle that may occur in the urban area

Fig.
FIG. 7 illustrates a situation in which a reference image restoration method is actually applied; FIG.
FIG. 8 illustrates a clear difference in function and performance from the reference image restoration method according to the present invention in the conventional method and the conventional application method.

)를 표적의 회전에 대비한 값으로 고정한 상관 추적기에 참조영상 복원 기법으로 명명한 클러터 제거 알고리즘을 적용하여,

를 설정치가 아닌 고정치(값)로 사용하고 기존 설정치 기반 방식이 가지는 성능 한계를 극복하는 방안을 제공한다. The present invention is based on weighting (

) Is applied to the correlation tracker, which is fixed to the rotation of the target, and the clutter removal algorithm named as the reference image restoration technique is applied,

Is used as a fixed value rather than a set value, and a method of overcoming the performance limit of the existing set value based method is provided.

The reference image reconstruction method according to the present invention uses a reference image robust against the shape change and an intelligent reference image that blocks inflow of the clutter, simultaneously blocking the inflow of the clutter in response to the rotation change of the target. In order to facilitate the intuitive understanding of the present invention, a reference image robust to the shape change is referred to as a reference reference image, and an intelligent reference image that blocks inflow of clutter is referred to as a restoration reference image. That is, the present invention uses the term 'restoration reference image' in the case of clutter or noise introduced into the basic reference image by the conventional repetitive reference image update to restore the state of the previous good reference reference image.

Hereinafter, a process of updating and replacing the restoration reference image and the basic reference image, which are the core of the reference image restoration technique, with the restoration reference image will be described.

Restoration reference image

이고, 상기 입력 템플릿은 정합영상(

)을 가산하여 생성한다. 상기 입력 템플릿(I)은 정합 영상(

)을 매 프레임(i)마다 누적하하여 생성되며 복원 참조영상 갱신 후에는 삭제된다. The input of the restoration reference image is input template

, And the input template is a registered image

). The input template (I)

) For each frame (i) and is deleted after updating the restored reference image.

)의 변화 패턴에 따라 결정된다. The maximum element value of the correlation matrix in which the update of the restoration reference image and the generation, update, and deletion of the input template occurs,

).

)의 변화에 따른 입력 탬플릿의 생성 및 갱신과 복원 참조 영상의 갱신을 나타낸 그래프이다. FIG. 3 shows the similarity (

) And the update of the restoration reference image.

)를 y로 볼 때, 시간에 따라 변화하는 유사도(

)는 함수

로 근사할 수 있다. 상기 함수에서, 복원 참조영상의 입력 템플릿이 생성되는 시점은 함수 C의 극소점(

)인 (c)점이다. 3, the frame number of the input image transmitted from the image sensor is x, the correlation degree of the reference image (

) Is represented by y, the degree of similarity (

) Is a function

. In this function, the point in time at which the input template of the restoration reference image is generated is the minimum point of the function C

(C).

)의 극대점(

)이 형성되는 (d)에서는 누적된 입력 템플릿을 소정 비율로 복원 참조영상에 더하여 갱신한다. The size of the input template is the size of the reference image, and the sum of the matching images added to the reference image during the (c) - (d) period, that is, the input template is updated. That is, in the period (c) - (d), the input template is updated by adding the matching image,

) Of the maximum point

(D), the accumulated input template is added to the restored reference image at a predetermined ratio and updated.

)의 극소점(c)에서 극대점(d)까지의 구간으로, 상기 유사도(

)가 상승하는(

) 구간을 의미한다. 상기 (a)-(c) 구간은 정합 영상에 클러터가 친입하여

가 감소하는 구간으로, 이 구간의 정합 영상은 버린다. Therefore, the matching image accumulated in the restoration reference image through the input template is similarity (

(C) to the maximum point (d) of the similarity degree

) Rise

) Section. The sections (a) - (c)

And the matching image of this section is discarded.

Since the input template accumulates the matching image in the same manner as the basic reference image update as in the following equation (2), when updating the restored reference image, the restored reference image is reduced using the ratio of the past reference image remaining The input template is added to update the restored reference image.

.......식(2)

... (2)

Here, P _C is the number of frames from the minimum point to the maximum point.

인 구간(a~c)의 정합 영상만 누적 대상에서 제외하여 클러터를 제거할 뿐 표적의 회전변화는 고려하지 않았다. 따라서, 표적의 회전변화 후에 클러터가 친입할 경우에 대비하기 위해서는 복원 참조영상도 회전변화가 발생할 때 다음의 식(3)과 같이 과거 참조영상의 잔존율(β)을 줄여 회전한 표적영상을 빠르게 반영해야 한다.However, the method of updating the restored reference image using Equation (2)

(A to c) are excluded from the accumulation object, and the clutter is removed, but the rotation change of the target is not considered. Therefore, in order to prepare for the case where the clutter is intruded after the rotation change of the target, the reconstructed reference image is obtained by reducing the residual rate (β) of the past reference image as shown in the following equation (3) It should be reflected quickly.

------ Equation (3)

)는 극대점의 값이며, (b)점의 값은 극대점의 값보다 작거나 같다. 즉, 극대점이 높게 형성될 경우에

는 늘어나고 β가 낮아져 빠르게 표적 형상을 변화시킨다. 상기 식(3)의 원리는

의 증가/감소 패턴에 기인하는데 표적이 느리게 회전할 경우에는

구간이 길어지고(유사도가 천천이 감소함) 양의 카운터(

) 구간은 짧아진다. 이는 표적의 변화를

구간에서 꾸준히 반영하였기 때문에

구간이 짧아진 경우이다.3,

) Is the value of the maximum point, and (b) the value of the point is less than or equal to the value of the maximum point. That is, when the maximum point is formed high

Is increased and beta is lowered to rapidly change the target shape. The principle of equation (3)

And the target rotates slowly when the target rotates slowly

The longer the interval (the similarity is reduced by a thousand) and the positive counter

) Section is shortened. This means that changes in the target

Because we constantly reflected in the section

This is the case where the interval is shortened.

구간의 표적의 회전변화를 반영하지 못하고 짧은

구간의 정합 영상만을 누적하여 입력 템플릿을 생성하기 때문에 변화한 표적에 빠르게 대응하지 못하게 된다. The restoration reference image using equation (2)

It does not reflect the rotation change of the target of the section,

Since the input template is generated by accumulating only the matching images of the sections, it can not cope with the changed target quickly.

까지 포함시켜 과거 참조영상의 잔존율(β)을 줄이고 회전변화를 마친 정합영상을 누적한 입력 템플릿(I)을빠르게 복원 참조영상에 반영함으로써 기본 참조영상보다 빠르게 회전변화에 대응할 수 있다.On the other hand, the restoration reference image using equation (3)

It is possible to quickly respond to the rotation change by reflecting the input template I in which the remaining ratio β of the past reference image is reduced and the registered change image after accumulating the rotated change to the reference reference image.

Update and replace the basic reference image

Through the above process, the restored reference image is periodically updated, and the updated restored reference image is replaced with the basic reference image only under a specific condition.

That is, the reference reference image is a reference image that the tracker recognizes as a target shape preferentially, and is an original of the reference image to be matched when calculating the position of the target in the input image. Although the basic reference image and the restored reference image may be matched to the input image as reference images, a basic reference image is used as a reference image to reduce computation load, and a restored reference image is overwritten on the basic reference image under specific conditions over-write).

가 복원 참조영상의

가 기본 참조영상의

보다 큰 경우이다. This particular condition may occur when the target appears after being obscured by the obstacle or at point (d) in Figure 3

Of the restoration reference image

Of the reference reference image

.

FIG. 4 is a graph showing that the correlation is leaped when replacement of a basic reference image occurs.

는 도 4의 (a)처럼 도약(jumping)한다.When a replacement of the basic reference image occurs, a restored reference image having a high degree of correlation is overwritten on the basic reference image,

As shown in FIG. 4 (a).

가 도 6의 (b)와 같이 변화하면 구간에서의

와

의 합은 프레임수와 같지만, 도 (c)와 같이 기본 참조영상의 대체가 발생하여

가 도약하면

는 작아진다. 그 이유는 복원 참조영상의 표적영상이 기본 참조영상보다 현재의 표적형상을 잘 반영하기 때문이다.Therefore, in a situation where there is no substitution of the basic reference image

Is changed as shown in FIG. 6 (b)

Wow

Is the same as the number of frames, but the substitution of the basic reference image occurs as shown in Fig.

When you jump

Becomes smaller. This is because the target image of the restoration reference image reflects the current target shape better than the basic reference image.

5 is a flowchart of a reference image restoration method for correlation tracking according to the present invention.

)을 측정한다. 상기 최대 원소값(

)은 참조 영상과 입력영상간의 유사도를 나타낸다.5, when the input image is transmitted from the image sensor, the correlation tracker extracts the template from the input image and matches the template extracted from the reference image stored in the memory. The maximum trace element of the correlation matrix generated by the template matching value(

). The maximum element value (

) Represents the degree of similarity between the reference image and the input image.

Another embodiment of the present invention is a method for measuring Cmax using characteristic matching such as SIFT (Scale Invariant Feature Transform) or SURF (Speed Up Robust Feature) and an optical search method such as LKT (Lucas-Kanade Tracker) .

를 이전 프레임에서 갱신된 참조영상의

와 비교하여, 상기

의 증감을 판단한다. 판단결과 상기

가 감소에서 증가하면(도 3의 c), 상관 추적기는 입력 영상에서 추출한 표적 영역 즉, 정합영역으로 입력 탬플릿을 생성하고, 상기

가 증가하는 동안(도 3의 c-d구간) 상기 입력 탬플릿에 정합 영상을 가산하여 입력 탬플릿을 갱신한다. The correlation tracker measures

Of the reference image updated in the previous frame

And

. As a result of the judgment

(C in FIG. 3), the correlation tracker generates an input template as a target area extracted from the input image, that is, a matching area,

(Cd section in Fig. 3), the input template is updated by adding the matching image to the input template.

의 증가가 멈추면(도 3의 d), 상기 상관 추적기는 입력 템플릿의 갱신을 중단하고 이제까지 갱신된 입력 템플릿을 복원 참조영상에 반영하여 복원 참조영상을 갱신한다. Then,

The correlation tracker stops updating the input template and updates the restored reference image by reflecting the updated input template to the restored reference image.

를 비교하여 상기 복원 참조 영상의

가 기본 참조영상의

보다 크면, 상기 복원 참조영상을 기본 참조영상으로 대체한다. 다시 말하면, 상관 추적기는 복원 참조영상과 기본 참조영상을 비교하여 현재의 표적 형상을 보다 잘 반영하는 참조영상을 기본 참조영상으로 선택한다. Wherein the correlation tracker is configured to determine whether the restoration reference image and the basic reference image

And outputs the restored reference image

Of the reference reference image

The restoration reference image is replaced with the basic reference image. In other words, the correlation tracker compares the restoration reference image with the basic reference image and selects a reference image that better reflects the current target shape as the reference reference image.

의 증가가 멈출 때 이외에, 복원 참조영상을 갱신하는 도중에 표적이 장애물에 가렸다가 다시 포착될 경우(기억 추적)에도 복원 참조영상과 기본 참조영상을 비교하여 현재의 표적 형상을 보다 잘 반영하는 참조영상을 기본 참조영상으로 선택하는 동작을 수행한다.In addition,

In addition to stopping the increase of the reference image, the reference image is compared with the reference reference image even when the target is obstructed by the obstacle while the restoration reference image is being updated (memory tracking) As a reference reference image.

의 증감을 관측하여 각 처리 순서를 결정한다. 는 매 프레임에 해당하는

의 값을 메모리에 저장하고 앞뒤 프레임과 비교하여 증가, 감소, 극대 및 극소를 판단하다, 각 프레임의

가 저장된 메모리 공간은

의 계산 후에 시스템으로 반환하여 메모리 포화를 막는다.In this way,

And the order of processing is determined. Corresponds to every frame

Is stored in the memory, and compared with the front and back frames to judge increase, decrease, maximum and minimum,

Is stored in

And returns to the system to prevent memory saturation.

를 산출해야 한다. The reference image updating method is performed independently as a whole, but is synchronized with the entire system in the replacement of the basic reference image. Since the replacement of the basic reference image must be completed before the matching of the next frame is performed, real-time property is required and efficient calculation is required. Since it is necessary to first determine whether to overwrite the restored reference image with the basic reference image, the reference image of the restored reference image is matched with the input image,

.

의 생성을 위해 기본 참조영상과 동일한 계산식을 사용한다. 만약 기준영상이 크다면 서브 샘플링을 수행할 수 있으며 코너 특징자를 이용하여 기준 영상을 축소할 수 있다. 기본 참조영상의 대체에서 정확한 정합보다 빠른 정합을 우선하는 이유는 대체 시점이 간헐적이고 빠른 정합으로 산출한

가 기존의

와 비교했을 때 충분히 커야

의 도약을 기대할 수 있기 때문이다.Considering that the calculation delay is caused by the added matching, the matching of the reference image extracted from the restored reference image is performed using a fast calculation formula such as MAD (Mean Absolute Difference), and the searching method is also a fast method such as TSS (Three Step Search) Lt; / RTI > When using the TSS, the 1 to 2-step may use MAD, but in the 3-step

The same calculation formula as the basic reference image is used. If the reference image is large, subsampling can be performed and the reference image can be reduced using the corner feature. The reason for giving priority to matching faster than exact matching in the substitution of the reference reference image is that the alternative time is intermittent and calculated by fast matching

Existing

Is big enough when compared to

Of the world's population.

The specific application range is correlation tracking using IR and CCD image sensor. The reference image used in the correlation tracker is effective for short-term clutter but is vulnerable to long-term clutter. The present invention minimizes the loss of tracking performance due to the accumulation of subsequent clutter by adding a robust restoration reference image to an existing clutter tracker.

The conventional tracking method is called the restoration tracking, and the restoration reference image is added to the restoration tracking.

의 변화를 나타낸다. FIG. 6 is a graph showing the relationship between a target moving at constant velocity and an obstacle that may occur in the urban area

.

의 변화는 특별한 상황을 제외하고는 완만한 곡선 형태를 보이나 직관적 표현을 위해 극소와 극대를 잇는 직선으로 표현하였다. As shown in Fig. 6,

Changes are represented by gentle curves except for special circumstances, but they are expressed in a straight line connecting the extreme and the maximum for intuitive expression.

가 도약하여 일반 추적에 비해

를

이상으로 회복하는 시점이 빠르다. 또한, 극대점이 높은 만큼

가

이하로 떨어지는 시점이 일반 추적보다 느리다.

는 참조영상 갱신을 수행할 때 정합의 신뢰성을 판단하는 기준으로 사용되며 추적을 유지하는 동안

가 높게 형성될 때, 신뢰성있는 추적을 유지한다고 볼 수 있다. (b)에서 일반 추적과 복원 추적의

는 모두

이상을 유지하나 복원 추적의 경우

가 도약하여 평균

가 일반추적에 비해 높게 나타난다. 6 (a), for example, restoration tracking through updating of the restoration reference image

Compared to regular tracking

To

Or more. Also, as long as the maximal point is high

end

Is slower than normal tracking.

Is used as a criterion for judging the reliability of the matching when performing the reference image update and while maintaining the tracking

It can be seen that reliable tracking is maintained. (b) of the generic tracking and restoration tracking

All

But in case of restoration tracking

The average

Is higher than that of general tracking.

FIG. 7 shows a situation in which the reference image restoration method is actually applied.

In FIG. 7, (a) represents a man-made structure, and (b) represents a target located in the field of view. In this case, the reference image must be prevented from being distorted by using a restoration reference image. (c) is a case of passing through obstacle similar to the target, and when the restoration reference image is not used, the reference image is distorted and the obstacle can be recognized as the target. (d) is a case where a target shape is blurred by a heat source such as a flame in an image tracking using an infrared (IR) image sensor. When the outline of the target is weak, the distortion of the fine reference image degrades the overall tracking performance. Therefore, the target image is continuously restored using the restored reference image.

FIG. 8 is a diagram showing a clear difference in function and performance from the reference image restoration method according to the present invention according to the conventional method and the conventional application method.

)를 사용하기 때문에 표적의 형상변화와 클러터 친입에 대한 대응력이 떨어진다. 이에 대한 해결책으로 가중치(

)가 다른 다수의 참조영상을 사용하거나 적응적으로

를 변화시키는 응용방법이 있을 수 있으나, 이 방법은 새로운 방식의 개발이기 보다는 구현 방식을 달리한 것일 뿐 근본 원리가 같아 큰 폭의 성능 향상을 기대할 수 없다. Referring to FIG. 8, a conventional reference image updating method includes a single weighting

), It is difficult to cope with the shape change of the target and clutter penetration. As a solution to this problem,

≪ / RTI > uses a number of different reference images or adaptively

However, this method is different from the implementation method rather than the development of the new method, but the fundamental principle is the same, so that the performance improvement can not be greatly improved.

를 크게 설정하여도 클러터의 친입을 탐지하는 양호한 참조 영상을 복원하여 적은 량의 클러터의 유입도 차단함으로써 표적 추적 성능을 향상시킬 수 있다. It is necessary to develop an algorithm for independently satisfying each of the trade-off relation between the contrast of rotation of the target and the minimization of the clutter inflow, and the reference image restoration method according to the present invention is applicable to the case of covering or rotating the target

It is possible to improve the target tracking performance by restoring a good reference image for detecting the clutter of the clutter and blocking the inflow of a small amount of clutter.

As described above, the present invention improves the target tracking performance of the image tracker by removing the clutter and noise from the stored reference image and updating the target information quickly by maintaining the stable and accurate tracking when the target rotates to change the shape. .

Also, according to the present invention, by comparing a restoration reference image with a previous reference reference image and selecting a reference image that reflects the current target shape as a reference reference image, a robust restoration reference image is added to the existing correlation tracker Therefore, it is possible to minimize the loss of tracking performance due to the accumulation of clutter that continues.

The method of restoring a reference image for correlation tracking according to the present invention described above can be applied to a configuration and a method of the embodiments described above in a limited manner without limiting the technical ideas or essential features It will be understood that the invention may be embodied in other specific forms. Therefore, it should be understood that the above-described embodiments are to be considered in all respects as illustrative and not restrictive.

Claims (7)

Matching a template extracted from a basic reference image with an input image;
Updating the restored reference image with the input template generated by cumulative addition of the target image extracted from the input image according to the maximum element value of the correlation matrix generated by the template matching; And
And comparing the updated restored reference image with a maximum element value of the previous basic reference image at a predetermined point of time to select a reference image having a largest maximum element value as a basic reference image. 2. The method according to claim 1,
Wherein the maximum value is increased to form a maximum point or a target is obstructed and re-captured. 2. The method of claim 1, wherein generating the restoration reference image comprises:
Measuring a maximum element value of the correlation matrix generated by the template matching and determining whether the maximum element value is increased or decreased;
Updating the input template by cumulatively adding the target images matched in the input image to the input template when the maximum element value increases; And
And stopping the updating of the input template when the increase of the maximum element value stops, and updating the restored reference image with the last updated input template. delete 2. The method according to claim 1, wherein, in updating the restoration reference image, a fixed value weight is set to a high value in order to prepare for rotation of the target and remove the clutter from the restoration reference image. Reference image restoration method. The method of claim 1, wherein when the restoration reference image is updated, the residual image of the past reference image is reduced to reflect the rotated target image. Matching a template extracted from a basic reference image with an input image;
Measuring a maximum element value of the correlation matrix generated by the template matching and determining whether the maximum element value is increased or decreased;
Updating the input template by cumulatively adding the target images matched in the input image to the input template when the maximum element value increases;
Stopping the update of the input template when the increase of the maximum element value stops, and updating the restored reference image with the last updated input template; And
And comparing the updated restored reference image with a maximum element value of the basic reference image to select a reference image having a larger maximum element value as a basic reference image.

Images