KR101591380B1 - Conjugation Method of Feature-point for Performance Enhancement of Correlation Tracker and Image tracking system for implementing the same - Google Patents

Abstract

More particularly, the present invention relates to a technique for tracking a single shape information of a target in order to secure stable tracking performance in an image having a low shape information level, And a method for predicting the position of a target.

Description

[TECHNICAL FIELD] The present invention relates to a feature tracking method and a tracking method, and more particularly,

More particularly, the present invention relates to an image tracking technology, and more particularly, to a method and apparatus for tracking trajectory information of a target in order to secure stable tracing performance in a low- A method for predicting the position of an actual target using a position, and a video tracking system for implementing the method.

In particular, the present invention relates to a method of combining a target window adjustment technique of a correlation trace that is robust against an image having a small shape information and a clustering technique of a feature point tracking that is robust against blurring of a target, and a video tracking system will be.

In video tracking, it is essential to maintain a stable tracking window in order to track targets located in complex backgrounds. In addition to the purpose of identifying the target in the image, the tracking window enhances the performance of the image processing algorithm for continuous target tracking and provides a reference for controlling the pan-tilt unit equipped with the image sensor.

A template matching method such as correlation tracking and a region segmentation method such as centroid tracking adjust the tracking window using the overall shape of the target and calculate the center of the target. But if the target is covered by an obstacle, the tracking window is reduced and an error occurs between the center of the actual target and the center of the calculated target.

In order to reduce this error, the method of dividing the whole shape information of the target into unit information is divided or patch tracking, and the regional shape information of the target is used to calculate the region of interest (ROI) and the displacement of the target.

The KLT (Kanade Lucas Tomasi) tracker is a popularized local tracking technology using feature points, which has a wide range of applications, but has a problem in that performance can not be guaranteed in an environment where the shape information level is low, such as IR (Infrared) images.

The image tracker is a device that calculates the trajectory of the target in real time using the image information of the target acquired by the image sensor. For example, a video tracker calculates the position of a target in an input image based on the target image stored in the memory and extracts the information of the target. The target image stored in the tracker is called a reference template. The tracker continuously reflects the target image extracted from the input image and maintains the reference image close to the current target shape.

Figure 1 shows a correlation tracker extracting a target image from an input image and storing it in a memory. The tracker first obtains the gradient magnitude of the brightness in the surrounding image of the target designated by the user, calculates the outline information of the target, and adjusts the position and size of the tracking window as shown in (a) of FIG.

When the tracking device receives the next input image from the image sensor, it searches for the target in the input image by matching the target image corresponding to the tracking window. (b), the target image is combined with the target image stored in the current tracking device in (c). This target image is referred to as a reference template update, and the target image for updating the reference image The process is as follows.

The reference image update is a technique of deriving the shape of the target in the image input from the image sensor and the reference image stored in the tracking device to be the same in consideration of the change of the posture of the target and the shape change. The method of quickly reflecting the inputted target image to the reference image is to replace the reference image with the current target image. However, in case that the current target image is severely contaminated by clutter and noise, To the reference image at a low rate.

Therefore, a series of processes reflecting the current target shape on the reference image is called a reference template update method and can be expressed as the following equation.

의 범위는

이며 0에 가까운 작은 값이다.

와

는 행렬성분이 이미지 화소값인 2차원 행렬로서,

은

번째(현재) 프레임에서 사용된 참조영상이고

은

번째 입력영상에서 표적영역(Region Of Interest, ROI)을 추출한 정합영상(matched template),

은 다음프레임에서 사용될 참조영상이다. 표적영역(ROI)은 입력영상에서 표적이 포함된 영역을 의미한다.here,

The range of

Which is a small value close to zero.

Wow

Is a two-dimensional matrix in which matrix elements are image pixel values,

silver

(Current) frame, and

silver

A matched template obtained by extracting a region of interest (ROI) from the input image,

Is a reference image to be used in the next frame. The target area (ROI) means an area including a target in the input image.

는

과

이 일치한 정도를 나타내는 척도이며 템플릿 정합(template matching)으로 생성되는 수치이다.

와

는 모두 현재의 표적형상을 참조영상에 반영하는 속도를 결정하지만, 자동으로 결정되는

와 달리

는 개발자 혹은 사용자의 주관에 따라 결정되는 설정치(parameter)이다.

The

and

Is a measure of the degree of agreement and is a number generated by template matching.

Wow

All determine the rate at which the current target shape is reflected in the reference image,

Unlike

Is a parameter that is determined by the developer or user's subjective opinion.

1 is a view for explaining a reference image update technique. FIG. 1 shows a process of repeating the updating of the reference image. It is shown that minimizing the distortion of the reference image and increasing the matching performance are methods for increasing the overall tracking performance.

As shown in FIG. 1, the erroneous reference image recognition of the correlation trace and the drift of the reference image can be mitigated by a recursive reference template update. However, due to two factors, impossible.

First, the background pixels (i.e., pixels) are introduced into the reference image and the tracking device recognizes the clutter as a target instead of the object designated as the target. Second, even with a precise pan-tilt control algorithm, the target is off center of the screen.

Also, when a feature point corresponding to a target is matched correctly in a continuous input image, if the feature point is called a valued feature point, the performance of the feature point tracking is determined by the number and distribution of the effective feature points based on the following .

First, in order to calculate the accurate movement amount of the target in the minutia tracking, it is necessary to secure enough effective minutia points to increase the reliability of the median values of minutiae.

Second, since the criterion for setting the ROI in the minutiae tracking is determined by the distribution of the effective minutiae points, it is possible to set the ROI including the entire target shape when the minutiae points are evenly distributed over the target. In addition, since the feature points to be tracked within the ROI area are updated when the rotation or size of the target changes, the correct ROI setting affects the tracking maintenance performance.

On the other hand, low-resolution images and infrared rays have less local feature information compared to general color images or black-and-white images. In particular, infrared images are scattered in the atmosphere, and the outline is not clear. The infrared image shows a monotonous shape using only temperature information, and the shape is often distorted due to sensitivity to temperature change. This phenomenon is even more pronounced when collecting target information located at a remote location, and most of the currently developed thermal tracing equipment is the reason for applying correlation tracking technology instead of feature tracking.

1. Korean Patent No. 10-1087592 2. Korean Patent No. 10-1348680

1. Seung-Yun Lee et al., "High-speed Track Merge Technique for Multi-Target Tracking in Complex Environments" Journal of Military Science and Technology 15 (4)

According to the above background, the feature point tracking is not efficient in terms of cost and performance in an infrared image which can not secure an effective feature point, and it is desirable to apply it as an auxiliary function of correlation tracking in order to maximize the performance of the apparatus using the feature points.

Therefore, the present invention has been proposed in order to solve the problem according to the above background art. In the infrared image which is difficult to obtain effective feature points, the feature point tracking is not efficient in terms of cost and performance. In order to maximize the performance of the apparatus using the feature points And to provide a method of utilizing a feature point for improving the performance of a correlation tracker applied as an auxiliary function of correlation tracking and a video tracking system implementing the feature point.

In order to achieve the above-mentioned object, in order to achieve the above-mentioned object, the feature point tracking is not efficient in terms of cost and performance in an infrared image hard to secure effective feature points, and is applied as an auxiliary function of correlation tracking in order to maximize the performance of the apparatus using the feature points. It provides a method to utilize feature points to improve the performance of correlation tracker.

The feature point utilizing method includes:

a) template matching a target image to an input image;

b) calculating correlations and target pixel numbers in a target area generated by template matching;

c) determining blindness of the target using the calculated correlations;

d) determining whether the occlusion of the target is partial occlusion or total occlusion using the calculated target pixel counts; And

and e) adjusting an actual target tracking window of the target area based on whether or not the target is occluded.

The step d) may include extracting feature points of the target in the target area, and matching valid feature points with the corresponding feature points in the next input image to determine valid feature points. And adjusting an actual target tracking window of the target area using the relative positions of the center of the screen and the effective feature points of the input image.

The actual target tracking window may be adjusted by controlling the control speed of the pan tilt unit according to coordinate information.

In addition, the control of the pan tilt unit may be performed using a PID (Proportional Integral Derivative) control method.

In addition, the step b) may include filtering the target pixel numbers using any one of an alpha-beta filter, a Kalman filter, an FIR filter, and an IIR filter.

In addition, the step c) may include a step of determining a partial or full occlusion using the rate of change of the number of pixels of the target pixel count.

Calculating center coordinates of the actual target tracking window if the partial clipping is performed; And comparing the maximum correlation among the correlations belonging to the real target tracking window with a preset reference value to determine the partial or complete occlusion.

(여기서, r_k는 화소수 변화율이고, k는 현재 시간이고,

는 노이즈가 제거된 화소수 변화속도이고,

는 노이즈가 제거된 표적 화소수를 나타낸다)로 정의되는 것을 특징으로 할 수 있다.Further, the pixel number change rate is calculated by the following equation

(Where r _k is the pixel number variation rate, k is the current time,

Is the pixel number variation rate at which noises are removed,

Represents the number of target pixels from which noise has been removed).

'에 해당한다.Also, the correlation diagrams may indicate the similarity between the stored target image and the target image of the input image. 1, the stored target image represents a 'reference image' in the description of FIG. 1, and the similarity between the reference image and the target image of the input image is'

'.

The step (c) may further include the step of calculating a center coordinate of the tracking window if the target is not covered.

On the other hand, an embodiment of the present invention provides an image sensor comprising: an image sensor; The target image is template-matched to the input image from the image sensor, the correlation values and the target pixel numbers are calculated in the target area generated by the template matching, and the blindness of the target is determined using the calculated target pixel numbers A signal processor; A controller for adjusting an actual target tracking window of the target area based on whether or not the target is occluded; And a pan tilt unit for driving the image sensor to adjust an actual target tracking window of the target area under the control of the control unit.

According to the present invention, a stable tracking window is maintained by combining the advantages of the correlation tracking tracking window adjustment technique with excellent performance in an image having a small amount of shape information and a high contrast ratio, and robust feature point tracking to cover the target.

Another advantage of the present invention is that the control of the pan-tilt unit equipped with the image sensor can be stably maintained so that the target can be stably trapped when the fast-moving target completely covers the target.

1 is a conceptual diagram showing a general correlation tracking process.
2 shows a general correlation tracking window adjustment technique.
3 shows a general feature point extraction method and feature point movement amount measurement characteristic.
FIG. 4 shows a method of measuring movement amount and ROI of general feature point tracking.
5A to 5C are conceptual diagrams illustrating a structure of an actual target tracking window according to an embodiment of the present invention.
6 is a graph illustrating experimental results of an actual target tracking window according to an embodiment of the present invention.
7 is a block diagram of a video tracking system 700 that utilizes feature points for improving performance of a correlation tracker according to an exemplary embodiment of the present invention.
FIG. 8 is a flowchart illustrating a blur tracking process for utilizing feature points for improving performance of a correlation tracker according to an exemplary embodiment of the present invention. Referring to FIG.
FIG. 9 shows an example in which an actual target tracking window technique and a pan-tilt control technique are applied according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

Like reference numerals are used for similar elements in describing each drawing.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. The term "and / or" includes any combination of a plurality of related listed items or any of a plurality of related listed items.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Should not.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout.

2 shows a general correlation tracking window adjustment technique. Referring to FIG. 2, a template 200 is an outline image of a target stored in a tracker, and is expressed by a gradient magnitude to emphasize an outline between a target and a background.

는 휘도 기울기 세기를 윤곽 영상으로 표현한 영상이고,

와

는 각각

와

축의 픽셀 좌표를 의미한다. 또한,

와

는 가우시안 스무딩(Gaussian smoothing) 필터로 노이즈를 제거한 표적 영상에 Sobel 마스크를 적용한 영상으로,

는 가로 방향의 엣지 성분을,

는 세로방향의 엣지 성분을 표현한다.here,

Is an image in which the luminance gradient intensity is expressed by an outline image,

Wow

Respectively

Wow

Means the pixel coordinates of the axis. Also,

Wow

Is a Sobel mask applied to a target image with noise removed by a Gaussian smoothing filter,

An edge component in the horizontal direction,

Represents an edge component in the vertical direction.

,

,

는 각각 세 영역의 평균 휘도 세기이다. 추적창의 4방향의

,

,

을 아래와 같이 비교하여 표적의 윤곽을 탐색하고 추적창을 조절한다.With continued reference to Figure 2,

,

,

Are the average luminance intensities of the three regions, respectively. The four-way

,

,

Is compared with the following to search the contour of the target and adjust the tracking window.

: 추적창 유지 a)

: Keep Tracking Window

이고

: 추적창 축소b)

ego

: Collapse Trace Window

이고

: 추적창 확대c)

ego

: Expanding the tracking window

 There are various local features for tracking, and it is very important to decide which features to use for tracking. A point having a large differential coefficient due to a sudden change in brightness value is a point existing around an edge in an image and a point where a large differential value is orthogonal to each other is called a corner.

The difference of the brightness value between the pixel and the surrounding pixels using the characteristic window is as follows.

는 도 3의 (a), (b), (c)에 나타낸 윈도우(300)다. 변화량을 나타내는 함수

의 값은 평탄한 영역에서는 낮은 값을 갖고, 코너 주변에서는 값이 커진다.

위치에서

만큼 이동된 위치에서의 밝기 값

은 테일러 급수를 이용해 다음과 같이 근사화할 수 있다.here

Is a window 300 shown in Figs. 3 (a), 3 (b), and 3 (c). A function representing the amount of change

Has a low value in a flat area and a large value around a corner.

From location

Brightness value at the moved position

Can be approximated as follows using Taylor series.

Substituting the above equation into the equation (2), it is as follows.

은 다시 다음과 같다.then,

Is again as follows.

의 고유값(eigen value)을

와

라고 한다면

와

의 값에 따라 해당 화소가 평탄한 영역인지, 경계선인지, 혹은 코너인지 구별된다. 만약

와

가 모두 작은 값을 가진다면, 이는 해당화소가 평탄한 점에 속한다고 볼 수 있다. If the matrix

The eigenvalue of

Wow

If you say

Wow

A boundary line, or a corner, depending on the value of the pixel. if

Wow

Is small, it can be said that the pixel belongs to a flat point.

와

중 하나는 크고 하나는 적은 값을 가진다면 이는 경계선 위에 있는 점이라고 할 수 있다. 마지막으로

와

모두 큰 값을 가진다면 주변 화소와 비교했을 때 모든 방향으로 차이가 많이 난다는 것을 의미하므로 코너에 위치한 점이라고 할 수 있다.if

Wow

If one of them is large and one has a small value, it can be said to be a point on the boundary line. Finally

Wow

If all of them have a large value, it means that there is a lot of difference in all directions when compared with surrounding pixels.

는 화소 좌표

,

의 시간

에서의 밝기 값을 의미하며,

,

는 각각

와

축으로의 이동량을 나타낸다.Optical flow is a common way of estimating object movement using corners. The optical flow estimates the amount of movement of a pixel using the surrounding brightness information of the corresponding pixel (i.e., pixel) on the assumption that the brightness value of a pixel belonging to an object is always constant even if the frame is changed. This can be expressed by the following equation

Pixel coordinates

,

Time of

Quot ;, " brightness "

,

Respectively

Wow

Represents the amount of movement to the axis.

The dashed lines in FIGS. 3 (d), 3 (e) and 3 (f) represent the pre-movement plane 310 (ie, the previous image), and the post-movement plane 320 (ie, the current image) is represented by a straight line. The circle means the window 302 to be estimated, and it tries to calculate the optical flow of this part.

The window 302 including the corners of FIG. 3 (d) will not have much difficulty in estimating the optical flow and will be able to estimate the portion such as the estimated area stably. FIG. 3 (e) can not calculate an accurate optical flow because an attempt is made to calculate the optical flow of the edge region, and the corresponding region of the same shape occurs on the edge on the edge. In FIG. 3 (f), the optical flow can not be found because it is a smooth area having no texture.

As shown in FIG. 4, the feature point tracking extracts feature points within a region of interest (ROI) region and moves ROIs by searching corresponding feature points in the input image. That is, the pre-movement ROI region 401 and the post-movement ROI region 402 are shown on the left side of FIG. 4 (a) for easy understanding.

In addition, when the target is changed in posture or size, the ROI size should be adjusted using the distribution of the feature points in the ROI. Feature points that determine the size of the ROI have a small forward-backward (FB) error and vectors similar to the travel of the entire target.

축 값들의 중앙값,

축 값들의 중앙값을 이용하여 생성한다.The FB error can be obtained by performing backward tracking and the amount of movement of the entire target can be obtained as shown in the right part of FIG. 4 (a), among the vector components of the feature points located in the ROI

The median of the axis values,

Axis values.

4 (b) shows the feature points in the ROI region.

In contrast, the correlation tracking measures the amount of movement of the target using the entire or partial shape of the target, so that the target can accurately measure the amount of movement of the actual target without blocking the target on the input image. In other words, in order to measure the movement amount of an actual target in a terrestrial environment in which various types of occlusion exist, it is necessary to divide and track the target. The minutiae are alternatives for dividing the target, and each minutiae has a relative positional relationship with the center of the target.

The purpose of applying the feature point to correlation tracking is to divide and track the target. In order to effectively divide the target image, it is desirable to induce the feature points to be evenly distributed within the target image.

영역)에 표적의 윤곽이 위치하도록 추적창을 조절하여 표적의 일부가 가릴 경우, 가린 위치의 추적창이 축소된다. 가림에 의한 추적창의 축소는 표적의 중심 위치를 왜곡하여 추적 성능을 크게 떨어뜨리는 요인이 된다.Next, an actual target tracking window will be described with reference to FIG. The tracking window used in the existing correlation tracking is located between the boundary 510 of the tracking window and the inner boundary 520

If a part of the target is obscured by adjusting the tracking window so that the outline of the target is located in the area, the tracking window of the obscured position is reduced. The shrinking of the tracking window due to occlusion distorts the center position of the target, thereby deteriorating the tracking performance significantly.

When the center of the target for judging the trajectory of the target is considered as the center of the tracking window, the real target tracking window adds minutia information to the existing tracking window to minimize the center distortion, thereby preventing the reduction of the tracking window due to occlusion.

In OpenCV and MATLAB 2013a and above, when extracting corner feature points, it is possible to extract feature points over the target image by designating the minimum distance between feature points, and it provides various matching algorithms to discriminate the position of feature points in a continuous image.

In the case of feature tracking in combination with feature extraction and matching algorithms in such images, there are various fields such as surveillance reconnaissance and environmental recognition. However, in case of using a single target at a distance target and a low resolution or IR image target, Falls.

) 프레임의 표적영역(500)에서 추출된 특징점을 나타낸다. 도 5b는 추출된 각각의 특징점들이 실표적 추적창(520)에 대응하는 벡터 정보이다. 도 5c에서 벡터

의

과

, 벡터

의

와

는 각각 실표적 추적창(520)의 4변(left, top, right, bottom)까지의 거리(521,522,531,532)를 나타내며, 실표적 추적창의 크기가 변화할 시에 표적의 가림 위치를 판별하여 실표적 추적창을 유지한다.FIG.

) Of the target region 500 of the frame. FIG. 5B shows vector information corresponding to each extracted feature point in the real target tracking window 520. FIG. 5C,

of

and

, Vector

of

Wow

Represents the distance (521, 522, 531, 532) to the four sides (left, top, right, bottom) of the real target tracking window 520. When the size of the real target tracking window changes, Keep the window.

The correlation tracker matches the stored target images from the input image, generates a correlation coefficient matrix of the tracking area, and returns the coordinates and correlation of the elements that are the maximum matching points.

,

프레임 동안

,

,

,

가 변화한 크기를

,

,

,

, 현재(

) 프레임의

의 크기를

라고 했을 때, 실표적 추적창 조절의 조건을 나타내는 Pseudo code는 도 10과 같다.The correlation shows the similarity between the stored target image and the target image of the input image, and is used as a criterion for determining the interval in which the shape change of the target occurs. Maximum correlation

,

During the frame

,

,

,

The size that changed

,

,

,

, Now(

) Of the frame

The size of

The pseudo code indicating the condition of the actual target tracking window adjustment is as shown in FIG.

6 (a) and 6 (b) show the feature tracking, (c) and (f) show correlation tracking, and (b) and (e) are tracking results using the real target tracking window. In case of feature tracking, FIG. 6 (a) shows a result of tracking failure by extracting a feature point from a wrong position when re-extracting the feature points that are subject to tracking due to occlusion, and (d) It is the result of failing. FIG. 6 (c) shows a result that an object similar to the target intersects with the target and enlarges the tracking window, and (f) shows an unstable size of the tracking window due to instantaneous high temperature air generation. FIG. 6 (b) shows a state in which the tracking window is prevented from being enlarged by using the minutia information, and (e) shows that the tracking window is stably maintained even when instantaneous high temperature air is generated.

7 is a block diagram of a video tracking system 700 that utilizes feature points for improving performance of a correlation tracker according to an exemplary embodiment of the present invention. 7, the image tracking system 700 includes an image sensor 740, template matching of the target image to the input image from the image sensor 740, correlation in the target area generated by template matching, A signal processor 710 for calculating the number of pixels and determining whether a target is occluded by using the calculated number of target pixels, a controller for adjusting the actual target tracking window of the target area based on whether or not the target is occluded A pan tilt unit 730 for driving the image sensor to adjust an actual target tracking window of the target area under the control of the controller, and the like.

The signal processing unit 710 includes a tracker 711 for template-matching a target image to an input image input from the image sensor 740, calculating correlations and target pixel numbers in a target area generated by template matching, A filtering unit 712 for filtering the number of target pixels using one of a filter, a Kalman filter, an FIR filter, and an IIR filter, and determines whether the target is occluded by using the calculated number of target pixels, A tracking normal determination unit 715 for determining whether the tracking target is traced, a position calculation unit 715 for calculating positional information such as a center coordinate of the tracking window, a center coordinate of the real target tracking window, a relative position of the center of the screen, And a Proportional Integral Derivative (PID) controller 719 for controlling the pan tilt unit 730 to adjust the position of the image sensor 740 using the calculated position information. It is configured to.

FIG. 8 is a flowchart illustrating a blur tracking process for utilizing feature points for improving performance of a correlation tracker according to an exemplary embodiment of the present invention. Referring to FIG. Further, it is a flowchart showing a process of performing memory tracking using the feature point tracking described above.

When the image tracking is performed with the pan-tilt driving, the tracker calculates the pan-tilt driving speed of the pan tilt unit (730 of FIG. 7) such that the target is positioned at the center of the input image. At this time, the distance between the center of the target and the center of the input image becomes an important input value for determining the pan-tilt driving speed, and it is important to predict the center of the actual target in case the target is obstructed by the obstacle.

Actual target tracking windows can be an alternative to predicting the actual center of a target when the target is partially obscured, but more precise metrology is required when the overall shape of the target is obscured. In other words, when the overall shape of the target is obscured, it is necessary to accurately measure the direction and speed of movement of the target rather than the center of the target, in order to predict the point at which the target is reacquired on the screen.

If the speed of the moving target is fast, the target may be completely obscured in a state where the control is not stabilized after switching to the real target tracking window. In this case, instead of switching to the real target tracking window, the current pan / tilt drive control speed should be stored as a relative position between the feature point and the center of the screen.

That is, if the target is completely covered, the current pan / tilt drive control speed must be maintained in case the target appears again on the screen. In order to judge that the target is rapidly covered, the area change of the target should be measured. The area of the target is Otsu technique, Kittler technique, Kapur technique and Quasi-Laplacian technique which are histogram-based area separation methods.

To calculate the number of pixels belonging to the target (i.e., the number of pixels), a statistical feature of the image in the tracking window is used to determine a threshold capable of separating the target and background. Even if the same target is tracked, the segmentation threshold of this process changes with time depending on the characteristics of the image of the background and the target, and consequently has a decisive influence on the performance of the target segmentation.

In a situation where tracking is normally performed, the number of pixels (i.e., the number of pixels) of the target area causes a temporal perturbation while maintaining a constant average value according to the change of the threshold value. Accordingly, in order to measure the rate of change of the number of pixels within the target, the target division result is largely erroneous when the target division result is directly applied. Therefore, the number of target pixels must be determined after filtering through the filtering (steps S810 and S820) .

Various digital filters such as an alpha-beta filter, a Kalman filter, a finite impulse response (FIR) or an infinite impulse response (IIR) filter can be applied as a method for filtering the number of pixels within a target. The number of pixels in the target that change as the target changes its posture or moves behind the obstacle changes little during the image input time interval for the target tracking and its rate of change can also be assumed to be constant.

If the covariance of the system and the covariance of the measurement are assumed to be constant values, the α-β filter can predict and regenerate the state variable using the steady-state gain instead of the Kalman filter update equation. Estimation prediction and renewal of such an? -Filter are as follows.

는 표적내 화소수이고,

는 화소수 변화속도이다. 또한, 여기서,

는 필터가 수행되는 시간 간격이며,

는 측정치로서

번째 측정된 화소수를 의미한다. here

Is the number of pixels in the target,

Is the pixel number variation speed. Also,

Is the time interval at which the filter is performed,

As a measure

Means the number of pixels measured.

  The coefficients? and? of the? -? filter are calculated by the following equations.

는 아래와 같다.Also, the covariance of the state variable estimates

Are as follows.

The design parameters of the α-β filter are determined by the selection of process noise and measurement noise. The number of pixels in the target is derived from the number of pixels occupied within a field of view (FOV) and the distance between a single target and a target. Therefore, in the shape change of the target image, the change with respect to the height is insufficient and the change mainly occurs in the width.

Measurement noise R is determined by the statistical difference between the application algorithm and the target / background, and can be obtained through experiments using real images. The set Q and R are as follows.

Q = 30 ² ; % variance of process noise

R = 100 ² ; % "measurement noise (standard deviation about 100 pixel error)"

When switching to the automatic tracking mode, the α-β filter for determining the occlusion of the center tracking should be initialized. Immediately after switching to the automatic tracking mode, the number of the first target pixels is set to the initial number of pixels (z ₀ ) of the? -? Filter, and the number of pixels change rate is initialized to zero.

The pixel number change rate is calculated as follows using the number of pixels and the pixel number change rate, which are states of the? -Filter.

여기서, r_k는 화소수 변화율이고, k는 현재 시간이고,

는 노이즈가 제거된 화소수 변화속도이고,

는 노이즈가 제거된 표적 화소수를 나타낸다. 부연하면, v는 현재의 화소수 변화속도이고,

는 노이즈가 제거된 v이다.

는 현재의 표적 화소수이고,

는 노이즈가 제거된

이다.

Here, r _k is the pixel number change rate, k is the current time,

Is the pixel number variation rate at which noises are removed,

Represents the number of target pixels from which noises have been removed. In other words, v is the current pixel number variation speed,

Is the noise removed v.

Is the current number of target pixels,

Noise-canceled

to be.

Accordingly, in step S820, the target pixel number filtering is performed to determine the type of occlusion when the occlusion of the target occurs (step S850). Further, in general, the shape change of the target due to occlusion is faster than the shape change due to the attitude change.

를 이용하여 형상 변화의 속도를 측정하고 표적의 가림이 판정되면 단계 S850에서 가림 정도를 측정한다(표적의 자세 변화 만으로는

가 0.75 이하로 떨어지는 경우가 드물며 떨어진

도 IIR 구조의 참조영상 갱신으로 인하여 빠르게 복구된다).Therefore, first, in step S830,

The speed of the shape change is measured by using the speed of the target and the degree of occlusion of the target is determined in step S850

Is less likely to fall below 0.75,

Is also quickly restored due to the reference image update of the IIR structure).

In the normal tracking situation, the number of pixels in the target area causes perturbations while maintaining a constant average value. On the other hand, in the case of occlusion tracking, the number of pixels in the target rapidly increases or decreases.

In the case of normal tracking, the center coordinates of the tracking window are calculated, and the pan tilt unit (730 of FIG. 7) is controlled by the PID (Proportional Integral Derivative) control method according to the calculated center coordinates (steps S840 and S880).

On the contrary, when the size of the tracking window changes, the target position of the target is discriminated and the actual target tracking window is maintained. If it is determined in step S830 that the tracking is performed in step S830, it is determined whether the pixel is partially or completely blocked using the pixel number change rate (step S850).

If it is determined in step S850 that the pixel number change rate is larger than a preset reference value (for example, -0.5), it is determined to be partial occlusion. In this case, the center coordinates of the real target tracking window are calculated and the pan tilt unit 730 ) Is controlled by the PID control method (steps S860 and S880).

On the other hand, if it is determined in step S850 that the pixel number change rate is less than the preset reference value (-0.5), it is determined to be complete blind. In this case, the relative position between the center of the screen and the minutiae points is calculated, and the pan tilt unit 730 ) Is controlled by the PID control method (steps S870 and S880).

Further, even if the pixel number change rate is larger than the reference value 2.0, it is judged as complete blindness. The rate of change in the number of pixels is greater than 2.0 when the area of the target area is increased more than two times per second. That is, when the target and the background are not smoothly divided and the outline of the target is ambiguous, the matching performance of the target and the tracking window adjustment performance are greatly deteriorated.

The correlation tracker matches the stored target images from the input image, generates a correlation coefficient matrix of the tracking area, and returns the coordinates and correlation of the elements that are the maximum matching points.

FIG. 9 shows an example in which an actual target tracking window technique and a pan-tilt control technique are applied according to an embodiment of the present invention. 9 (a) and 9 (b) show an example in which the tracking window is distorted due to an obstacle. In general, since the target of the IR image has a higher temperature than the background, a strong outline is formed between the target and the background to guarantee the performance of the tracking window algorithm.

However, if the target is covered by sporadic obstacles such as (a) and (b), the image of the dark obstacles will invade into the target image, obscuring the boundary between the target and the background, and degrading the performance of the tracking window algorithm. By applying the feature point utilization method according to the present invention, even if a strong outline of the target and the background can not be secured, the relative position of the tracking window is memorized to prevent the tracking window from being distorted.

One embodiment of the present invention adjusts the position and size of the tracking window so that the target is located at the center of the reference image by correcting the center and size of the target calculated every frame.

In addition, the tracking window adjustment method according to an embodiment of the present invention combines the advantages of the correlation tracking tracking window adjustment technique with excellent performance in the image with low shape information and high contrast ratio, and the advantage of robust feature tracking in covering the target, . Also, the control of the pan / tilt unit equipped with the image sensor is stably maintained so that the target can be stably trapped when the rapidly moving target completely covers.

700: Image tracking system
710:
720:
730: a pan tilt unit
740: Image sensor

Claims (11)

a) template matching a target image to an input image;
b) calculating correlations and target pixel numbers in a target area generated by template matching;
c) determining blindness of the target using the calculated correlations;
d) determining whether the occlusion of the target is partial occlusion or total occlusion using the calculated target pixel counts; And
e) adjusting an actual target tracking window of the target area based on whether the target has occluded,
The step c)
And determining a partial or full occlusion using the rate of change of the number of pixels of the target pixel,
The step d)
Extracting feature points for the target in the target area and matching valid feature points with the next input image to determine valid feature points; And
And adjusting an actual target tracking window of the target area using the relative positions of the screen center of the input image and the effective feature points. delete The method according to claim 1,
Wherein the real target tracking window is adjusted by controlling the speed of the pan tilt unit according to the coordinate information.
The method of claim 3,
Wherein the control of the pan tilt unit is performed using a PID (Proportional Integral Derivative) control method.
The method according to claim 1,
The step b)
filtering the target pixel numbers using one of an alpha-beta filter, a Kalman filter, a finite impulse response (FIR) filter and an infinite impulse response (IIR) filter. How to use minutiae for.
delete The method according to claim 1,
Calculating center coordinates of the actual target tracking window if the partial clipping is performed; And
And comparing the maximum correlation among the correlations belonging to the real target tracking window with a preset reference value to determine that the partial or full occlusion is to be performed.
The method according to claim 1,
The pixel number change rate is calculated using Equation

(Where r _k is the pixel number variation rate, k is the current time,

Is the pixel number variation rate at which noises are removed,

Wherein the number of pixels is defined as the number of target pixels from which noises have been removed.
The method according to claim 1,
Wherein the correlation maps represent similarity between the stored target image and the target image of the input image, and whether the target is occluded by using the difference in similarity is used.
The method according to claim 1,
The step c)
And calculating center coordinates of the tracking window when the target does not occlude the occlusion of the target.
Image sensor;
The target image is template-matched to the input image from the image sensor, the correlation values and the target pixel numbers are calculated in the target area generated by the template matching, and whether the target is blurred using the calculated target pixel numbers is determined A signal processor;
A controller for adjusting an actual target tracking window of the target area based on whether or not the target is occluded; And
And a pan tilt unit for driving the image sensor to adjust an actual target tracking window of the target area under the control of the controller,
The signal processing unit,
And determining a partial or full occlusion using the rate of change of the number of pixels of the target pixel,
Wherein,
The method comprising the steps of: extracting feature points for a target in the target area and matching valued feature points with the corresponding feature points in a next input image; determining, based on the relative position of the center of the input image and the effective feature points, Wherein the real target tracking window of the image tracking system is adjusted.

Images