KR102339727B1 - Robust visual object tracking based on global and local search with confidence estimation - Google Patents

Abstract

A global and local search method performed by a tracking system according to an embodiment comprises: constructing a region proposal network in a multi-scale-based target recognition detector; performing global and local searches by cooperatively combining the DCF-based tracking model and the multi-scale-based target recognition detector; and estimating the location of the target object through the performed global and local searches.

Description

ROBUST VISUAL OBJECT TRACKING BASED ON GLOBAL AND LOCAL SEARCH WITH CONFIDENCE ESTIMATION

The description below relates to object tracking technology.

Visual object tracking is an open and challenging problem, requiring online trackers to be able to track target objects for long periods of time, even in complex scenarios such as target movement and background closure. The differential correlation filter (DCF) excelled in short-term target tracking problems thanks to its circular density sampling mechanism and fast computation through discrete Fourier transforms. However, it tends to deform from the target when it encounters a sharp deformation, rapid movement, or background occlusion. This can lead to incorrect model updates because the tracker searches for the target in the local area of where the target was located in the previous frame. There is no recovery mechanism for re-targeting and re-locating.

The dominant paradigm in modern object detection is based on standard benchmarks such as PASCAL VOC 2007/2012 and a two-stage detector that consistently achieves the highest accuracy on recently challenging COCO data sets. The high-speed R-CNN network takes the full image and object proposal set as input. The network first propagates the input image forward to produce a CNN feature map, and then applies a region of interest (ROI) pooling layer to extract a fixed-length feature vector from the feature map when each object is proposed. Each feature vector is then fed into multiple fully connected layers for final object classification and bounding box regression analysis. For faster training and testing speed, and higher accuracy, a Region Proposal Network has been proposed to propose candidate object bounding boxes in faster R-CNN. It uses shared convolution features for object proposal generation, object classification, and bounding box regression analysis for faster inference.

Therefore, it is necessary to explore the potential weaknesses of the existing state-of-the-art correlation filter (CF)-based tracking methods. In other words, because of the simple synthetic circular sampling mechanism, the target tends to move in extreme cases such as sharp deformation, rapid movement, or background closure. When a tracker movement problem occurs, there is no recovery mechanism for retargeting and relocation, resulting in tracking failures.

We propose a global and local search technique that jointly applies a differential correlation filter (DCF)-based tracking model together with a novel target recognition detector.

A global and local search method performed by a tracking system includes: constructing a region proposal network in a multi-scale-based target recognition detector; performing global and local searches by cooperatively combining the DCF-based tracking model and the multi-scale-based target recognition detector; and estimating the location of the target object through the performed global and local searches.

In the estimating of the location of the target object, a peak-to-sidelobe ratio (EPSR) for measuring the reliability of a tracking model according to a tracking result of the current frame is proposed, and based on the peak-to-sidelobe ratio Determining whether to single-track or joint-track, and update the tracking model for computational efficiency.

In the estimating of the location of the target object, the multi-scale-based target recognition detector generates a similar object candidate with spatial and scale constraints, and distinguishes foreground and background interference through the DCF-based tracking model. It may include reordering the ranking of object candidates for reconfirming objects by performing a role.

The step of estimating the location of the target object may include selecting an object proposal having an object recovery rate equal to or greater than a preset detection score in order to ensure that the tracking object is detected by the multi-scale-based target recognition detector, applying a spatial constraint by calculating the relative distance between the centroid and the centroid of a previously estimated tracking object, and applying a scale constraint by an intersection combination function to the integration function between the selected object proposal and the estimated tracking output. may include

In the estimating of the location of the target object, a target object candidate is generated by applying the spatial constraint and the scale constraint, and a candidate bounding box with a high probability of becoming a tracking target object in a new frame within the generated target object candidate , and calculating the correlation reliability of each candidate bounding box to estimate the final target object.

A tracking system for global and local searches includes: a network builder configured to construct a region proposal network in a multi-scale-based target recognition detector; a search execution unit that cooperatively combines the DCF-based tracking model and the multi-scale-based target recognition detector to perform global and local searches; and a location estimator for estimating the location of the target object through the performed global and local searches.

It can effectively help avoid drift problems in extreme scenarios and can significantly improve tracking robustness without sacrificing execution time.

For future work, more efficient model update methodologies can be explored and collaboration mechanisms can be optimized to further improve performance.

1 is a block diagram for explaining the configuration of a tracking system according to an embodiment.
2 is a diagram illustrating a multi-scale regional proposal network according to an embodiment.
3 is a diagram for explaining an overall process of a global and local search algorithm, according to an embodiment.
4 is an example showing a ground-true bounding box, nine anchor boxes generated by Algorithm 1, and a final object region regressed from the anchor box, according to an embodiment.
5 is a diagram for explaining a space constraint and a scale constraint according to an embodiment.
6 is a diagram illustrating an improved peak-to-sidelobe ratio (EPSR) value and a detection score of DCF, according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

The tracking model proposed in the embodiment performs a local search process with high tracking reliability, and when instability and confidence fluctuations of the tracking model are detected by the proposed tracking system, it retargets and locates the target through global search in the entire frame. To do this, you can run a target recognition detector. In addition, an improved peak-to-sidelobe ratio (EPSR) is designed for reliability estimation to indicate the degree of instability and variation. Accordingly, the local tracking model and the target recognition detector may be jointly applied to the state estimation of the final target and the updating process of the estimation model. This not only avoids model corruption from bad updates, but also avoids moving problems for long-term tracking.

1 is a block diagram for explaining the configuration of a tracking system according to an embodiment.

The tracking system 100 is for performing global and local searches, and may include a network building unit 110 , a search performing unit 120 , and a location estimating unit 130 .

The network construction unit 110 may build a local proposal network in a multi-scale-based target recognition detector.

The search performing unit 120 may perform global and local searches by cooperatively combining a DCF-based tracking model and a multi-scale-based target recognition detector.

The location estimator 130 may estimate the location of the target object through the performed global and local searches. The position estimator 130 proposes a peak-to-sidelobe ratio (EPSR) for measuring the reliability of the tracking model according to the tracking result of the current frame, and single tracking or joint tracking based on the peak-to-sidelobe ratio It can determine whether or not to update the tracking model for computational efficiency. The location estimator 130 performs a role for the multi-scale-based target recognition detector to generate similar object candidates with spatial and scale constraints, and to distinguish foreground and background interferences through a DCF-based tracking model to reconfirm the object. For this purpose, the ranking of object candidates can be rearranged. The location estimator 130 selects an object proposal having an object recovery rate greater than or equal to a preset detection score in order to ensure that the tracking object is detected in the multi-scale-based target recognition detector, and the center point of the selected object proposal and the previously estimated It is possible to apply a spatial constraint by calculating the relative distance between the center points of the tracking object, and a scale constraint by the intersection combination function to the integration function between the selected object proposal and the estimated tracking output. The position estimator 130 generates a target object candidate by applying a spatial constraint and a scale constraint, and indicates a candidate bounding box that is highly likely to become a tracking target object in a new frame within the generated target object candidate, and each candidate The final target object may be estimated by calculating the correlation reliability of the bounding box.

3 is a diagram for describing an overall process of a global and local search algorithm according to an embodiment.

The tracking system according to the embodiment may configure a multi-scale RPN-based target recognition detector in the tracking field. Instead of using a single RPN on top of the last convolutional feature map, we can add one local proposal network on top of the pass-through layer through the second convolutional layer. In this way, the regional suggestion network can learn both higher spatial resolution information and rich differential information to better localize the target object.

The multi-scale RPN-based target recognition detector can process both the fine features of the convolutional layer and the semantic features of the deep convolutional layer. The feature of the shallow layer is excellent for accurate object location estimation, and the feature of the deep convolutional layer can efficiently distinguish between the target object and the background.

The tracking system according to the embodiment may propose a global and local search algorithm that cooperatively combines the DCF-based tracking model and the proposed target recognition detector for long-term object tracking. In addition, the improved peak-to-sidelobe ratio (EPSR) can be designed as a model reliability estimator representing the degree of tracking model reliability. The tracking algorithm switches between global and local searches and local-based searches expressed as EPSR values for computational efficiency.

A tracking system according to an embodiment may establish a collaborative model update mechanism performed as a global and local search process. When tracking reliability fluctuations or model instability are detected, the most probable target candidates proposed by the target recognition detector can be used to update the DCF model instead of the self-supervised update approach applied to other DCF-based tracking algorithms. This can prove important for maintaining model purity and tracking robustness.

A global and local search-based visual tracking operation in the tracking system according to the embodiment will be described. Specifically, Fig. 3 is intended to explain the overall process of the proposed global and local search algorithm, in which the current frame is fed to a multi-scale region proposal network to generate object proposals, and Colorname and Hog functions as representations of DCF-based tracking models. You can truncate the local search region in the current frame for extraction. Post-constraint tracking algorithms can perform object re-identification and model updates in a collaborative manner.

2 is a diagram illustrating a multi-scale regional proposal network according to an embodiment.

It may consist of five convolutional (conv) layers, one pass-through layer following activation of the second convolutional layer, and two regional proposal layers RPN 1 and RPN 2 . Here, the regional proposal layer RPN 1 may be built on top of the pass confirmation mapping layer. RPN 2 may be built on the convolutional feature map of the fifth convolutional layer. RPN 1 can generate regional proposals from a subdivided feature map. Refinement features can help localize small objects. RPN 2 generates a regional proposal from a rough feature map, and this feature map can provide general information. After RPN 1 and RPN 2, a region of interest pooling layer can be applied on top of each ROI position. Then, two fully-connected layers can be used as more precise bounding box regressors. The input to the network is a single scale full frame, and the image can be scaled so that the shorter size = 600 pixels. 2 is an example showing an input of a fixed size of 1000 × 600 pixels.

The regional proposal network may generate regional proposals regressed by anchor boxes. In order to give better precedence to anchor boxes and to make it easier to converge local proposal networks for better predicting tight bounding boxes, we first use a standard K-means clustering algorithm to determine the ground truth boundaries among the training data. A box can be clustered into m (m is a natural number) clusters. Here, m cluster centers in all bounding boxes can be selected as anchor boxes. The training set used in the embodiment includes M (M = 19, 780) ground-true bounding boxes, and these ground-true bounding boxes may have different scales and aspect ratios (aspect ratios). Accordingly, these bounding boxes should be clustered in N clusters in terms of scale and aspect ratio.

로 정의된다. 이때,

, j,

임계값인 모든 그라운드 트루 경계 박스에서 N개의 박스

를 랜덤으로 선택하여 N 중심부를 초기화할 수 있다. 앵커 박스 클러스터링 알고리즘은 다음의 알고리즘 1에 언급되어 있으며, m스케일과 가로 세로비를 가진 앵커 박스를 생성할 수 있다. If b _i denotes the i-th bounding box and c _j is the center of the j-th cluster, then the distance metric is

is defined as At this time,

, j,

N boxes from all ground true bounding boxes that are thresholds

can be randomly selected to initialize N centers. The anchor box clustering algorithm is mentioned in Algorithm 1 below, and can generate anchor boxes with m-scale and aspect ratio.

In Fig. 4, the left image includes 19,780 object-based ground-true bounding boxes from the training set, and the middle image shows the 9 anchor boxes generated by Algorithm 1. Each anchor box represents one cluster centroid, and the image on the right represents the final object generated by the multi-scale regional proposal network.

Table 1 shows the size of the anchor box. The first row is the height and the second row is the width (width) of the anchor box.

Table 1:

A multi-domain learning mechanism of the tracking system according to an embodiment will be described.

The tracking system can design the K branch at the end of the network as a classification layer. Each branch represents one specific video domain, and each domain represents one specific training sequence. Apply the standard stochastic gradient descent (SGD) method in the network training process, and when the training is at the k (k is a natural number) iteration, the k-th branch of the classifier until the network converges or reaches a predefined maximum number of iterations. Only layers can be used to update the network. Specifically, in the case of training of a multi-scale regional proposal network, the objective function can be minimized with a multi-task loss. The multi-task loss function is defined as

Equation 1:

는 분류 손실과 박스 회귀 손실 사이의 중요성을 제어하기 위한 하이퍼 파라미터이다. 실시예에서는 정확한 경계 박스 회귀에 더 많은 초점을 맞추고 있기 때문에 훈련 과정에서

=20으로 설정한 것을 예를 들어 설명하기로 한다. N_cls와 N_reg는 각각 미니 bach 크기와 앵커 위치 수를 나타낸다. 회귀 손실은 p*=1인 앵커에 대해서만 활성화될 수 있다. t_i와 t_i*는 예측된 좌표 파라미터와 양(positive)의 앵커와 관련된 그라운드 트루이다. Here, {p*} represents the label of the anchor box including the object. When IoU (anchor box, ground true box) ≥ 0.7, p*=1. When IoU≤0.3, p*=0. and

is a hyperparameter to control the importance between classification loss and box regression loss. Since the example focuses more on accurate bounding box regression, the training process

A setting of =20 will be described as an example. N _cls and N _reg represent the mini-bach size and number of anchor positions, respectively. Regression loss can only be activated for anchors with p*=1. t _i and t _i * are the predicted coordinate parameters and the ground true associated with the positive anchor.

In order to improve the final bounding box, non-patent document 1<R. Girshick, Fast r-CNN, in: IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1440-1448.> follow the method suggested. Since there is no need to predict the object class during tracking, the learning process can be simplified because the loss function only focuses on bounding box regression. The loss function is

Equation 2:

이다. here,

to be.

Scale-invariant translation of predicted bounding box and ground true for log space height/width shift and object proposal t _i and t _i *(i = x , y , w , h ) are defined as do.

Equation 3:

In this case, x , x* and x _p represent prediction, ground true, and regional proposal adjustment, respectively. Here, x _p is the output result of the local proposal network.

In the embodiment, the maximum number of iterations is 100K, the convolutional layer ^{has a learning rate of 10 -4} , and the fully connected layer ^{has a learning rate of 10 -3} , and the momentum and weight decay during training can be set to ^{0.9 and 5.0 Х10 -4, respectively.} have. During testing, domain-specific layers may be replaced with new classifier layers for test sequences.

The tracking system according to the embodiment may provide a DCF-based tracker through model reliability estimation. Accordingly, we describe a differential correlation filter-based tracking model and propose a new method for model reliability estimation.

를 분류기 필터와 외관 모델을 업데이트하기 위한 학습 샘플로 간주할 수 있다. 구체적으로, x _i 는 프레임 i에서 타겟을 중심으로 한 크기의 M×N의 단일 이미지 패치로서, 그 모든 주기적 이동

,

는 분류기를 훈련시키는 훈련 사례로 사용할 수 있다. 그것들은 Gaussian 함수 y로 라벨이 표시되어 있으므로,

은

의 라벨이다. 필터 f는 다음과 같은 목적 함수를 최소화하여 학습될 수 있다.First, a differential correlation filter-based tracking model will be described. Since it has been proven to yield excellent results for object recognition, object detection, and pose estimation, we can apply color features that indicate the appearance of objects. A correlation filter operator can be applied for a powerful classifier. Patch all target object shapes extracted from frame 1 to frame t to update the tracking model

can be considered as a training sample for updating the classifier filter and appearance model. Specifically, x _i is a single image patch of size M×N centered on the target in frame i, with all its periodic shifts

,

can be used as a training example to train the classifier. Since they are labeled with the Gaussian function y,

silver

is the label of The filter f can be learned by minimizing the following objective function.

Equation 4:

은 커널

에 의해 유도된 Hilbert 공간에 색상 이름(CN)을 매핑하여, 내적을

로 정의한다. 해결책은 입력의 선형 조합

으로 확장할 수 있으며

는 정규화 파라미터이다. 이러한 비용 함수는 다음을 통해 최소화된다.here

silver kernel

By mapping the color names (CN) to the Hilbert space derived by

is defined as The solution is a linear combination of inputs

can be expanded to

is a normalization parameter. This cost function is minimized by

Equation 5:

를 이산 푸리에 변환 연산자로 정의할 수 있다. 분류기의 필터는 푸리에 도메인인

에서 학습되며, 또한 푸리에 도메인의 가우스 라벨을

로 변환할 수 있다. 푸리에 변환된 커널 출력은

로 정의되며, 여기서

로 정의될 수 있다. 현재 프레임 t에서 추적할 타겟 객체의 신뢰 점수는 크기 M ×N의 단일 패치 z에 대한 분류기 응답으로 계산될 수 있다.At this time,

can be defined as a discrete Fourier transform operator. The filter of the classifier is a Fourier domain

is learned from, and also the Gaussian label of the Fourier domain.

can be converted to Fourier transformed kernel output is

is defined as, where

can be defined as The confidence score of the target object to be tracked in the current frame t can be calculated as the classifier response to a single patch z of size M × N.

Equation 6:

와

이다. 이때,

는 복수 개의 프레임에 걸쳐 이전 타겟 외관의 학습된 모델을 나타낸다. 새 프레임의 타겟 위치는 탐지 점수

를 최대화하여 추정할 수 있다.here,

Wow

to be. At this time,

represents the learned model of the previous target appearance over a plurality of frames. The target position of the new frame is the detection score

can be estimated by maximizing

으로 업데이트 되고, 이전의 모든 타겟 외관 저장을 피하고 추적 속도를 최적화하기 위해 비특허문헌 2< M. Danelljan, F. Khan, M. Felsberg , J. Weijer, Adaptive color attributes for re- al-time visual tracking, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1090-1097 .>의 업데이트 개요를 채택할 수 있다.

를

(분자 부분)와

(분모 부분)의 분할 결과로 간주하며,

는 다음과 같이

와

를 별도로 업데이트하여 갱신할 수 있다.As the adversary experiences closure, deformation and sudden movements, the target model must be adaptively updated to match the state of the oncoming target, but also retain the previous target characteristics. Accordingly, the appearance model of the target object has a learning rate

In order to avoid all previous target appearance storage and optimize tracking speed, non-patent document 2 < M. Danelljan, F. Khan, M. Felsberg , J. Weijer, Adaptive color attributes for re-time visual tracking , in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. The update outline of 1090-1097 .> can be adopted.

cast

(molecular part) and

It is regarded as the result of division of (the denominator part),

is as follows

Wow

can be updated separately.

Equation 7:

,

이고, 추정 타겟 상태

는 모델 안정성 및 추적의 강력함을 유지하기 위해 사용될 수 있다.here,

,

, and the estimated target state

can be used to maintain model stability and robustness of tracking.

A model reliability mechanism in the tracking system according to the embodiment will be described. Most existing online learning trackers update the appearance model and correlation filters at each frame with a high risk of model corruption. In particular, if the target object responds to a dramatic appearance change, background occlusion, or fast movement, these factors lead to false updates. A bad up-to-date model can prevent the tracker from tracking the target object in subsequent frames.

는 각 비디오에 대한 첫 번째 프레임의 초기 EPSR 값에 대한 현재 프레임의 EPSR 값의 규모 요인을 나타내는 EPSR 값의 사전 정의된 임계값이며, 실시예에서는

= 0.8로 설정한 것으로 예를 들어 설명하기로 한다. 신뢰도 값이 사전 정의된 임계값

보다 클 경우, 추적 출력을 새 프레임에서 최종 타겟의 위치로 간주할 수 있다. 신뢰도 값이 사전 정의된 임계값

보다 크지 않을 경우, 제안된 타겟 인식 탐지기를 사용하여 객체 재탐지를 수행한 다음, re-ranking 알고리즘을 적용하여 진실 양성 확률을 가진 타겟 객체 후보들을 다시 정렬시킬 수 있다. 각 프레임에서 기준 DCF 추적기는 각 샘플링 패치에 대한 이산 신뢰도 점수

를 추정하며 이상적으로는 한 프레임에 있는 MХN 샘플링 패치 중 하나의 급격한 피크가 있어야 한다. In an embodiment, the improved peak-to-sidelobe ratio (EPSR) shown in FIG. 6 may be proposed to measure the reliability to determine whether the tracking result of the current frame is sufficiently reliable. here

is a predefined threshold value of EPSR value representing the scale factor of the EPSR value of the current frame with respect to the initial EPSR value of the first frame for each video, in the embodiment

= 0.8, which will be described as an example. Thresholds with predefined confidence values

If greater, the tracking output can be considered as the position of the last target in the new frame. Thresholds with predefined confidence values

If it is not greater than that, object re-detection is performed using the proposed target recognition detector, and then target object candidates with true positive probabilities can be rearranged by applying a re-ranking algorithm. At each frame, the baseline DCF tracker scores a discrete confidence score for each sampling patch.

, and ideally there should be an abrupt peak of one of the MХN sampling patches in a frame.

However, when there are two or more peaks in the confidence map, the tracking result is not convincing as the actual detection result. Accordingly, it is possible to explore the confidence detection mechanism of an improved peak-to-sidelobe ratio to remove hints of background occlusion or tracking failure, prevent bad updates, and re-identify target objects with the target-aware detector proposed in the embodiment. The reliability detection value (EPSR) may be defined as follows.

Equation 8:

와

는 신뢰도 점수 S_m,n의 평균값과 공분산을 나타낸다. PVPR은 1차 대 2차 피크 비율을 나타내며, 초 단위 피크까지의 1차 피크의 억제 정도를 나타낸다. PSVR 값이 커질 수록 위치의 정확도가 향상될 수 있다. PSR 값은 피크 대 sidelobe 비율이며, 타겟 객체가 정확히 피크 위치에 존재하는지에 대한 신뢰도를 나타낸다.where S _max , S _max2 are the maximum (main peak) and sub-peak values,

Wow

is the mean value and covariance of the confidence score S _m,n. PVPR indicates the ratio of the primary to secondary peaks, and indicates the degree of suppression of the primary peak up to the peak in seconds. As the PSVR value increases, positioning accuracy may be improved. The PSR value is the peak-to-sidelobe ratio, and indicates the reliability of whether the target object is exactly at the peak position.

A target object position and a model update operation in the tracking system according to the embodiment will be described. A strong tracker must be computationally efficient and have differential power. The tracking system according to the embodiment is modeled as a tracking-by-tracking mechanism by taking advantage of the high object recovery rate of the proposed multi-scale regional proposal network, and the tracker selects all object candidates based on the spatial and scale state and unique pattern of the target object. Distinguish between the target and the non-correct option. The tracking process can be divided into two phases: spatial and scale constraints and object proposal reordering.

의 경계 박스 풀을 생성한다. 각 경계 박스는 객체가 속하는 범주를 나타내는 탐지 점수

로 할당된 타겟성을 제안한다. 기준 j의 하한 임계값을 나타내는

를 이용할 수 있다.

는 타겟성 확률이고,

는 스케일 제약 기준이며,

은 공간 제약 기준이다.

를 이용하여 각각 기준

,

의 하한 임계값을 나타낸다. The tracking system may apply a first step (spatial and scale constraints). The original multi-scale RPN-based target recognition detector D _ta is

Create a bounding box pool of Each bounding box has a detection score indicating which category the object belongs to.

We propose the targetability assigned to representing the lower threshold of criterion j

is available.

is the target probability,

is the scale constraint criterion,

is a space constraint criterion.

Each standard using

,

represents the lower threshold of .

를 가진 객체 제안을 선택하고,

는

의 하한 임계값이며,

로 설정할 수 있다. 후속 프레임에서 추적된 객체가 다음과 같은 이전 위치 근처에 위치해야 한다는 가정에 근거하여 타겟 객체가 나타난 것을 motion smoothness라고 하며,

에 의해 객체 제안의 중심점과 이전에 추정된 추적 객체의 중심점 사이의 상대적 거리를 계산함으로써 공간 제약을 적용할 수 있다. 연속 프레임에서 추적 객체의 스케일 변화는 스케일 범위 일관성을 만족시키기 위하여 특정 범위 내에서 제어되어야 하며, 객체 제안과 추정된 추적 출력 사이의 통합 함수에 대한 교차점 조합 함수

에 의한 스케일 제한으로 정의될 수 있다. 공간적, 스케일 제약을 만족시키는 객체 제안을 매우 유사한 객체 후보

로 유보할 수 있다.Here, the detection score for a high recovery rate to ensure that the tracked object is detected by the target-aware detector

Select the object proposal with

Is

is the lower threshold of

can be set to Based on the assumption that the tracked object in the subsequent frame should be located near the previous position as follows, the appearance of the target object is called motion smoothness,

The spatial constraint can be applied by calculating the relative distance between the center point of the object proposal and the center point of the previously estimated tracking object. The scale change of the tracking object in successive frames should be controlled within a certain range to satisfy the scale range consistency, and the intersection combination function for the integration function between the object proposal and the estimated tracking output

It can be defined as a scale limit by Object proposals that satisfy spatial and scale constraints are very similar object candidates

can be withheld by

Equation 9:

와

는 각각 0.60과 0.50으로 설정된 공간 제약과 스케일 제약의 사전 정의된 하한 임계값이다. 도 5는 공간과 스케일 제약의 프로세스를 나타낸 것이다.here

Wow

are the predefined lower thresholds of the space constraint and scale constraint set to 0.60 and 0.50, respectively. Figure 5 shows the process of space and scale constraints.

를 유지하며, 세트

내의 각 요소는 n에서 새로운 프레임에서 추적 타겟 객체가 될 가능성이 가장 높은 상위 K의 후보 경계 박스를 나타낸다. 그리고 나서, 각 후보들은 외관 모델

와의 상관도 점수 재계산을 위한 검색 영역 R_t ^k를 확립하기 위하여 즉각적인 샘플로 제공되며, 각 후보자들의 상관 신뢰도는

로 계산될 수 있다. 최대 상관도 점수를 가진 후보자는 다음을 통해 최종 추적 타겟 객체로 추정된다.The tracking system may perform a second step (re-identification with target location). After the above-mentioned spatial and scale constraint process, target object candidates generated by global search-based detection

maintain and set

Each element in n represents the top K candidate bounding box that is most likely to be the tracking target object in a new frame at n. Then, each candidate has an appearance model

_{is provided as an immediate sample to establish a search region R t} ^k for recalculating the correlation score with , and the correlation confidence of each candidate is

can be calculated as The candidate with the maximum relevance score is estimated as the final tracking target object through

Equation 10:

라 할 때 다음과 같이 업데이트될 수 있다. We update the tracking model with weighted instant object candidates for model stability and tracking robustness, and the weight is an exponential function of the relevance score. This update mechanism can avoid model corruption and helps the tracking algorithm avoid movement problems. The new target appearance model is

It can be updated as follows.

Equation 11:

The global and local search-based tracking algorithms proposed in the embodiment are presented in Algorithm 2.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. may be embodied in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (6)

A global and local search method performed by a tracking system, comprising:
constructing a region proposal network in a multi-scale-based target-aware detector;
performing global and local searches by cooperatively combining a differential correlation filter (DCF)-based tracking model and the multi-scale-based target recognition detector; and
estimating the location of the target object through the performed global and local searches;
including,
The step of estimating the location of the target object,
Suggests a peak-to-sidelobe ratio (EPSR) for measuring the reliability of a tracking model according to the tracking result of the current frame, and determines whether single tracking or joint tracking is performed based on the peak-to-sidelobe ratio, and calculates The tracking model is updated for efficiency, the multi-scale-based target recognition detector generates similar object candidates with spatial and scale constraints, and foreground and background interference is detected through the differential correlation filter (DCF)-based tracking model. To perform a role to distinguish and rearrange the ranks of object candidates for object reconfirmation, and to ensure that the tracking object is detected by the multi-scale-based target recognition detector, the object recovery rate is higher than a preset detection score Select, apply a spatial constraint by calculating the relative distance between the centroid of the selected object suggestion and the centroid of a previously estimated tracking object; Applying scale constraints by
Global and local search methods including. delete delete delete According to claim 1,
The step of estimating the location of the target object,
A target object candidate is generated by applying the spatial constraint and scale constraint, and a candidate bounding box that is likely to become a tracking target object in a new frame within the generated target object candidate is indicated, and the correlation reliability of each candidate bounding box estimating the final target object by calculating
Global and local search methods including. A tracking system for global and local searches, comprising:
a network construction unit for constructing a region proposal network in a multi-scale-based target recognition detector;
a search performing unit that cooperatively combines a differential correlation filter (DCF)-based tracking model and the multi-scale-based target recognition detector to perform global and local searches; and
A location estimator for estimating the location of the target object through the performed global and local searches
including,
The location estimation unit,
Suggests a peak-to-sidelobe ratio (EPSR) for measuring the reliability of a tracking model according to the tracking result of the current frame, and determines whether single tracking or joint tracking is performed based on the peak-to-sidelobe ratio, and calculates The tracking model is updated for efficiency, the multi-scale-based target recognition detector generates similar object candidates with spatial and scale constraints, and foreground and background interference is detected through the differential correlation filter (DCF)-based tracking model. To perform a role to distinguish and rearrange the ranks of object candidates for object reconfirmation, and to ensure that the tracking object is detected by the multi-scale-based target recognition detector, the object recovery rate is higher than a preset detection score Select, apply a spatial constraint by calculating the relative distance between the centroid of the selected object suggestion and the centroid of a previously estimated tracking object; to apply a scale constraint by
tracking system.

Images