CN111968155B - A target tracking method based on segmentation target mask update template - Google Patents

Abstract

The invention discloses a target tracking method based on segmentation target mask update template, comprising the following steps: constructing a basic network framework for target tracking; initializing the network, obtaining foreground information in the regression frame, generating an initialization target template with outstanding foreground, and then combining with Initialize the linear superposition of the target template; input the above linear superposition result into the target template branch of the tracking module to obtain the center position and size of the target in the next frame; every m frame, the network will calculate the target center point and regression frame parameters of the corresponding frame. The target template corresponding to the frame, and the frame target template is input into the above-mentioned mask segmentation module, and the subsequent frame target template of the prominent foreground is generated; the linearly superimposed initialization target template, the initialized target template of the foreground and the subsequent frame target template of the prominent foreground are superimposed. , generate a new target template used in the next frame; input the target template into the tracking network framework, and calculate the center position and size of the target in the next frame.

Description

A target tracking method based on segmentation target mask update template

technical field

The invention relates to the field of deep neural networks, in particular to a target tracking method based on a segmentation target mask update template under a deep Siamese network framework.

Background technique

With the increasing development of artificial intelligence, computer vision is more and more widely used in various fields and people's daily life. As an important branch of computer vision, target tracking technology plays a pivotal role in different application fields such as automatic driving, human-computer interaction, pedestrian detection and weapon precision strike. It also has a very broad application prospect and far-reaching effect. Significance.

The basic task of target tracking can be briefly summarized as: given the size and position of the target in the first frame of the video, the algorithm calculates the center position and size of the target frame by frame in the subsequent frames, thereby realizing the target tracking in the video. According to the number of video frames and whether the target is single, the target tracking algorithm is mainly divided into: long video multi-target tracking, long video single target tracking, short video multi target tracking and short video single target tracking. Although the target tracking algorithm has developed rapidly in recent years, it still faces many challenging problems, such as: interference of the same target, non-rigid deformation of the target, change of target size, and rotation of the target in and out of the plane, etc. These problems will affect to varying degrees. The performance of the target tracking algorithm; at the same time, the practical application of target tracking requires the algorithm to be real-time, so the speed of the algorithm is also a key indicator to measure the performance of the algorithm.

There are two important branches of target tracking algorithm, namely correlation filtering algorithm and deep learning algorithm. In recent years, among deep learning algorithms, target tracking algorithms based on deep Siamese convolutional neural networks have been favored by many researchers due to their better balance between speed and accuracy and better stability than related filtering algorithms. Follow and develop. This type of algorithm usually uses a large-scale dataset to train the model end-to-end and track offline, so it lacks an online update strategy for adjusting the filter template similar to related filtering algorithms. In the application scenarios of , the stability of the tracking algorithm will decrease to varying degrees. Therefore, it is necessary to increase the online update mechanism of target templates under the framework of deep Siamese convolutional neural network and improve the adaptability of the algorithm to complex application scenarios.

The deep mask network (DeepMask ^[1] ) is an instance segmentation model based on the VGG network. The network implements front and background segmentation, foreground semantic segmentation and foreground instance segmentation, and the network outputs the mask of the image foreground. As we all know, the target tracking algorithm can be essentially understood as a binary classification problem of the image foreground and background in the search area, and the single target tracking network requires much more attention to the image foreground than the image background. Therefore, by segmenting the image foreground, namely The mask of the target highlights the target to be tracked in the target template, which is beneficial for the target to be more easily noticed by the network during the tracking process; at the same time, combined with the online update mechanism, it is very beneficial to improve the target tracking algorithm's ability to adapt to complex scenes and the stability of the algorithm operation.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a target tracking method based on segmentation target mask update template in order to overcome the deficiencies in the prior art. The target tracking method improves the network's attention to the target in the initialization frame by segmenting the target mask in the target template; at the same time, the template online update strategy is added in subsequent frames to effectively improve the algorithm's adaptation to challenging problems in the tracking process. ability.

The purpose of this invention is to realize through the following technical solutions:

A target tracking method based on segmentation target mask update template, comprising the following steps:

Build the basic network framework for target tracking;

Initialize the network, input the initialized and obtained target regression frame parameters into the mask segmentation module based on the DeepMask network framework in the basic network framework to obtain the foreground information in the regression frame, generate an initialized target template that highlights the foreground, and then linearly superimpose with the initialized target template. ;

Input the above linear superposition result into the target template branch of the tracking module to obtain the center position and size of the target in the next frame; every m frame, the network will calculate the target template corresponding to the frame through the corresponding frame target center point and regression frame parameters, and input the frame target template into the above-mentioned mask segmentation module to generate the subsequent frame target template of the prominent foreground;

Linearly superimpose the initial target template, the initial target template highlighting the foreground, and the subsequent frame target template highlighting the foreground, to generate a new target template used in the next frame;

Input the target template into the tracking network framework, and calculate the center position and size of the target in the next frame.

Further, the basic network framework is:

Based on the basic tracking framework of SiamRPN++, a mask segmentation module based on the DeepMask network framework is added to the front end of the framework.

Further, the new target template is:

T _i+1 =T ₀ +αA ₀ +βA _i

Among them, T ₀ represents the initialization target template, A ₀ represents the initialization target template that highlights the foreground, A _i represents the subsequent frame target template that highlights the foreground, α and β are hyperparameters, and T _i+1 represents the next frame tracking. New target template.

further,

A ₀ =F _crop (DeepMask(x ₀ , y ₀ , bbox ₀ ))

Among them, x ₀ , y ₀ , and bbox ₀ are the initial center coordinates and regression box parameters generated in the target initialization stage, respectively; F _crop represents the cropping function, which is used to crop the target template in a certain frame of the video; DeepMask represents the network framework based on DeepMask. Mask segmentation module; A ₀ represents the initialization target template that highlights the foreground;

A _i =F _crop (DeepMask( _xi , y _i , bbox _i ) _m|i )

Among them, m represents that the target template of the subsequent frame of the prominent foreground is updated every m frames, x _i , y _i , and bbox _i represent the center coordinates and regression frame parameters of the target in this frame when updating, respectively, and F _crop represents cropping function, DeepMask represents the mask segmentation module, and A _i represents the subsequent frame target template that highlights the foreground.

Compared with the prior art, the beneficial effects brought by the technical solution of the present invention are:

1. The present invention takes SiamRPN++ ^[2] as the base tracking framework, uses the DeepMask ^[1] network framework to segment the target mask in the initialization process, and uses the target mask to generate attention on the foreground to improve the target initialization frame. Weight, so the network pays attention to the tracked target object, so as to correctly predict the target position during the tracking process.

2. The present invention alleviates the negative impact caused by the inability of the tracking algorithm to adapt to challenging problems such as large morphological changes and occlusion of the target object when the tracking algorithm performs offline tracking by adding an online update mechanism of the target template. This mechanism uses the mask segmentation module based on the DeepMask ^[1] network framework to segment the target mask of the subsequent frame, and generates the mask information related to the target in the subsequent frame, which effectively improves the adaptability of the new target template to motion changes.

3. The present invention not only uses more accurate target mask information than the regression frame parameters parallel to the coordinate axis to generate foreground attention in the initialization stage, but also increases the linear superposition of target-related foreground information in the subsequent stage, so that the newly generated target The template enriches target-related motion information in subsequent frames, thereby improving the algorithm's ability to adapt to target motion. In the general benchmark dataset, the method proposed in the present invention has better experimental results.

Description of drawings

Fig. 1 is a kind of flow chart of the target tracking method based on segmentation target mask update template;

Figure 2 is a block diagram of the target mask segmentation network based on the DeepMask ^[1] network framework;

Figure 3 is a block diagram of a tracking algorithm for updating target templates based on segmentation target masks.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

1. An embodiment of the present invention proposes a target tracking method based on a segmentation target mask update template. Referring to FIG. 1, the method includes the following steps:

101: Build the basic network framework of the target tracking algorithm: that is, first build the basic tracking framework based on SiamRPN++ ^[2] , and then add a mask segmentation module based on the DeepMask ^[1] network framework before the target template branch at the input of the network. Complete basic network framework.

Among them, the DeepMask ^[1] network framework consists of a VGG feature extraction network, a mask prediction branch and a class scoring branch. The present invention omits the output of the category score branch, and only uses the output of the mask prediction branch as the preliminary result of segmenting the foreground. The mask segmentation module is connected with the target template branch part of the basic tracking network SiamRPN++ ^[2] to form the basic network framework of the method of the present invention.

102: The present invention firstly performs mask segmentation of the initialization target template, that is, inputting the initialization information into the foreground highlighting module based on DeepMask ^[1] to generate an initialization target template highlighting the foreground.

The method first obtains the target center position and regression frame parameters in the initialization process, and inputs them into the mask prediction branch in the DeepMask ^[1] framework to generate the mask segmentation result, and then generates the initialization target template that highlights the foreground through calculation. Items are superimposed linearly with the initialization target template.

103: During the tracking process, the present invention adds a template online update strategy that is beneficial for the target template to adapt to changes in the target motion. The strategy mainly realizes the online update of the target template by segmenting the foreground information in the target template of subsequent frames, and updating part of the information in the new target template every m frames. After the tracking starts, the algorithm will count the subsequent frames. When the interval frame number is m, obtain the target center position and regression box information of the frame, input the mask extraction module to generate the subsequent frame target template that highlights the foreground, and use the template to It is linearly superimposed with the initialization target template and the above-mentioned target template with prominent foreground to generate a new template for the next frame. Until the next update, the target template of the subsequent frame of the prominent foreground remains unchanged. In the present invention, m=30;

104: In the testing phase, first initialize the algorithm to obtain the target template of the first frame and the search area of the subsequent frames, generate two parts of the relevant items that highlight the foreground through the mask extraction module of the initialized target template, and then cut the new template and the target template into The fixed-size search area is also input into the feature extraction network for feature matching and calculation, calculating the position offset and size of the target in the next frame, completing the tracking of the target in the next frame, and counting the tracking. Subsequent frame target template items that highlight the foreground in the template are updated.

To sum up, the embodiment of the present invention designs a target tracking method based on segmentation target mask update template through steps 101 to 104, increases the proportion of foreground information in the initialization process, and effectively improves the feature extraction network. The degree of attention to the target; at the same time, since the target motion has a certain continuity in time and space, the target motion changes in recent frames are relatively similar, and the previous frame information has a certain guiding role for the subsequent frames. Taking advantage of this feature, the present invention adds an online update strategy of the target template in the tracking process, and adds the latest motion change information of the target to the target template. This strategy effectively improves the adaptability of the new target template to changes in object motion, thereby effectively improving the performance of the algorithm.

2. The technical solutions of the above-described embodiments are further introduced below, and are described in detail below:

201: Generally speaking, in the task of short video single target tracking, the center position and size of the target to be tracked in the first frame are usually given first, and the algorithm calculates the target position offset and the target frame by frame in subsequent frames. Dimensions vary. In recent years, target tracking algorithms based on deep Siamese convolutional neural networks usually use a two-branch feature matching framework. to match. Therefore, the information contained in the target template has a direct impact on the final target tracking results. The target template is usually composed of the target to be tested as the center, combined with a small amount of background information around the target, that is, the foreground and background in the target template. The target templates used by the commonly used Siamese network are all directly generated during the initialization process, and there is no weight assigned to the foreground and background information in the target template, so the network pays almost the same attention to the foreground and background in the target template. The invention uses the network framework based on DeepMask ^[1] to highlight the foreground information in the target template, so that the proportion of the target to be tracked increases, so that the network can more easily pay attention to the target to be tracked, thereby improving the performance of the tracking algorithm.

The present invention adopts the deep Siamese convolutional neural network SiamRPN++ ^[2] as the basic framework of the tracking network, and adds a mask segmentation module based on the DeepMask ^[1] network framework before its target template branch. The input of this module is the center position of the target to be tracked and the parameters of the regression box, which are input into the DeepMask ^[1] network. The network consists of three parts: the basic feature extraction network VGG, the mask prediction branch and the score prediction branch. The present invention omits the output of the score prediction branch, so only the two parts of the network are used. The VGG network contains eight 3×3 convolutional layers, four 2×2 max pooling layers, and the mask prediction branch contains one 1×1 convolutional layer and an upsampling layer based on bilinear interpolation , and the output of the network is the target image in the regression box that highlights the foreground. The basic network framework of target tracking, SiamRPN++ ^[2] , uses ResNet50 to extract features, and outputs the final tracking results through three series RPN (Region Proposal Network, Region Proposal Network) networks. In the present invention, the target template branch input of the SiamRPN++ ^[2] network is a new target template composed of a linearly superimposed initialized target template, an initialized target template that highlights the foreground, and a subsequent frame target template that highlights the foreground.

202: In the initialization process of the algorithm, the center position of the target and the parameters of the regression box are given by the first frame, and input it into the mask segmentation module based on the DeepMask ^[1] network, segment the foreground information of the image in the regression box, and obtain the highlight. The image in the regression frame of the foreground is made to cover the image in the original first frame of the regression frame, and then the first frame after the segmentation mask is divided by the crop function F _crop , and the target template that highlights the foreground is obtained. The target template is the prominent foreground The initialization target template of . The target template size formula for F _crop cropping is expressed as follows:

A=s(w+p)×s(h+p)

Among them, A is the size of the target template after cropping, that is, 127×127; w and h are the width and height of the regression box, respectively; s is the scaling factor, p is the expansion factor, p=(w+h)/2. Note that the initial target template for highlighting the foreground is A ₀ .

During the tracking process, the shape and motion state of the target to be tracked will change to a certain extent, so the information in the target template of the initial frame is less timely and often cannot adapt to the changes of the target template well. In order to improve the adaptability of the target template to the morphological changes of the target in subsequent frames and the network's attention to the target, the present invention adds an online update mechanism of the target template. This mechanism mainly relies on mask segmentation of the target in the subsequent frames, which not only increases the information of the target in the nearest frame in the initial target template, but also highlights the foreground in the subsequent frame, which further enhances the network's attention to the target. Similar to the initialization target template for generating a prominent foreground, the center position and regression box parameters of the target in the frame output by the algorithm are obtained every 30 frames, and they are input into the mask segmentation module based on the DeepMask ^[1] network to obtain the frame regression frame. The foreground of the inner image, and then cover the position of the regression frame in the frame, and finally obtain the target template of the frame highlighting the foreground through F _crop cropping, which is recorded as A _i .

203: During the testing process, due to the continuity of the target motion in the time domain and the space domain, the target template online update strategy of the present invention is performed every 30 frames. First, after the initialization phase, the initial target template and the initial target template highlighting the foreground are generated from the first frame, linearly superimposed, and input into the target template branch of SiamRPN++ ^[2] to extract features. When the tracking reaches the 30th frame, input the target center position and regression box output based on the SiamRPN++ ^[2] tracking framework into the mask segmentation module based on the DeepMask ^[1] network to obtain the target template that highlights the foreground of the frame, and then linearly stack it. In the above template, the formula generated by the new template is expressed as follows:

T _i+1 =T ₀ +αA ₀ +βA _i

Among them, T ₀ represents the initialization target template, A ₀ represents the initialization target template that highlights the foreground, A _i represents the subsequent frame target template that highlights the foreground, α and β are hyperparameters, and T _i+1 represents the next frame tracking. The new target template, in the present invention, α=0.03, β=0.005. Input the new target template into the target template branch of the tracking network, and calculate the position of the target in the next frame and the size of the regression box. The formula of the whole algorithm is expressed as follows:

(x, y, Δs) _i+1 =S(T _i+1 , R _i+1 )

Wherein, T _i+1 represents the new target template that contains the initial frame target template that highlights the foreground and the subsequent frame target template that highlights the foreground, R _i+1 represents the search area of the next frame, S represents the tracking algorithm, ( x, y, Δs) _i+1 represents the target position in the next frame and the change size of the regression frame.

3. Below in conjunction with specific experimental data, the scheme of the above-described embodiment is evaluated for effect, as described in detail below:

301: Data Composition

The test set consists of all the video sequences in the VOT2016 and OTB100 datasets. Among them, the VOT2016 dataset contains 60 videos, all of which are color sequences; the OTB100 dataset contains 100 videos, including 75 color sequences and 25 grayscale sequences.

302: Evaluation Criteria

The present invention uses different evaluation indicators to evaluate the performance of the algorithm on the VOT2016 data set and the OTB100 data set respectively.

On the VOT2016 data set, three evaluation indicators are mainly used, namely Accuracy (accuracy rate), Robustness (robustness) and EAO (average overlap expectation, ExpectAverage Overlap rate) to evaluate the performance of the target tracking algorithm: Accuracy is a measure Tracking algorithms track metrics of accuracy. By calculating the IoU (overlap rate, Intersection over Union) between the predicted target regression frame and the real target regression frame of each frame in the video sequence,

表示真实目标回归框，

表示预测目标回归框。为了保证实验中测试的准确性，计算时会多次重复测量每帧的准确率，最后将所有结果取平均，从而获取跟踪算法最终的准确率值。所述Accuracy值越大，表明跟踪结果准确性越好；Robustness是一种衡量跟踪算法稳定性的指标，计算跟踪过程出现跟丢情况的帧的数量。所述Robustness值越大，表明跟踪算法跟丢的帧数越多，算法越不稳定；EAO值则用来综合评估算法的准确性和鲁棒性，是总体评估跟踪算法性能的重要指标。其值计算过程如下：首先将所有视频序列按照长度分类，将待测的跟踪器在长度为N_s的序列上测试，从第一帧到最后一帧，没有跟踪失败后重新初始化的机制，得到每一帧的准确率(Accuracy)φ_i，之后对每一帧取平均，得到该序列上的准确率

将所有长度为N_s的序列全部测评一遍并求平均，得到跟踪器在长度为N_s序列上的EAO值

其他长度的序列也按照同样的方法计算EAO。对不同N_s值对应的

再次求平均，在序列长度范围为[N_lo，N_hi]上得到一个定值：in,

represents the true target regression box,

Represents the prediction target regression box. In order to ensure the accuracy of the test in the experiment, the accuracy of each frame will be measured repeatedly during the calculation, and finally all the results are averaged to obtain the final accuracy value of the tracking algorithm. The larger the Accuracy value is, the better the accuracy of the tracking result is; Robustness is an indicator for measuring the stability of the tracking algorithm, and the number of frames that are tracked and lost during the tracking process is calculated. The larger the Robustness value, the more frames lost by the tracking algorithm, and the more unstable the algorithm is; the EAO value is used to comprehensively evaluate the accuracy and robustness of the algorithm, and is an important indicator for evaluating the overall performance of the tracking algorithm. The calculation process of its value is as follows: First, all video sequences are classified according to their lengths, and the tracker to be tested is tested on a sequence of length N _s . From the first frame to the last frame, there is no mechanism for re-initialization after tracking failure. Accuracy rate of each frame (Accuracy) φ _i , and then average each frame to get the accuracy rate on the sequence

All sequences of length N _s are evaluated and averaged to obtain the EAO value of the tracker on the sequence of length N _s

For sequences of other lengths, EAO is calculated in the same way. Corresponding to different N _s values

Average again to get a constant value over the range of sequence lengths [N _lo , N _hi ]:

On the OTB dataset, the one-pass calculation precision plot (Precision Plot) and success rate plot (Success Plot) are mainly used. In the accuracy rate graph, the distance between the center point of the target predicted position and the actual value is usually calculated, and the ratio of the number of frames with a distance smaller than the set threshold to the total number of frames in the sequence is counted. Different threshold settings have different accuracy rates. In the experiment of the present invention, the threshold is set to 0-50; in the success rate graph, the IoU between the predicted value and the true value of the target regression frame is usually calculated to obtain the coincidence rate. The overlap score is calculated, and the ratio of frames with a score greater than the set threshold to the total number of frames in the sequence is counted. In the experiment of the present invention, the threshold is set to 0-1.

303: Comparison Algorithms

In the comparative experiment, the present invention is compared with seven mainstream algorithms on the VOT2016 data set, including two related filtering algorithms and five deep learning algorithms. The correlation filtering algorithms include: C-COT ^[3] , ECO ^[4] . This type of algorithm has a faster tracking speed, but its accuracy is lower than that of deep learning algorithms. The deep learning algorithms include: SiameseFC ^[5] , SiameseRPN ^[6] , DaSiamRPN ^[7] and SiamRPN++ ^[2] . This kind of algorithm balances the speed and accuracy of tracking, which not only meets the requirements of real-time tracking, but also has good accuracy and stability.

On the OTB100 dataset, it is compared with eight mainstream algorithms, including 6 related filtering algorithms and 5 deep learning algorithms. The correlation filtering algorithms include: ECO ^[4] , SRDCF ^[8] , BACF ^[9] , SRDCFdecon ^[10] , Staple ^[11] and LMCF ^[12] . The deep learning algorithms include: SiamFCRes22 and Memtrack ^{[14] in SiamDW [13] .}

Table 1 and Table 2 respectively show the objective evaluation results of this method and related comparison algorithms on the VOT2016 data set and the OTB100 data set (the best evaluation results are shown in bold). It can be seen from Table 1 that in the evaluation results obtained by most deep learning methods and the method proposed by the present invention, the Accuracy is generally higher than the related filtering tracking algorithm; the Robustness is generally slightly higher than the related filtering algorithm; the EAO index is generally higher. Significantly higher than related filtering algorithms. Compared with the deep learning class comparison algorithm, this method has achieved higher results on the three objective evaluation indicators, which objectively shows that this method has better performance than the comparison algorithm on the VOT2016 data set. It can be seen from Table 2 that the difference between the related filtering algorithm and the deep learning algorithm in the Success and Presion indicators is not obvious, but the performance of the deep learning algorithm is generally higher than that of the related filtering algorithm, and the algorithm proposed by the present invention is generally higher. The two objective indicators tested on the OTB100 dataset have better objective results than related filtering algorithms and deep learning algorithms.

After the experimental verification of the two databases, the experimental results of the algorithm of the present invention are better, which also shows that the algorithm proposed by the present invention can reduce the probability of tracking failure in the tracking process to a certain extent, and effectively improve the accuracy of the tracking results. , to enhance the adaptability of the tracker when faced with problems such as non-rigid deformation, occlusion and rapid changes of the target object, thereby improving the accuracy and stability of the tracking algorithm.

Table 1

Table 2

references

[1]Pinheiro P O,Collobert R,Dollar P.Learning to Segment ObjectCandidates[C]//Advances in Neural Information Processing Systems,Dec.7-12,2015,Montreal,Canada,1990-1998

[2] Li B, Wu W, Wang Q, et al.SiamRPN++:Evolution of Siamese VisualTracking with Very Deep Networks[C]//IEEE International Conference on Computer Vision and Pattern Recognition,Jun.16-20,2019,Long Beach, CA, USA. 2019, pp: 4277-4286.

[3]Danelljan M,Robinson A,Khan F S,et al.Beyond Correlation Filters:Learning Continuous Convolution Operators for Visual Tracking[C]//Proceedingsof the 2016European Conference on Computer Vision,Oct.8-16,2016.Amsterdam,theNetherlands .2016, 472-488.

[4] Danelljan M, Gavves G, Khan F S, et al.Eco:Efficient convolutionoperators for tracking[C]//2017IEEE Conference on Computer Vision and PatternRecognition,Jul.21-26,2017,Honolulu,HI,USA:IEEE, 2017, 79: 6931-6939.

[5] Bertinetto L, Valmadre J, Henriques J F, et al.Fully-convolutional Siamese networks for object tracking[C]//2016European Conference on ComputerVision Workshop,Oct.8-10,2016,Amsterdam,The Netherlands,2016,9914: 850-865.

[6] Li B, Yan J, Wu W, et al. High Performance Visual Tracking with Siamese Region Proposal Network [C]; 2018IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp.8971-8980 .

[7] Zhang Z, Peng H. Deeper and wider siamese networks for real-time visual tracking[C]//IEEE International Conference on Computer Vision and Pattern Recognition,Jun.16-20,2019,Long Beach,CA,USA.2019, pp:4591-4600.

[8] Danelljan M, Hager G, Shahbaz Khan F, et al.Learning spatiallyregularized correlation filters for visual tracking[C]//IEEE InternationalConference on Computer Vision,Dec.13-16,2015,Santiago,Chile.2015,pp: 4310-4318.

[9] Kiani Galoogahi H, Fagg A, Lucey S. Learning background-awarecorrelation filters for visual tracking[C]//IEEE International Conference onComputer Vision,Oct.22-29,2017,Venice,Italy.2017,pp:1135- 1143.

[10] Danelljan M, Hager G, Khan F S, et al.Adaptive decontamination of the training set: A unified formulation for discriminative visual tracking[C]//IEEE International Conference on Computer Vision and PatternRecognition,Jun.27-30,2016, Las Vegas, NV, USA: IEEE, 2016, 59: 1430-1438.

[11] Bertinetto L, Valmadre J, Golodetz S, et al.Staple: Complementarylearners for real-time tracking[C]//IEEE International Conference on ComputerVision and Pattern Recognition,Jun.27-30,2016,Las Vegas,NV, USA: IEEE, 2016, 237: 1401-1409.

[12] Wang M M, Liu Y, Huang Z. Large margin object tracking with circulant feature maps[C]//IEEE International Conference on Computer Vision and Pattern Recognition, Jul. 21-26, 2017, Honolulu, HI, USA. 2017, pp :4021-4029.

[13] Zhang Z, Peng H. Deeper and wider siamese networks for real-time visual tracking[C]//IEEE International Conference on Computer Vision and Pattern Recognition, Jun.16-20,2019,Long Beach,CA,USA.2019, pp:4591-4600.

[14] Yang T Y, Chan Antoni B. Learning Dynamic Memory Networks for Object Tracking[C]//European Conference on Computer Vision, Sep, 8-14, 2018, Munich, Germany. 2018, pp: 152-167.

The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solutions of the present invention, and the above-mentioned specific embodiments are only illustrative and not restrictive. Without departing from the spirit of the present invention and the protection scope of the claims, those of ordinary skill in the art can also make many specific transformations under the inspiration of the present invention, which all fall within the protection scope of the present invention.

Claims (4)

1. A target tracking method for updating a template based on a segmented target mask is characterized by comprising the following steps:

constructing a basic network framework of target tracking;

initializing a network, inputting target regression frame parameters obtained by initialization into a mask segmentation module based on a DeepMask network frame in the basic network frame to obtain foreground information in the regression frame, generating an initialization target template with prominent foreground, and linearly overlapping the initialization target template;

inputting the linear superposition result into a target template branch of a tracking module to obtain the central position and the size of the target of the next frame; every m frames, the network calculates the target template corresponding to the frame through the target center point of the corresponding frame and the regression frame parameters, and inputs the target template of the frame into the mask segmentation module to generate the subsequent frame target template with the prominent foreground;

linearly superposing the initialization target template, the initialization target template with the outstanding foreground and the subsequent frame target template with the outstanding foreground to generate a new target template used by the next frame;

and inputting the target template into a tracking network frame, and calculating the central position and the size of the target of the next frame.

2. The method of claim 1, wherein the basic network framework is:

and a mask segmentation module based on a DeepMask network framework is added at the front end of the framework based on a basic tracking framework of the SimRPN + +.

3. The method of claim 1, wherein the new target template is:

T_i+1＝T₀+αA₀+βA_i

wherein, T₀Indicating the initialization target template, A₀Initial target template representing a prominent foreground, A_iThe target template of the subsequent frame showing the outstanding foreground, alpha and beta are both hyper-parameters, T_i+1Representing the new target template used for the next frame tracking.

4. The method of claim 3, wherein the step of updating the template of the object tracking based on the mask of the segmented object comprises,

A₀＝F_crop(DeepMask(x₀，y₀，bbox₀))

wherein x is₀、y₀、bbox₀Respectively generating a horizontal coordinate and a vertical coordinate of an initial center and a regression frame parameter in a target initialization stage; f_cropRepresenting a clipping function for clipping a target template in a frame of a video; the DeepMask represents a mask segmentation module based on a DeepMask network framework; a. the₀An initialization target template representing a salient foreground;

A_i＝F_crop(DeepMask(x_i，y_i，bbox_i)_m|i)

wherein m represents updating the target template of the subsequent frame of the salient foreground once every m frames, x_i、y_i、bbox_iRepresenting the center coordinates of the object in this frame and the regression box parameters, F, respectively, when updated_cropRepresenting a clipping function, DeepMask representing a mask segmentation Module, A_iA subsequent frame object template representing a salient foreground.

Images