CN105654505B - A kind of collaboration track algorithm and system based on super-pixel - Google Patents

Abstract

The invention relates to a superpixel-based collaborative tracking algorithm and system. The method provided by the present invention will combine global judgment and local judgment to determine whether the candidate image contains the target area, so it can solve the tracking problem that the target area is occluded. Various appearance changes during the process, and its accuracy and applicability are greatly improved.

Description

A Collaborative Tracking Algorithm and System Based on Superpixels

technical field

The present invention relates to the field of computer vision target tracking, and more specifically, to a superpixel-based cooperative tracking algorithm and system.

Background technique

With the development and popularization of computers, people are increasingly expecting computers to have the same perception and recognition capabilities as humans. One of the directions of effort is to have a visual perception system similar to humans. Computer vision is to process the input image information through the computer, simulate the perception and recognition of visual information by the human eye, and complete tasks such as target recognition and tracking. With the improvement of computer performance and the popularization of cameras, we can obtain a large amount of video image information every day, and it is still growing, which makes people's demand for automatic processing of visual information increasing.

Object tracking is to detect a pre-selected object of interest in a set of image sequences and track the object frame by frame. According to the number of tracking targets, target tracking algorithms can be simply divided into single target tracking algorithms and multi-target tracking algorithms; according to the number of cameras used in the tracking process, target tracking algorithms can be divided into single-camera tracking and multi-camera tracking. The present invention is mainly aimed at the tracking problem of a single camera and a single target. Object tracking itself is an applied technology in computer vision, and it is the basis for other advanced applications. Some typical applications of object tracking include: human-computer interaction, security monitoring, traffic detection, intelligent robot navigation, etc. However, object tracking is a complex process, and there are still many challenges in this field, such as partial occlusion, appearance changes, light changes, violent motion, object reappearance after disappearing in the field of view, background influence, etc. during object tracking.

Contents of the invention

In order to solve the above defects of the prior art, the present invention provides a superpixel-based collaborative tracking algorithm, which can deal with common problems in target tracking such as occlusion and appearance changes, and has good stability and robustness .

For realizing above-mentioned purpose of the invention, the technical scheme that adopts is:

A collaborative tracking algorithm based on superpixel segmentation is used to solve the tracking problem of a single camera and a single target, including the following steps:

1. Training stage

S1. Build a global discriminant model, the global discriminant model is used to extract the Haar_Like feature of the target area, then construct a global classifier GC according to the extracted Haar_Like feature, and determine the parameters of the global classifier GC;

S2. Use the fragmentation method based on overlapping sliding windows to fragment the target area to obtain N sub-regions, and then construct N local discriminant models, and the N local discriminant models are used to extract Haar_Like features for the N sub-regions respectively, Then build local classifiers according to the extracted Haar_Like features, and determine the parameters of the local classifiers;

S3. Constructing an adaptive generative model, and confirming the model parameters of the adaptive generative model, the specific steps are as follows:

Carry out superpixel segmentation on the target area, and extract the feature vector of each superpixel separately, and then use the K-means algorithm to cluster all the superpixels in the target area, so as to determine the model parameters suitable for the generated model;

2. Tracking stage

S4. Input the candidate image p _i into the global discriminant model, the global discriminant model extracts the Haar_Like feature of the candidate image p _i , and then uses the global classifier GC to classify the Haar_Like feature of the candidate image p _i , GC(pi ₎ means The classification result of the candidate image p _i ;

S5. Use the method of step S2 to divide the candidate image p _i into N sub-regions, then make N local discriminant models extract Haar_Like features for the N sub-regions, and then use N local classifiers to classify the Haar_Like features of the N sub-regions respectively , LC _j (p _i ) represents the classification result of the jth local classifier for the sub-region;

S6. Combining the classification results of the global classification model and the local classification model, it is judged whether the candidate image contains the target area:

thr _GC and thr _LC represent the two thresholds of global classification and local classification respectively, when y(pi ₎ =1, it means that the candidate image p _i contains the target area;

S7. Perform the operations of steps S4 to S6 on all candidate images to determine whether they contain the target area, and then input all candidate images that are determined to contain the target area into the adaptive generative model;

S8. For each candidate image, adapt the generated model to perform superpixel segmentation, then extract the feature vector of each superpixel, then use the K-means algorithm to cluster the feature vectors of all superpixels, and calculate the candidate image clustering confidence; then select the candidate image with the highest confidence as the tracking result to output, the output data includes the confidence conf _T of the current tracking result and the matching area area _T of the target area, where where A _i is the area of each superpixel, N represents the number of superpixels contained in the candidate image slice,

The above formula shows that when the superpixel is close to the cluster center in the feature space, and the relative position of the template superpixel in the cluster is also similar to the target area, and the target/background confidence of the cluster to which it belongs, this patent considers that the superpixel of this type is Pixels can more fully describe the appearance information of the current target and have strong discriminative ability, where _g'i represents the superpixels contained in the candidate image slice, _k'i represents the cluster to which the superpixel belongs, and _S'i represents the distance of the cluster to which the superpixel belongs , Indicates the target/background confidence of k′ _i , R′ _j represents the cluster radius, conf _i ′ represents the confidence of g′ _i , L _i represents each superpixel in the candidate image slice and the template superpixel in the cluster to which it belongs The minimum spatial distance between, a _s is the weight factor to control the weight of the spatial distance, Represents g′ _i and the template superpixel of the cluster it belongs to the spatial distance in the target area, Indicates that a _s is the base, and the is the power operation of the exponent;

in

A' _j represents the number of pixels of each superpixel in the current tracking result, Represents the number of pixels in the target area contained in each superpixel cluster, and M represents the total number of superpixels;

3. Detection stage

S9. Build a template library generation model, and make the template library generation model detect the target area in the current frame, return the confidence degree conf _D of the detection result, and then estimate the current position of the target area according to the output results of the adaptation generation model and the template library generation model :

1) When area _T ≥ thr _PL and conf _T ≥ thr _TH

Among them, thr _TH and thr _PL represent the confidence threshold and the matching area threshold respectively. At this time, the tracking result of the adaptive generative model has high confidence and matching area, and the adaptive generative model works normally and adapts to the appearance of the target area, so the adaptive generative model The output of the model is output as the target position; then according to the update strategy, update the parameters of the global classifier GC, local classifier, and adaptive generation model according to area _T and conf _T ;

2) When area _T < thr _PL and conf _T ≥ thr _TH

At this time, the matching area of the tracking result of the adaptive generative model is low, but the confidence of the tracking result is still higher than the threshold, so the output result of the adaptive generative model is still output as the target position; then according to the update strategy according to area _T , conf _T Update the parameters of the global classifier GC, local classifier, and adaptive generation model;

3) When area _T ≥ thr _PL and conf _T < thr _TH

At this time, the tracking result of the adaptive generative model has low confidence, but has a high matching area, so the output result of the adaptive generative model is still output as the target position; and then the global classification is performed according to the area _T and conf _T according to the update strategy GC, local classifiers, parameters adapted to generate models are updated;

4) When area _T < thr _PL , conf _T ≥ thr _TH and conf _D ≥ thr _DH

thr _DH represents the threshold of the confidence of the detection result. At this time, the confidence of the tracking result and the matching area of the adaptive generation model are lower than the preset threshold, and the template library generation model detects a target position with a high degree of confidence. The detection result of the template library generation model is output as the target position, and then the global classifier GC, local classifier, and adaptive generation model are reinitialized.

Model update is the key to enable the tracking algorithm to adapt to changes in the appearance of the target. The discriminant model uses an incremental update method similar to that in the RealTime Compressive Tracking literature (note that it has nothing to do with this patent, so it will not be described in detail), and the generation model uses a method based on sliding windows. update method. In the tracking process, every U frame of image, we add a frame of image to the model and perform superpixel segmentation, feature extraction, and clustering. In order to ensure the real-time performance of the algorithm, we use a fixed-size window, and at each update, if the number of image frames in the window is greater than the predetermined size, the image that has the least impact on the generation model will be discarded according to a certain strategy.

At the same time, the present invention also provides a system for applying the cooperative tracking algorithm, and its specific scheme is as follows: it includes a tracking module, a detection module and a position estimation module, wherein the tracking module includes a global discriminant model, a local discriminant model and an adaptive A generation model, the detection model includes a template library generation model, and the position estimation module is used to estimate the current position of the target area according to output results of the adaptive generation model and the template library generation model.

Compared with prior art, the beneficial effect of the present invention is:

The superpixel-based collaborative tracking algorithm provided by the present invention can deal with common problems in target tracking such as occlusion and appearance change, and has good stability and robustness.

Description of drawings

Figure 1 is the frame diagram of this method.

Figure 2 is a schematic diagram of the training of the discriminant model.

Figure 3 is a schematic diagram of the training of the adaptive generative model.

Detailed ways

The accompanying drawings are for illustrative purposes only and cannot be construed as limiting the patent;

The present invention will be further elaborated below in conjunction with the accompanying drawings and embodiments.

Example 1

A collaborative tracking algorithm based on superpixel segmentation is used to solve the tracking problem of a single camera and a single target, including the following steps:

1. Training stage

S1. Build a global discriminant model, the global discriminant model is used to extract the Haar_Like feature of the target area, then construct a global classifier GC according to the extracted global compressed Haar_Like feature, and determine the parameters of the global classifier GC, specifically as shown in Figure 2 ;

S2. Use the fragmentation method based on overlapping sliding windows to fragment the target area to obtain N sub-regions, and then construct N global discriminant models respectively, and the N local discriminant models are used to extract Haar_Like features respectively for the N sub-regions, Then construct local classifiers according to the extracted local compressed Haar_Like features, and determine the parameters of the local classifiers, as shown in Figure 2;

S3. Constructing an adaptive generative model, and confirming the model parameters of the adaptive generative model, the specific steps are as follows:

Carry out superpixel segmentation on the target area, and extract the feature vector of each superpixel separately, and then use the K-means algorithm to cluster all the superpixels in the target area, so as to determine the model parameters suitable for the generated model, as shown in Figure 3 shown;

2. Tracking stage

S4. Input the candidate image p _i into the global discriminant model, the global discriminant model extracts the Haar_Like feature of the candidate image p _i , and then uses the global classifier GC to classify the globally compressed Haar_Like feature of the candidate image p _i , GC(p _i ) represents the classification result of the candidate image p _i ;

S5. Use the method of step S2 to divide the candidate image p _i into N sub-regions, then make N local discriminant models extract Haar_Like features for N sub-regions, and then use N local classifiers to compress Haar_Like features for N sub-regions respectively Classify, LC _j (p _i ) represents the classification result of the jth local classifier for the sub-region. When the target is occluded, the global discriminant model may not be able to correctly identify the target area, but there are usually one or more local classifiers whose corresponding areas are not occluded in the N local discriminant models, which can correctly identify the target area.

S6. Combining the classification results of the global classification model and the local classification model, it is judged whether the candidate image contains the target area:

thr _GC and thr _LC represent the two thresholds of global classification and local classification respectively, when y(pi ₎ =1, it means that the candidate image p _i contains the target area;

In the above solution, when the target area is blocked, the global discrimination model cannot work normally. In order to avoid such defects, the method provided by the present invention will combine global judgment and local judgment to determine whether the candidate image contains the target area, and its accuracy , The applicability is greatly improved.

S7. Perform the operations of steps S4 to S6 on all candidate images to determine whether they contain the target area, and then input all candidate images that are determined to contain the target area into the adaptive generative model;

S8. For each candidate image, adapt the generated model to perform superpixel segmentation, then extract the feature vector of each superpixel, then use the K-means algorithm to cluster the feature vectors of all superpixels, and calculate the candidate image clustering confidence; then select the candidate image with the highest confidence as the tracking result to output, the output data includes the confidence conf _T of the current tracking result and the matching area area _T of the target area, where where A _i is the area of each superpixel, N represents the number of superpixels contained in the candidate image slice,

The above formula shows that when the superpixel is close to the cluster center in the feature space, and the relative position of the template superpixel in the cluster is also similar to the target area, and the target/background confidence of the cluster to which it belongs, this patent considers that the superpixel of this type is Pixels can more fully describe the appearance information of the current target and have strong discriminative ability, where _g'i represents the superpixels contained in the candidate image slice, _k'i represents the cluster to which the superpixel belongs, and _S'i represents the distance of the cluster to which the superpixel belongs , Represents the target/background confidence of each cluster, R′ _j represents the cluster radius, conf _i ′ represents the confidence of g′ _i , L _i represents the relationship between each superpixel in the candidate image slice and the template superpixel in the cluster to which it belongs. The minimum spatial distance between pixels, a _s is the weight factor to control the weight of the spatial distance, Represents g′ _i and the template superpixel of the cluster it belongs to Spatial distance in the target area;

Where A _target represents the sum of the number of pixels belonging to the target area for all class members in each cluster, and A _background represents the sum of the number of pixels in the background area;

in

A' _j represents the number of pixels of each superpixel in the current tracking result, Represents the number of pixels in the target area contained in each superpixel cluster, and M represents the total number of superpixels;

3. Detection stage

S9. Build a template library generation model, and make the template library generation model detect the target area in the current frame, return the confidence degree conf _D of the detection result, and then estimate the current position of the target area according to the output results of the adaptation generation model and the template library generation model :

1) When area _T ≥ thr _PL and conf _T ≥ thr _TH

Among them, thr _TH and thr _PL represent the confidence threshold and the matching area threshold respectively. At this time, the tracking result of the adaptive generative model has high confidence and matching area, and the adaptive generative model works normally and adapts to the appearance of the target area, so the adaptive generative model The output of the model is output as the target position; then according to the update strategy, update the parameters of the global classifier GC, local classifier, and adaptive generation model according to area _T and conf _T ;

2) When area _T < thr _PL and conf _T ≥ thr _TH

At this time, the matching area of the tracking result of the adaptive generative model is low, but the confidence of the tracking result is still higher than the threshold, so the output result of the adaptive generative model is still output as the target position; then according to the update strategy according to area _T , conf _T Update the parameters of the global classifier GC, local classifier, and adaptive generation model;

3) When area _T ≥ thr _PL and conf _T < thr _TH

At this time, the tracking result of the adaptive generative model has low confidence, but has a high matching area, so the output result of the adaptive generative model is still output as the target position; and then the global classification is performed according to the area _T and conf _T according to the update strategy GC, local classifiers, parameters adapted to generate models are updated;

4) When area _T < thr _PL , conf _T ≥ thr _TH and conf _D ≥ thr _DH

thr _DH represents the threshold of the confidence of the detection result. At this time, the confidence of the tracking result and the matching area of the adaptive generation model are lower than the preset threshold, and the template library generation model detects a target position with a high degree of confidence. The detection result of the template library generation model is output as the target position, and then the global classifier GC, local classifier, and adaptive generation model are reinitialized.

In the above scheme, the template library generation model determines the current working status and target position of each model according to a certain strategy and outputs them, and feeds back to the global classifier GC, local classifiers, and adaptive generation models at the same time, and updates the global classifier GC, The local classifier is updated in the adaptive generative model, so that the method can adapt to various appearance changes of the target area during the tracking process.

Example 2

The present invention also provides a system for applying the collaborative tracking algorithm, as shown in Figure 3, and its specific scheme is as follows:

Including a tracking module, a detection module and a position estimation module, wherein the tracking module includes a global discriminant model, a local discriminant model and an adaptive generation model, the detection model includes a template library generation model, and the position estimation module is used to generate the model according to the adaptation and The output of the template library generation model estimates the current location of the target region.

Apparently, the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, rather than limiting the implementation of the present invention. For those of ordinary skill in the art, other changes or changes in different forms can be made on the basis of the above description. It is not necessary and impossible to exhaustively list all the implementation manners here. All modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the claims of the present invention.

Claims (2)

1. A collaborative tracking algorithm based on superpixel segmentation, used to solve the tracking problem of single camera single target, is characterized in that: comprise the following steps: 1. Training stage S1. Build a global discriminant model, the global discriminant model is used to extract the Haar_Like feature of the target area, then construct a global classifier GC according to the extracted Haar_Like feature, and determine the parameters of the global classifier GC; S2. Use the fragmentation method based on overlapping sliding windows to fragment the target area to obtain N sub-regions, and then construct N local discriminant models, and the N local discriminant models are used to extract Haar_Like features for the N sub-regions respectively, Then build local classifiers according to the extracted Haar_Like features, and determine the parameters of the local classifiers; S3. Constructing an adaptive generative model, and confirming the model parameters of the adaptive generative model, the specific steps are as follows: Carry out superpixel segmentation on the target area, and extract the feature vector of each superpixel separately, and then use the K-means algorithm to cluster all the superpixels in the target area, so as to determine the model parameters suitable for the generated model; 2. Tracking stage S4. Input the candidate image p _i into the global discriminant model, the global discriminant model extracts the Haar_Like feature of the candidate image p _i , and then uses the global classifier GC to classify the Haar_Like feature of the candidate image p _i , GC(pi ₎ means The classification result of the candidate image p _i ; S5. Use the method of step S2 to divide the candidate image p _i into N sub-regions, then make N local discriminant models extract Haar_Like features for the N sub-regions, and then use N local classifiers to classify the Haar_Like features of the N sub-regions respectively , LC _j (p _i ) represents the classification result of the jth local classifier for the sub-region; S6. Combining the classification results of the global classification model and the local classification model, it is judged whether the candidate image contains the target area: thr _GC and thr _LC represent the two thresholds of global classification and local classification respectively, when y(pi ₎ =1, it means that the candidate image p _i contains the target area; S7. Perform the operations of steps S4 to S6 on all candidate images to determine whether they contain the target area, and then input all candidate images that are determined to contain the target area into the adaptive generative model; S8. For each candidate image, adapt the generated model to perform superpixel segmentation, then extract the feature vector of each superpixel, then use the K-means algorithm to cluster the feature vectors of all superpixels, and calculate the candidate image clustering confidence; then select the candidate image with the highest confidence as the tracking result to output, the output data includes the confidence conf _T of the current tracking result and the matching area area _T of the target area, where where A _i is the area of each superpixel, N represents the number of superpixels contained in the candidate image slice, Among them _{, g'i} represents the superpixels contained in the candidate image slice, _k'i represents the cluster to which the superpixel belongs, _S'i represents the distance of the cluster to which the superpixel belongs, Indicates the target/background confidence of k′ _i , R′ _j represents the cluster radius, conf _i ′ represents the confidence of g′ _i , L _i represents each superpixel in the candidate image slice and the template superpixel in the cluster to which it belongs The minimum spatial distance between, a _s is the weight factor to control the weight of the spatial distance, a _s ∈ (0,1), Represents the template superpixels of g′ _i and the cluster it belongs to The spatial distance in the target area, Indicates that a _s is the base, and the is the power operation of the exponent; in A' _j represents the number of pixels of each superpixel in the current tracking result, Represents the number of pixels in the target area contained in each superpixel cluster, and M represents the total number of superpixels; 3. Detection stage S9. Build a template library generation model, and make the template library generation model detect the target area in the current frame, return the confidence degree conf _D of the detection result, and then estimate the current position of the target area according to the output results of the adaptation generation model and the template library generation model : 1) When area _T ≥ thr _PL and conf _T ≥ thr _TH Among them, thr _TH and thr _PL represent the confidence threshold and the matching area threshold respectively. At this time, the tracking result of the adaptive generative model has high confidence and matching area, and the adaptive generative model works normally and adapts to the appearance of the target area, so the adaptive generative model The output of the model is output as the target position; then according to the update strategy, update the parameters of the global classifier GC, local classifier, and adaptive generation model according to area _T and conf _T ; 2) When area _T < thr _PL and conf _T ≥ thr _TH At this time, the matching area of the tracking result of the adaptive generative model is low, but the confidence of the tracking result is still higher than the threshold, so the output result of the adaptive generative model is still output as the target position; then according to the update strategy according to area _T , conf _T Update the parameters of the global classifier GC, local classifier, and adaptive generation model; 3) When area _T ≥ thr _PL and conf _T < thr _TH At this time, the tracking result of the adaptive generative model has low confidence, but has a high matching area, so the output result of the adaptive generative model is still output as the target position; and then the global classification is performed according to the area _T and conf _T according to the update strategy GC, local classifiers, parameters adapted to generate models are updated; 4) When area _T < thr _PL , conf _T ≥ thr _TH and conf _D ≥ thr _DH thr _DH represents the threshold of the confidence of the detection result. At this time, the confidence of the tracking result and the matching area of the adaptive generation model are lower than the preset threshold, and the template library generation model detects a target position with a high degree of confidence. The detection result of the template library generation model is output as the target position, and then the global classifier GC, local classifier, and adaptive generation model are reinitialized. 2. A system of collaborative tracking algorithms based on superpixel segmentation according to claim 1, characterized in that: comprising a tracking module, a detection module and a position estimation module, wherein said tracking module includes a global discriminant model, a local discriminant model and The adaptive generation model, the detection module includes a template library generation model, and the position estimation module is used to estimate the current position of the target area according to the output results of the adaptation generation model and the template library generation model.