CN102999921A - Pixel label propagation method based on directional tracing windows - Google Patents

Abstract

The invention discloses a method for propagating pixel labels based on directional narrow and long tracking windows. The specific steps are as follows: Step 1: Determine the area to be marked in the target image; Step 2: Arrange the tracking window in the target image so that the tracking window covers the area to be marked Marking the area; step 3: taking the pixels covered by the tracking window in the input image as samples, and establishing a Gaussian mixture model for each type of label; step 4: calculating the probability density that each pixel to be marked covered by the tracking window belongs to each label; Step 5: Calculate the confidence of the estimated probability of each pixel to be marked by the tracking window; Step 6: Process all tracking windows in all directions; Step 7: According to the probability that the pixels to be marked covered by each window belong to each label and the confidence level to determine the label to which the pixel to be marked belongs. The invention has the advantages of effectively utilizing spatial context relations and reducing errors caused by ambiguous features.

Description

Pixel Label Propagation Method Based on Directional Tracking Window

technical field

The invention relates to a pixel label propagation method, in particular to a pixel label propagation method based on a directional tracking window.

Background technique

Label propagation between video frames is a common problem in video processing, especially video editing. Labels can usually represent the results of video processing, and label propagation can be understood as the process of solving the results of other frames when the results of a certain frame are known. For example, in region tracking and foreground segmentation, users can obtain the results of a certain frame through interaction, and then use label propagation to obtain the results of other video frames. Label propagation generally adopts the following three methods:

1. Method based on image matching

The method based on image matching first registers the image of the input frame and the target frame, and then copies the pixel label of the input frame to the target frame according to the corresponding relationship of pixels. Therefore, this class of label propagation methods is equivalent to performing image matching. Image matching is a classic problem in computer vision, which is generally performed by optical flow tracking. Due to the effects of occlusion, edge blurring, etc., exact image matching is difficult to obtain. Optical flow methods based on local features are not suitable for flat areas of images, while methods based on global optimization are sensitive to video discontinuities caused by occlusions, etc. Therefore, although label propagation can be equivalent to image matching in theory, in practice this type of method is rarely applied independently, and is usually only used to obtain an initial result.

2. The method based on the global classifier

The method based on the global classifier first extracts the features of each pixel, and completes the label propagation in the feature space according to the distance and adjacency of the pixels in the feature space. The so-called global classifier means that all pixels of the target frame share the same classifier regardless of the position of the pixel. A typical example of this type of method is video segmentation based on global color distribution. This method uses pixel color as a feature. First, the foreground and background pixel colors with known labels are used as samples to obtain the distribution function of the foreground and background in the color space. The unknown pixels are then classified based on the distribution function. The method based on the global classifier ignores the spatial position relationship of the pixels, and directly propagates the label in the feature space, which makes it easy to make mistakes in areas where the features are ambiguous, such as in areas where the foreground and background colors are similar, based on Video segmentation methods with global color distributions generate a large number of errors. However, due to ignoring the spatial position relationship of pixels and sampling in a larger range, the method based on the global feature distribution can better deal with temporal discontinuities in the video (that is, due to occlusion, topological changes, fast motion etc. resulting in new areas).

3. Methods based on local classifiers

The newly introduced RotoBrush tool in Adobe After Effects 5 uses a local classifier for label propagation. The classifier only covers a local region of the target image, while the samples for training the local classifier come from the corresponding region of the input image. This is actually a way of exploiting the spatial positional relationship of pixels. On the other hand, since the local classifier covers a much smaller area than the global classifier, the feature distribution is also relatively simple, which further reduces its possibility of error.

The technical problem solved by the present invention is different from common visual tracking, a non-parametric model visual tracking method, No.200910080381.8 and video target marking real-time multi-target marking and centroid calculation method, No.200510047785.9, and feature point tracking A multi-feature point tracking method for microscopic sequence images, No.201010516768.6. Both visual tracking and object labeling can be attributed to the problem of labeling regions, while pixel label propagation needs to label each pixel, so it is more closely related to video segmentation. The invention can also be directly used for video segmentation. Feature point tracking belongs to the method of image matching, but it only processes a small number of pixels that are easy to track in the video, and cannot be used for pixel label propagation. The directional window adopted in the present invention is mainly to make better use of color distribution, so it is essentially different from feature tracking and image matching.

A critical step in employing local classifiers is to define the coverage area of each classifier, the tracking window. The larger the tracking window, the more complex the distribution of features in each window, and the greater the possibility of containing ambiguous features, which will lead to similar problems with the global classifier; The robustness will be better, but at the same time it will be sensitive to local discontinuities between video frames, and it is more prone to errors in fast motion and emerging areas; existing local classifiers use regular-shaped tracking windows, that is, square or Circular tracking window, but it is difficult to obtain satisfactory results when facing the problem of ambiguous features and inter-frame discontinuity at the same time; the directional tracking window disclosed by the present invention will help to overcome this shortcoming of the local classifier .

Contents of the invention

The object of the present invention is to solve the above-mentioned problems and provide a pixel label propagation method based on a directional tracking window, which has the advantages of effectively utilizing the spatial context relationship and reducing errors caused by ambiguous features.

In order to achieve the above object, the present invention adopts the following technical solutions:

A pixel label propagation method based on a directional tracking window, the specific steps are

Step 1: expand the area to be propagated in the input image by 30-70 pixels, and the result is used as the area to be marked in the target image;

Step 2: For all specified directions, arrange tracking windows along each direction in the target image, so that the tracking windows in each direction completely cover the area to be marked;

Step 3: For each tracking window, take the pixels covered by the tracking window in the input frame as samples, and use the pixel color as a feature, and establish a corresponding Gaussian mixture model p(x|L) for each label L to represent its color Distribution, x is the color of the pixel to be marked; the label represents the mark of the class that the pixel is divided into, and each class is marked with a label;

Step 4: For each tracking window, calculate the probability that each pixel covered by it belongs to each label;

Step 5: For each tracking window, calculate its confidence degree for the estimated probability of each pixel to be marked;

Step 6: Process all tracking windows in all directions in turn;

Step 7: For each pixel to be marked, record the probability and confidence calculated by all tracking windows covering the pixel, and determine the label of the pixel with the probability output by the window with the highest confidence.

The tracking window is a tracking window with a fixed width and variable length, and the width of the tracking window is W pixels.

The specific steps of step two are:

(2-1) First arrange the horizontal tracking window, scan from top to bottom to the first row containing the area to be marked, denoted as r ₀ ; take r _0th row and r ₀ + W-1 row as the first tracking The upper and lower ends of the window; calculate the start column and end column of the pixels to be marked in these rows, that is, scan from left to right, the column containing the first pixel to be marked is the start column, and the column containing the last pixel to be marked is the end column, and is set to the left end and right end of the first tracking window respectively; the r ₀ +2W/3th line is the starting line of the second tracking window, and the r ₀ +2(k-1)W/ 3 is the starting line of the kth tracking window (there is an overlapping area of W/3 between adjacent tracking windows), and the subsequent tracking windows are arranged in the same way until all pixels to be marked are completely covered, k is Natural number;

(2-2) For any other direction θ, first rotate the target image clockwise by θ degrees, arrange the tracking window according to the method of arranging the horizontal tracking window in step (2-1), and then rotate the target image counterclockwise by θ degrees, Get the tracking window in the θ direction.

其中N为正态分布，π_k,σ_k分别为其均值和方差，ω_k为第k项的权重，K为高斯项的个数，一般取为3-5之间，参数π_k,σ_k,ω_k都可以通过期望值最大化（Expectation-Maxmization）算法得到。The specific form of the Gaussian mixture model p(x|L) in step 3 is

Among them, N is a normal distribution, π _k , σ _k are their mean and variance respectively, ω _k is the weight of the kth item, K is the number of Gaussian items, generally between 3-5, the parameters π _k , σ Both _k and ω _k can be obtained through the Expectation-Maximization algorithm.

The specific steps of step four are:

(4-1) Record the color distribution of the pixel labeled l in the tracking window as p(x|L=l), which is calculated by the Gaussian mixture model obtained in step 3;

(4-2) Suppose the number of labels is M, then the probability that the pixel i to be marked belongs to the label l is:

$\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}$

The specific method of step five is: the confidence degree of the probability of label l in the tracking window estimated by each pixel i to be marked covered by the tracking window is:

$\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}$

Where p _max ( _xi ) and p _min ( _xi ) are the maximum value and minimum value in p( _xi |L=j),j=1,...,M respectively, and j refers to a certain value in the tracking window a label; ε is a constant, usually the value is 1e-3; when the maximum value and the minimum value of the probability density of pixel i are very large or very small, it will be assigned a lower confidence, and pixel i belongs to the tracking window The probability density of each label in the tracking window is very large, which corresponds to the case that the color of the label in the tracking window is similar, and the probability density of each label in the tracking window for pixel i is very small, corresponding to the temporal discontinuity area, find in the input frame Less than associated samples;

j指跟踪窗口内的某个标号，即使用最大概率值所对应的标号来标记像素i。The specific method of step seven is: each tracking window will output a probability p(x _i |L=l) and confidence c(x _i ) belonging to each label l to the pixels covered by each tracking window, denote p′(x i _i |L=l) is the probability corresponding to the one with the highest confidence in all tracking windows covering pixel i, then the label of pixel i is

j refers to a label in the tracking window, that is, the label corresponding to the maximum probability value is used to mark pixel i.

Beneficial effects of the present invention: the present invention adopts a rectangular tracking window, which maintains a relatively small coverage area while ensuring a sufficiently large span. A sufficiently large span can make use of image correlation information that is far apart, thereby achieving processing with discontinuity between frames; while a small coverage area reduces ambiguous features and can maintain a relatively simple feature distribution, thereby reducing The error rate of the classifier. The tracking windows are arranged in different directions, which can realize effective processing of different directions of motion and different shapes of regions to be marked, and can more effectively use the spatial context relationship to further reduce errors caused by ambiguous features.

Description of drawings

Figure 1 is a schematic diagram of video foreground segmentation based on pixel label propagation, where the question mark represents the segmentation result to be solved;

Fig. 2 is a schematic diagram of a traditional square tracking window;

Fig. 3 (a) is the directional horizontal tracking window provided by the present invention;

Figure 3(b) is the directional 45-degree tracking window provided by the present invention;

Figure 3(c) is the tracking window in different directions centered on the same pixel provided by the present invention;

Figure 4(a) is a schematic diagram of the present invention using long-distance image association to process discontinuity between frames;

Figure 4(b) is a schematic diagram of the best tracking window where the present invention uses directionality to deal with different situations and where each position of the target image is located;

Figure 5(a) is a schematic diagram of the combination of results obtained by the horizontal tracking window;

Figure 5(b) is a schematic diagram of the merging of the results obtained by the 45° tracking window;

Figure 5(c) is a schematic diagram of the merging of the results obtained by the 90° tracking window;

Figure 5(d) is a schematic diagram of the merging of the results obtained by the 135° tracking window;

Figure 5(e) is a schematic diagram of the final output of the results of the tracking window in all directions;

Figure 5(f) is a schematic diagram of the direction selected at each pixel;

Figure 6(a) is the input frame image;

Figure 6(b) is the target frame image;

Figure 6(c) is the rendering of a square window used for video matting;

Figure 6(d) is the rendering of the directional tracking window used for video matting.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

As shown in FIG. 1 , video segmentation is taken as an example below, where the question mark represents the result to be solved, and the present invention will be further described in conjunction with the accompanying drawings.

For video segmentation based on label propagation, the key problem to be solved is to use the segmentation result of the previous frame (input frame) to segment the current frame (target frame). The segmentation result is expressed as a binary image, the foreground pixel value is 255, and the background pixel value is 0. The main difficulty in this process is to overcome the contradiction between the color similarity between the foreground and the background and the temporal discontinuity of the video. The method of foreground and background segmentation using directional tracking window is as follows:

1) Expand the segmentation result of the input frame by 30-70 pixels (can be adjusted appropriately according to the speed of the foreground movement), and use the expanded foreground area as the area to be marked in the target image, and other areas are considered as the background;

2) Set the width of the tracking window to W pixels (W is generally taken as 15), and first arrange the horizontal tracking window. The specific method is: scan from top to bottom to the first row containing the region to be marked, denoted as r ₀ ; respectively Take the r _0th line and the r ₀ +W-1th line as the upper and lower ends of the first tracking window; calculate the start and end columns of the pixels to be marked in these lines, and set them as the left end and the end of the first tracking window respectively Right end; take r ₀ +2W/3 as the starting line of the second tracking window, and take r ₀ +2(k-1)W/3 as the starting line of the kth tracking window (adjacent tracking There is an overlapping area of W/3 between the windows), and the subsequent tracking windows are arranged in the same way until all pixels to be marked are completely covered; the resulting horizontal tracking window is shown in Figure 3(a);

3) Rotate the target image 45 degrees clockwise, arrange the horizontal tracking window in the method of 2), and then rotate 45 degrees counterclockwise to obtain the tracking window in the direction of 45 degrees, the result is shown in Figure 3(b);

4) Arrange tracking windows of 90 degrees and 135 degrees in the manner of 3) as shown in Figure 3(c);

5) For each tracking window, use the RGB colors of the foreground and background pixels covered in the input frame as samples, and train the Gaussian Mixture Model (GMM) p(x|F) of the foreground and background color distribution ) and p(x|B), where x is the color of the pixel to be marked;

6) For each tracking window, calculate the probability density p( _xi |F) and p( _xi |B) of each pixel i covered by it in the foreground and background GMM, and then calculate its foreground Probability:

$\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span>}$

7) For each tracking window, calculate its confidence for the estimated probability of each pixel i to be marked:

$\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>}{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}$

Where ε is a constant, usually 1e-3. The above formula means that when the probability density of the color of pixel i in the foreground and background distributions is very large or very small, it will be assigned a lower confidence level, and the probability density of the color of pixel i in the foreground and background distributions is very high Large corresponds to the case where the foreground and background colors are similar, while the color of pixel i has a small probability density in both the foreground and background corresponds to a temporal discontinuity region where no associated samples are found in the input frame;

8) Process all tracking windows in all directions sequentially;

9) Since each pixel will be covered by multiple tracking windows, the classifier of each window will output a probability p( _xi ) and confidence c(xi ₎ belonging to the foreground for the pixel, so one of them needs to be selected as the final output, as shown in Figure 5. Denote p′(xi ₎ as the probability corresponding to the one with the highest confidence among all tracking windows covering pixel i, then if p′(xi ₎ >0.5, mark pixel i as foreground; otherwise, mark it as background.

Figure 4(a) is a schematic diagram of how rectangular tracking uses long-distance image correlation to deal with inter-frame discontinuity. Figure 4(b) is a schematic diagram of how to use directionality to deal with different situations. Figure 4(b) shows where each position is The best tracking window at .

Schematic diagram of merging the results of tracking windows in different directions, as shown in Figure 5(a), Figure 5(b), Figure 5(c), Figure 5(d), as shown in Figure 5(e), for all directions The schematic diagram of the final output of the results obtained by the tracking window of ; Figure 5(f) is a schematic diagram of the direction selected at each pixel;

Comparing the effects of rectangular tracking window and square window for video matting, the rectangular window can correctly identify the newly-appeared area between the legs, while the square window can wrongly identify the area as the foreground, as shown in Figure 6(a), Fig. 6(b), Figure 6(c), and Figure 6(d).

As shown in Figure 2, it is a traditional square tracking window and its problems when dealing with discontinuities between video frames.

Although the specific implementation of the present invention has been described above in conjunction with the accompanying drawings, it does not limit the protection scope of the present invention. Those skilled in the art should understand that on the basis of the technical solution of the present invention, those skilled in the art do not need to pay creative work Various modifications or variations that can be made are still within the protection scope of the present invention.

Claims (8)

1. A pixel label propagation method based on directional tracking window, characterized in that, the concrete steps are: Step 1: expand the area to be propagated in the input image by 30-70 pixels, and the result is used as the area to be marked in the target image; Step 2: For all specified directions, arrange tracking windows along each direction in the target image, so that the tracking windows in each direction completely cover the area to be marked; Step 3: For each tracking window, take the pixels covered by the tracking window in the input frame as samples, and use the pixel color as a feature, and establish a corresponding Gaussian mixture model p(x|L) for each label L to represent its color distributed; Step 4: For each tracking window, calculate the probability that each pixel covered by it belongs to each label; Step 5: For each tracking window, calculate its confidence degree for the estimated probability of each pixel to be marked; Step 6: Process all tracking windows in all directions in turn; Step 7: For each pixel to be marked, record the probability and confidence calculated by all tracking windows covering the pixel, and determine the label of the pixel with the probability output by the window with the highest confidence. 2. the pixel label propagation method based on directional tracking window as claimed in claim 1, is characterized in that, the width of described tracking window is determined, the directional window of variable length, and the width of described tracking window is W pixels. 3. the pixel label propagating method based on directional tracking window as claimed in claim 1, is characterized in that, the concrete steps of described step 2 are: (2-1) First arrange the horizontal tracking window, scan from top to bottom to the first row containing the area to be marked, denoted as r ₀ ; take r _0th row and r ₀ + W-1 row as the first tracking The upper and lower ends of the window; calculate the start column and end column of the pixels to be marked in these rows, that is, scan from left to right, the column containing the first pixel to be marked is the start column, and the column containing the last pixel to be marked is the end column, and is set to the left end and right end of the first tracking window respectively; the r ₀ +2W/3th line is the starting line of the second tracking window, and the r ₀ +2(k-1)W/ 3 is the starting line of the k-th tracking window, and there is an overlapping area of W/3 between adjacent tracking windows. Follow-up tracking windows are arranged in the same way until all pixels to be marked are completely covered, and k is a natural number ; (2-2) For any other direction θ, first rotate the target image clockwise by θ degrees, arrange the tracking window according to the method of arranging the horizontal tracking window in step (2-1), and then rotate the target image counterclockwise by θ degrees, Get the tracking window in the θ direction. 3. the pixel label propagation method based on directional tracking window as claimed in claim 1, is characterized in that, the specific form of the Gaussian mixture model p(x|L) of described step 3 is

Where N is a normal distribution, π _k , σ _k are their mean and variance respectively, ω _k is the weight of the kth item, K is the number of Gaussian items, and the parameters π _k , σ _k , ω _k are all maximized by the expected value Algorithm to get. 4. the pixel label propagation method based on directional tracking window as claimed in claim 1, is characterized in that, the concrete steps of described step 4 are: (4-1) Record the Gaussian mixture model of the color distribution of the pixel labeled l in the tracking window as p(x|L=l); (4-2) Suppose the number of labels is M, then the probability that the pixel i to be marked belongs to the label l is: $\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}$ 5. The pixel label propagation method based on directional tracking window as claimed in claim 1, characterized in that, the specific method of said step 5 is: each pixel i to be marked that is covered by the tracking window is estimated to belong to the label in the tracking window The confidence level of the probability is: $\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}$ Where p _max ( _xi ) and p _min ( _xi ) are the maximum and minimum values in p( _xi |L=j),j=1,...,M respectively; ε is a constant, usually taking the value is 1e-3; when the maximum value and the minimum value of the probability density of pixel i are both very large or very small, it will be given a lower confidence, and the probability density of each label in the tracking window belonging to pixel i is very large Corresponds to the case where the labels in the tracking window have similar colors, and the probability density of pixel i belonging to each label in the tracking window is small. Corresponds to temporally discontinuous regions, and no associated samples can be found in the input frame. 6. The pixel label propagation method based on directional tracking window as claimed in claim 1, characterized in that, the specific method of said step 7 is: each tracking window will output a pixel label belonging to each label l to its covered pixel. The probability p( _xi |L=l) and the degree of confidence c( _xi ), denote p′( _xi |L=l) as the probability corresponding to the one with the highest confidence degree among all the tracking windows covering pixel i, Then the label of pixel i is

That is, the pixel i is marked with the label corresponding to the maximum probability value. 7. The method for propagating pixel labels based on directional tracking windows according to claim 1, wherein the windows in the same direction are parallel to each other. 8. The pixel label propagation method based on directional tracking windows as claimed in claim 1, wherein there is no gap between adjacent windows in the same direction and there is a certain overlapping area.

Images