CN102509308A - Motion segmentation method based on mixtures-of-dynamic-textures-based spatiotemporal saliency detection - Google Patents

Abstract

The invention discloses a motion segmentation method based on mixtures-of-dynamic-textures-based spatiotemporal saliency detection. The method comprises the following steps of: 1, modeling a background by adopting mixtures of dynamic textures; 2, defining a spatiotemporal saliency discrimination function by utilizing a Kullback-Leibler (KL) divergence, and calculating a saliency map; and 3, processing a saliency threshold value to obtain a motion segmentation result. By the method, a motional object can be accurately segmented in a complex environment with a highly-dynamic background and a motional camera; and compared with the conventional method, the invention greatly improves complex scene processing and noise suppression, has relatively higher robustness and can be applied to various complex motion scenes.

Description

The motion segmentation method that conspicuousness detects during based on mixing dynamic texture sky

Technical field

The present invention relates to a kind of moving Object Segmentation method of technical field of information processing, specifically is a kind of based on mixing dynamic texture (Mixtures of Dynamic Textures, the motion segmentation method that conspicuousness detects when MDT) empty.

Background technology

The task of motion segmentation is from video sequence, as far as possible intactly to extract moving target.This task has two difficult points: (1) high dynamic background, and motion segmentation is run into the situation that background is also moved through regular meeting, such as the branch that rocks, sleet, pedestrian, water wave etc.; (2) camera motion, in a lot of practical applications, camera is not fixed.Classic method is made following hypothesis usually and simplified this problem: (1) camera is actionless; Perhaps (2) camera motion parameter can be obtained or calculate; Perhaps (3) background satisfies given model, like the mixed Gaussian background model.These supposition have reduced the complex nature of the problem, but in actual scene, these supposition might not be satisfied.Classic method is applied in the dynamic background, and the segmentation result noise is a lot, disturbs bigger.

Conspicuousness detects to solving these difficult problems and brings new thinking, and the task that conspicuousness detects is meant out that two ways becomes the focus that human vision is paid close attention to more easily in the scene.Different with motion segmentation, conspicuousness detects and does not distinguish prospect and background, does not provide precise results, and provides the conspicuousness figure of similar gray-scale map.The tradition conspicuousness detects and roughly can be divided into 3 types.The first kind is most popular method, and it regards the conspicuousness problem as certain perceptual property characteristic, and the shortcoming of this method is can not excellent popularization, and specific property detector effect in different scenes differs greatly.Second class methods are the complexities that conspicuousness are defined as image.An its very big advantage is exactly its dirigibility, can choose any bottom attribute and come computation complexity; But conspicuousness is defined as image complexity does not obtain checking biologically, have sizable difference with the artificial result who discerns so detect the result who obtains in many instances.Last class methods are based on the model of biological vision, and a common problem of these class methods is to lack the criterion of weighing testing result, and this makes troubles for improving to wait.

To the deficiency that traditional conspicuousness detects, some new methods have been suggested.D.Gao etc. have proposed differentiation conspicuousness method; This method is set up a discriminant function; Classification problem has been introduced the vision significance field, conspicuousness has been detected under complex environment, can obtain good result, but how to define the difficult point that a good discriminant function is this method [D.Gao and N.Vasconcelos.Discriminant saliency for visual recognition from cluttered scenes.In Proc.NIPS; Pages 481-488,2004.].In handling video sequence,, the method that conspicuousness detects when empty has been proposed in order to utilize the information between the frame.V.Mahadevan and Nuno Vasconcelos have proposed a kind of conspicuousness detection method when empty; This method is carried out modeling with dynamic texture to background earlier; Then at the center utilize information calculations conspicuousness [V.Mahadevan and Nuno Vasconcelos.Spatiotemporal Saliency in Dynamic Scenes.IEEE transactions on Pattern Analysis and Machine Intelligence when empty under the neighbour structure framework; Vol.32; No.1, p171-177,2009.].Experimental result shows that people's such as V.Mahadevan method has greatly improved than classic method.But this method is based on single mode dynamic texture model, and in real scene, dynamic background usually is not a single mode, but multi-modal, the single mode dynamic texture can't the accurate description real scene.

Summary of the invention

The present invention is directed to the deficiency that above-mentioned prior art exists; Propose a kind of based on mixing the dynamic texture motion segmentation method that conspicuousness detects when empty; This method still can be partitioned into moving target more accurately among the complex environment of high dynamic background and camera motion; Be greatly improved aspect complex scene processing and the squelch than classic method, and having stronger robustness, can be adapted to various compound movement scenes.

The present invention realizes that through following technical scheme it comprises three steps:

The first step: adopt the mixing dynamic texture that background is carried out modeling;

Second step: conspicuousness discriminant function and calculate conspicuousness figure when utilizing the KL distance definition empty;

The 3rd step: conspicuousness figure threshold process is obtained the motion segmentation result.

Below the inventive method each several part is carried out detailed explanation:

1, adopt the mixing dynamic texture that background is carried out modeling

In order to solve these two difficult points of high dynamic background and camera motion, the present invention adopts mixing dynamic texture model that background is carried out modeling.Dynamic texture is meant, some stability property that each frame of video sequence is showed on time and space.Common dynamic texture scene has: water wave, smog, branch, sandy beach, crowd, sleet etc.The dynamic texture model can be realized the modeling to these complex scenes.

Given video sequence Y, it represents observed sequence frame (y ₁..., y _τ), given K dynamic texture { Θ ₁..., Θ _K, the probability that generates video Y is respectively { α ₁..., α _K, and

The Y generative process can be modeled as:

1) at first from probability distribution { α ₁..., α _KMiddle sampling dynamic texture component Θ _j, its probability is α _j

2) at dynamic texture component Θ _jLast sampling generates video sequence Y, and its probability can be expressed as conditional probability p (Y| Θ _j).Can calculate by total probability formula

$p\left(Y\right)=\underset{j=1}{\overset{K}{\mathrm{\Σ}}}{\mathrm{\α}}_{j}p\left(Y\right|{\mathrm{\Θ}}_j)---\left(2\right)$

Formula (2) is mixes the dynamic texture model.In this model, the parameter that needs to confirm is Θ={ Θ ₁..., Θ _K, α ₁..., α _K.

In order to obtain mixing dynamic texture parameter Θ, adopt the maximum likelihood method for parameter estimation, utilize video sequence Y that parameter is learnt.Choose maximization expectation algorithm (EM) during practical implementation, it progressively obtains the optimum solution of parameter with the mode of iteration.

Conspicuousness discriminant function and calculate conspicuousness figure when 2, utilizing the KL distance definition empty

In order to utilize temporal information, V is made as three-dimensional matrice with video sequence, and each element representation is l (m, n, t) ∈ L=R ³, wherein (m n) is coordinate position in the frame of video, and t is a time coordinate, is unit with the frame.If x is the proper vector (extracting direction character, color characteristic etc. like pixel value, Gabor wave filter) that each l ∈ L is extracted, the classification information at l place be C (l) ∈ 0,1}, wherein 1 expression this locate to be moving target, 0 representes to belong to background.Because employing center-neighborhood framework, two windows setting element to represent respectively center and neighborhood, use

The window of expression central area,

The window in expression neighborhood zone, total window definition does

The characteristic of central area Use conditional probability p _{Y|c (l)}(y|1) expression, the characteristic of neighborhood

Use conditional probability p _{Y|c (l)}(y|0) expression.Define conspicuousness S (l) as follows:

$S\left(l\right)=\underset{c=0}{\overset{1}{\mathrm{\Σ}}}{p}_{c\left(l\right)}\left(c\right)\mathrm{KL}\left(p_{Y|c\left(l\right)}\left(y\right|c)\right|\left|p_Y\left(y\right)\right)---\left(3\right)$

Wherein KL () expression KL distance is called relative entropy again, generally is used for weighing two differences between the probability distribution, is defined as:

$\mathrm{KL}\left(p\right|q)={\∫}_X{p}_X\left(x\right)\log \frac{p_X\left(x\right)}{q_X\left(x\right)}\mathrm{dx}---\left(4\right)$

(3) the formula definition shows that bigger conspicuousness shows that this center, place and neighborhood characteristics differ greatly, and can distinguish target and background with less differentiation error, so the classification C of center (l) is likely moving target, rather than the background of dynamic texture.Like this, the size of pixel conspicuousness value has reflected that this place belongs to the size of moving target probability.

3, conspicuousness figure threshold process is obtained the motion segmentation result

Conspicuousness figure S is the image of half-tone information, in order to obtain the result of motion segmentation, it is made threshold process: the pixel that is lower than threshold value as a setting; And being higher than its value of reservation of threshold value, it belongs to the size of moving target probability the size reflection of value.

The present invention's remarkable result compared with prior art is: proposed based on the time and space significance detection method of dynamic texture model with utilize its significant result to make motion segmentation, realized under the complex environment of high dynamic background and camera motion, cutting apart the target of motion.Comparison with the conventional motion dividing method; Institute of the present invention extracting method is introduced vision significance and is cut apart; Because vision significance is based on the biological vision system principle; Its very important characteristic be exactly the center the neighborhood framework, just can obtain conspicuousness figure through calculating local feature difference, so insensitive to dynamic background and camera motion.And this conspicuousness method combines when empty conspicuousness and differentiates these two kinds of newer methods of conspicuousness, and they can improve the precision that conspicuousness detects, and are obvious to interference suppressioning effect.The inventive method has realized under the complex environment of dynamic background and camera motion, cutting apart the task of moving target basically.

Description of drawings

Fig. 1 is a Surfers cycle tests in the embodiment of the invention;

Fig. 2 is a Surf cycle tests in the embodiment of the invention;

Fig. 3 is a Cyclist cycle tests in the embodiment of the invention.

Embodiment

Elaborate in the face of embodiments of the invention down, present embodiment has provided detailed operating process being to implement under the prerequisite with technical scheme of the present invention.

The present invention provides a kind of motion segmentation method that conspicuousness detects when empty based on MDT, and the flow process of this method is:

Given video sequence V, each element are l (m, n, t) ∈ L=R ³, the dimension of state space x is n, the size of center window is n _cIndividual pixel is used for the topology window size of modeling dynamic texture and is n _p, time window is the τ frame;

Each pixel of each frame is carried out following operation:

Get the center window Size be n _c* n _c* τ, the neighborhood window Size is 4n _c* 4n _c* τ;

With texture modeling window n _p* n _p* τ covers the center window

Go up and move,, use the EM algorithm that its parametric texture is estimated then as texture sampling to the central area.The neighborhood window

With total window w _lCarry out same operation, learn the parameter of mixing dynamic texture separately respectively;

After obtaining mixing the dynamic texture parameter, design conditions probability density function and KL distance by formula (3), obtain the conspicuousness S (l) at this place;

Figure makes threshold process to conspicuousness, the pixel that is lower than threshold value as a setting, and be higher than threshold value keep its value as prospect.

For understanding technical scheme of the present invention better, this method is further described in conjunction with embodiment.

1, adopt the mixing dynamic texture that background is carried out modeling

(1) sets up parameter model

The given video sequence V that comprises dynamic background and/or camera motion puts on 3 dimension coordinate l (m, n, t) ∈ L=R for each location of pixels l ³, wherein (m n) is the coordinate position of pixel in frame, and t is the sequence number of place frame.Certain frame y with sequence _tBe regarded as observation variable, extracting special to this frame is x as its corresponding state variable _t, intrinsic dimensionality n is generally about 10.Choose 2～4 dynamic texture components and set up mixing dynamic texture model, the parameter that needs like this to estimate is the parameter Θ of each component _i={ A, Q, C, R, μ, S} and its corresponding probability of happening α _i

(2) take training sample

1) selected window

This step will be chosen 3 windows: center window, neighborhood window, total window.Each window is with the parameter estimation of mixing the dynamic texture model respectively, and promptly each window is described with different mixing dynamic texture.

For the conspicuousness value at calculated for given pixel l place, access time window τ=11, before and after promptly the frame at pixel place adds each 5 frame totally 11 frames form the time window of training; The size of center window is n _cIndividual pixel promptly is that n is chosen at this frame in the center with l _c* n _cThe window of size, and be stretched to other frame in the time window, obtain the center window

Its size is n _c* n _c* τ considers computation complexity, n _cGenerally be taken as 10～20, this instance gets 16, and promptly obtaining the center window is 16 * 16 * 11.

In like manner choose total window w _l, size 64 * 64 * 11.

Total window w _lWith the center window

Difference be the neighborhood window.

2) choose sample

Selected texture modeling window; Size is 8 * 8 * 11; It is covered that center window

is gone up and moves, and each position is promptly as a sample of central area texture.Obtain 81 samples so altogether, these samples will be used for the mixing dynamic texture parameter of training centre's window.

Selected window corresponding sample in neighborhood window and total window in like manner.

(3) estimated parameter

Adopt the maximum likelihood method for parameter estimation, utilize sample to set up likelihood function.The EM algorithm of utilization iteration progressively obtains the optimum solution of parameter.

Conspicuousness discriminant function and calculate conspicuousness figure when 2, utilizing the KL distance definition empty

After the mixing dynamic texture parameter of 3 windows of difference; Computing center's window and always the KL distance of window respectively; The KL distance of neighborhood window and total window; The method of calculating K L distance is pressed V.Mahadevan and the method for Nuno Vasconcelos in " Spatiotemporal Saliency in Dynamic Scenes ", is added up by formula (3) at last and obtains conspicuousness figure.

3, conspicuousness figure threshold process is obtained the motion segmentation result

The pixel that is lower than threshold value as a setting, and be higher than threshold value keep its value as prospect.Because this method is apparent in view to the inhibition effect of disturbing,, generally between 30～60, and just can obtain good segmentation result after the simple threshold values so selection of threshold can be chosen lower threshold value.

Fig. 1,2,3 is respectively to 3 not homotactic test results, has contrasted the difference of the motion segmentation of classic method and the inventive method in the table.Can find out that from comparing result traditional classical way can not well be described the complex situations that high dynamic background and camera motion combine, the segmentation result effect is bad, have in addition can not provide significant segmentation result; And method of the present invention is owing to choose MDT model as a setting, and combines when empty conspicuousness and differentiate conspicuousness, so inhibition complex environment that can be stronger disturbs, under various test scenes, can both obtain preferable performance.

Claims (6)

1. one kind based on mixing the dynamic texture motion segmentation method that conspicuousness detects when empty, it is characterized in that comprising three steps:

The first step: adopt the mixing dynamic texture that background is carried out modeling;

Second step: conspicuousness discriminant function and calculate conspicuousness figure when utilizing the KL distance definition empty;

The 3rd step: conspicuousness figure threshold process is obtained the motion segmentation result;

Said employing mixes dynamic texture background is carried out modeling, is specially:

Given video sequence Y, it represents observed sequence frame (y ₁..., y _τ), given K dynamic texture { Θ ₁..., Θ _K, the probability that generates video Y is respectively { α ₁..., α _K, and

The Y generative process is modeled as:

1) at first from probability distribution { α ₁..., α _KMiddle sampling dynamic texture component Θ _j, its probability is α _j

2) at dynamic texture component Θ _jLast sampling generates video sequence Y, and its probability tables is shown conditional probability p (Y| Θ _j); Calculate by total probability formula

$p\left(Y\right)=\underset{j=1}{\overset{K}{\mathrm{\Σ}}}{\mathrm{\α}}_{j}p\left(Y\right|{\mathrm{\Θ}}_j)---\left(2\right)$

Formula (2) is mixes the dynamic texture model, and in this model, the parameter that needs to confirm is Θ={ Θ ₁..., Θ _K, α ₁..., α _K.

2. according to claim 1 based on mixing the dynamic texture motion segmentation method that conspicuousness detects when empty; It is characterized in that said mixing dynamic texture parameter Θ; Adopt the maximum likelihood method for parameter estimation; Utilize video sequence Y that parameter is learnt, choose maximization expectation algorithm during practical implementation, it progressively obtains the optimum solution of parameter with the mode of iteration.

3. according to claim 1 based on mixing the dynamic texture motion segmentation method that conspicuousness detects when empty, it is characterized in that said dynamic texture is meant, some stability property that each frame of video sequence is showed on time and space; The dynamic texture scene comprises the water wave; Smog, branch, sandy beach; Crowd and sleet, the dynamic texture model can be realized the modeling to these complex scenes.

4. according to claim 1 based on mixing the dynamic texture motion segmentation method that conspicuousness detects when empty, conspicuousness discriminant function and calculate conspicuousness figure when it is characterized in that the said KL of utilization distance definition is empty is specially:

In order to utilize temporal information, V is made as three-dimensional matrice with video sequence, and each element representation is l (m, n, t) ∈ L=R ³, wherein (m n) is coordinate position in the frame of video, and t is a time coordinate, is unit with the frame; If x is the proper vector that each l ∈ L is extracted, the classification information at l place be C (l) ∈ 0,1}, wherein 1 expression this locate to be moving target, 0 representes to belong to background; Employing center-neighborhood framework, two windows setting element to represent respectively center and neighborhood, use

The window of expression central area,

The window in expression neighborhood zone, total window definition does

The characteristic of central area

Use conditional probability p _{Y|c (l)}(y|1) expression, the characteristic of neighborhood

Use conditional probability p _{Y|c (l)}(y|0) expression; Define conspicuousness S (l) as follows:

$S\left(l\right)=\underset{c-0}{\overset{1}{\mathrm{\Σ}}}{p}_{c\left(l\right)}\left(c\right)\mathrm{KL}\left(p_{Y|c\left(l\right)}\left(y\right|c)\right|\left|p_Y\left(y\right)\right)---\left(3\right)$

Wherein KL () expression KL distance is used for weighing two differences between the probability distribution, is defined as:

$\mathrm{KL}\left(p\right|q)={\∫}_X{p}_X\left(x\right)\log \frac{p_X\left(x\right)}{q_X\left(x\right)}\mathrm{dx}---\left(4\right)$

(3) the formula definition shows; Bigger conspicuousness shows that this center, place and neighborhood characteristics differ greatly, and can distinguish target and background with less differentiation error; So the classification C of center (l) is likely moving target; Rather than the background of dynamic texture, like this, the size of pixel conspicuousness value has reflected that this place belongs to the size of moving target probability.

5. according to claim 1 based on mixing the dynamic texture motion segmentation method that conspicuousness detects when empty; It is characterized in that said conspicuousness figure threshold process being obtained the motion segmentation result; Be specially: conspicuousness figure S is the image of half-tone information; In order to obtain the result of motion segmentation, it is made threshold process, that is: the pixel that is lower than threshold value is as a setting; And being higher than its value of reservation of threshold value, it belongs to the size of moving target probability the size reflection of value.

6. according to claim 1 or 5 described, it is characterized in that said selection of threshold is between 30～60 based on mixing the dynamic texture motion segmentation method that conspicuousness detects when empty.

Images