CN105701823A - Method of using occlusion relation to recover depth order - Google Patents

Abstract

The invention discloses a method of using an occlusion relation to recover a depth order. The method comprises the following steps of carrying out background scene segmentation; carrying out moving object segmentation, based on a background scene segmentation graph, updating frames of superpixel images of a scene one by one, which means that a superpixel area corresponding to a moving object is determined through a background subtraction method in the background scene where the moving object appears, and acquiring a segmentation graph of the moving object in the background scene; constructing an area-level Markov random field; based on a space-time graph, carrying out depth order inference of a pairing area so as to acquire an occlusion matrix; based on the above occlusion matrix, carrying out global depth order inference so as to acquire a depth order relation of the background image. The method can realize an advantage of enhancing a depth level recovery effect.

Description

The method being recovered depth order by hiding relation

Technical field

The present invention relates to computer vision, area of pattern recognition, in particular it relates to a kind of method being recovered depth order by hiding relation。

Background technology

Have from the degree of depth hierarchical relationship of two-dimensional image sequence restoration scenario and be widely applied very much。Video monitoring is commonly installed a fixing photographic head, one or more moving objects in record scene。Along with object of which movement is through scene, researcher can the block information of restoration scenario and hierarchy representation accordingly。But, what still camera was difficult to obtain corresponding scene image is effectively matched clue, for instance edge and illumination, thus causing that object edge detection and degree of depth level inferred results are inaccurate。

Method based on study is used for estimating that the degree of depth of scene and order have made exploration before from single image。Later, object of study was transferred to image sequence from single image by researcheres。Fouhey et al. is it is found that obtain in room scene areas substantially by being in the human posture in mixed and disorderly room。

During according to moving object through scene, its federation and the part generation hiding relation in scene, thus G.Brostow et al. obtains sparse paired level orbution by moving object traverse static scene。Although these clues are effective, but sparse so that the target that each pixel carries out tight depth order sequence becomes difficulty。A.Schodl make use of the hierarchical relationship blocking clue in pairs to obtain moving object and the background area of its process too。But, moving object can not pass all regions in background scene, thus the region functioned is limited, causes that some region is not assigned to degree of depth hierarchical sequence。Additionally, the problem that also can there is over-segmentation。

It is an extremely important and classical problem in computer vision that the degree of depth level of image is estimated。But, due to the limitation of tradition estimating method, for instance what still camera was difficult to obtain corresponding scene image has only used single clue etc. when being effectively matched clue or depth reasoning, result in the inference that result is barely satisfactory。

Summary of the invention

It is an object of the invention to, for the problems referred to above, it is proposed to a kind of method being recovered depth order by hiding relation, to realize the advantage strengthening degree of depth level recovery effects。

For achieving the above object, the technical solution used in the present invention is:

A kind of method being recovered depth order by hiding relation, including:

Step 1, background scene are split, and utilize still camera to obtain background image, and background image is carried out over-segmentation by recycling Meanshift dividing method, obtains the background scene segmentation figure that super-pixel block zonule is formed；

Step 2, moving meshes, based on above-mentioned background scene segmentation figure, remove the super-pixel image of more new scene one frame frame, namely in the background scene occur moving object, delete that method determines the super-pixel region that moving object is corresponding by background, obtain moving object segmentation figure in background scene；

Step 3, structure realm rank markov random file, based on the moving object obtained above segmentation figure in background scene, using each region in segmentation figure as a node, if two regions are adjacent namely edge, then corresponding two nodes are connected, for the moving object on each two field picture of still camera shooting, also serve as a node；Meanwhile, adding time edge and connect the node of the moving object on consecutive frame, thus constructing the space-time diagram with regional occlusion relation, this space-time diagram had both contained spatial information, contained again moving object information under different time；

Step 4, carry out the depth order reasoning in paired region based on above-mentioned space-time diagram, thus obtaining blocking matrix；

Step 5, carry out global depth order reasoning according to the above-mentioned matrix that blocks, thus drawing the depth order relationships of background image。

Preferably, in step 2 in the background scene occur moving object by background delete method determine super-pixel region corresponding to moving object particularly as follows:

First the scene of background being expressed and be modeled, each of which pixel all obeys the Gauss distribution Ap centered by its average color in all two field pictures, after given Ap, for every two field picture, estimate this pixel and belong to the probability of background, if probability is more than 90%, then it is assumed that be background pixel；If probability is lower than 10%, think the pixel of moving object。

Preferably, described step 4 based on above-mentioned space-time diagram carry out paired region depth order reasoning particularly as follows:

Super-pixel based on scene is expressed, set up one and block matrix O for judging the hiding relation of a pair region i, j, wherein Oi, j ∈ {+1,-1,0}, corresponding region i occlusion area j, region i are blocked by region j and unobstructed clue these three situation respectively, and the depth order in region is gone to infer by two kinds of clues, block matrix O thus updating frame by frame；

First block clue according to motion and go to judge the depth order of moving object and background area, according to moving meshes, obtain the boundary pixel of moving object and the boundary pixel of background area, thus inferring that moving object is before this background area or below, what then update correspondence position blocks matrix O；

Then, utilize monocular clue to carry out regional occlusion judgement for the region that the space not updated in scene is adjacent, and update correspondence position block matrix O。

Preferably, described step 5 according to above-mentioned block matrix carry out global depth order reasoning particularly as follows:

For each the super-pixel region in background scene distribute deep tag 1,2 ..., L}, wherein L is presetting, L numeral show more greatly from photographic head more away from；Therefore the segmentation problem of multi-tag is converted to the energy minimization problem based on space-time diagram model, space-time diagram comprises n+F node, n super-pixel Area Node of n correspondence background scene, corresponding each the moving object node under the F two field picture of video of F, thus target is exactly try to achieve the arrangement X={X of a sounding mark₁,...,X_n+F}；

Define an energy function based on MRF space-time diagram as follows:

$E\left(X\right)=\underset{i\∈1,...,n+F}{\Σ}{E}_i\left(X_i\right)+\underset{(i,j)\∈N_S}{\Σ}{E}_{ij}^S(X_i,X_j)+\underset{(i,j)\∈N_T}{\Σ}{E}_{ij}^T(X_i,X_j);$

Wherein, E_iRepresent the cost distributing to a certain Area Node depth order label, Xi ∈ X, Xj ∈ X；

For the paired item in space, N_SRepresent synergistic region in background scene,

Wherein,

$\begin{array}{c}{E}_{ij}^{S,f}(X_i,X_j)=\left\{\begin{array}{cc}-log(c_{ij}^f\&times;\frac{1+O_{i,j}^f+\mathrm{\&epsiv;}}{2})& \&ForAll;X_i<X_j\\ \mathrm{\&gamma;}& X_i=X_j\\ -log(c_{ij}^f\&times;\frac{1+O_{i,j}^f+\mathrm{\&epsiv;}}{2})& \&ForAll;X_i>X_j\end{array}\right.\\ E_{ij}^S(X_i,X_j)=\underset{f\&Element;F}{\&Sigma;}\left(E_{ij}^{S,f}\right(X_i,X_j)\&times;\exp (-{\mathrm{\&delta;}}_{i,j}^f\left)\right)\end{array}$

It is the paired space item of f two field picture,It is the hiding relation of region i and j under f frame,The confidence level of corresponding hiding relation, ε and γ is known coefficient,Represent and utilize relevant provincial characteristics to judge the coplanar probability of region i, j each frame f；

For time paired item, N_TRepresent the time edge of moving object。

Technical scheme has the advantages that

(1) it is proposed by first combining and blocks clue according to the motion of moving object gained and blocked the clue of deduction by monocular image static scene is carried out the framework of degree of depth level deduction, before making up, only use the deficiency of single depth reasoning clue；(2) knowledge of graph theory is utilized, construct the MRF based on adjacent area and middle edge thereof, and the moving object node between two field picture is attached by time edge, it is ensured that moving object video auspicious between the fluency of movement so that the MRF of structure has spatiotemporal；(3) Moving Objects in the middle of method is arbitrary, it is possible to be people can also be other can paleocinetic object, and be generalized to multiple moving object and under scene, carry out level judge to stand good；(4) global depth order infer time by moving object through whole video all auspicious and set up MRF structure energy minimization function solve, item weak for deduction ability own is combined, constitute strong overall situation function so that the result that ultimate depth is inferred is more efficient。

Technical scheme, to carrying out depth order reasoning and again look back from blocking clue, and deals with to the limitation before it and improves。In the technical program, degree of depth level segmentation problem is changed into and processes based on image sequence discrete markers problem under spatiotemporal markov random file (MRF), for degree of depth level recovery effects, the method being better than similar research now, and employ the correctness of L.Guan et al. two sets of video data SET-A (single movement object) issued and SET-B (multiple moving object) verification algorithm。The technical program method is simple, and effect is better。

Below by drawings and Examples, technical scheme is described in further detail。

Accompanying drawing explanation

Fig. 1 is the flow chart of the method being recovered depth order by hiding relation described in the embodiment of the present invention；

Fig. 2 is background scene segmentation schematic diagram；

Fig. 3 is the segmentation schematic diagram of moving object；

Fig. 4 is the MRF model schematic of region class；

Fig. 5 is paired depth reasoning schematic diagram；

Fig. 6 is the paired item schematic diagram in space；

Fig. 7 is time paired item schematic diagram；

Fig. 8 is the depth reasoning schematic diagram of multiple moving object。

Detailed description of the invention

Below in conjunction with accompanying drawing, the preferred embodiments of the present invention are illustrated, it will be appreciated that preferred embodiment described herein is merely to illustrate and explains the present invention, is not intended to limit the present invention。

As it is shown in figure 1, a kind of method being recovered depth order by hiding relation, including:

Step 1, background scene are split, and utilize still camera to obtain background image, and background image is carried out over-segmentation by recycling Meanshift dividing method, obtains the background scene segmentation figure that super-pixel block zonule is formed；

Step 2, moving meshes, based on above-mentioned background scene segmentation figure, remove the super-pixel image of more new scene one frame frame, namely in the background scene occur moving object, delete that method determines the super-pixel region that moving object is corresponding by background, obtain moving object segmentation figure in background scene；

Step 3, structure realm rank markov random file, based on the moving object obtained above segmentation figure in background scene, using each region in segmentation figure as a node, if two regions are adjacent namely edge, then corresponding two nodes are connected, for the moving object on each two field picture of still camera shooting, also serve as a node；Meanwhile, adding time edge and connect the node of the moving object on consecutive frame, thus constructing the space-time diagram with regional occlusion relation, this space-time diagram had both contained spatial information, contained again moving object information under different time；

Step 4, carry out the depth order reasoning in paired region based on above-mentioned space-time diagram, thus obtaining blocking matrix；

Step 5, carry out global depth order reasoning according to the above-mentioned matrix that blocks, thus drawing the depth order relationships of background image。

Below in conjunction with accompanying drawing, detailed description of the invention is carried out as described below:

1. background scene segmentation:

The scene not having moving object in each two field picture of video is exactly background scene。Thus, background image is obtained first with still camera, background image is carried out over-segmentation by recycling Meanshift dividing method, obtains the background scene segmentation figure of about 300 super-pixel block zonules formation as the background template of this series of frame images, is namely applied on each two field picture。Do so had both largely remained the marginal information of background scene, the region shape of formation also more systematicness, it is simple to moving object is processed through background scene。The image obtained is as shown in Figure 2。

2. moving meshes

In simple terms, when the segmentation of moving object is exactly moving object traverse background scene, moving object and scene can be distinguished。After given background scene template, remove the super-pixel image of more new scene a frame frame, namely in the background scene occur moving object, delete that method determines the super-pixel region that moving object is corresponding by background。Specific practice is: first the scene of background is expressed being modeled, and each of which pixel all obeys the Gauss distribution A centered by its average color in all two field pictures_p。Given A_pAfter, for every two field picture, estimate this pixel and belong to the probability of background。If probability is more than 90%, then can fully think background pixel；If lower than 10%, fully think the pixel of moving object。So, using the result the estimated result as preliminary motion segmentation。Then, accordingly a model of place is learnt for background and moving object both are made a distinction。The Graph-cuts dividing method utilizing iteration updates the scene color model of background and moving object to carry out the segmentation of moving object, and each of which time iteration is similar to C.Rother et al. GrabCut algorithm done。So, the moving meshes result under the super-pixel figure of background scene and each two field picture is just obtained。The image obtained is as shown in Figure 3。

3. the markov random file (MRF) of structure realm rank:

Here a non-directed graph model is mainly constructed。Based on the super-pixel figure of background scene, using each region (super-pixel) as a node, if two regions are adjacent namely edge, then corresponding two nodes are connected。For the moving object on each two field picture, also serve as a node。Meanwhile, also added time edge to connect the node of the moving object on consecutive frame。Structure result is as shown in Figure 4, the region of background scene is in Fructus Citri tangerinae color square frame, and f represents frame number, and a node represents the moving object under each frame, connect limit a2 and represent that two regions exist a pair depth order relationships, connect the fluency that limit a1 is then the motion model in order to strengthen moving object。The graph model being constructed such that out is a kind of space-time diagram, and it had both contained spatial information, contained again moving object information under different time。

4. the depth order reasoning in paired region:

When moving object is through background scene, or block object scene, or blocked by object scene, claim it to block event do motion。Super-pixel based on scene is expressed, and sets up one and blocks matrix O for judging the hiding relation of a pair region i, j, wherein O_i,j{+1 ,-1,0}, corresponding region i occlusion area j, region i are blocked and unobstructed clue these three situation ∈ by region j respectively。Obviously, O is an antisymmetric matrix。Going to infer the depth order in region here by two kinds of clues, blocking matrix O thus updating frame by frame。

First block clue according to G.Brostow et al. motion proposed and go to judge the depth order of moving object and background area。According to moving meshes, the boundary pixel of moving object and the boundary pixel of background area can be obtained, thus inferring that moving object is before this background area or below, what then update correspondence position blocks matrix O according to its clue。Specifically, this clue has transitivity。Such as moving object region is m, and when triggered motion blocks event, background area k has blocked moving object m, and moving object m has blocked again region s simultaneously, then according to its transitivity it can be inferred that region k has blocked region s。Additionally, due to k and s is not limited to adjacent region, the edge being therefore at non-conterminous interregional long scope is also suitable。

But, owing to moving object can not be overlapping with each region on background image, utilizing D.Hoiem et al. monocular clue proposed to carry out regional occlusion judgement thus for other region in scene, the region being wherein primarily directed to the space not updated before those adjacent carries out level deduction。But, the reliability of monocular clue blocks clue not as motion, thus does not have transitivity。The image obtained is as shown in Figure 5。

5. global depth order reasoning:

Be can be seen that it is the reasoning of a kind of local hierarchy by the paired regional depth order reasoning of previous step, for consistency constraint local inferred results in the overall situation, establish an energy function, and obtain final result by solving its minimization problem。

For each the super-pixel region in background scene distribute deep tag 1,2 ..., L}, wherein L is that predefined is good, numeral show more greatly from photographic head more away from。So, the segmentation problem of multi-tag is converted to the energy minimization problem based on space-time diagram model。Space-time diagram comprises n+F node, n super-pixel Area Node of n correspondence background scene, corresponding each the moving object node under the F two field picture of video of F, thus target be exactly try to achieve the arrangement X={X of a sounding mark₁,...,X_n+F}。For this, define an energy function based on MRF space-time diagram as follows:

$E(-X)=\underset{i\&Element;1,...,n+F}{\&Sigma;}{E}_i\left(X_i\right)+\underset{(i,j)\&Element;N_S}{\&Sigma;}{E}_{ij}^S(X_i,X_j)+\underset{(i,j)\&Element;N_T}{\&Sigma;}{E}_{ij}^T(X_i,X_j)---\left(1\right)$

This functional expression comprises three parts:

Unitary item E_iRepresent the cost distributing to a certain Area Node depth order label。Owing to moving object can move between neighboring background region, if now the label in the two region is connected to, then the label can not had to distribute to moving object。In order to avoid this phenomenon, background area is only distributed odd number label, i.e. mould 2 label, thus can guarantee that and vacated level label between neighboring background level to moving object。Thus, if background area has been assigned to even number label, then it is added infinitely-great punishment, namely cost is infinitely great。

The paired item in spaceCorresponding to synergistic region N_SThe cost of the paired regional depth order (blocking matrix O) that clue is blocked in the motion obtained and monocular clue deduces。Assume O_i,jFor+1, represent region i occlusion area j, thus the label of i should less than j。Now, if the label of i distribution is more than j, then need it is made huge punishment。For this, according to A.Kowdle et al. work, define following formula:

$\begin{array}{c}{E}_{ij}^{S,f}(X_i,X_j)=\left\{\begin{array}{cc}-log(c_{ij}^f\&times;\frac{1+O_{i,j}^f+\mathrm{\&epsiv;}}{2})& \&ForAll;X_i<X_j\\ \mathrm{\&gamma;}& X_i=X_j\\ -log(c_{ij}^f\&times;\frac{1+O_{i,j}^f+\mathrm{\&epsiv;}}{2})& \&ForAll;X_i>X_j\end{array}\right.\\ E_{ij}^S(X_i,X_j)=\underset{f\&Element;F}{\&Sigma;}\left(E_{ij}^{S,f}\right(X_i,X_j)\&times;\exp (-{\mathrm{\&delta;}}_{i,j}^f\left)\right)\end{array}---\left(2\right)$

WhereinIt is the paired space item of f two field picture,It is the hiding relation of region i and j under f frame,The confidence level of corresponding hiding relation, ε and γ is parameter,Then come from the coplanar grader of one of design in A.Kowdle et al. research work,Represent each frame f utilizes relevant provincial characteristics judge the coplanar probability of region i, j is much。Making those edge confidence degrees being associated with moving object is 1, other edge then according to D.Hoiem et al. utilize block edge strength as confidence level, thus the paired item under each frame linearly being sued for peace just can effectively capture two Area Nodes block information in whole image sequence。Dotted line 2 region mean terms as shown in Figure 6, if region i occlusion area j, is then implemented zero punish that the label that i is distributed is less than j by the item in dotted line 1 region is added bigger punishment by the paired item schematic diagram in space。

Time paired itemThe time edge N of moving object under corresponding different frame_TGo to update, be used for punishing the situation that the label of Moving Objects in Image Sequences node is inconsistent, it is ensured that its fluency moved。Owing to moving object is to move through scene, thus will not there is suddenly great variety in the degree of depth level of object。Thus, for this situation, it is necessary to it is made bigger punishment。The schematic diagram of time paired item as it is shown in fig. 7, penalty term position from diagonal distance more away from, punishment numerical value is more big, thus ensureing that the degree of depth level of moving object is slowly varying。

Above three respective deduction abilities are more weak, thus, combine them, constitute strong global depth order estimating method。Minimum of a function is solved, utilizes the TRW-S method that V.Kolmogorov proposes, thus drawing the deep tag arrangement of final scene areas, i.e. degree of depth hierarchical sequence。

6. extend to multiple moving object problem:

Picture depth level estimation is carried out for one moving object of above-mentioned use, it is possible to be generalized to multiple moving object and estimate。Implementation is similar with a moving object problem。For 2 moving objects, schematic diagram is as shown in Figure 8, carrying out background scene segmentation still according to the first step, then second step moving meshes, now moving object has two, then the 3rd step structure space-time diagram is carried out again, only now except a node, add g node, corresponding two moving objects, if having hiding relation between two moving objects, connect with edge equally。Then, the 4th step utilizes two kinds of clues to carry out regional occlusion thread reasoning, updates and blocks matrix。Finally setting up energy function, the situation of the same moving object of its form is the same, utilizes TRW-S method to solve。

Table 1 and table 2 have gone out the inventive method and have employed the result of part clue for other comparison。Need exist for illustrate be, four evaluation index values in form the 2nd, 3 row are more big, represent that the result of depth order reasoning is more accurate, and two evaluation indexes of last string, propose according to the research of M.Kendall and G.E.Noether two people respectively, be used for evaluating the depth interpretation effect in paired region。The former numerical value is more big, and effect is more good；The latter's numerical value is more little, and effect is more good。Thus, it will be seen that the inventive method is all better than remaining method from table。Wherein (I) only employs motion and blocks clue, (II) the monocular clue that study is arrived only is employed, (III) it is that the simple linear combination that motion is blocked clue and monocular clue is as judging clue, (IV) it is the comprehensive clue utilizing recommendation method gained, but the time edge not including strengthening motion model seriality and adding connects, and last column is then utilize to add, on (IV) basis, the evaluation of result that the framework of time edge and technical solution of the present invention draws。

Table 1, the accuracy of depth order reasoning based on SET-A (Singlemovingobject)。

Table 2, the accuracy of depth order reasoning based on SET-B (Multiplemovingobjects)。

In sum, depth order reasoning problems is carried out review and conclude again by technical solution of the present invention, and on this basis the limitation before it is dealt with and improve。

For estimating the depth order problem of scene from single image by the method based on study, it is improved to and utilizes the image set of one group of moving object captured by still camera to go to estimate, and moving object is also no longer limited to people, namely in advance for moving object and scene type and be unaware of, thus make problem more general。Simultaneously, motion for existing in the research of G.Brostow and A.Schodl is blocked the Sparse Problems of clue and fails to travel through whole image-region and carry out estimation of Depth problem, utilizing sparse effective motion to block, those regions not judged are carried out level judgement in conjunction with monocular clue by the basis that clue judges those synergistic regions, thus this unified framework judges to be generalized in the middle of whole scene by effectively blocking level, more reasonable compared to inference method before。Generally speaking, degree of depth level segmentation problem is changed into and processes based on image sequence discrete markers problem under spatiotemporal markov random file (MRF), for degree of depth level recovery effects, the method being better than similar research now。

It is an object of the invention to the research of the depth order inference method of the many orders image for general scene and realization, namely carry out studying realization to the depth reasoning of background scene through the image sequence of scene by moving object under different frame。With the method that the part picture of L.Guan et al. two sets of video data SET-A (single movement object) issued and SET-B (multiple moving object) illustrates us, the depth order reasoning of other scene can realize completely according to this。

Last it is noted that the foregoing is only the preferred embodiments of the present invention, it is not limited to the present invention, although the present invention being described in detail with reference to previous embodiment, for a person skilled in the art, technical scheme described in foregoing embodiments still can be modified by it, or wherein portion of techniques feature carries out equivalent replacement。All within the spirit and principles in the present invention, any amendment of making, equivalent replacement, improvement etc., should be included within protection scope of the present invention。

Claims (4)

1. the method being recovered depth order by hiding relation, it is characterised in that including:

Step 1, background scene are split, and utilize still camera to obtain background image, and background image is carried out over-segmentation by recycling Meanshift dividing method, obtains the background scene segmentation figure that super-pixel block zonule is formed；

Step 2, moving meshes, based on above-mentioned background scene segmentation figure, remove the super-pixel image of more new scene one frame frame, namely in the background scene occur moving object, delete that method determines the super-pixel region that moving object is corresponding by background, obtain moving object segmentation figure in background scene；

Step 3, structure realm rank markov random file, based on the moving object obtained above segmentation figure in background scene, using each region in segmentation figure as a node, if two regions are adjacent namely edge, then corresponding two nodes are connected, for the moving object on each two field picture of still camera shooting, also serve as a node；Meanwhile, adding time edge and connect the node of the moving object on consecutive frame, thus constructing the space-time diagram with regional occlusion relation, this space-time diagram had both contained spatial information, contained again moving object information under different time；

Step 4, carry out the depth order reasoning in paired region based on above-mentioned space-time diagram, thus obtaining blocking matrix；

Step 5, carry out global depth order reasoning according to the above-mentioned matrix that blocks, thus drawing the depth order relationships of background image。

2. the method being recovered depth order by hiding relation according to claim 1, it is characterised in that in step 2 in the background scene occur moving object by background delete method determine super-pixel region corresponding to moving object particularly as follows:

First the scene of background being expressed and be modeled, each of which pixel all obeys the Gauss distribution Ap centered by its average color in all two field pictures, after given Ap, for every two field picture, estimate this pixel and belong to the probability of background, if probability is more than 90%, then it is assumed that be background pixel；If probability is lower than 10%, think the pixel of moving object。

3. according to claim 1 by hiding relation recover depth order method, it is characterised in that described step 4 based on above-mentioned space-time diagram carry out paired region depth order reasoning particularly as follows:

Super-pixel based on scene is expressed, set up one and block matrix O for judging the hiding relation of a pair region i, j, wherein Oi, j ∈ {+1,-1,0}, corresponding region i occlusion area j, region i are blocked by region j and unobstructed clue these three situation respectively, and the depth order in region is gone to infer by two kinds of clues, block matrix O thus updating frame by frame；

First block clue according to motion and go to judge the depth order of moving object and background area, according to moving meshes, obtain the boundary pixel of moving object and the boundary pixel of background area, thus inferring that moving object is before this background area or below, what then update correspondence position blocks matrix O；

Then, utilize monocular clue to carry out regional occlusion judgement for the region that the space not updated in scene is adjacent, and update correspondence position block matrix O。

4. according to claim 3 by hiding relation recover depth order method, it is characterised in that described step 5 according to above-mentioned block matrix carry out global depth order reasoning particularly as follows:

For each the super-pixel region in background scene distribute deep tag 1,2 ..., L}, wherein L is presetting, L numeral show more greatly from photographic head more away from；Therefore the segmentation problem of multi-tag is converted to the energy minimization problem based on space-time diagram model, space-time diagram comprises n+F node, n super-pixel Area Node of n correspondence background scene, corresponding each the moving object node under the F two field picture of video of F, thus target is exactly try to achieve the arrangement X={X of a sounding mark₁,...,X_n+F}；

Define an energy function based on MRF space-time diagram as follows:

$E\left(X\right)=\underset{i\&Element;1,...,n+F}{\mathrm{\&Sigma;}}{E}_i\left(X_i\right)+\underset{(i,j)\&Element;N_S}{\mathrm{\&Sigma;}}{E}_{ij}^S(X_i,X_j)+\underset{(i,j)\&Element;N_T}{\mathrm{\&Sigma;}}{E}_{ij}^T(X_i,X_j);$

Wherein, E_iRepresent the cost distributing to a certain Area Node depth order label, Xi ∈ X, Xj ∈ X；

For the paired item in space, N_SRepresent synergistic region in background scene,

Wherein,

$E_{ij}^{S,f}(X_i,X_j)=\left\{\begin{array}{cc}-log(c_{ij}^f\&times;\frac{1+O_{i,j}^f+\mathrm{\&epsiv;}}{2})& \&ForAll;X_i<X_j\\ \mathrm{\&gamma;}& X_i=X_j\\ -log(c_{ij}^f\&times;\frac{1+O_{j,i}^f+\mathrm{\&epsiv;}}{2})& \&ForAll;X_i>X_j\end{array}\right.$

$E_{ij}^S(X_i,X_j)=\underset{f\&Element;F}{\&Sigma;}\left(E_{ij}^{S,f}\right(X_i,X_j\left)\exp \right(-{\mathrm{\&delta;}}_{i,j}^f\left)\right)$

It is the paired space item of f two field picture,It is the hiding relation of region i and j under f frame,The confidence level of corresponding hiding relation, ε and γ is known coefficient,Represent and utilize relevant provincial characteristics to judge the coplanar probability of region i, j each frame f；

For time paired item, N_TRepresent the time edge of moving object。