CN104935832A - Video matting method aiming at depth information - Google Patents

Abstract

The present invention discloses a video matting method aiming at depth information. The method comprises the steps of calculating the three-parted graph of each frame image; segmenting a video to keep the frame-to-frame consistency in each video segment; obtaining the optimal foreground and transparency estimation values of each pixel in an unknown area of each frame; obtaining a global optimal solution of the pixels in all unknown areas in the video segments; finishing the matting processing of a whole segment of video. The method of the present invention is suitable for carrying out the rapid and efficient foreground object extraction on a video sequence having the frame-to-frame consistency, can keep the space-time consistency of the video matting, and enables the flicker and visual mutation to be reduced and the matting calculation efficiency to be improved.

Description

For the video keying method of band depth information

Technical field

The invention belongs to image processing field, particularly for the video keying method with depth information.

Background technology

Image take as in computer vision field one there is the concern that many multiduty technology enjoy scholar and enterprise always, have a large amount of successfully practical application in a lot of fields.And the technology taking interesting target from the image sequence of video single image is scratched to the more deep structure research of picture, though be in the starting stage at present, because of its widely application prospect make increasing scholar start to be devoted to the research in this field.

The foreground pixel value F of each pixel in the stingy picture process of single-frame images and computed image, background pixel value B and alpha transparence value are underconstrained problem, calculation of complex.And video keying is the stingy picture to image sequence, it is more complicated that institute involves problem, and the difficult point that there is several aspect needs to solve, and emphasis has: 1. pending data volume is huge, needs a large amount of pixels in efficient process video sequence, improve and scratch picture efficiency; 2. video keying needs to keep the space-time consistency between sequence, reduces and scratches picture generation flicker and vision sudden change;

At present, existing video keying method mainly divides following a few class:

Stingy picture algorithm frame by frame.This algorithm regards as independently picture frame the image sequence in video, then applies existing single image to each two field picture and takes algorithm to realize stingy picture process.This method is comparatively convenient, easy realization, can owing to causing the correlation properly do not considered between consecutive frame to the independent process of every two field picture, pixel transparent angle value corresponding between continuous print sequence image can be made to produce difference, the interframe continuity taking result cannot be guaranteed, produce flicker and vision sudden change.

The stingy image space method of 3 D stereo.This type of algorithm regards a three-dimensional solid as video sequence, stereoscopic for three-dimensional geometry work entirety is carried out scratching picture process.This method often needs to carry out multistep and scratches picture and obtain an ideal result, and due to it, tentatively to scratch effect that picture result obtains accurate not, needs the optimization process continued to obtain better effect, therefore it is lower to scratch picture efficiency.

The stingy picture algorithm of successive frame.These class methods apply some comparatively ripe single width equally and scratch image space method, scratching between consecutive frame are applied in the stingy picture step of present frame as the restrictive condition of result as interframe continuity simultaneously.Owing to considering the association between the consecutive frame of front and back, can obtain and scratch picture effect relatively preferably.But owing to all adopting single frames to scratch picture to all frames, spent by these class methods, the time is more.

The present invention is based on the interframe continuity utilizing video, design estimation and the optimization method of a kind of quick prospect and transparency, the foreground target interested in video sequence is extracted.Automatic three components employed in the method based on depth information generate and video segmentation, fast prospect and transparence value estimation, and have carried out optimizing and the optimization of interframe boundling in frame, the stingy picture effect that final acquisition is desirable.

Summary of the invention

The object of the invention is to the above-mentioned problem solving this field existing, provide a kind of sequence of video images to scratch image space method fast, the foreground target be suitable for the video sequence with interframe continuity carries out rapidly and efficiently extracts, and comprises the following steps:

Calculate three components of every two field picture, be divided into foreground area, background area and zone of ignorance therebetween by every two field picture; Segmentation is carried out to video, makes the interframe in each video segmentation keep coherent; The prospect of each pixel optimum and transparency estimated value in the zone of ignorance obtaining each frame; Obtain the global optimization solution of pixel in all zone of ignorances in this video segmentation; Complete the stingy picture process of whole section of video.

As preferably, every two field picture is divided into foreground area, background area and zone of ignorance therebetween, is realized by following steps:

1) a selected depth threshold, carries out segmentation acquisition binarization segmentation result to depth map, is less than the region of threshold value as prospect;

2) morphological erosion operation is carried out to the foreground area of binaryzation, corrosion after obtain region as the foreground area R determined _f;

3) carry out morphological dilation to binaryzation foreground area and negate, gained region is as the background area R determined _b;

4) zone of ignorance R is between the prospect determined, background area _u, calculate its area S _u.

As preferably, segmentation is carried out to video, makes the interframe in each video segmentation keep coherent and realize in the following manner:

Area ratio P overlapping between the zone of ignorance before and after calculating between successive frame _u=(S _u ^t∩ S _u ^t-1)/Max (S _u ^t, S _u ^t-1), choose P _uthe frame of <0.8 is key frame Fr _key; By certain key frame Fr _{key, i}with next key frame Fr _{key, i+1}between image sequence be grouped into same segmentation wherein Fr _t, t=1,2..., n are the normal frames between key frame.

As preferably, in the zone of ignorance obtaining each frame, the prospect of each pixel optimum and transparency estimated value, realize: scratch picture to the first and last two frame application closed type of video segmentation in the following manner; Based on the kinematic parameter of optical flow method principle solving pixel; The information of pixel in each frame zone of ignorance is estimated frame by frame in conjunction with interframe continuity; With simulated annealing to estimating the parameter optimization obtained in each frame.

Further, the described first and last two frame application closed type to video segmentation scratches picture, realizes by the following method: to Seg [Fr _{key, i}] in key frame Fr _{key, i}, scratch picture (ClosedForm Matting) method based on classical closed type and obtain its zone of ignorance R _uin the prospect of each pixel, background and transparency (F, B, α) value.

Further, the described kinematic parameter based on optical flow method principle solving pixel, realizes by the following method:

Make I be the gray value of image, D is the depth value of image, according to the general principle of optical flow method, all meets following two formula to each pixel:

I _x·u+I _y·v+I _t＝0 (1)

D _x·u+D _y·v+D _t＝0 (2)

Wherein I _x, I _y, I _tand D _x, D _y, D _tbe respectively gray value and the partial derivative of depth value on x, y direction and time t, all directly can be calculated by the color diagram of frame sequence and depth map;

for this pixel is along the velocity component on x, y direction; By separating above-mentioned two formula, the kinematic parameter of each pixel can be tried to achieve wherein representation speed size, representation speed direction, as shown in Figure 2.

Further, the described information estimating pixel in each frame zone of ignorance in conjunction with interframe continuity frame by frame, realizes by the following method:

For present frame Fr _tzone of ignorance R _u ^tin a kth pixel p _k ^t(x, y), according to its kinematic parameter calculate it at Fr _t-1in position p _k ^t-1(x ', y '); Use p _k ^t-1the background value B at (x, y) place _k ^t-1(x, y) is as p _k ^tthe background estimating value of (x, y) use p _k ^t-1in (x ', y ') place 3 × 3 neighborhood, given value estimates p _k ^tthe prospect value at (x, y) place and transparence value

$\stackrel{\&OverBar;}{B}={B_k}^{t-1}(x,y)$

$\stackrel{\&OverBar;}{F}=\underset{i,j\&Element;[-\mathrm{1,1}]}{\mathrm{\&Sigma;}}{\mathrm{\&lambda;}}_{i,j}\&CenterDot;{F_k}^{t-1}(x^{\&prime;}+i,y^{\&prime;}+j)$

$\stackrel{\&OverBar;}{\mathrm{\&alpha;}}=\underset{i,j\&Element;[-\mathrm{1,1}]}{\mathrm{\&Sigma;}}{\mathrm{\&lambda;}}_{i,j}\&CenterDot;{{\mathrm{\&alpha;}}_k}^{t-1}(x^{\&prime;}+i,y^{\&prime;}+j)$

${\mathrm{\&lambda;}}_{3*3}=\left[\begin{array}{ccc}0.05& 0.1& 0.05\\ 0.1& 0.4& 0.1\\ 0.05& 0.1& 0.05\end{array}\right]$

As preferably, described with simulated annealing to estimating in each frame that the parameter obtained is optimized, realize by the following method:

Assuming that p _k ^tthe pixel background color value B at (x, y) place is constant, utilizes simulated annealing to foreground pixel value and transparence value be optimized;

Wherein, the solution that simulated annealing optimization algorithm uses is S:

$S=\left\{({\mathrm{\&alpha;}}^{*},F^{*})\right|{\mathrm{\&alpha;}}^{*}=\stackrel{\&OverBar;}{\mathrm{\&alpha;}}\&PlusMinus;i\&CenterDot;\mathrm{\&Delta;}{\mathrm{\&sigma;}}_{\mathrm{\&alpha;}},F^{*}=\stackrel{\&OverBar;}{F}\&PlusMinus;j\&CenterDot;\mathrm{\&Delta;}{\mathrm{\&sigma;}}_F;i,j=\mathrm{0,1},...,N\}$

Wherein △ σ _α=σ _α/ (3N), △ σ _f=σ _f/ (3N); σ _fand σ _αbe respectively p _k ^t-1the variance of prospect and transparence value in (x ', y ') place 3 × 3 neighborhood; N is constant, is used for step size;

The evaluation function C (S) that simulated annealing optimization algorithm uses is:

$C\left(S\right)={\mathrm{\&beta;}}_1\&CenterDot;\left|\right|{\stackrel{\&OverBar;}{\mathrm{\&alpha;}}}^{*}\&CenterDot;\stackrel{\&OverBar;}{F}+(1-{\stackrel{\&OverBar;}{\mathrm{\&alpha;}}}^{*})\&CenterDot;\stackrel{\&OverBar;}{B}-\mathrm{Color}(x,y)\left|\right|+{\mathrm{\&beta;}}_2\&CenterDot;\left|\right|{\stackrel{\&OverBar;}{\mathrm{\&alpha;}}}^{*}-\stackrel{\&OverBar;}{\mathrm{\&alpha;}}\left|\right|+{\mathrm{\&beta;}}_3\&CenterDot;\left|\right|{\stackrel{\&OverBar;}{F}}^{*}-\stackrel{\&OverBar;}{F}\left|\right|$

Wherein β ₁, β ₂, β ₃for constant factor, the color RGB vector that Color (x, y) is pixel, with for the estimated value of initial background, prospect and transparency, with the prospect obtained for current iteration and transparence value.

As preferably, obtaining the global optimization solution of pixel in all zone of ignorances in this video segmentation, being undertaken by asking minimum the dissolving of following energy equation:

$\begin{array}{c}E=\underset{t\&Element;[{\mathrm{Fr}}_{\mathrm{key}},{\mathrm{Fr}}_m),k\&Element;{R_U}^m}{\mathrm{\&Sigma;}}{\mathrm{\&epsiv;}}_t\&CenterDot;|\mathrm{\&Delta;}{{\mathrm{\&alpha;}}_k}^{t-1\&RightArrow;t}\&CenterDot;\mathrm{\&Delta;}{F_k}^{t-1\&RightArrow;t}|+\underset{t^{\&prime;}\&Element;({\mathrm{Fr}}_m,{\mathrm{Fr}}_n],k\&Element;{R_U}^m}{\mathrm{\&Sigma;}}{\mathrm{\&epsiv;}}_{t^{\&prime;}}\&CenterDot;|\mathrm{\&Delta;}{{\mathrm{\&alpha;}}_k}^{t^{\&prime;}\&RightArrow;t^{\&prime;}-1}\&CenterDot;\mathrm{\&Delta;}{F_k}^{t^{\&prime;}\&RightArrow;t^{\&prime;}-1}|\\ +\underset{k\&Element;{R_U}^m}{\mathrm{\&Sigma;}}{\mathrm{\&epsiv;}}_m\&CenterDot;|({{\mathrm{\&alpha;}}_k}^m-{{\mathrm{\&alpha;}}_k}^{m^{\&prime;}})\&CenterDot;({F_k}^m-{F_k}^{m^{\&prime;}})|\end{array}$

Wherein Fr _mfor the picture frame in step 3, with represent Fr respectively _mzone of ignorance R _u ^min the prospect of pixel undetermined and the frame difference of transparence value in (afterwards) derivation forward, F _k ^m, F _k ^{m '}, α _k ^mand α _k ^{m '}for Fr _mmiddle kth pixel undetermined to be derived the prospect value and transparence value that obtain respectively by (afterwards) forward; for meeting the coefficient of normal distribution, at Fr _mplace reaches peak value; N is for controlling constant.

The present invention has following beneficial effect: the foreground target be suitable for the video sequence with interframe continuity carries out rapidly and efficiently extracts, and can keep the space-time consistency of video keying, reduces flicker and vision sudden change, improves and scratch picture computational efficiency.

Accompanying drawing explanation

Fig. 1 is the flow chart of the video keying method for band depth information;

Three components of Fig. 2 video segmentation frame and zone of ignorance pixel are at the motion change figure of interframe.

Embodiment

Below in conjunction with specific embodiment, and by reference to the accompanying drawings, technical scheme of the present invention is further described:

Embodiment 1: use depth camera (Kinect as Microsoft) capture video, obtain the image sequence being with depth information, namely image is except color Color value, also with degree of depth Depth value.

Parsing obtains image sequence, to this image sequence application stingy image space method of the present invention, carries out in accordance with the following steps:

(1) three components of every two field picture in sequence of computed images;

(2) key frame extraction and video segmentation; According to area ratio P overlapping between the zone of ignorance between consecutive frame _uthe frame of <0.8 is key frame Fr _key.By certain key frame Fr _{key, i}with next key frame Fr _{key, i+1}between image sequence be grouped into same segmentation Seg [Fr _{key, i}];

(3) every frame zone of ignorance R in video segmentation is calculated frame by frame _uinterior pixel prospect, background and transparency (F, B, α) value.The simulated annealing optimization algorithm flow adopted in reckoning process is:

1. initialization: initial temperature T, initial value C _{(x, y)}, the iterations L of each T value;

2. to k=1 ..., L, performs the 3rd to the 6th step:

3. from solution space, search for new solution (α ', F '), (α ', F ') ∈ S;

4. incremental computations $\mathrm{\&Delta;}{t}^{\&prime;}=\mathrm{Cost}({\mathrm{\&alpha;}}^{\&prime;},F^{\&prime;})-C_{(x,y)},\mathrm{Cost}({\mathrm{\&alpha;}}^{\&prime;},F^{\&prime;})={\mathrm{\&alpha;}}^{\&prime;}{F}^{\&prime;}+(1-{\mathrm{\&alpha;}}^{\&prime;})\stackrel{\&OverBar;}{B};$

5. if Δ t ' < 0, then accept (α ', F ') as current new explanation, otherwise with probability exp (-Δ t '/T)

Accept (α ', F ') as current new explanation.

If 6. meet end condition, current solution is optimal solution, terminator.End condition is continuously all unaccredited situation of several new explanations.

7.T reduces, and T trends towards 0, turns the 2nd step.

(4) interframe boundling is optimized.Boundling optimizing process solves following equation to carry out by using gradient descent method:

$\begin{array}{c}(F^{\&prime;},{\mathrm{\&alpha;}}^{\&prime;})=\underset{{{\mathrm{\&alpha;}}_k}^t,{F_k}^t}{\arg \min }(\underset{t\&Element;[{\mathrm{Fr}}_{\mathrm{key}},{\mathrm{Fr}}_m),k\&Element;{R_U}^m}{\mathrm{\&Sigma;}}{\mathrm{\&epsiv;}}_t\&CenterDot;|\mathrm{\&Delta;}{{\mathrm{\&alpha;}}_k}^{t-1\&RightArrow;t}\&CenterDot;\mathrm{\&Delta;}{F_k}^{t-1\&RightArrow;t}|\\ +\underset{t^{\&prime;}\&Element;({\mathrm{Fr}}_m,{\mathrm{Fr}}_n],k\&Element;{R_U}^m}{\mathrm{\&Sigma;}}{\mathrm{\&epsiv;}}_{t^{\&prime;}}\&CenterDot;|\mathrm{\&Delta;}{{\mathrm{\&alpha;}}_k}^{t^{\&prime;}\&RightArrow;t^{\&prime;}-1}\&CenterDot;\mathrm{\&Delta;}{F_k}^{t^{\&prime;}\&RightArrow;t^{\&prime;}-1}|+\underset{k\&Element;{R_U}^m}{\mathrm{\&Sigma;}}{\mathrm{\&epsiv;}}_m\&CenterDot;|({{\mathrm{\&alpha;}}_k}^m-{{\mathrm{\&alpha;}}_k}^{m^{\&prime;}})\&CenterDot;({F_k}^m-{F_k}^{m^{\&prime;}})|)\end{array}$

(5) according to the global optimum of all video segmentations (F ', α ') value, the background substantially remained unchanged is added value, completes the stingy picture process of whole section of video.

Compared with background technology, innovation of the present invention is:

1) video segmentation.Utilize depth information automatically to calculate three components, the degree of overlapping based on consecutive frame zone of ignorance carries out segmentation to video, at utmost maintains the interframe continuity in video segmentation;

2) single frames scratches the reckoning of picture+interframe.In video segmentation, only need to use single frames to scratch picture technology to first and last two frame, all the other frames carry out prospect based on kinematic parameter and transparency is worth the mode estimated and optimize to carry out stingy picture, improve and scratch picture computational efficiency.

3) double optimization flow process.In independent picture frame, make use of the thought of simulated annealing, carry out the optimal screening of prospect colour and transparent value; Between the sequence frame of video segmentation, carry out boundling optimization, foreground and transparence value keep space-time consistency in interframe, and robust is reliable more to make final stingy picture result.

Above-mentioned embodiment is used for explaining and the present invention is described, instead of limits the invention, and in the protection range of spirit of the present invention and claim, any amendment make the present invention and change, all fall into protection scope of the present invention.

Claims (9)

1., for the video keying method of band depth information, it is characterized in that, comprise the following steps:

S01 calculates three components of every two field picture;

S02 carries out segmentation to video, makes the interframe in each video segmentation keep coherent;

The prospect of each pixel optimum and transparency estimated value in the zone of ignorance that S03 obtains each frame;

S04 obtains the global optimization solution of pixel in all zone of ignorances in this video segmentation;

S05 completes the stingy picture process of whole section of video.

2., according to the video keying method for band depth information according to claim 1, it is characterized in that: three components described in step S01 refer to image is divided into three subregions, the foreground area R determined _f, the background area R that determines _band zone of ignorance R therebetween _u, this method calculates three components automatically based on depth information:

1) a selected depth threshold, carries out segmentation acquisition binarization segmentation result to depth map, is less than the region of threshold value as prospect;

2) morphological erosion operation is carried out to the foreground area of binaryzation, corrosion after obtain region as the foreground area R determined _f;

3) carry out morphological dilation to binaryzation foreground area and negate, gained region is as the background area R determined _b;

4) zone of ignorance R is between the prospect determined, background area _u, calculate its area S _u.

3. according to the video keying method for band depth information according to claim 1, it is characterized in that, step S02 comprises: area ratio P overlapping between the zone of ignorance before and after calculating between successive frame _u=(S _u ^t∩ S _u ^t-1)/Max (S _u ^t, S _u ^t-1), choose P _uthe frame of <0.8 is key frame Fr _key; By certain key frame Fr _{key, i}with next key frame Fr _{key, i+1}between image sequence be grouped into same segmentation wherein Fr _t, t=1,2..., n are the normal frames between key frame.

4., according to the video keying method for band depth information according to claim 1, it is characterized in that, step S03 comprises: scratch picture to the first and last two frame application closed type of video segmentation; Based on the kinematic parameter of optical flow method principle solving pixel; The information of pixel in each frame zone of ignorance is estimated frame by frame in conjunction with interframe continuity; With simulated annealing to estimating the parameter optimization obtained in each frame.

5. according to the video keying method for band depth information according to claim 4, it is characterized in that, the described first and last two frame application closed type to video segmentation scratches picture, realizes by the following method: to Seg [Fr _{key, i}] in key frame Fr _{key, i}, scratch picture (Closed FormMatting) method based on classical closed type and obtain its zone of ignorance R _uin the prospect of each pixel, background and transparency (F, B, α) value.

6. according to the video keying method for band depth information according to claim 4, it is characterized in that, the described kinematic parameter based on optical flow method principle solving pixel, realize by the following method:

Make I be the gray value of image, D is the depth value of image, according to the general principle of optical flow method, all meets following two formula to each pixel:

I _x·u+I _y·v+I _t＝0 (1)

D _x·u+D _y·v+D _t＝0 (2)

Wherein I _x, I _y, I _tand D _x, D _y, D _tbe respectively gray value and the partial derivative of depth value on x, y direction and time t, all directly can be calculated by the color diagram of frame sequence and depth map;

for this pixel is along the velocity component on x, y direction; By separating above-mentioned two formula, the kinematic parameter of each pixel can be tried to achieve wherein representation speed size, representation speed direction, as shown in Figure 2.

7. according to the video keying method for band depth information according to claim 4, it is characterized in that, the described information estimating pixel in each frame zone of ignorance in conjunction with interframe continuity frame by frame, realize by the following method:

For present frame Fr _tzone of ignorance R _u ^tin a kth pixel p _k ^t(x, y), according to its kinematic parameter calculate it at Fr _t-1in position p _k ^t-1(x ', y '); Use p _k ^t-1the background value B at (x, y) place _k ^t-1(x, y) is as p _k ^tthe background estimating value of (x, y) use p _k ^t-1in (x ', y ') place 3 × 3 neighborhood, given value estimates p _k ^tthe prospect value at (x, y) place and transparence value

$\stackrel{\&OverBar;}{B}={B_k}^{t-1}(x,y)$

$\stackrel{\&OverBar;}{F}=\underset{i,j\&Element;[-\mathrm{1,1}]}{\mathrm{\&Sigma;}}{\mathrm{\&lambda;}}_{i,j}\&CenterDot;{F_k}^{t-1}(x^{\&prime;}+i,y^{\&prime;}+j)$

$\stackrel{\&OverBar;}{\mathrm{\&alpha;}}=\underset{i,j\&Element;[-\mathrm{1,1}]}{\mathrm{\&Sigma;}}{\mathrm{\&lambda;}}_{i,j}\&CenterDot;{{\mathrm{\&alpha;}}_k}^{t-1}(x^{\&prime;}+i,y^{\&prime;}+j)$

${\mathrm{\&lambda;}}_{3*3}=\left[\begin{array}{ccc}0.05& 0.1& 0.05\\ 0.1& 0.4& 0.1\\ 0.05& 0.1& 0.05\end{array}\right]$

8. according to according to claim 4 for band depth information video keying method, it is characterized in that, described with simulated annealing to estimating in each frame that the parameter obtained is optimized, by the following method realize:

Assuming that p _k ^tthe pixel background color value at (x, y) place constant, utilize simulated annealing to foreground pixel value and transparence value be optimized;

Wherein, the solution that simulated annealing optimization algorithm uses is S:

$S=\left\{({\mathrm{\&alpha;}}^{*},F^{*})\right|{\mathrm{\&alpha;}}^{*}=\stackrel{\&OverBar;}{\mathrm{\&alpha;}}\&PlusMinus;i\&CenterDot;\mathrm{\&Delta;}{\mathrm{\&sigma;}}_{\mathrm{\&alpha;}},F^{*}=\stackrel{\&OverBar;}{F}\&PlusMinus;j\&CenterDot;\mathrm{\&Delta;}{\mathrm{\&sigma;}}_F;i,j=\mathrm{0,1},...,N\}$

Wherein △ σ _α=σ _α/ (3N), △ σ _f=σ _f/ (3N); σ _fand σ _αbe respectively p _k ^t-1the variance of prospect and transparence value in (x ', y ') place 3 × 3 neighborhood; N is constant, is used for step size;

The evaluation function C (S) that simulated annealing optimization algorithm uses is:

$C\left(S\right)={\mathrm{\&beta;}}_1\&CenterDot;\left|\right|{\stackrel{\&OverBar;}{\mathrm{\&alpha;}}}^{*}\&CenterDot;{\stackrel{\&OverBar;}{F}}^{*}+(1-{\stackrel{\&OverBar;}{\mathrm{\&alpha;}}}^{*})\&CenterDot;\stackrel{\&OverBar;}{B}-\mathrm{Color}(x,y)\left|\right|+{\mathrm{\&beta;}}_2\&CenterDot;\left|\right|{\stackrel{\&OverBar;}{\mathrm{\&alpha;}}}^{*}-\stackrel{\&OverBar;}{\mathrm{\&alpha;}}\left|\right|+{\mathrm{\&beta;}}_3\&CenterDot;\left|\right|{\stackrel{\&OverBar;}{F}}^{*}-\stackrel{\&OverBar;}{F}\left|\right|$

Wherein β ₁, β ₂, β ₃for constant factor, the color RGB vector that Color (x, y) is pixel, with for the estimated value of initial background, prospect and transparency, with the prospect obtained for current iteration and transparence value.

9., according to the video keying method for band depth information according to claim 4, it is characterized in that, step S04 is undertaken by asking minimum the dissolving of following energy equation:

$\begin{array}{c}E=\underset{t\&Element;[{\mathrm{Fr}}_{\mathrm{key}},{\mathrm{Fr}}_m),k\&Element;{R_U}^m}{\mathrm{\&Sigma;}}{\mathrm{\&epsiv;}}_t\&CenterDot;|\mathrm{\&Delta;}{{\mathrm{\&alpha;}}_k}^{t-1\&RightArrow;t}\&CenterDot;\mathrm{\&Delta;}{F_k}^{t-1\&RightArrow;t}|+\underset{t^{\&prime;}\&Element;({\mathrm{Fr}}_m,{\mathrm{Fr}}_n],k\&Element;{R_U}^m}{\mathrm{\&Sigma;}}{\mathrm{\&epsiv;}}_{t^{\&prime;}}\&CenterDot;|\mathrm{\&Delta;}{{\mathrm{\&alpha;}}_k}^{t^{\&prime;}\&RightArrow;t^{\&prime;}-1}\&CenterDot;\mathrm{\&Delta;}{F_k}^{t^{\&prime;}\&RightArrow;t^{\&prime;}-1}|\\ +\underset{k\&Element;{R_U}^m}{\mathrm{\&Sigma;}}{\mathrm{\&epsiv;}}_m\&CenterDot;|({{\mathrm{\&alpha;}}_k}^m-{{\mathrm{\&alpha;}}_k}^{m^{\&prime;}})\&CenterDot;({F_k}^m-{F_k}^{m^{\&prime;}})|\end{array}$

Wherein Fr _mfor the picture frame in step 3, with represent Fr respectively _mzone of ignorance R _u ^min the prospect of pixel undetermined and the frame difference of transparence value in (afterwards) derivation forward, F _k ^m, F _k ^{m '}, α _k ^mand α _k ^{m '}for Fr _mmiddle kth pixel undetermined to be derived the prospect value and transparence value that obtain respectively by (afterwards) forward; for meeting the coefficient of normal distribution, at Fr _mplace reaches peak value; N is for controlling constant.