CN104268866A - Video sequence registering method based on combination of motion information and background information - Google Patents

Abstract

The invention discloses a video sequence registering method based on combination of motion information and background information. The method mainly solves the problem that in the prior art, video sequences in different view points cannot be accurately registered. The method includes the steps that (1) separation of background images and motion targets is carried out on two input video sequences; (2) feature point matching pairs of the background images are obtained, and basis matrixes between the background images are calculated; (3) four sets of matching point pairs corresponding to four spatial points not on the same plane in a three-dimensional space are selected from the background images; (4) intersection points of projection lines, in another video sequence, of points in motion tracks and bipolar lines are obtained; (5) motion track points are matched and a point pair set corresponding to candidate time is obtained; (6) a time line is fitted and time transformation parameters are recovered. The time relation between video sequences can be accurately recovered, registering accuracy is improved and the method can be used for registering video sequences with static backgrounds.

Description

Based on the video sequence method for registering that movable information combines with background information

Technical field

The invention belongs to technical field of image processing, further relate to a kind of video sequence method for registering, its target calibrates the video sequence taken from different visual angles or different time, can be applicable in the video sequence registration of static background.

Background technology

Video sequence registration is a key areas of image procossing, it is by the space-time transformation parameter between the same scene different visual angles of the multiple sensor acquisition of calculating or the similar video of different time, and then these videos are carried out in time synchronous, spatially carry out geometric calibration.Generally, the method of feature based can be adopted, especially be that the method for feature carries out video sequence registration with movement objective orbit, these video sequence method for registering based on movement locus mainly consider the Epipolar geometry relation between the shape of movement locus itself or view, but the spatial relation that often have ignored between the background characteristics of video sequence and moving target, thus cause the location of tracing point accurate not, corresponding matching track point can not be obtained accurately and effectively.

Such as, Padua F L C, Carceroni R L, paper " Linear sequence-to-sequence alignment " (the IEEE Transactions on Pattern Analysis and Machine Intelligence that et al. delivers, 2010,32 (2): 304-320) a kind of video sequence method for registering based on timeline constraint is disclosed in.First the method extracts background, utilizes Background matching point to the fundamental matrix between estimated background image; Then extract the track of moving target, utilize the Epipolar geometry relation between view to search out candidate tracks point pair; RANSAC algorithm is finally used to carry out timeline matching.The deficiency that the method exists is, because the method only considers the Epipolar geometry relation between view, track is utilized to find out corresponding tracing point with to the intersection point of polar curve, therefore accurate not to the location of tracing point, and when movement locus be similar to be straight line and with it, polar curve overlapped, finding can be a lot of to the intersection point of polar curve and track, produces a large amount of error hiding tracing point pair, thus reduces registration accuracy.

Nunziati W, Sclaroff S, paper " An Invariant Representation for Matching Trajectories Across Uncalibrated Video Streams " (the Springer Berlin Heidelberg that and Del Bimbo delivers, 2005, pp.318-327) in disclose the video sequence method for registering of a kind of movement locus point projection invariant representation.First the method extracts the track of moving target in video, is carried out the Local approximation of track by the cubic spline interpolation method of least square fitting; Then, to each sampled point of geometric locus obtained, calculate the double ratio of projection that four sampled points in it and neighborhood form not structure changes, obtain the constant descriptor of projection of track self; Finally, utilize the statistical property of double ratio to carry out distance metric to the track described, obtain the orbit segment mated.The deficiency that the method exists is, because the method only considers the information of track self, the local feature in tracing point neighborhood is utilized to be described tracing point, therefore when there is multiple moving target in scene and movement locus is similar, or when there is the similar orbit segment of a large amount of local feature in movement locus, the track descriptor obtained does not have good distinction, and in these cases, the registration result error of the video sequence that the method obtains even cannot complete registration comparatively greatly.

Summary of the invention

The object of the invention is to the shortcoming for above-mentioned prior art, a kind of video sequence method for registering combined with background information based on movable information is proposed, to obtain the tracing point pair of coupling better from the video sequence of input, reduce registration error, recover the time relationship between video sequence and spatial relationship more exactly.

For achieving the above object, technical scheme of the present invention is: in conjunction with the spatial relation of background characteristics and moving target, utilize the corresponding point information of background characteristics, and set up space coordinates and Epipolar geometry relation by spatial point not coplanar in three dimensions, thus obtain time-varying parameter.Its concrete steps comprise as follows:

(1) reference video sequence l and video sequence l ' subject to registration is inputted respectively;

(2) being respectively separated of background image and moving target is carried out to two input video sequence, obtain with reference to movement locus P and movement locus P ' subject to registration;

(3) adopt feature point detection and the method for mating, the Feature Points Matching of background extraction image is to (X _i, X ' _i), 300≤i≤500;

(4) utilize the Feature Points Matching of the background image obtained to (X _i, X ' _i), calculate the basis matrix F between background image;

(5) from the background image Feature Points Matching obtained to (X _i, X ' _i) in, choose four groups of matching double points (x not coplanar in three dimensions _j, x ' _j), j=1..4:

5.1) background image Feature Points Matching is calculated to (X _i, X ' _i) in the induction of three groups of matching double points singly answer H;

5.2) the Transfer Error Er of residue one group of matching double points under this homography induced H is calculated;

5.3) above-mentioned Transfer Error Er and setting threshold value E=10 are compared, if Er > is E, retain matching double points, otherwise, repeat step 5.1) and step 5.2), until meet the threshold value E that Er is greater than setting;

(6) projection line of point (p, q) in another video sequence in movement locus and the intersection point (p (t), q (t)) to polar curve is obtained respectively:

6.1) obtain with reference to the projection line k of some p in video sequence subject to registration in movement locus;

6.2) obtain with reference to the some p in movement locus to polar curve l:

L=Fp, wherein, F is basis matrix;

6.3) obtain with reference to the projection line of some p in video sequence subject to registration in movement locus and intersection point q (t) to polar curve;

6.4) projection line of some q in reference video sequence in movement locus subject to registration and intersection point p (t) to polar curve is obtained:

6.4.1) the projection line k ' of some q in reference video sequence in video motion track subject to registration is obtained;

6.4.2) obtain the some q in movement locus subject to registration to polar curve l ':

L '=F ^-1q, wherein, F ^-1it is the inverse matrix of basis matrix;

6.4.3) projection line of some q in reference video sequence in movement locus subject to registration and intersection point p (t) to polar curve is obtained;

(7) set with reference to candidate time corresponding point in movement locus and movement locus subject to registration is obtained:

7.1) projection line of the some p in computing reference movement locus in video sequence subject to registration with to intersection point q (t) of polar curve and the distance of all tracing point q subject to registration, select the tracing point q (t that distance is less than setting threshold value e _i) as the candidate tracks point of video sequence subject to registration, and record movement locus time point t subject to registration _i;

7.2) each movement locus point q (t subject to registration is calculated _i) projection line in reference video sequence and the distance to intersection point p (t) of the polar curve reference locus point p corresponding with it, select the tracing point p (t that distance is less than setting threshold value e _j) as the candidate tracks point with reference to video sequence, and record is with reference to movement locus time point t _j;

(8) with movement locus time point t subject to registration _ifor ordinate, with reference movement locus time point t _jfor horizontal ordinate, utilize RANSAC algorithm fit time line, recover time-varying parameter [α, t], wherein α is the slope of timeline, and t is timeline and longitudinal axis t _iintercept.

Tool of the present invention has the following advantages:

First, present invention utilizes the description of three-dimensional coordinate system to tracing point and there is this principle of projective invariance, for movement locus be similar to be straight line and with its situation that polar curve is overlapped, the present invention can search exactly and the unique point matched with reference to movement locus point in movement locus subject to registration, improve prior art and there is the right defect of a large amount of error hiding to during above-mentioned situation coupling, make the present invention compared with prior art can obtain more accurate video sequence registration effect.

Second, the strategy that the background characteristics that present invention employs video sequence is described tracing point, for there is multiple moving target in scene and movement locus is similar, or there is the situation of the similar orbit segment of a large amount of local feature in movement locus, unique description can both be set up to tracing point, overcome in prior art and only utilize track self character and unique defect described cannot be set up to the tracing point of above-mentioned situation, the present invention is made compared with prior art to have higher ga s safety degree to the description of movement locus point, thus more accurate video sequence registration result can be obtained.

Accompanying drawing explanation

Fig. 1 is process flow diagram of the present invention;

Fig. 2 is the back-up video sequence registration simulated effect figure undertaken by the inventive method;

Fig. 3 is another group back-up video sequence registration simulated effect figure undertaken by the inventive method;

Fig. 4 is with the existing back-up video sequence registration simulated effect figure carried out based on timeline bounding algorithm;

Fig. 5 is with the existing back-up video sequence registration simulated effect figure carried out based on movement locus point projection invariant representation algorithm;

The scene video sequence registration simulated effect figure that Fig. 6 synthesizes when being and being similar to and being straight line with the movement locus that the inventive method is carried out and overlapping to polar curve with it;

Fig. 7 is that another group movement locus that the inventive method is carried out is similar to the scene video sequence registration simulated effect figure synthesized when being straight line and overlapping to polar curve with it;

The scene video sequence registration simulated effect figure that Fig. 8 synthesizes when being and being similar to and being straight line with the existing movement locus carried out based on timeline bounding algorithm and overlapping to polar curve with it;

The scene video sequence registration simulated effect figure that Fig. 9 synthesizes when being and being similar to and being straight line with the existing movement locus carried out based on movement locus point projection invariant representation algorithm and overlapping to polar curve with it.

Specific implementation method

Below in conjunction with accompanying drawing, the present invention will be further described.

With reference to Fig. 1, concrete steps of the present invention are as follows:

Step 1, inputs reference video sequence l and video sequence l ' subject to registration respectively, and carries out being separated of background image and moving target to these two input video sequence respectively, obtains with reference to movement locus P and movement locus P ' subject to registration.

Step 2, adopt feature point detection and the method for mating to the background image obtained, the Feature Points Matching of background extraction image is to (X _i, X ' _i), 300≤i≤500.

Step 3, utilizes the Feature Points Matching of the background image obtained to (X _i, X ' _i), calculate the basis matrix F between background image;

Step 4, from the background image Feature Points Matching obtained to (X _i, X ' _i) in, choose four groups of matching double points (x that four spatial point not coplanar in three dimensions are corresponding _j, x ' _j), j=1,2,3,4:

4.1) from the Background matching point obtained to (X _i, X ' _i) in choose arbitrarily four groups of matching double points (X _j, X ' _j), induce a list to answer with three groups of matching double points wherein:

H＝[x′ ₁,x′ ₂,x′ ₃][x ₁,x ₂,x ₃] ^-1

Wherein x ₁, x ₂, x ₃the background image unique point that in reference video sequence three are different respectively, x ' ₁, x ' ₂, x ' ₃the background image unique point that in video sequence subject to registration three are different respectively;

4.2) residue one group of matching double points (x is calculated ₄x ' ₄) Transfer Error Er under this homography induced:

Er＝d(x′ ₄,Hx ₄)，

Wherein d (x ' ₄, Hx ₄) represent some x ' ₄to a Hx ₄euclidean distance, x ₄, x ' ₄be respectively the background image unique point of reference video sequence and the unique point of video frequency sequence background image subject to registration;

4.3) above-mentioned Transfer Error Er and setting threshold value E=10 are compared, if Er > is E, retain matching double points, otherwise, repeat step 5.1) and step 5.2), until meet the threshold value E that Er is greater than setting.

Step 5, obtains the projection line of point (p, q) in another video sequence in movement locus and the intersection point (p (t), q (t)) to polar curve respectively.

5.1) obtain with reference to the projection line k of some p in video sequence subject to registration in movement locus;

Suppose that the four groups of matching double points back projection's point in space obtained is X _i, i=1,2,3,4, utilize these four not coplanar point set up space coordinates, and with X ₁for initial point, construct 3 not coplanar networks by remaining three point, be respectively (X ₂-X ₁), (X ₃-X ₁) and (X ₄-X ₁), coordinate expression can be carried out under the coordinate system set up with reference to the some p in movement locus:

p＝x ₁+x(x ₂-x ₁)+y(x ₃-x ₁)+z(x ₄-x ₁)

Wherein x ₁, x ₂, x ₃, x ₄be respectively the imaging point that in reference video sequence four are different, (x, y, z) is the coordinate of reference locus point p in the space coordinates set up;

5.2) by 5.1) relational expression derive coordinate (x, y, z) relational expression:

$x=\frac{{\mathrm{\Δ}}_{34}}{{\mathrm{\Δ}}_{23}}z+\frac{{\mathrm{\Δ}}_3}{{\mathrm{\Δ}}_{23}},y=\frac{{\mathrm{\Δ}}_{24}}{{\mathrm{\Δ}}_{32}}z+\frac{{\mathrm{\Δ}}_2}{{\mathrm{\Δ}}_{32}}$

Wherein, ${\mathrm{\Δ}}_{34}=\det \begin{array}{c}\left(\begin{array}{cc}{x}_{3x}-x_{1x}& x_{4x}-x_{1x}\\ x_{3y}-x_{1y}& x_{4y}-x_{1y}\end{array}\right)\end{array},{\mathrm{\Δ}}_{23}=\det \left(\begin{array}{cc}{x}_{2x}-x_{1x}& x_{3x}-x_{1x}\\ x_{2y}-x_{1y}& x_{3y}-x_{1y}\end{array}\right)$

${\mathrm{\Δ}}_{32}=\det \begin{array}{c}\left(\begin{array}{cc}{x}_{3x}-x_{1x}& x_{2x}-x_{1x}\\ x_{3y}-x_{1y}& x_{2y}-x_{1y}\end{array}\right)\end{array},{\mathrm{\Δ}}_2=\det \left(\begin{array}{cc}{p}_x-x_{1x}& x_{2x}-x_{1x}\\ p_y-x_{1y}& x_{2y}-x_{1y}\end{array}\right)$

${\mathrm{\Δ}}_3=\det \begin{array}{c}\left(\begin{array}{cc}{p}_x-x_{1x}& x_{3x}-x_{1x}\\ p_y-x_{1y}& x_{3y}-x_{1y}\end{array}\right)\end{array},{\mathrm{\Δ}}_{24}=\det \left(\begin{array}{cc}{x}_{2x}-x_{1x}& x_{4x}-x_{1x}\\ x_{2y}-x_{1y}& x_{4y}-x_{1y}\end{array}\right),$

(x _1x, x _1y), (x _2x, x _2y), (x _3x, x _3y), (x _4x, x _4y) and (p _x, p _y) be respectively the point of imaging point x and p in homogeneous coordinates in reference video sequence;

5.3) coordinate (x, y, z) relational expression is brought into following formula:

$\left\{\begin{array}{c}X=x_{1x}^{\′}+x(x_{2x}^{\′}-x_{1x}^{\′})+y(x_{3x}^{\′}-x_{1x}^{\′})+z(x_{4x}^{\′}-x_{1x}^{\′})\\ Y=x_{1y}^{\′}+x(x_{2y}^{\′}-x_{1y}^{\′})+y(x_{3y}^{\′}-x_{1y}^{\′})+z(x_{4y}^{\′}-x_{1y}^{\′})\end{array}\right.$

Wherein x ' _1x, x ' _2x, x ' _3x, x ' _4x, x ' _1y, x ' _2y, x ' _3y, x ' _4yfor the point of sequential projection point subject to registration in homogeneous coordinates, X, Y are the coordinates of projection line k in image subject to registration;

5.4) by 5.3) relational expression obtain the expression formula of projection line k in video sequence subject to registration:

$Y=\frac{g_3}{g_1}X+\frac{g_1{x}_{1y}^{\′}-g_3{x}_{1x}^{\′}+g_1{g}_4-g_2{g}_3}{g_1},$ Wherein:

$g_1=\frac{{\mathrm{\Δ}}_{34}}{{\mathrm{\Δ}}_{23}}(x_{2x}^{\′}-x_{1x}^{\′})+\frac{{\mathrm{\Δ}}_{24}}{{\mathrm{\Δ}}_{32}}(x_{3x}^{\′}-x_{1x}^{\′})+(x_{4x}^{\′}-x_{1x}^{\′}),g_2=\frac{{\mathrm{\Δ}}_3}{{\mathrm{\Δ}}_{23}}(x_{2x}^{\′}-x_{1x}^{\′})+\frac{{\mathrm{\Δ}}_2}{{\mathrm{\Δ}}_{32}}(x_{3x}^{\′}-x_{1x}^{\′})$

$g_3=\frac{{\mathrm{\Δ}}_{34}}{{\mathrm{\Δ}}_{23}}(x_{2y}^{\′}-x_{1y}^{\′})+\frac{{\mathrm{\Δ}}_{24}}{{\mathrm{\Δ}}_{32}}(x_{3y}^{\′}-x_{1y}^{\′})+(x_{4y}^{\′}-x_{1y}^{\′}),g_4=\frac{{\mathrm{\Δ}}_3}{{\mathrm{\Δ}}_{23}}(x_{2y}^{\′}-x_{1y}^{\′})+\frac{{\mathrm{\Δ}}_2}{{\mathrm{\Δ}}_{32}}(x_{3y}^{\′}-x_{1y}^{\′});$

5.5) obtain with reference to the some p in movement locus to polar curve l:

L=Fp, wherein, F is basis matrix;

5.6) obtain with reference to the projection line of some p in video sequence subject to registration in movement locus and intersection point q (t) to polar curve;

5.7) projection line of some q in reference video sequence in movement locus subject to registration and intersection point p (t) to polar curve is obtained:

5.7.1) the projection line k ' of some q in reference video sequence in video motion track subject to registration is obtained;

5.7.2) obtain the some q in movement locus subject to registration to polar curve l ':

L '=F ^-1q, wherein, F ^-1it is the inverse matrix of basis matrix;

5.7.3) projection line of some q in reference video sequence in movement locus subject to registration and intersection point p (t) to polar curve is obtained;

Step 6, movement locus Point matching, obtains candidate time corresponding point to set.

6.1) projection line of the some p in computing reference movement locus in video sequence subject to registration with to intersection point q (t) of polar curve and the distance of all tracing point q subject to registration, select the tracing point q (t that distance is less than setting threshold value e _i) as the candidate tracks point of video sequence subject to registration, and record movement locus time point t subject to registration _i;

6.2) each movement locus point q (t subject to registration is calculated _i) projection line in reference video sequence and the distance to intersection point p (t) of the polar curve reference locus point p corresponding with it, select the tracing point p (t that distance is less than setting threshold value e _j) as the candidate tracks point with reference to video sequence, and record is with reference to movement locus time point t _j;

The calculating formula of threshold value e is as follows:

$e=\frac{1}{N}\underset{i}{\overset{N}{\mathrm{\Σ}}}(d(x_i^{\′},x_i^c)+d(x_i+x_i^{\′c})),$

Wherein, N represents matching double points number 60≤N≤80 that static background image is manually selected, x _i, x ' _irepresent the background image unique point of reference video sequence and video sequence subject to registration respectively, with represent some x respectively _iwith an x ' _iprojection line in another video sequence and the intersection point to polar curve, represent some x ' _ito point euclidean distance, represent some x ' _ito point euclidean distance.

Step 7, with movement locus time point t subject to registration _ifor ordinate, with reference movement locus time point t _jfor horizontal ordinate, utilize RANSAC algorithm fit time line, recover time-varying parameter [α, t], wherein α is the slope of timeline, and t is timeline and longitudinal axis t _iintercept.

Below in conjunction with analogous diagram, effect of the present invention is further described.

For verifying validity of the present invention and correctness, the video sequence that have employed two groups of different scenes carries out registration emulation experiment.All emulation experiments all adopt Matlab R2010a software simulating under Windows 7 operating system.

Emulation 1, by the inventive method, registration emulation is carried out to the back-up video from different viewing angles, result as Fig. 2, wherein:

The static background image of Fig. 2 (a) reference video sequence,

Fig. 2 (b) is the static background image of video sequence subject to registration.The movement objective orbit that in figure, the some labelled notation representative of white is extracted, the circle labelled notation of white represents four groups of corresponding matching double points of four spatial point not coplanar in the three dimensions selected;

Fig. 2 (c) is the timeline recovered by the inventive method, and wherein, the slope of timeline is 1.0009, and the intercept of timeline on the longitudinal axis is 5.2483.

Emulation 2, the basis of emulation 1 is selected four some the inventive method again not coplanar in one group of three dimensions and emulates, result as Fig. 3, wherein:

Fig. 3 (a) is the static background image of reference video sequence,

Fig. 2 (b) is the static background image of video sequence subject to registration,

Fig. 3 (c) is the timeline recovered by the inventive method, and wherein, the slope of timeline is 1.0028, and the intercept of timeline on the longitudinal axis is 4.7441.

As can be seen from Fig. 2 and Fig. 3, as long as select four groups of Background matching points are not coplanar to back projection's point in three dimensions, just can Describing Motion tracing point accurately, and time-varying parameter more accurately can be obtained.

Emulation 3, with existing based on timeline bounding algorithm, carry out registration emulation to the back-up video from different viewing angles, result is as Fig. 4, and wherein, the slope of the timeline that this algorithm recovers is 1.0081, and the intercept of timeline on the longitudinal axis is 2.5264;

As seen from Figure 4, the timeline application condition that the algorithm based on timeline constraint recovers is large, this is because it considers the Epipolar geometry relation between view, causes the location of tracing point accurate not.

Emulation 4, with existing based on movement locus point projection invariant representation algorithm, carries out registration emulation to the back-up video from different viewing angles, result is as Fig. 5, wherein, the slope of the timeline that this algorithm recovers is 1.0037, and the intercept of timeline on the longitudinal axis is 4.2787.

As seen from Figure 5, although the timeline application condition recovered based on movement locus point projection invariant representation algorithm is little, but there is a large amount of Mismatching points pair in the tracing point centering obtained, this is because there is the similar orbit segment of some local features in track, to the description of tracing point, not there is good distinction.

Emulation 5, to be similar to movement locus by the inventive method method and to be straight line and to carry out registration emulation with it to the overlap video sequence of synthesis scene of situation of polar curve, simulation result as Fig. 6, wherein:

The static background image of Fig. 6 (a) reference video sequence, Fig. 6 (b) is the static background image of video sequence subject to registration.The movement objective orbit that in figure, the some labelled notation representative of white is extracted, the circle labelled notation of white represents four groups of corresponding matching double points of four spatial point not coplanar in the three dimensions selected, the line markings of black represent first tracing point to polar curve;

Fig. 6 (c) is the timeline recovered by the inventive method, and wherein, the slope of timeline is 1.0031, and the intercept of timeline on the longitudinal axis is 14.9148.

Emulation 6, the basis of emulation 5 is chosen four some the inventive method again not coplanar in one group of three dimensions and emulates, simulation result as Fig. 7, wherein,

Fig. 7 (a) is the static background image of reference video sequence,

Fig. 7 (b) is the static background image of video sequence subject to registration,

Fig. 7 (c) is the timeline adopting the inventive method to recover, and wherein, the slope of timeline is 1.0080, and the intercept of timeline on the longitudinal axis is 14.8106.

As can be seen from Fig. 6 and Fig. 7, unique description can be set up to tracing point with the background characteristics that the inventive method is extracted, and time-varying parameter more accurately can be obtained.

Emulation 7, with existing based on timeline bounding algorithm, movement locus is similar to and is straight line and with it, registration emulation is carried out to the overlap video sequence of synthesis scene of situation of polar curve, result is as Fig. 8, wherein, the slope of the timeline that this algorithm recovers is 1.0352, and the intercept of timeline on the longitudinal axis is 12.7555;

Emulation 8, with existing based on movement locus point projection invariant representation algorithm, movement locus is similar to and is straight line and with it, registration emulation is carried out to the overlap video sequence of synthesis scene of situation of polar curve, result is as Fig. 9, wherein, the slope of the timeline that this algorithm recovers is 1.0362, and the intercept of timeline on the longitudinal axis is 14.2077.

As can be seen from Fig. 8 and Fig. 9, when movement locus be similar to be straight line and with it, polar curve overlapped, all there is a large amount of Mismatching points pair in the tracing point centering of two kinds of algorithm acquisitions, this is because track is too many and there is the similar orbit segment of a large amount of local feature in track to the intersection point of polar curve to it, registration accuracy is caused to reduce, and the algorithm that the present invention proposes still uniquely can describe tracing point in the case, eliminate a large amount of error hiding tracing points pair, thus time-varying parameter more accurately can be obtained.

Claims (5)

1., based on the video sequence method for registering that movable information combines with background information, comprise the steps:

(1) reference video sequence l and video sequence l ' subject to registration is inputted respectively;

(2) being respectively separated of background image and moving target is carried out to two input video sequence, obtain with reference to movement locus P and movement locus P ' subject to registration;

(3) adopt feature point detection and the method for mating, the Feature Points Matching of background extraction image is to (X _i, X ' _i), 300≤i≤500;

(4) utilize the Feature Points Matching of the background image obtained to (X _i, X ' _i), calculate the basis matrix F between background image;

(5) from the background image Feature Points Matching obtained to (X _i, X ' _i) in, choose four groups of matching double points (x not coplanar in three dimensions _j, x ' _j), j=1,2,3,4:

5.1) background image Feature Points Matching is calculated to (X _i, X ' _i) in the induction of three groups of matching double points singly answer H;

5.2) the Transfer Error Er of residue one group of matching double points under this homography induced H is calculated;

5.3) above-mentioned Transfer Error Er and setting threshold value E=10 are compared, if Er > is E, retain matching double points, otherwise, repeat step 5.1) and step 5.2), until meet the threshold value E that Er is greater than setting;

(6) projection line of point (p, q) in another video sequence in movement locus and the intersection point (p (t), q (t)) to polar curve is obtained respectively:

6.1) obtain with reference to the projection line k of some p in video sequence subject to registration in movement locus;

6.2) obtain with reference to the some p in movement locus to polar curve l:

L=Fp, wherein, F is basis matrix;

6.3) obtain with reference to the projection line of some p in video sequence subject to registration in movement locus and intersection point q (t) to polar curve;

6.4) projection line of some q in reference video sequence in movement locus subject to registration and intersection point p (t) to polar curve is obtained:

6.4.1) the projection line k ' of some q in reference video sequence in video motion track subject to registration is obtained;

6.4.2) obtain the some q in movement locus subject to registration to polar curve l ':

L '=F ^-1q, wherein, F ^-1it is the inverse matrix of basis matrix;

6.4.3) projection line of some q in reference video sequence in movement locus subject to registration and intersection point p (t) to polar curve is obtained;

(7) set with reference to candidate time corresponding point in movement locus and movement locus subject to registration is obtained:

7.1) projection line of the some p in computing reference movement locus in video sequence subject to registration with to intersection point q (t) of polar curve and the distance of all tracing point q subject to registration, select the tracing point q (t that distance is less than setting threshold value e _i) as the candidate tracks point of video sequence subject to registration, and record movement locus time point t subject to registration _i;

7.2) each movement locus point q (t subject to registration is calculated _i) projection line in reference video sequence and the distance to intersection point p (t) of the polar curve reference locus point p corresponding with it, select the tracing point p (t that distance is less than setting threshold value e _j) as the candidate tracks point with reference to video sequence, and record is with reference to movement locus time point t _j;

(8) with movement locus time point t subject to registration _ifor ordinate, with reference movement locus time point t _jfor horizontal ordinate, utilize RANSAC algorithm fit time line, recover time-varying parameter [α, t], wherein α is the slope of timeline, and t is timeline and longitudinal axis t _iintercept.

2. the video sequence method for registering combined with background information based on movable information according to claim 1, wherein step 5.1) described in calculating background image Feature Points Matching to (X _i, X ' _i) in the induction of three groups of matching double points singly answer H, by following formulae discovery:

H＝[x′ ₁,x′ ₂,x′ ₃][x ₁,x ₂,x ₃] ^-1

Wherein x ₁, x ₂, x ₃the background image unique point that in reference video sequence three are different respectively, x ' ₁, x ' ₂, x ' ₃the background image unique point that in video sequence subject to registration three are different respectively.

3. the video sequence method for registering combined with background information based on movable information according to claim 1, wherein step 5.2) described in calculating remain one group of match point (x ₄, x ' ₄) to the Transfer Error Er under this homography induced H, by following formulae discovery:

Er＝d(x′ ₄,Hx ₄)，

Wherein d (x ' ₄, Hx ₄) represent some x ' ₄to a Hx ₄euclidean distance, x ₄, x ' ₄be respectively the background image unique point of reference video sequence and the unique point of video frequency sequence background image subject to registration.

4. the video sequence method for registering combined with background information based on movable information according to claim 1, wherein step 6.1) described in acquisition with reference to the projection line k of some p in video sequence subject to registration in movement locus, carry out according to the following steps:

Four groups of matching double points back projection's point in space that 6.1a) hypothesis obtains is X _i, i=1,2,3,4, utilize these four not coplanar point set up space coordinates, and with X ₁for initial point, construct 3 not coplanar networks by remaining three point, be respectively (X ₂-X ₁), (X ₃-X ₁) and (X ₄-X ₁);

6.1b) be expressed as under the space coordinates set up with reference to the some p in movement locus:

p＝x ₁+x(x ₂-x ₁)+y(x ₃-x ₁)+z(x ₄-x ₁)

Wherein x ₁, x ₂, x ₃, x ₄be respectively the imaging point that in reference video sequence four are different, (x, y, z) is the coordinate of reference locus point p in the space coordinates set up;

6.1c) calculate the relational expression of coordinate (x, y, z):

Wherein,

(x _1x, x _1y), (x _2x, x _2y), (x _3x, x _3y), (x _4x, x _4y) and (p _x, p _y) be respectively the point of imaging point x and p in homogeneous coordinates in reference video sequence;

6.1d) relational expression of coordinate (x, y, z) is substituted into following formula:

Wherein x ' _1x, x ' _2x, x ' _3x, x ' _4x, x ' _1y, x ' _2y, x ' _3y, x ' _4yfor the point of sequential projection point subject to registration in homogeneous coordinates, X, Y are the coordinates of projection line k in image subject to registration;

6.1e) by 6.1d) relational expression obtain the expression formula of projection line k in video sequence subject to registration:

Wherein,

5. the video sequence registration combined with background information based on movable information according to claim 1, wherein said step 7.1) and 7.2) in threshold value e, set by following formula:

Wherein, N represents matching double points number 60≤N≤80 that static background image is manually selected, x _i, x ' _irepresent the background image unique point of reference video sequence and video sequence subject to registration respectively, with represent some x respectively _iwith an x ' _iprojection line in another video sequence and the intersection point to polar curve, represent some x ' _ito point euclidean distance, represent some x ' _ito point euclidean distance.