CN105303518A - Region feature based video inter-frame splicing method - Google Patents

Abstract

The present invention discloses a region feature based video inter-frame splicing method. The method comprises: using a video camera and an inertial sensor and acquiring three-dimensional information of a target scene image and a motion parameter of the video camera, thereby estimating global optical flow of the image; using local fitting of the global optical flow to extract a local feature region which accords with global motion of the image; and in the local feature region, based on a modified FAST feature extraction operator, accurately estimating a global motion transformation matrix of the image, and combining an image fusion technology to complete inter-frame splicing. According to the region feature based video inter-frame splicing method provided by the present invention, on the premise of ensuring precision, a calculation amount is reduced, real-time performance of an algorithm is improved, and meanwhile, the effect of environment change on the image splicing is reduced.

Description

Joining method between a kind of frame of video based on provincial characteristics

Technical field

The invention belongs to technical field of image processing, be specifically related to joining method between a kind of frame of video based on provincial characteristics.

Background technology

Along with unmanned plane in military surveillance, fight calamities and provide relief and the widespread use in the field such as remote sensing remote measurement, the image mosaic technology of video of taking photo by plane causes the attention of Chinese scholars expert.Use image mosaic technology that the video sequence that unmanned plane obtains is spliced into a width and cover FR panorama sketch, be conducive to the accurate location of target, detection and tracking.The core of image mosaic and emphasis are image registration, and the precision of image registration and algorithm amount directly affect precision and the real-time of image mosaic.Taking photo by plane in video, the global motion of image typically refers to the motion of the image background caused due to aircraft and Airborne camera, and the target of motion is usually less, and therefore the global motion of image is occupied an leading position.

The joining method of existing video of taking photo by plane mainly is divided into direct method and characteristic method two class.Wherein direct method compares has what represent meaning to be that attiato realizes the overall motion estimation of image by least square method and then realizes image registration on the basis to image local block motion estimation, the method achieves good effect, but the method easily lost efficacy when image local difference is less, and larger by the impact of environment.By contrast, the application of characteristic method is comparatively extensive, and such as brown and lowe adopts SIFT to achieve the registration of different images and the image mosaic under being applied to off-line state, and the method precision is higher, but algorithm length consuming time, poor real; Steedly adopts MOPMulti-scaleOrientedPatches to realize registration counting yield between frame of video in video-splicing, but its real-time still cannot meet unmanned plane takes photo by plane the requirement that video splices fast.

Summary of the invention

The present invention proposes joining method between a kind of frame of video based on provincial characteristics, under the prerequisite ensureing precision, reduces calculated amount, improves the real-time of algorithm, reduces environmental change to the impact of image mosaic simultaneously.

In order to solve the problems of the technologies described above, the invention provides joining method between a kind of frame of video based on provincial characteristics, comprising the following steps:

Step one: synchro control video camera and inertial sensor work, uses video camera to obtain the depth information of each pixel in image to be spliced, uses inertial sensor to obtain the kinematic parameter of video camera, and calculates the global optical flow of image to be spliced;

Step 2: in image to be spliced, uses light stream fitting process in local to extract the local characteristic region meeting image global motion;

Step 3: in local characteristic region, uses the FAST unique point of FAST feature extraction operator extraction image;

Step 4: use SAD operator to carry out registration to the FAST unique point in reference picture and image to be spliced, calculates the global motion projective transformation matrix of image to be spliced based on FAST unique point in conjunction with RANSAC algorithm;

Step 5: combine and be fade-in gradually to go out Weighted Average Algorithm and complete the seamless spliced of image to be spliced and reference picture.

The present invention compared with prior art, its remarkable advantage is: in (1) practical application, because scene of taking photo by plane is comparatively complicated, the interference of the uncertainty that regional movement is estimated and moving target reduces the estimated accuracy of global motion, the present invention proposes the rapid registering algorithm that the image global motion based on provincial characteristics is estimated, improve precision; (2) hardware platform that the present invention is based on inertial sensor and Kinect video camera composition completes the calculating of image overall light stream, not only reduce calculated amount of the present invention, improve real-time, also eliminated the impact of environment for optical flow computation, there is higher environmental robustness; (3) the present invention carries out by extracting local characteristic region in entire image calculated amount and the computing time that image registration mode reduces image mosaic significantly; (4) the SIFT operator that the present invention adopts the FAST of improvement to extract operator replacement classical carries out feature calculation, again reduces the computation complexity of algorithm.

Accompanying drawing explanation

Fig. 1 be the present invention is based on provincial characteristics frame of video between joining method process flow diagram;

Fig. 2 is the FAST feature extraction operator schematic diagram that the present invention uses;

Fig. 3 is that being fade-in of using of the present invention gradually goes out algorithm schematic diagram.

Embodiment

As shown in Figure 1, joining method between the frame of video that the present invention is based on provincial characteristics, performing step is as described below:

Step one: synchro control video camera and inertial sensor work, uses video camera to obtain the depth information of each pixel in image to be spliced, uses inertial sensor to obtain the kinematic parameter of video camera, and calculates the global optical flow of image to be spliced;

Step 2: in image to be spliced, uses light stream fitting process in local to extract the local characteristic region meeting image global motion;

Step 3: in local characteristic region, uses the FAST unique point of FAST feature extraction operator extraction image;

Step 4: use SAD operator to carry out registration to the FAST unique point in reference picture and image to be spliced, calculates the global motion projective transformation matrix of image to be spliced based on FAST unique point in conjunction with RANSAC algorithm;

Step 5: combine and be fade-in gradually to go out Weighted Average Algorithm and complete the seamless spliced of image to be spliced and reference picture.

Being implemented as follows of above-mentioned steps one:

Step 1.1: the work of synchro control inertial sensor and Kinect video camera, video camera can adopt Kinect video camera, Kinect video camera comprises depth image simultaneously and obtains CCD and coloured image acquisition CCD, can obtain coloured image and depth image simultaneously; Inertial sensor comprises gyroscope and accelerometer.

Inertial sensor is used to obtain motion Eulerian angle [α, beta, gamma], the three-shaft displacement [t of video camera _x, t _y, t _z], and join matrix [R, T] outward according to the motion that the beginning parameter transform model of video camera goes out video camera, wherein α is angle of nutation, β is angle of precession, γ is angle of rotation, t _xfor the displacement in x direction, t _yfor the displacement in y direction, t _zfor the displacement in z direction; R is motion outer ginseng matrix rotation matrix, and T is the translation matrix of motion outer ginseng matrix, and the computing formula of rotation matrix R and translation matrix T is such as formula shown in (1), (2):

$R=\left[\begin{array}{ccc}\cos \mathrm{\γ}\cos \mathrm{\β}& \cos \mathrm{\γ}\sin \mathrm{\β}\sin \mathrm{\α}-\sin \mathrm{\γ}& \cos \mathrm{\γ}\sin \mathrm{\β}\cos \mathrm{\α}-\sin \mathrm{\α}\sin \mathrm{\γ}\\ \sin \mathrm{\γ}\cos \mathrm{\β}& \sin \mathrm{\α}\sin \mathrm{\β}\sin \mathrm{\γ}+\cos \mathrm{\γ}\cos \mathrm{\α}& \sin \mathrm{\γ}\sin \mathrm{\β}\cos \mathrm{\α}+\cos \mathrm{\γ}\sin \mathrm{\α}\\ -\sin \mathrm{\β}& \cos \mathrm{\β}\sin \mathrm{\α}& \cos \mathrm{\β}\cos \mathrm{\α}\end{array}\right]---\left(1\right)$

T＝[t _x,t _y,t _z]'(2)

In formula (2), T is vector [t _x, t _y, t _z] transposed form,

Video camera is used to obtain the depth information u of each pixel in image to be spliced,

Step 1.2: use light stream formulae discovery to go out the global optical flow d (u, v) of image to be spliced, wherein light stream formula is such as formula shown in (3), (4):

$\frac{u_2}{u_1}{m}_2={\mathrm{KRK}}^{-1}{m}_1+\frac{1}{u_1}\mathrm{KT}---\left(3\right)$

d(u,v)＝m ₂-m ₁(4)

Wherein, the pixel coordinate m in reference picture is calculated by formula (3) ₁pixel coordinate m in image to be spliced ₂, calculate pixel coordinate m according to formula (4) ₁global optical flow d (u, v), wherein u is pixel coordinate m ₁the global optical flow of horizontal direction, v is pixel coordinate m ₁the global optical flow of vertical direction, u ₁, u ₂be respectively m ₁and m ₂depth information, K is the internal reference matrix of video camera.

The patented claim that step one implementation procedure specifically can be 201410027349.4 see application number, name is called the method for registering images based on inertial sensor and Kinect video camera.

Being implemented as follows of above-mentioned steps two:

Step 2.1: become by Iamge Segmentation to be spliced size to be the multiple pieces of regions of w × w, w is the number of pixel in block region; In each piece of region, estimate the local affine transformations relation in each piece of region in conjunction with four-point method use global optical flow d (u, v), local affine transformations relation is as shown in formula (5):

$\left\{\begin{array}{c}u=a_{1}x+a_{2}y+a_3\\ v=a_{4}x+a_{5}y+a_6\end{array}\right.----\left(5\right)$

In formula (5), the pixel coordinate that (x, y) is reference picture; Parameter a ₁, a ₂, a ₃, a ₄, a ₅and a ₆respectively as the partial projection transformation matrix H of the parameter factors structure agglomerated regions for matrix _block, as shown in formula (6),

$H_{\mathrm{block}}=\left[\begin{array}{ccc}{a}_1& a_2& a_3\\ a_4& a_5& a_6\\ 0& 0& 1\end{array}\right]---\left(6\right)$

Step 2.2: global optical flow d (u, v) calculated all in block region are substituted into partial projection transformation matrix H respectively _blockin, calculate to obtain and estimate global optical flow value d'(u', v'), as shown in formula (7),

$\left\{\begin{array}{c}{u}^{\′}=a_{1}x+a_{2}y+a_3\\ v^{\′}=a_{4}x+a_{5}y+a_6\end{array}\right.---\left(7\right)$

Step 2.3: according to estimation global optical flow value d'(u', v') and global optical flow value d (u, v) calculate error of fitting σ, as shown in formula (8),

$\mathrm{\σ}=\frac{1}{w\×w}\underset{[u,v]\∈w\×w}{\mathrm{\Σ}}{|d(u,v)-d^{\′}(u^{\′},v^{\′})|}^2---\left(8\right)$

Step 2.4: when the error of fitting σ value in block region is less than the threshold value δ preset, extracts this block region; Otherwise, give up this block region; Finally using the local characteristic region of all block regions extracted as image mosaic.

The present invention has made improvement to the FAST feature extraction operator in step 3, is implemented as follows:

In local characteristic region, in order to obtain the relatively uniform FAST unique point of distribution, the present invention is that step-length scans in local characteristic region with D, is namely expert at on column direction, choose pixel centered by the pixel of D pixel, carry out judgement and the extraction of FAST unique point.The judgement of FAST unique point and extracting mode are as shown in Figure 2, pixel centered by scanning element, 16 pixels around it are selected to be pixel to be compared, wherein row is set to two pixels with the radius on column direction, the radius of diagonal is set to 1 pixel, the Pixel gray difference of computing center's pixel and surrounding 16 points, in the Pixel gray difference of these 16 points, the threshold value T (usual value is 12) preset is greater than if difference is positive quantity, or difference is that negative quantity is greater than the threshold value T preset, then think that this center pixel is FAST unique point.

In order to the impact getting rid of marginal point is gone forward side by side the low operand of a step-down, the gray scale difference value of the present invention first computing center's pixel and Fig. 2 mid point 1, point 9, point 5, point 13, the i.e. first gray scale difference value of four summit pixels on computing center's pixel and line direction and column direction, if difference is that positive quantity is less than 3 in these four gray scale difference values, or difference is that negative quantity is less than 3, then judge that this center pixel is not FAST unique point, without the need to carrying out the calculating of the gray scale difference value of center pixel and other pixels; If difference is that positive quantity is more than or equal to 3 in these four gray scale difference values, or difference is that negative quantity is more than or equal to 3, then proceed the calculating of center pixel and other 12 pixel gray scale difference values, and contrast with setting threshold value T, confirm whether this point is FAST unique point with this.

Being implemented as follows of above-mentioned steps four:

Step 4.1: use SAD operator to complete the registration of FAST unique point in the two field picture of front and back two, the basis for estimation of image registration adopts the computing method of Euclidean distance;

Step 4.2: complete asking for of image overall projection matrix H based on FAST feature point pairs in conjunction with RANSAC algorithm, wherein RANSAC algorithmic procedure is as described below:

In FAST feature point pairs, extract four pairs of feature point pairs arbitrarily, adopt four-point method to ask for overall projection matrix H, then bring remaining feature point pairs into formula (9), the unique point [x' of the rear two field picture estimated by overall projection matrix H _q, y' _q], if the unique point [x' estimated _q, y' _q] meeting formula (10), then this feature point pairs is set to intra-office point, the point namely meeting overall projection matrix H is right; If do not meet formula (10), then this feature point pairs is set to point not in the know, the point namely not meeting overall projection matrix H is right.

[x' _q,y' _q]＝H*[x _p,y _p](9)

In formula (9), [x _p, y _p], [x _q, y _q] be respectively reference picture and treat the unique point coordinate of reference picture,

|[x' _q,y' _q]-[x _q,y _q]|≤T _ran(10)

In formula (10), [x' _q, y' _q] for treating the estimated value of reference picture unique point, T _ranfor weighing the estimated value [x' treating reference picture unique point _q, y' _q] and treat reference picture unique point actual coordinate [x _q, y _q] threshold value, be generally set as [0.1,0.1],

After calculating terminates, the unique point logarithm Num of point in statistics bureau, and judge whether the optimization model exporting overall affine matrix H, be specially: if Num>=M, wherein M is intra-office point number threshold value, then terminate RANSAC algorithm, and using current overall affine matrix H as optimization model; If Num < is M, and the cycle index of RANSAC algorithm is at setting threshold value N _rANin, then in intra-office point, select four to try to achieve to putting and repeating step 4 calculating the new overall affine matrix H meeting formula (10) again, otherwise, using current overall affine matrix H as optimization model.

The present invention gradually to go out Weighted Average Algorithm and has done part to being fade-in in step 5 and improve, and is implemented as follows:

Suppose m ₁(x, y), m ₂(x, y) is respectively the grey scale pixel value of front frame reference picture and present frame image to be spliced, m (x, y) be merge after grey scale pixel value, be then fade-in gradually go out Weighted Average Algorithm calculating such as formula shown in (11):

$m(x,y)=\left\{\begin{array}{cc}{m}_1(x,y)& (x,y)\&Element;m_1\\ k_1{m}_1(x,y)+k_2{m}_2(x,y)& (x,y)\&Element;(m_1\&cap;m_2)\\ m_2(x,y)& (x,y)\&Element;m_2\end{array}\right.---\left(11\right)$

Wherein, k ₁, k ₂for weights, and meet k ₁+ k ₂=1.In order to make the overlapping region of image seamlessly transit better, present invention improves over k ₁, k ₂value, as shown in formula (12),

$k_1=\frac{m_2}{m_1+m_2},k_2=\frac{m_1}{m_1+m_2}---\left(12\right).$

In conjunction with formula (12), and as shown in Figure 3, according to the pixel distance relation of pixel and image to be spliced in the pixel distance of pixel in overlapping region and reference picture and overlapping region, the normalized mode of adoption rate carries out pixel value distribution, thus realize seamlessly transitting of image overlapping region pixel value, thus complete the seamless spliced of image.

Claims (7)

1. based on provincial characteristics frame of video between a joining method, it is characterized in that, comprise the following steps:

Step one: synchro control video camera and inertial sensor work, uses video camera to obtain the depth information of each pixel in image to be spliced, uses inertial sensor to obtain the kinematic parameter of video camera, and calculates the global optical flow of image to be spliced;

Step 2: in image to be spliced, uses light stream fitting process in local to extract the local characteristic region meeting image global motion;

Step 3: in local characteristic region, uses the FAST unique point of FAST feature extraction operator extraction image;

Step 4: use SAD operator to carry out registration to the FAST unique point in reference picture and image to be spliced, calculates the global motion projective transformation matrix of image to be spliced based on FAST unique point in conjunction with RANSAC algorithm;

Step 5: combine and be fade-in gradually to go out Weighted Average Algorithm and complete the seamless spliced of image to be spliced and reference picture.

2. as claimed in claim 1 based on provincial characteristics frame of video between joining method, it is characterized in that, in step one,

Inertial sensor is used to obtain motion Eulerian angle [α, beta, gamma], the three-shaft displacement [t of video camera _x, t _y, t _z], and join matrix [R, T] outward according to the motion that the beginning parameter transform model of video camera goes out video camera, wherein α is angle of nutation, β is angle of precession, γ is angle of rotation, t _xfor the displacement in x direction, t _yfor the displacement in y direction, t _zfor the displacement in z direction; R is motion outer ginseng matrix rotation matrix, and T is the translation matrix of motion outer ginseng matrix, and the computing formula of rotation matrix R and translation matrix T is such as formula shown in (1), (2):

$R=\left[\begin{array}{ccc}\cos \mathrm{\&gamma;}\cos \mathrm{\&beta;}& \cos \mathrm{\&gamma;}\sin \mathrm{\&beta;}\sin \mathrm{\&alpha;}-\sin \mathrm{\&gamma;}& \cos \mathrm{\&gamma;}\sin \mathrm{\&beta;}\cos \mathrm{\&alpha;}-\sin \mathrm{\&alpha;}\sin \mathrm{\&gamma;}\\ \sin \mathrm{\&gamma;}\cos \mathrm{\&beta;}& \sin \mathrm{\&alpha;}\sin \mathrm{\&beta;}\sin \mathrm{\&gamma;}+\cos \mathrm{\&gamma;}\cos \mathrm{\&alpha;}& \sin \mathrm{\&gamma;}\sin \mathrm{\&beta;}\cos \mathrm{\&alpha;}+\cos \mathrm{\&gamma;}\sin \mathrm{\&alpha;}\\ -\sin \mathrm{\&beta;}& \cos \mathrm{\&beta;}\sin \mathrm{\&alpha;}& \cos \mathrm{\&beta;}\cos \mathrm{\&alpha;}\end{array}\right]---\left(1\right)$

T＝[t _x,t _y,t _z]'(2)

In formula (2), T is vector [t _x, t _y, t _z] transposed form,

The computing method of the global optical flow of described image to be spliced are such as formula shown in (3) and (4):

$\frac{u_2}{u_1}{m}_2={\mathrm{KRK}}^{-1}{m}_1+\frac{1}{u_1}\mathrm{KT}---\left(3\right)$

d(u,v)＝m ₂-m ₁(4)

Wherein, the pixel coordinate m in reference picture is calculated by formula (3) ₁pixel coordinate m in image to be spliced ₂, calculate pixel coordinate m according to formula (4) ₁global optical flow d (u, v), wherein u is pixel coordinate m ₁the global optical flow of horizontal direction, v is pixel coordinate m ₁the global optical flow of vertical direction, u ₁, u ₂be respectively m ₁and m ₂depth information, K is the internal reference matrix of video camera.

3. as claimed in claim 1 based on provincial characteristics frame of video between joining method, it is characterized in that, the implementation procedure of step 2 is:

Step 2.1: become by Iamge Segmentation to be spliced size to be the multiple pieces of regions of w × w, w is the number of pixel in block region; In each piece of region, estimate the local affine transformations relation in each piece of region in conjunction with four-point method use global optical flow d (u, v), local affine transformations relation is as shown in formula (5):

$\left\{\begin{array}{c}u=a_{1}x+a_{2}y+a_3\\ v=a_{4}x+a_{5}y+a_6\end{array}\right.----\left(5\right)$

In formula (5), the pixel coordinate that (x, y) is reference picture; Parameter a ₁, a ₂, a ₃, a ₄, a ₅and a ₆respectively as the partial projection transformation matrix H of the parameter factors structure agglomerated regions for matrix _block, as shown in formula (6),

$H_{\mathrm{block}}=\left[\begin{array}{ccc}{a}_1& a_2& a_3\\ a_4& a_5& a_6\\ 0& 0& 1\end{array}\right]---\left(6\right)$

Step 2.2: global optical flow d (u, v) calculated all in block region are substituted into partial projection transformation matrix H respectively _blockin, calculate to obtain and estimate global optical flow value d'(u', v'), as shown in formula (7),

$\left\{\begin{array}{c}{u}^{\&prime;}=a_{1}x+a_{2}y+a_3\\ v^{\&prime;}=a_{4}x+a_{5}y+a_6\end{array}\right.---\left(7\right)$

Step 2.3: according to estimation global optical flow value d'(u', v') and global optical flow value d (u, v) calculate error of fitting σ, as shown in formula (8),

$\mathrm{\&sigma;}=\frac{1}{w\&times;w}\underset{[u,v]\&Element;w\&times;w}{\mathrm{\&Sigma;}}{|d(u,v)-d^{\&prime;}(u^{\&prime;},v^{\&prime;})|}^2---\left(8\right)$

Step 2.4: when the error of fitting σ value in block region is less than the threshold value δ preset, extracts this block region; Otherwise, give up this block region; Finally using the local characteristic region of all block regions extracted as image mosaic.

4. as claimed in claim 1 based on provincial characteristics frame of video between joining method, it is characterized in that, the implementation procedure of step 3 is:

Choose pixel centered by the pixel of D pixel, scan in local characteristic region, pixel centered by scanning element, 16 pixels around it are selected to be pixel to be compared, wherein row is set to two pixels with the radius on column direction, the radius of diagonal is set to a pixel, computing center's pixel and the Pixel gray difference of 16 pixels around it, if difference is positive quantity or difference is negative quantity, the threshold value T having one of them to be greater than to preset, then using this center pixel as FAST unique point.

5. as claimed in claim 4 based on provincial characteristics frame of video between joining method, it is characterized in that,

The gray scale difference value of four summit pixels on first computing center's pixel and line direction and column direction, if in these four gray scale difference values, difference is positive quantity or difference is that negative quantity has one of them to be less than threshold value 3, then judge that this center pixel is not FAST unique point, no longer carry out the calculating of center pixel and the gray scale difference value of remaining 12 pixels of surrounding and compare; If in these four gray scale difference values difference be just or difference be negative quantity one of them be more than or equal to threshold value 3, then proceed the calculating of center pixel and remaining 12 the pixel gray scale difference values of surrounding.

6. as claimed in claim 1 based on provincial characteristics frame of video between joining method, it is characterized in that, rapid four be implemented as follows:

Step 4.1: use SAD operator to complete reference picture and treat the registration of FAST unique point in reference picture, the basis for estimation of image registration adopts the computing method of Euclidean distance;

Step 4.2: complete asking for of image overall projection matrix H based on FAST feature point pairs in conjunction with RANSAC algorithm, wherein RANSAC algorithmic procedure is as described below:

In FAST feature point pairs, extract four pairs of feature point pairs arbitrarily, adopt four-point method to ask for overall projection matrix H, then bring remaining feature point pairs into formula (9), estimated the facial feature estimation value [x' of image to be spliced by overall projection matrix H _q, y' _q], if the unique point [x' estimated _q, y' _q] meet formula (10), then this feature point pairs is set to intra-office point; If do not meet formula (10), then this feature point pairs is set to point not in the know,

[x' _q,y' _q]＝H*[x _p,y _p](9)

In formula (9), [x _p, y _p], [x _q, y _q] be respectively reference picture and treat the unique point coordinate of reference picture,

|[x' _q,y' _q]-[x _q,y _q]|≤T _ran(10)

In formula (10), [x' _q, y' _q] for treating the estimated value of reference picture unique point, T _ranfor weighing the estimated value [x' treating reference picture unique point _q, y' _q] and treat reference picture unique point actual coordinate [x _q, y _q] threshold value, be generally set as [0.1,0.1],

After calculating terminates, the unique point logarithm Num of point in statistics bureau, if Num>=M, wherein M is intra-office point number threshold value, then terminate RANSAC algorithm, and is exported as optimization model by current overall affine matrix H; If Num < is M, and judge the threshold value the N whether cycle index of RANSAC algorithm is presetting further _rANin, if the threshold value N that the cycle index of RANSAC algorithm is presetting _rANin, then in intra-office point, select four to try to achieve new overall affine matrix H to putting and repeating step 4 again; Otherwise, current overall affine matrix H is exported as optimization model.

7. as claimed in claim 1 based on provincial characteristics frame of video between joining method, it is characterized in that, be fade-in in step 5 gradually go out Weighted Average Algorithm calculating such as formula shown in (11):

$m(x,y)=\left\{\begin{array}{cc}{m}_1(x,y)& (x,y)\&Element;m_1\\ k_1{m}_1(x,y)+k_2{m}_2(x,y)& (x,y)\&Element;(m_1\&cap;m_2)\\ m_2(x,y)& (x,y)\&Element;m_2\end{array}\right.---\left(11\right)$

Wherein, m ₁(x, y), m ₂(x, y) is respectively the grey scale pixel value of front frame reference picture and present frame image to be spliced, and m (x, y) is the grey scale pixel value after merging, k ₁, k ₂for weights, and meet k ₁+ k ₂=1, k ₁, k ₂value as shown in formula (12),

$k_1=\frac{m_2}{m_1+m_2},k_2=\frac{m_1}{m_1+m_2}---\left(12\right).$