TWI254257B - Method and system for non-iterative global motion estimation - Google Patents

Abstract

A fast non-iterative global motion estimation (GME) algorithm is disclosed for estimating the perspective transform global motion parameters from the motion vectors (MV) obtained from the block matching process that includes grouping a plurality of motion vectors in the input video stream into a predetermined number of groups of motion vectors, calculating a set of global motion parameters from each of the predetermined groups of the motion vector, and processing the set of global motion parameters generated from the calculation to obtain a final estimation.

Description

1254257 玖, invention description: [Miao Ming belongs to the technical field] Domain mobile estimation method mobile estimation method I invention is related to the whole method and system in the image sequence ^ 乂 &, 4, especially a non-repetitive global domain And system. L Prior Art 10 To 1 uses a video camera or a digital camera (DS| is known as a scene. The scene recorded by the camera = stored as a sequence of images. The sequence of images is controlled by the camera. The time interval is obtained by taking individual images or frames. When the time interval is sufficiently small, the continuous playback of 1 can fully reproduce the dynamics of the recorded scene.

In general, the movement in the image sequence or the 5 differences in the continuous frame is caused by the movement of the recorded object or by the movement of the camera. The movement of the camera originates from the user's function (such as zooming in) or unintentional movement, or jitter. Due to the movement of the camera, a global movement occurs in the image sequence, which is the entire scene shift and movement. This and the moving of the recorded object relative to the stationary background are different. The global movement caused by jitter is generally unintentional, and it is really unwanted during the recording process. Some systems and methods have been proposed to estimate and compensate for global mobility. In the prior art, the global movement in the image sequence often simulates 5 1254257 into a two-dimensional parameter conversion. Global Motion Estimation (GME) estimates the conversion parameters from images. Global Mobile Estimation (GME) is an important tool and is widely used in computer vision, image processing and other related fields. For example, for MPEG4 global motion estimation, global mobility is described using parametric forms, which can be modeled from a simple model with only two parameters to a perspective model with eight parameters. Among these models, a model with eight parameters is the most general in the global motion estimation of MPEG4. According to this model, in the global motion (GM) of the 10 reference frame and the current frame, the coordinates calculated by the following formula are expressed: m0x+ mxy+ m2 m3x+ m4y+ m5 x ^ y ————-- m6x+ ΐΥΐηγΛ -1 m6x+ rr^y + 1 . Global Mobile (GM) can pass all eight parameters m of the calculation frame. ~m7 and get. In the field of pixels or compression, many algorithms have been proposed to estimate the global movement of MPEG4. Although acceptable performance can be achieved through repeated calculations, such computational cost is not allowed for instant encoding or for wireless devices with limited computing power. In addition, the conventional Global Motion Estimation (GME) algorithm is considered to be the most time consuming and inefficient operation in MPEG4 Advanced 20 Simple Profile (ASP) image coding. For some applications involving Global Mobile Estimation (GME), once the computational cost is considered the primary consideration, a new 1254257 technique is needed to reduce the computational complexity. SUMMARY OF THE INVENTION The object of the present invention is to provide a non-repetitive global mobile 5-motion estimation method for estimating global motion between multiple image frames of an input image data stream, which includes the image data stream The plurality of motion vectors in the group are grouped into a plurality of packets having a predetermined number of motion vectors, and secondly, the global motion vector parameter values of the groups are determined according to the motion vectors in each group, and finally, each of the calculated calculations 10 A set of global motion vector parameter values is obtained to obtain a final global motion vector. In a preferred embodiment, in each predetermined group, the motion vector grouping step groups the motion vectors in the group based on a fixed spatial distance between the motion vectors. At the same time, in accordance with the present invention, there is provided an estimated global motion method between a reference image frame and a current image frame, comprising using a perspective consideration model having eight global motion parameters (m〇-m7), The global mobility parameter (m〇-m7) can be described by the following formula: , .. ,.m0x + mxy + m2 = /;(χ,}|ιη) = ^-1-f- m6x + m7y +1 ,r ( , x m3x + m4y + m5 2〇7 =/>, less Im)= ¥ + ¥ + 1, where (x, _y) and (x', /) are in the reference image frame and the current image The coordinates of the frame, m is the set of eight global mobility parameters (m〇-m7), also 1254257 ie m = [m〇,···,%]. The set m with eight global moving parameters (mO - m7) can be calculated by using the following algebraic distance formula: /=〇Ν-\ =Σ Λ: }\τη6χι +1) -(^ +/^) ,,·(—ς+哪+1)-(-;+卿+巧)—vhm +Vx ·% +Vx, word rvmw,,+>;'>: .w where ν=Λ^· +'·And brother, =M^+ brother. In an embodiment, the 'algebraic distance' formula in an over-determined linear system can be: % where 1 0 0 0 ~ y〇V, 0 0 0 ^ Λ 1 -3⁄43⁄41凡凡, V Μ Μ Μ Μ Μ Μ Μ Μ = Μ 1 0 0 0 ϋ / Uw ' m4 , 0 0 0 1 -yN.xyNA\ 10 According to the present invention, a non-repetitive global mobile estimation method is provided. Estimating global motion between a plurality of image frames of an input image data stream, comprising grouping the plurality of motion vectors in the image data stream into a plurality of packets having a predetermined number of motion vectors, followed by each The motion vector in the group 'determines the global motion vector parameter values of the groups. Finally, the calculated global motion vector parameter values of each group are processed to obtain a final global motion vector. The calculation step seeks the global motion vector parameter value of the group to include the use of algebraic distance (Cai (10) dlstance) to describe the ten-home global mobility parameter, and to use the over-determined 15 1254257 (over-deternnned) linear system. Calculating Algebraic Distance (Mg Coffee) In accordance with the present invention, a non-repetitive global motion estimation system is provided for estimating a global 5 movement between a plurality of image frames of an input image data stream, which includes - The grouping device divides the plurality of motion vectors in the input image data stream into groups having a predetermined number of motion vectors; and the counting is based on the motion vectors in each group to obtain the groups The global motion vector parameter value of the group {m mountain~^, the per-global motion vector estimate contains eight global mobility parameters (7). ~ 7); A post-processing device' processes the group global motion vector parameter values for each group calculated in the computing device to obtain a final global motion vector. [Embodiment] Before a detailed description of one or more preferred embodiments, a person skilled in the art will correctly evaluate that the present invention is not limited to the application described in the specification, or the component arrangement, or the detailed description. Or the arrangement of the illustration. The present invention is a fast non-repetitive global motion estimation (Global 20 Motion Estimati〇n, GME) method, which is obtained by a plurality of motion vectors (M〇ti〇n Ve ctors, MV) obtained by the block motion estimation method. To estimate the global movement parameters of the perspective. The present invention uses a global motion estimation (GME) method based on non-repetitive moving directions: !: to estimate eight 1254257 global movements in the most general painting model (G1〇bal M〇tion, gm )parameter. The method of the present invention uses a plurality of motion vectors of an input image stream to estimate global motion (GM). In the present invention, the motion vector (Mv) based method can reduce the computational complexity of the Global Motion Estimation (GME) while maintaining the most image distortion. Furthermore, the method of the present invention can be implemented after the block motion estimation (bl〇ck_baad motion estimati〇n, BME) in the conventional MPEG-4 encoder. In particular, the motion vector (MV) method of the present invention is linear and non-repetitive, so that perspective global motion (G M ) can be estimated efficiently and strongly. In accordance with a preferred embodiment of the present invention, the Global Motion Estimation (GME) of the motion vector (Mv) method performs a perspective global motion (gm) estimation of a general perspective model from a sample motion vector (mv) domain. Block Motion Estimation (BME) first performs block motion vector (MV) estimation on a portion of the image frame, and then block motion vector 15 (MV) is used to estimate global motion (GM) parameters. . A preferred embodiment of the present invention provides an estimation method for estimating global movement between a plurality of image frames of an input image stream. According to the preferred embodiment, the plurality of motion vectors (MVs) of the input image data stream are divided into a group motion vector. The estimation method calculates the global motion estimation (GME) of the J group based on the motion vector in each group. The group's Global Mobile Estimation (GME) is further processed to obtain a final estimate. The present invention further provides a global motion estimation system for estimating global motion between a plurality of image frames of an input image data stream. The system includes a grouping device, a computing device, and a 10 1254257 post-processing device. The sub-series ^ ^ ^ , and the table is divided into a plurality of motion vectors in the input image, F ^ α ζ π # ^ Θ 枓 £ ^ £ into J group of motion vectors. The bucket device is based on the J group shifting quantity, and obtains the movement vector parameter value {nij} of the group, and the global mobile vector estimate; I丨1 '1 , then the value m] contains eight global mobile springs (to.....). The post-processing device processes the calculated j-group global mobility parameter values to obtain - 畀, ^^ θ moves to the last global domain to ϊ. Ίο 15

In accordance with the present invention, an image frame recorded by a digital camera is first divided into a plurality of blocks, each block comprising a matrix of one pixel and one. The motion vector of each block is first estimated using the block motion estimation method, and the obtained motion vector is processed to obtain a post-acquisition estimate. Unlike the conventional estimation method of estimating one pixel by pixel, the conventional estimation method is repeated, so that a lot of time is wasted. However, the present invention performs global motion estimation (GME) for each block, thereby reducing the number of calculation steps and calculating the complexity.

For example 'consider a point on a moving object in three dimensions. Its position can be expressed as a three-dimensional coordinate x = (U, z) r j3, where (X(7), 0 can define the 20 trajectories of the point in time in three-dimensional space. An image capture system projects the three-dimensional representation to two Dimension image plane, and then sample the two-dimensional image plane at a certain interval x = (x, 3;) r ei? 2. According to the two-dimensional image projection, a two-dimensional motion trace (X(1), 〇 can be obtained. In general, a moving vector field (MV field) is a function of the 1 1254 257 of the moving trajectory of a continuous space coordinate. In practical applications, the vector function is generally used by the parameter table. This parameter type field is used to convert A, and parameters, or to use the moving track of some reference points. In the MPEG-4 standard, a large number of two-dimensional parameter models, with eight parameters The transmissive M ^ thinks that the model is the most common. It can be defined as: 6*^ + 7 less + 1 m6x + m7y +1 where (4) and (Ο,) are respectively in the reference image frame and current The coordinates of the image frame 'm are eight global movement parameters (m.-m7) The set, which is also the preferred embodiment of the present invention, focuses on the most common perspective consideration conversion in the MPEG-4 standard. Figure 1 is a schematic diagram showing global motion compensation in a perspective consideration model. It shows the relationship between the current frame and the reference frame. According to a preferred embodiment of the present invention, each of the image frames recorded by the digital camera is used using the conventional Global 15 Motion Estimation (GME) method. The region is divided into a plurality of motion vector (MV) blocks. In the MPEg_2 to PEG PEG-4 video conversion coding application, multiple motion vectors (μ v) can be obtained by a block-matching method. The motion vector obtained by the estimation (BME) method has noise sampling in the MV field of the moving vector. The purpose of the actual global motion estimation (GME) method is to achieve a fast and robust estimation of the global motion parameters. 12 1254257 The global domain using the motion vector (M v )::: is from the moving to the motion vector (Mv) can be obtained from the image compression data stream, and - meaning {, ·,,,)}_," Suspect, 仃 frame ^) 帛i The number of motion vectors, (4) the total number of motion vectors. Using the distance formula of the Euclidian space, since the perspective model is a nonlinear system, the parameter m can be calculated by the following nonlinear least squares (10) problem. Nl N m = argmin{Ellr/|2} = argmin{ymi=0 m to MVxi " I m) + X.-force; (~ is I m) + brother (2)

However, in order to solve the above-described nonlinear least squares problem, a repetitive optimization procedure is required. When the recurring computational burden increases the cost of Global Mobile Estimation (GME), for many applications, such procedures are not used because they are too costly. In accordance with a preferred embodiment of the present invention, an algebraic distance is used as a function of the target, which can be linearized and expressed as follows:

7V-1/=Σ /=0 /=0 ^ '{m +ι) Αρψι +m +^)_yt +^. +1) Ατψ, +/^) (3) where, and yes·=Λ^+ It is. Those skilled in the art will know that the least squares formula in equation (3) is an outlier, and the least squares formula is huge due to the inaccuracy of the block motion estimation (ΒΜΕ) and the region shift. 13 20 1254257 Many robust degradation algorithms (such as Μ-estimators) were built to solve the anomaly problem of the least squares formula. The preferred embodiment of the present invention avoids the use of / repetitive algorithms, but still handles outliers. Equation (3) can be solved using the following over-determined linear system: % % Less than 1 0 0 0 0 0 0 3⁄4 Μ Μ Μ Μ yNx ί 〇-w - Λ i - W u〇, MM Μ M 0 0 -Λμ%-】 5

mA

V V M W UW or A2iVx8m8xl = b2_ (4)

It is equivalent to ArAm = A,. The matrix equation can be solved by at least one standard inverse matrix program, or by a Singular Value Decomposition (SVD) virtual inverse matrix program. 10 Using this algebraic distance in the target function, the estimation problem in the prior art is completely simplified. Although this algebraic distance may cause some accuracy deviation, the simulation results of Fig. 5 and Fig. 6 which will be described later show that the = energy degradation is not obvious.

2 is a flow chart of the non-repetitive global motion estimation method of the present invention, 15 = first in step 31, the plurality of motion vectors 3 in the input image data stream are divided into J group motion vectors. Figure 4 is a schematic illustration of the grouping step. There are at least 4 or more motion vectors in each group. In step 32, the global motion estimate (gme) for each group is solved by the formula (4) V Am = Arb using the singular value decomposition (s vd) virtual inverse matrix process 14 1254257. In an embodiment, A is an 8 by 8 matrix. Finally, a set of global mobility parameters can be obtained (7). Mountain=1. In general, the more separated the motion vectors in a group, the more mobile.

Inwardly, the better the separation energy of the global motion parameters is estimated. The above can be explained by the fact that the A 5 pair inversion system is so inappropriate, that is, the larger the spatial distance, the smaller the number of conditions of the matrix A (which is the same as A and the number of conditions). In particular, in singular value decomposition (s VD), the number of conditions is defined as the ratio of the largest singular value of a matrix to the smallest singular value. When the number of conditions is large enough, the matrix is close to singularity and its inverse 10 matrix becomes unreliable. The method of the present invention can pick any four motion vectors to produce a king domain mobility parameter. For example, the four motion vectors can be selected from four corners that are close to the ~ image. However, in order to prevent the selected four moving directions from being destroyed by the area moving, a preferred embodiment 15 of the present invention groups all the moving vectors, and each group contains at least four moving directions to achieve a robust estimation. Measurement. As shown in Figure 4, the sub-, and is based on the fixed space pattern of the input motion vector. In each group, 'all motion vectors are fixed space distances, while each group has the same and largest spatial separation. 20 Referring back to FIG. 2, in step 3 3, the /est estimated value is calculated from the set of global mobility parameters = u obtained in step 32. According to a preferred embodiment of the invention, the final estimate is obtained by performing a histogram post-processing method. That is, (mj丨』=1; 1 histogram uses 15 1254257 four partitions (bln) in each dimension of the eight dimensions, and then selects one with the largest number of divisions. The number is based on the number of groups. Here: and the simulation system has four areas = (bin) in each of the eight dimensions. Finally, the calculation is performed by the selected one in the interval. The final estimate. The more the number of bins used in each dimension, the more accurate the results are. However, the simulation results show that the simulation results for the four bins have been shown. Rarely accurate results. By grouping the motion vectors, the present invention divides the input motion vector into 10 sub-groups without overlap, and obtains a global = parameter from each sub-group. Some global mobile parameters are called abnormalities. The situation is destroyed, but more likely to be close to the true global mobile value. The method of the present invention described above can be implemented by the system of Figure 3. Referring to Figure 3, system 40 includes a block-type matching device 41 that inputs human-image data: 15 401 and generate multiple motion vectors. The multiple movement directions The quantity is further divided into two groups of devices 42 having a predetermined number of motion vectors, each group containing at least four motion vectors, the grouping being based on the input motion vector fixed spatial pattern. In each group, all motion vectors The system is a fixed spatial distance. The system 40 further includes a computing device 43 that calculates a global mobile vector parameter value for each of the electricity. The calculation is based on a perspective model having eight global motion parameters (m. ~ 7). 'It can be described by the formulas (1) to (4). After the calculation, the J group global motion vector parameter value can be obtained {丨丨\〆16 1254257 The J group global motion vector parameter value {Π1』}" = 1; Further processed by a post-processing device 44. In accordance with a preferred embodiment of the present invention, the post-processing device 44 first computes a histogram from the j-group {mj} j = i, which is used in each of the eight dimensions. Four partitions (bin). Then select a segment with the largest number of %, and finally average the mj in the selected segment to obtain the final estimated value of 410. The final estimated value 4 0 2 and then output to a processor (Figure not Figure 5 (a), 5 (b) and Figure 6 show simulation results of the method and system of the present invention compared to conventional methods and systems. Global Mobile Estimation (GME) of the present invention The method is used as a fast estimate global motion (GM) in MPEG-4 encoding to replace the predetermined pixel repeat global motion estimation (GME) method in a reference software Momusys. The Global Mobile Estimation (GMe) of the present invention. The method is labeled as 15 MV-GME' where the motion vector of integer pixel units generated by the full search block motion estimation (BME) based on the 16x16 macroblock is input to the fast global motion estimation (GME) method. in. Some CIF-sized image streams are used in simulations, including camera zooming and moving. Figures 5(a) and 5(b) show a comparison of the rate-distortion comparison between the pixel-based global motion estimation (GME) method and the MV-GME. In the figure, MV-GME is very close to the pixel-based Global Motion Estimation (GME) method in coding efficiency. Figure 6 shows a comparison of the bits used when encoding the first 3 frames of the ΜIT image sequence, where the fixed QP has a PSNR of approximately 3 〇 ^. 17 1254257 Computing device Input image data stream 43 Post-processing device 44 401 Final estimated value 402

20

Claims (1)

1254257 Picking up, patenting scope · 1 · A non-repetitive global mobile estimation method for estimating global movement between multiple image frames of an input image data stream, the method comprising: · a grouping step Dividing the plurality of motion vectors 5 in the image data stream into groups having a predetermined number of motion vectors; the counting step is to determine the global movement of the groups according to the motion vectors in each group The vector parameter value; and - the processing step 'processes the group global motion vector parameter value of each group calculated in the calculation step to obtain - the last global 10 motion vector. 2. The global motion estimation method according to claim 2, wherein, in the grouping step, each group has #N motion vectors, and N is an integer greater than or equal to 4. Calling no one to describe the mobile domain estimation method, : 'The multiple mobile vectors in the input image data stream are obtained using the block motion estimation method. The global mobile estimation method, 20:, the δ hai grouping step is divided into groups having a predetermined number of motion vectors according to the fixed spatial distance of the motion vector interval. 5. The global motion vector obtained by the 'in this calculation step' in the global mobile f as described in the scope of the patent application, and the calculation of the global motion vector estimation system based on the I Μ 王 王 移动 移动 移动(The so-called perspective is described by the following formula: 4 chess type 21 1254257 m0x + mxy + m2 m6x + m7y +1 , ^ , , , m3x + m4y + ms y =fy(x,y\^) =-r - 〇, 少) and (/, y) are the coordinates of the reference image frame and the current image frame, respectively, m is the set of eight global motion parameters (m〇-m7), that is, ^^^. ,···,%], m of the eight global mobility parameters (m〇 - m7) can be calculated by the following algebraic distance formula: V·(啊+哪+1) +你+%) ,·'.(啊+哪+1)-(叫+哪+,)_ hm -π + V· + V wV 2 - π V % + + (Where, and brother. '= From ^ + brother. 6. The global motion estimation method according to claim 5, wherein in an over-determined linear system, the algebraic distance is 10 The formula for algebraic distance can be: '^ Λ 1 〇0 0 0 3⁄4 Μ Μ Μ Μ XN-l JVl 1 〇00 ^-1 0 0 -W - less 〇V y〇1 -W -•M), Μ Μ Μ M 0 0 - -Wam ~yN-lXN-\ Λμ 1 · —Un-i ~"3Vi3V-i V π π =Μ ΧΝ-1 7· The global motion estimation method as described in claim 6, wherein the matrix equation can be solved by at least one standard inverse matrix program or singular value decomposition (Singular Value Decomposition) 15 virtual The inverse matrix program solves. 22 1254257 8. The global motion estimation method according to claim 5, further comprising: a histogram calculation step using four partitions in each dimension of eight dimensions ( Bin) the histogram 5 'calculates the global motion vector parameter = ij ; a selection step to select a segment with the largest amount of %; and an averaging step to average the mj in the segment selected by the selection step, Get the final estimate. 10 9· A method for estimating a global motion vector is an estimated global motion vector between a reference image map and a current image frame, the method comprising: a step of using a perspective consideration model, which uses one with eight global domains A perspective consideration model for the movement parameter (m〇-m?), which is described by the following formula: xf = fx(x,y\m) = y' - fy(x,y I m ) = 15 mox + total reduction + ιηβχ + ιηΊγ + \ m6x + m7y + l where '(4) and (w) are the coordinates of the reference image frame and the current image frame respectively, m is the global movement parameter ( a set of mQ_(d), that is, a calculation step that uses the following algebraic distance (4) gehie distance formula to calculate a set m with a global moving parameter d%): 23 20 1254257 using an over-determined linearity The system calculates the algebraic distance; and a processing step of processing the group global motion vector parameter values of each group calculated in the calculating step to obtain a final global shifting motion vector. The global mobile estimation method according to claim 11, wherein the processing step further comprises calculating a histogram of a global motion vector parameter. 10 15 20 13 - A system for estimating global motion for estimating global motion between multiple image frames of an input image stream, the system comprising: a grouping device for streaming the input image data The plurality of motion vectors in the group are divided into groups having a predetermined number of motion vectors; the global motion vector parameter values of the groups are called, and the global motion vector estimation value mj includes eight global motion parameters. 7); and. Each group of nasal discharges, a final one-time post-processing device, processes the group global motion vector parameter values of the groups counted in the computing device to obtain a domain motion vector. - The system described in item 13 of the patent scope, #, each group; and having N moving vectors, (4) - an integer greater than or equal to 4. In the case of the application, the system and system described in item 13 of the patent application, in which: The plurality of motion vectors in the stream uses the block motion estimation method: 25 1254257 16 as set forth in claim 13 wherein the inter-division distance is divided into fixed-group 5 devices according to the motion vector interval. Group of empty number of motion vectors 17. As in the system of claim 13 of the patent application, the calculation of the bean is calculated by obtaining a set of global motion vector parameters 2: the calculation of the global motion vector estimation system has A perspective consideration model for eight global motion parameters (m. ~ 7), which is described by the following formula: m6x + m7y-{-l m3x + m4y + i m6x + m7^ + l 10 (A less) and (/ , less ') are the coordinates of the reference image frame and the current image frame, respectively, m is the set of eight global motion parameters (m〇-m7), that is, m = [〇,····,7 〇 18. The system of claim 17 wherein the m of the set of eight global mobility parameters (m0 - m7) can be calculated by the following algebraic distance formula: m ^=1 /= 〇 where ^ '{m +1) -(^ +ΐΨι _γ. ^ +ι) +^). :Μί^ + · Easy and brother '= + · brother. 26