DE4112235A1 - Separating zoom, rotational and translational motor parameters in image sequences - dividing motion vector data and storing in matrix memory, forming histograms and determining global parameters - Google Patents

Abstract

A high definition t.v. transmission system uses separation of zoom, rotation and translation parameters. The first stage in the process consists of storing the motion vectors (Vxi,Vyi) in a memory (A). The values stored are then set in relation to the different coordinates (B) and the different levels in the memory matrix used (C) to generate a 3D histogram (h). Global parameters (axglob,ayglob) are determined (D) for pure translational motion and a separate parameter (rglob) for rotational movement. A global zoom parameter (bglob) is generated by separate units (EFG). ADVANTAGE - Simultaneous generation of zoom, rotation and translational motion.

Description

The invention relates to a method for separating Zoom, rotation and translation parameters for the Calculation of global motion parameters in moving TV picture sequences. When coding image sequences for image transmission - e.g. B. HDTV or Telephony - or with a high quality Change of norms for moving picture scenes - e.g. B. from 60 Hz 50 Hz - motion vectors play a central role. By comparing two successive images can move an object from one to the next Image can be determined. Here, the picture is usually divided into blocks with block sizes from 8 · 8 to 16 · 16 Pixels. However, the block size plays in the following not matter. The so obtained or "measured" Displacement or motion vectors are at Bit rate reduction techniques from sender to receiver transmitted and in the receiver for reconstruction of an image used on the basis of the previous one. At Norm changes are used to temporally to create intermediate images.

The motion can be derived from these motion vectors Determine parameters that are no longer the movement of a are assigned to a small block, but the movement of one larger object or the background (e.g. Camera panning or zoom).

So far, complex recursive methods, such as. B. linear regression used around translational movements separate from zoom and rotation and so the movement Determine parameters of objects and background.  

The speed of convergence depended on the Picture template from and in critical cases that failed Process (see: Goetze, M: "Generation of motion vector fields for motion compensated interpolation of HDTV signals ", signal processing of HDTV, Elsevier Science Publishers B.V. (North Holland), 1988, pp. 383-391).

The following is a much simpler procedure specified by which the movement parameters of larger Objects or background from the measured movement vector gates determined by 2 to 3-dimensional histograms can be. In contrast to the usual methods all recursive calculations are omitted. It will be one direct investigation of the local motion vectors carried out.

The motion vectors can be determined using various methods described in the literature. The most illustrative is the so-called block matching method, which is shown schematically in FIG. 4. The block matching procedure is based on the following process:

• A) Delay by two fields or one frame
• B) Block-by-frame image decomposition; the two-dimensional blocks include e.g. 4 x 4 or 8 x 8 or 16 x 16 Pixels.
• C) definition of a two-dimensional search area, e.g. B. of ± 17 · ± 17 pixels.
• D) Shift of a block corresponding to B by one pixel or one pixel in the Neighbor line in the search area per processing step.  
• E) Determine the correlation between the blocks B and D; in the simplest training there will only be one Difference between the picture elements of the correct sponding blocks of B and D measured and over the total block totaled (block difference).
• F) Determination of the highest determined in the search area Correlation or the smallest block difference. This results in the displacement of the block Reason of movement.
• G) vectorial representation of the displacement (movement).

The method according to the invention enables at the same time according to the appearance of zoom, rotation and translation Separation of these three movements. Via post processing the motion vectors can then be global Determine movement parameters of each of the three movements.

The procedure is independent of whether in the presentation the motion vectors a pel or subpel resolution is chosen. Pel-precise resolution means that the measured vectors refer only to the original two-dimensional and in general also orthogonal Obtain the scanning grid. At subpel-precise resolution intermediate grid points are also allowed, but from the original sampling grid must be interpolated.

The global motion estimation is based on the Statistics of motion vectors in general on subpel Accuracy even when assuming pel-precise vectors becomes.

If zoom, rotation and translation occur simultaneously, see above the following equation can be assumed:

The horizontal and vertical direction of the image is designated x and y and (v _x , v _y ) is the "measured" displacement vector belonging to the pixel (x, y). The zoom parameter sought is denoted by b, the parameter for the rotation with r and the sought parameter for the translatory movement in the x and y directions with a _x and a _y .

If the centers of the zoom and rotation are not in the Origin of the coordinate system and this is becoming more normal as the case may be, the additional terms that occur also interpreted as translational sham movements will.

These apparent movements appear in the transla Including toric movement and thus can without Limitation of the general validity of the coordinates origin (e.g. in the middle of the picture) and Rotation and zoom can be defined with regard to this point what happened in Equation 1.

You can now either use the zoom parameter b or to eliminate the rotation parameter r. At Elimination of the zoom parameter b results

v _y x - v _x y = a _y x - a _x y + (x² + y²) r (2)

If the quantities a _x , a _y and r are understood as axes of a coordinate system in three-dimensional space, equation 2 forms a plane in this space. The set of all planes intersect at one point with the rotation and translation parameters sought. The levels do not intersect in a straight line. If the rotation parameter r is understood as a function of (a _x , a _y ), the slope of the plane in the directions a _x or a _y depends on x or y (-x / (x² + y²) or y / (x² + y²)) and is therefore different for each value of (x, y).

In space (a _x , a _y , r) the values of the points on the respective levels are increased by 1. The histogram created in this way is three-dimensional, the maximum represents the value sought.

The motion vectors are usually not in the case of television pictures exactly determinable, but with a certain fluctuation wide afflicted. Then there will be no clean intersections give more but areas where the levels are cut frequently. The maximum of this range is then interpreted as the global movement.

To determine the zoom parameter b, the rotation parameter r can be eliminated in equation 1 and, analogously to the above, a three-dimensional histogram with a _x , a _y and b as coordinate axes can be formed. However, it is simpler if the values obtained from a _x , a _y and r are used in equations 1 and the zoom parameter b is determined using a simple histogram.

Just as well, of course, can only be eliminated from equation 1 and then be determined.

The determination of the global parameters for translation, rotation and zoom is shown schematically in FIG . Fig. 1, the following sequence underlies:

• A: Storage of the vectors determined in the entire figure (vector field memory)
  x _i , y _i : pixel coordinates
  v _xi , v _yi : assigned vectors.
• B: Insert the values x _i , y _i , v _xi , v _yi in the plane equation 0 = l _i (a _x , a _y , r) = -v _y x + v _x y + a _y x - a _x y + (x² + y²) approx. H. Address calculation of all the memory elements of a memory matrix that are assigned to the points of this level.
• C: Accumulation of the levels in the (a _x , a _y , r) storage matrix gives a three-dimensional histogram h (a _x , a _y , r).
• D: Peak value _detection gives the global parameters a _xglob , a _yglob for pure translatory movement and r _glob for rotary movement.
• E: Determination of the zoom parameter b _i from the equations bx _i = v _xi - a _xglob + x _i r _glob and by _i = v _yi - a _yglob + x _i r _glob ie address calculation of the memory element associated with the point b _i is.
• F: Accumulation of the points in the b _i memory to form the one-dimensional histogram h (b).
• G: Peak value _detection gives the global zoom parameter b _glob . An analogous procedure can also be used if only one zoom and translation are present in one image sequence. If a zoom is overlaid with a translational movement, the movement in the image plane can be described as follows:

The horizontal and vertical direction of the image are denoted by x and y and (v _x v _y ) is the displacement vector belonging to the pixel (x, y) and is "measured". The zoom parameter is denoted by b and the parameters sought for the translatory movement in the x and y directions are denoted by a _x and a _y, respectively.

If the center of the zoom is not in the center of the image, the additional terms that occur are interpreted as translational sham movement and combined with the already existing translational movement to form a term a _x , a _y .

If the zoom parameter b is eliminated in equation 3, the result is yourself:

(a _y - v _y ) x = (a _x - v _x ) y (4)

For each image coordinate point (x _i , y _i ), the motion vectors (v _xi , v _yi ) are considered to be known, since they are measured quantities. If the quantities (a _x , a _y ) are now treated as variables, this equation establishes a linear assignment between a _x and a _y . A line in (a _x , a _y ) space belongs to each pixel with the associated motion vectors. All pixels that move at the same translatory speed are assigned straight lines that meet at one point (a _xglob , a _yglob ). The two speeds (a _xglob , a _yglob ) then represent the translational speed _parameters in the x or y direction. A line according to equation 4 must therefore be drawn for each pixel with the associated motion vectors. Intersections of several straight lines then result in the translatory movement even when the zoom is present.

The motion vectors are usually not in the case of television pictures exactly determinable, but with a certain fluctuation  wide afflicted. Then there will be no clean intersections give more, but areas where the straight lines cut frequently. The maximum of this range is called interpreted the translational movement.

If the maximum found (or the maximums found) is now used in the (a _x , a _y ) plane in equation 3, the result is

bx = v _x - a _xglob
by = v _y - a _yglob (5)

the zoom parameter b. The value of b is calculated for each pixel with the associated “measured” motion vectors (v _xi , v _yi ). A histogram h (b) then makes it easy to determine the value of b that is most commonly assumed. This maximum of h (b) gives the desired zoom parameter b _glob .

The determination of the global parameters is shown schematically in FIG . The following procedure is to be assumed.

• A: Storage of the vectors determined in the entire figure (vector field memory)
  x _i , y _i : pixel coordinates
  v _xi , v _yi : assigned vectors.
• B: Insert the values x _i , y _i , v _xi , v _yi in the line equation: 0 = f _i (a _x , a _y ) = (a _y - v _x ) x - (a _x - v _x ) yd. H. Address calculation of all the storage elements of a storage matrix which are assigned to the points of this straight line.
• C: Accumulation of the straight lines in the (a _x , a _y ) storage matrix gives a three-dimensional histogram h (a _x , a _y ).
• D: Peak value _detection gives the global parameters a _xglob and a _yglob for pure translational movement.
• E: Determination of the zoom parameter b _i from the equations bx = v _xi - a _xglob and by = v _yi - a _yglob, ie address calculation of the memory element of a memory matrix which is assigned to point b _i .
• F: Accumulation of the b _i values to the one-dimensional histogram h (b).
• G: Peak value _detection gives the global zoom parameter b _glob .

The same can be done if in a sequence of images There is rotation and translation at the same time.

The local motion vector can be represented as a superposition of the translatory motion (a _x , a _y ) and the rotary motion with the rotation parameter r.

If the origin of the coordinate system does not agree with that Center of rotation, so this sub again be interpreted as a sham movement and can be offset against the translational movement.

The procedure is now the same as for the separation of zoom and translatory movement. The elimination of the rotation parameter r gives a linear relationship between a _x and a _y with known local motion vectors (v _x , v _y ) and location (x, y):

(v _y - a _y ) y = - (v _x - a _x ) x (7)

Again, the intersection of the straight lines of different local motion vectors and locations is the desired value of the translatory motion. To determine the point of intersection, a two-dimensional histogram is created in the speed plane (a _x , a _y ), in which the value is increased along each straight line.

With known translatory movement parameters can then with the two equations 6 of the rotation parameters r can be determined using a histogram.

The determination of the global parameters for translation and rotation is shown schematically in FIG . Fig. 3, the following flow underlies:

• A: Storage of the vectors determined in the entire FIG. 3 (vector field memory)
  x _i , y _i : pixel coordinates
  v _xi , v _yi : assigned vectors.
• B: Insert the values x _i , y _i , v _xi , v _yi in the line equation 0 = g _i (a _x , a _y ) = (a _y -v _y ) y + (a _x -v _x ) xd. H. Address calculation of the memory elements of a memory matrix which are assigned to the points of the straight line.
• C: Accumulation of the straight lines in the (a _x , a _y ) storage matrix results in a two-dimensional histogram h (a _x , a _y ).
• D: Peak value _detection gives the global parameters a _xglob and a _yglob for pure translational movement.
• E: Determination of the rotation _parameter r _i from the equations rx _i = v _yi - a _yglob and ry _i = v _xi - a _xglob , ie address calculation of the memory element of a memory _matrix which is assigned to the point r _i .
• F: Accumulation of the points r _i memory to the one-dimensional histogram h (r).
• G: Peak value _detection gives the global rotation parameter r _glob .

Claims (5)

1. A method for separating zoom, rotation and translation parameters in the determination of global motion parameters in image sequences (full image or partial image), characterized in that
that the global motion parameters obtained by postprocessing the motion vectors are determined by
that the plane a _y xa _x y + (x ² + y ² ) rv _y x + v _x y = 0 assigned to each image coordinate point (x _i , y _i ), which was obtained from the equations of motion by eliminating the zoom parameter (b), is accumulated in a memory matrix using a three-dimensional histogram,
that with subsequent peak value _detection , the global parameters for the translatory movement (a _xglob , a _yglob ) and the rotary movement (r _glob ) are determined, that each image coordinate point (x _i , y _i ) with known translatory movement (a _xglob , a _yglob ) and with known rotary movement (r _glob ) associated zoom movement (b) is accumulated in a memory matrix by means of a one-dimensional histogram and that the global zoom parameter (b _glob ) is detected with subsequent peak value detection. 2. The method according to claim 1, characterized in that for the separation of rotation and translation parameters, the global motion parameters obtained by postprocessing the motion vectors are determined in that the straight line v _y ya _y y + assigned to each pixel (x _i , y _i ) v _x xa _x x = 0, which was obtained from the equations of motion by eliminating the rotation parameter (r), is accumulated in a memory matrix using a two-dimensional histogram,
that with subsequent peak value _detection the global parameters for the translational movement (a _xglob , a _yglob ) are determined and that the rotatory movement assigned to each image coordinate point (x _i , y _i ) with known translatory movement (a _xglob , a _yglob ) (right _glob ) is determined using a one-dimensional histogram and subsequent peak value detection. 3. The method according to claim 1, characterized in that for the separation of zoom and translation parameters, the global motion parameters obtained by postprocessing the motion vectors are determined in that the straight line a _x vv _x ya _y assigned to each pixel (x _i , y _i ) x + v _y x = O, which was obtained from the equations of motion by eliminating the zoom parameter (b), is accumulated in a memory matrix using a two-dimensional histogram,
that with subsequent peak detection, the global parameters for the translatory movement (a _xglob, a _yglob) are determined, and that the each image coordinate point (x _i, y _{i) (xglob} a, a _yglob) at a known translational movement associated zoom movement (b _glob) is determined by means of a one-dimensional histogram and subsequent peak value detection. 4. The method according to claim 1 to 3, characterized records that to represent the motion vectors a pel resolution is used. 5. The method according to claim 1 to 3, characterized records that to represent the motion vectors a subpel resolution is chosen.