JPH04227184A - Motion vector evaluator - Google Patents

Abstract

PURPOSE: To quickly, accurately, and stably obtain estimated values by adding a candidate motion vector generator to the already known system. CONSTITUTION: A delaying element 16 forms a delayed output motion vector constituting an estimated vector and a candidate motion vector generator 18 connected to the element 16 computes at least first and second candidate motion vectors by adding very small increments to the estimated vector. A matching deciding means 14 feeds back the candidate motion vectors to the generator 18 through the element 16. The first increment is added to the estimated motion vector in a first selected direction and, at the same time, the second increment having a different value than the first increment has is added to the estimated motion vector in a second selected direction. When this means is used, quick convergence and accurate and stable results can be synthesized with a correct value by means of one estimator without increasing the number of actuating times per pixel.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector evaluation device for video images, and more particularly to a motion vector evaluation device for standard conversion methods.

0002

BACKGROUND OF THE INVENTION Motion vectors are used, for example, to estimate the position of a portion (generally a very small portion) of a video image occupied by a field to be interpolated between two input fields. The processing in this case can be done at the transmitting end and/or at the receiving end, where a portion of the image (referred to here as a block) is a single pixel as small as possible, or
Alternatively, the pixels may be a plurality of pixels adjacent to each other. Generally, for a given block, an estimated vector, e.g., a motion vector from a previous block, is provided, and a search centered on the estimated area of the second field is performed to determine the second field 1
Enables finding the best match of pixel values across blocks. Such a system with iterations based on the components I
A document entitled “New Algorithms for Motion Estimation” presented by Gerard de Haan and Hank Huygen in Turin, 1989, Proceedings of the Third International Workship on HDTV, sponsored by EEE, SMPTE, etc.
It is described in.

0003

[Problem to be solved by the invention] Also, this document contains two
Dimensional convergence configurations are also included. The contents of this document are detailed below, and the above system is proposed to have a limited search area and obtain stable estimates, but the convergence speed can be improved.

It is an object of the invention to provide a motion vector estimation device of the above-mentioned type, which maintains the stability of the known system, but allows the speed of convergence to be increased.

[0005]

SUMMARY OF THE INVENTION The present invention provides that the pixel values of a block of a given field are shifted relative to the associated block of the given field by the x and y values that constitute the components of the output motion vector. A block-matched motion vector estimator for matching pixel values of blocks of a field, comprising: a delay element for forming a delayed output motion vector constituting an estimated vector;
A motion vector evaluation comprising: a candidate motion vector generator for generating at least first and second candidate motion vectors by adding small increments to the estimated vector; and a selection circuit for selecting an output motion vector from the candidate motion vectors. In the apparatus, the candidate motion vector generator adds a first increment to the estimated motion vector in a first selected direction and a second increment having a different value than the first increment to the estimated motion vector. A feature is that the vector is added in the second selection direction.

[0006]

[Operation] According to this means, a large update length, that is,
There is the advantage that fast convergence to the correct value and short update length, ie accurate and stable results, can be combined in one estimator without increasing the number of operations per pixel.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows a basic block diagram of a motion vector estimator suitable for implementing the present invention and the motion vector estimation method of the above-mentioned document.

In FIG. 1, the interpolation input signal is filtered by a low pass filter 10. This low pass filter 1
A zero output signal is provided to field delay element 12 and stage 14 for determining the best match. This best matching determination stage 1
The output of 4 is a vector or vector component, so as to obtain the best match of the pixel values of the blocks in the first and second fields. For example, the best match may be the match that yields the smallest mean absolute value difference when comparing the lowest mean squared errors taken on pixel values across the entire block or similar criteria. Also, as shown in FIG. 1, the vector or vector component that produces the best match is fed back through the delay element 16 so that an estimation program for the next block can be constructed. In the known circuit, a search to determine the best match is performed over four blocks equidistant from the estimated position in the +x and -x directions and in the +y and -y directions. In other words, the candidate motion vector Di can be defined by the following table.

Table 1 Di1=Di-1+L*xu Di2=Di-1+L*xu Di3=Di-1+L*yu Di4=Di-1+L*yu Di-1=Estimated vector x obtained by delay element 16
u = x-axis unit vector yu = y-axis unit vector L = update length (integer value)

The error value is, for example, the difference (M
AD) Assign to these candidate vectors using criteria. Error (Di) = MAD = ΣxΣy | sk(x)-sk-
1(x-di) | Also, mean square error (MSE) standard error (Di) = MSE-ΣxΣy(sk(x)-sk-
1(x-di))2 where sk(x): signal value of the current field at position x sk-1(x): signal value of the previous field at position x x and y: of the horizontal and vertical blocks, respectively. size

For each of the candidate motion vectors Di obtained by the candidate vector calculation stage 18, the difference in mean absolute value or mean squared error is therefore determined by the position of the incoming data and the pixels of the incoming block shifted according to the candidate motion vector. Calculate between the pixel values of the block at the positions corresponding to the values. The calculation of the error and the determination of the calculated error are performed at stage 14.
The minimum value that occurs in

The motion vector producing the minimum error is low-pass filtered in signal processing stage 20. It is clear that known systems allow for symmetric updates in the +x and -x directions as well as in the +y and -y directions.

[0013] The above document suggests the use of separate convergence directions. The convergence direction referred to here is defined as a direction in which a vector related to the output of one block is used as an estimated vector for another block. Each block of pixels is subdivided into two sub-sample blocks as shown in FIG. The first subsampled block shows only samples marked 1, and the second subsampled block shows samples marked 2. Thus, the total number of actuations per pixel remains unchanged, but the convergence directions for the two subsampled blocks are different. Therefore, the composite vector that produces the least block error is assigned to the complete block. Using two subsample blocks in this manner means that the circuit of FIG. 1 overlaps one array for one sample and one array for two samples. This possible modification will be ignored in the following description.

According to the principles of the invention, in the candidate vector calculation stage 18, the update length L of at least one candidate vector Di is made different from the update length of at least one other candidate vector.

In one example of the motion vector evaluation device of the present invention,
Preferably, there are two asymmetric update groups and these are different for odd and even blocks along the transform line. Preferably, the second update group is a mirror image of the first update group. This is expressed as shown in Table 2 below.

Table 2 Di1o=Di-1+L1*xu Di2o=Di-1-L2*xu Di3o=Di-1+L1*yu Di4o=Di-1-L2*yu Di1e=Di-1+L1*xu Di2e=Di-1- L2*xu Di3e=Di-1+L1*yu Di4e=Di-1-L2*yu Dio=Candidate vector for odd blocks Die=Candidate vector for even blocks Di-1=Estimated vector obtained by delay device xu=X-axis Unit vector yu=y-axis unit vector Ln=update length (integer value) For example, L1 can be small (for example, 1) and L2 can be large (for example, 3).

The search area obtained for a given value is shown in FIG. 3, and the search area for odd-numbered blocks is denoted by A.
The search area for even blocks is denoted by B. According to the table above, the presently preferred embodiment can be obtained. However, the unit vectors xu and yu need not be perpendicular to each other, and the candidate vector with L2 factor can be defined as L1
There is also no need for a 180° angle for candidate vectors with factors.

Furthermore, the update lengths in the x and y directions are not equal. The search strategy shown in FIG. 3 still applies, assuming that different unit vectors are used in the x and y directions, respectively. For example, a unit vector has a distance of 1 pixel in the x direction and a distance of 2 pixels in the y direction.

FIG. 4 schematically shows the motion vector evaluation device of the present invention.
Shown below. This example also uses the same low-pass filter 10 and field delay device 12 as shown in FIG. However, instead of the four blocks Di1, Di2, Di3 and Di4, the block 18' corresponding to the block 18 of FIG. delay element.

As shown in detail with respect to FIG. 5, the pixel values taken from the search pattern shown in FIG. 3A for odd-numbered blocks appear on the line labeled "0" at the output of block 18' in FIG. Multiplexer 4 these lines
This multiplexer selects the "o" input for odd blocks and the "e" input for even blocks under the control of the signal provided at terminal 42. Select. This causes block 18'
These signals at the "o" outputs of the circuit appear on lines 44a-44d and are then compared with the current value of the signal applied to the best matched block 14 at terminal 15. As in the prior art, the absolute difference between the pixel value supplied via lines 44a-44d and the pixel value supplied to terminal 15 is determined, and the direction that produces the pixel value with the smallest difference is selected as the output vector. Then, it is selected as the estimated vector for the next block. The odd and even blocks are shown alternating in the convergence direction as indicated by o and e in block 18'.

One of the biggest problems in motion estimation is the need to obtain motion vectors that represent the actual motion of an object having a movable periodic structure, such as a movable lattice door. It is clear that if the bar of the grid door is located in the image, for example, 14 pixels from one side to the other, the motion vector of +6 pixels will be as close as possible to the motion vector of -8 pixels. However, it is important that these motion vectors represent the true motion if they are to be used to interpolate an intermediate image between two current images.

In the next example of the invention, the performance of periodic structures can be improved when the structures have sharp boundaries. As the algorithm starts to converge at the boundary of a moving (periodic) object, it ensures an improved spatially harmonious motion estimation algorithm and therefore does not select other vectors after convergence at this boundary. This improved spatial harmonization can be achieved by adding a "penalty" to the selection of candidate vectors that depends on the update length. Such a penalty consists in adding an amount by block 14 to the alignment error of the candidate vector. For example, the estimated vector itself is not penalized for alignment errors, a small horizontal update of length 1 is penalized by 10 units, and a large horizontal update of length 5 is penalized by 5 units.
Vertical updates of length 2 are penalized by 0 units, and vertical updates of length 2 are
Take a penalty of 0 units. As described in European Patent Application EP-A 0,415,491, if temporary candidate vectors from previous fields are also used, such temporary candidate vectors are subject to a penalty of 10 units. . These units are 1 in a block of 16 pixels with 8-bit luminance quantization.
Associated with alignment errors with a maximum value of 6 x 255 = 4080 units.

[0023] In the example described above, the update length and/or penalty depends on the spatial frequency. Short updates are preferred in highly detailed areas. In particular, the horizontal slope is used to control the horizontal update length and/or the penalty imposed thereon, and the vertical slope is used to control the vertical update length and/or the penalty imposed thereon, in detail areas. Candidate vectors with large update lengths for selection have significantly lower alignment errors than candidate vectors with short update lengths. To determine the amount of detail, the absolute value of the difference between two adjacent pixels is determined once every 8 pixels. The absolute differences thus obtained are then fed into a cyclic filter to reduce the noise so that it only affects the performance of the motion estimator when the area of detail is above a certain minimum degree.

Boxes Di1o-Di4 of block 18
The operation within each of e is shown in more detail in FIG. In FIG. 5, under the control of the write address generated by the address counter 52, video information is sent through the FIFO 51 to R.
Store in AM50. The FIFO 51 is inserted to minimize the size of the RAM 50, and is provided with a delay element having a one-field delay equal to the value obtained by subtracting an offset value K, which will be described later, from the integer line. Then, for each block of video information stored in RAM 50, the pixel values at the four relevant search positions for the given block are stored in RAM 50.
It is necessary to generate a corrected read address to appear at output 54 of 50. For this purpose, an adder 56 takes into account the maximum x value of the motion vector, the maximum candidate motion vector, and the number of pixels per line (pixels/line).
and add an offset value K equal to the sum of the selected motion vector maximum y value to cover the minimum desired offset between the write and read addresses. If the FIFO 51 is omitted, the offset value K needs to correspond to an integer field delay of the line. This sum is added to the estimated vector x in adder 58.
Add the components and y components algebraically. Therefore, the read address from the estimated vector P appears at the output of this adder 58. This address value needs to be increased as shown in part A of FIG. 3 for odd-numbered blocks and part B of FIG. 3 for even-numbered blocks. The output of the summing stage 58 is therefore fed to one input of an adder 60, the other input of which is fed the output of a multiplexer 62. This multiplexer 62
is switched for odd-numbered blocks (o) and even-numbered blocks (e). For odd-numbered blocks, multiplexer 62 is replaced by multiplexer 64.
is supplied to the second input terminal of the adder 60. These outputs are L1 times the update length of the negative unit vector in the x direction and L1 the update length of the positive unit vector in the x direction.
2 times the update length L1 of the negative unit vector in the y direction
Update length L of positive unit vector in double and y direction
That's twice as much. The unit vector in the y direction is equal to the number of pixels per line (pixels/line).

The second inputs supplied to the adder 60 via multiplexers 62 and 66 for even blocks are the update length L2 of the unit x vector in the negative direction and the update length L1 of the unit vector in the x direction. , the update length L2 of the unit vector in the negative direction and L1 times the update length of the unit vector in the y direction.

Therefore, the output of the adder 60 is
This is a corrected read address for 0. line 5
The resulting value at 4 is compared with the current value on line 68 in match determining stage 14' and the resulting vector is transferred to other processing stages 20 and delay element 16 by the minimum difference.
Output to.

The asymmetric search method described above can be combined with subsampling as shown in FIG. In particular, it is possible to improve the asymmetric search measures described above for each subsampled block for a given direction of transformation.

The present invention has been described for clarity in terms of specific examples, with certain stages shown individually that can be easily combined into specific examples. One example is adder stages 58 and 56. Also, various multiplexers can be realized in various different ways. The key point of the invention is to make the search plan asymmetric, which is the selection of blocks, preferably odd and even blocks along the convergence line have different search patterns, so that The advantages of a stable but slow convergence system,
can be combined with the benefits of rapid convergence.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of a vector estimator.

FIG. 2 is an explanatory diagram illustrating a known method for estimating along individual convergence directions.

FIG. 3 is an explanatory diagram showing an asymmetric search area by the motion vector evaluation device of the present invention.

FIG. 4 is a block circuit diagram illustrating the configuration of an apparatus for implementing an asymmetric search strategy according to the present invention.

FIG. 5 is a block circuit diagram showing the configuration of the device in FIG. 4 in more detail.

[Explanation of symbols]

10 Low pass filter 12 Field delay element 14 Best matching decision stage 14' Matching decision stage 15 Terminal 16 Delay element 18 Candidate vector calculation stage 18' Candidate vector calculation stage 20 Signal processing stage 40 Multiplexer 42 Terminal 44a-44d line 50 RAM 51 FIFO 52 Address counter 54 Output line 56 Addition stage 58 Addition stage 60 Adder 62 Multiplexer 64 Multiplexer 66 Multiplexer 68 line

Claims (9)

[Claims] 1. The pixel values of a block of a given field are shifted with respect to the associated block of this given field by the x and y values that constitute the components of the output motion vector. A matching block matching motion vector estimation apparatus comprising: a delay element for forming a delayed output motion vector constituting an estimated vector; Second
A motion vector evaluation device comprising a candidate motion vector generator that generates candidate motion vectors and a selection circuit that selects an output motion vector from these candidate motion vectors, wherein the candidate motion vector generator converts the first increment to the estimated motion vector. The estimated motion vector is added to the motion vector in a first selection direction, and a second increment having a value different from the first increment is added to the estimated motion vector in the second selection direction. motion vector evaluation device. 2. The candidate motion vector generator further generates a third increment in a third selection direction and a fourth increment having a different value than the third increment in a fourth selection direction. in addition to the third and fourth
2. The motion vector evaluation device according to claim 1, wherein candidate motion vectors are generated. 3. The motion vector evaluation device according to claim 2, wherein the first and third increments are equal. 4. The motion vector evaluation device according to claim 2, wherein the second and fourth increments are equal. 5. The candidate motion vector generator causes the directions of the first and second increments to be reversed for a first selected block of the blocks. 2. The motion vector evaluation device according to 2. 6. The candidate motion vector generator according to claim 2, wherein the direction of the third and fourth increments is reversed for a second selected block of the blocks. 5. The motion vector evaluation device according to 5. 7. The first and second of the blocks
7. The motion vector evaluation device according to claim 5, wherein the selected blocks are alternate blocks in the focusing direction. 8. The selection circuit comprises means for determining a matching error for each of the candidate motion vectors, and means for selecting the candidate motion vector having the lowest matching error, wherein the determining means selects the candidate motion vector. 2. The motion vector evaluation device according to claim 1, wherein a penalty is added to the alignment error depending on the length of the increment. 9. The motion vector evaluation device according to claim 8, wherein the increment length and/or the penalty depend on the spatial content of the predetermined field.