JP3192698B2 - Motion vector evaluation device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector evaluation apparatus for a video image, and more particularly to a motion vector evaluation apparatus for a standard conversion system.

[0002]

2. Description of the Related Art Motion vectors are used, for example, to estimate the position of a portion (generally a very small portion) of a video image occupied by a field to be interpolated between two input fields. The processing in this case can be performed at the transmitting end and / or at the receiving end, where one part of the image (referred to here as a block) is a single pixel as small as possible,
Alternatively, a plurality of pixels can be adjacent to each other.
In general, for a given block, an estimated vector, for example, a motion vector from the previous block is provided, and a search centered on the estimated area of the second field is performed to obtain a second vector for the pixel value of the block of the first field. Field one
Try to find the best match of pixel values for the whole block. Such a system with component-based repetition is I
Proceedings of the Third International Workship on HDTV, sponsored by EEE, SMPTE and others, described in "New Algorithm for Motion Estimation" published by Herald de Hahn and Hank Huygen in Turin, 1989.

[0003]

[0005] Further, in this document,
A dimensional convergence configuration is also included. The content of this document is detailed below, and the system has a limited search area and has been proposed to obtain stable estimates, but convergence speed can be improved.

It is an object of the present invention to provide a motion vector estimator of the kind described above, which maintains the stability of the known system but allows the speed of convergence to be increased.

[0005]

SUMMARY OF THE INVENTION The present invention provides a method for converting a pixel value of a block of a predetermined field into an associated block of the predetermined field by shifting the pixel value by the x and y values constituting the components of the output motion vector. In a block matching motion vector estimating apparatus that matches pixel values of blocks in a field of a field, a delay element that forms a delayed output motion vector that constitutes an estimated vector, and a delay element that is connected to the delay element and has different directions with respect to the estimated vector. Have
A candidate motion vector generator for adding at least the first and second increment vectors for generating at least the first and second candidate motion vectors; and a selection circuit for selecting the output motion vector from the candidate motion vectors. , The first increment vector is the first
And the second increment vector has a second vector length different from the first vector length.

[0006]

According to such means, a large update length, that is,
The advantage is that fast convergence to the correct value and short update lengths, ie accurate and stable results, can be combined in one estimator without increasing the number of operations per pixel.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows a basic block diagram of a motion vector estimator suitable for implementing the motion vector estimation method of the present invention and the above-mentioned literature.

In FIG. 1, the internal insertion force signal is filtered by a low-pass filter 10. This low-pass filter 1
The zero output signal is provided to a field delay element 12 and a stage 14 that determines the best match. This best match determination stage 1
The output of 4 is a vector or vector component, which ensures the best match of the pixel values of the blocks in the first and second fields. For example, the best match may be one that produces a difference in the minimum mean absolute value when comparing pixel values over the entire block or the lowest mean squared error taken on a similar basis. Also, as shown in FIG. 1, a vector or a vector component that produces the best matching is fed back via the delay element 16 so that an estimation program for the next block can be constructed. In known circuits, a search to determine the best match is performed on four blocks located equidistant in the + x and -x directions and in the + y and -y directions from the estimated position. In other words, the candidate motion vector Di can be defined by the following table.

Table 1 Di1 = Di−1 + L * xu Di2 = Di−1 + L * xu Di3 = Di−1 + L * yu Di4 = Di−1 + L * yu Di−1 = estimated vector obtained by delay element 16 cu = x-axis Unit vector of yu = unit vector of y axis L = update length (integer value)

The error value is, for example, the difference between the average absolute values (M
AD) Assign to these candidate vectors using the criterion. Error (Di) = MAD = {xΣy | sk (x) -sk-1 (x-di) | Further, mean square error (MSE) standard error (Di) = MSE-ΣxΣy (sk (x) -sk-1 ( x-di)) 2 where sk (x): signal value of current field at position x sk-1 (x): signal value of previous field at position x and y: size of horizontal and vertical blocks, respectively

For each of the candidate motion vectors Di obtained by the candidate vector calculation stage 18, for example, the mean absolute difference or the mean squared error is determined by the position of the incoming data and the pixels of the incoming block, which have been shifted according to the candidate motion vector. The calculation is performed between the pixel values of the block at the position corresponding to the value. The calculation of the error and the determination of the calculated error are performed in step 14.
Is the minimum value that occurs.

The motion vector that generates the minimum error is low-pass filtered in the signal processing stage 20. It is clear that known systems consider symmetric updates in the + x and -x directions and in the + y and -y directions.

The above document suggests using individual convergence directions. The convergence direction is defined as a direction in which a vector related to the output of one block is used as an estimated vector for another block. Each block of pixels is subdivided into two sub-sample blocks as shown in FIG. The first sub-sampled block shows only the sample represented by 1 and the second sub-sampled block shows the sample represented by 2. Because of this, the total number of operations per pixel remains unchanged, but the convergence direction for the two subsampled blocks is different. Thus, the composite vector that produces the least block error is assigned to the complete block. Using two sub-sample blocks in this way means that the circuit of FIG. 1 overlaps one array for one sample and one array for two samples. I will ignore this possible change in the following description.

According to the principles of the present invention, the update length L of at least one candidate vector Di is made different from the update length of at least one other candidate vector in the candidate vector calculation stage 18.

In one example of the motion vector evaluation device of the present invention,
Preferably, two asymmetric update groups are provided and differ for odd and even blocks along the transformation line. Preferably, the second update group is a mirror image of the first update group. This is represented as shown in Table 2 below.

Table 2 Di1o = Di−1 + L1 * xu Di2o = Di−1−L2 * xu Di3o = Di−1 + L1 * yu Di4o = Di−1−L2 * yu Di1e = Di−1 + L1 * xu Di2e = Di−1− L2 * xu Di3e = Di−1 + L1 * yu Di4e = Di−1−L2 * yu Dio = candidate vector for odd block Die = candidate vector for even block Di−1 = estimated vector obtained by delay device xu = x axis Unit vector yu = y-axis unit vector Ln = update length (integer value) For example, L1 can be small (for example, 1) and L2 can be large (for example, 3).

The search area obtained for a given value is shown in FIG. 3, the search area for odd-numbered blocks is indicated by A,
The search area for the even-numbered block is denoted by B. According to the above table, the presently preferred embodiment can be obtained. However, unit vectors xu and yu need not be at right angles to each other, and a candidate vector having an L2 factor is L1
It is not necessary to set 180 ° for the candidate vector having the factor.

Further, the update lengths in the x and y directions are not made equal. The search strategy shown in FIG. 3 still applies, and it is assumed that various unit vectors are used in the x and y directions, respectively. For example, a unit vector has a one pixel distance in the x direction and a two pixel distance in the y direction.

FIG. 4 schematically shows the motion vector evaluation apparatus of the present invention.
Shown in In this example, the same low-pass filter 10 and field delay device 12 as those shown in FIG. 1 are used. However, the block 18 'corresponding to the block 18 in FIG.
Instead of one block Di1, Di2, Di3 and Di4, it comprises eight other blocks, namely odd and even blocks, in particular different delay elements for each of the blocks of FIG.

As shown in detail in FIG. 5, pixel values extracted from the search pattern shown in FIG. 3A for the odd-numbered blocks appear on the output side of block 18 'in FIG. These lines are connected to multiplexer 4
0, and the multiplexer selects the "o" input for odd blocks and the "e" input for even blocks under control of the signal supplied to terminal 42. Select This is why block 18 '
These signals at the "o" output of are appearing on lines 44a-44d and are then compared at terminal 15 to the current value of the signal supplied to the best matched block 14. As in the case of the prior art, the absolute difference between the pixel value supplied via lines 44a-44d and the pixel value supplied to terminal 15 is determined, and the direction in which the pixel value is generated with the minimum difference is selected as the output vector. Then, the next block is selected as an estimation vector. Odd and even blocks are indicated by alternating blocks in the convergence direction, as indicated by o and e in block 18 '.

One of the biggest problems in motion estimation is that it is necessary to obtain a motion vector that indicates the actual motion of an object having a movable periodic structure such as a movable lattice door. Obviously, if the bar of the lattice door is located 14 pixels from one side to the other in the image, the motion vector of +6 pixels is as close as possible to the motion vector of -8 pixels. However, it is important that these motion vectors have true motion vectors if they are used to interpolate an intermediate image between two current images.

In the next example of the present invention, the performance of a periodic structure can be improved if the structure has well-defined boundaries. As the algorithm begins convergence at the boundary of a moving (periodic) object, it guarantees an improved spatially harmonized motion estimation algorithm, and thus does not select other vectors after convergence at this boundary.
This improved spatial reconciliation can be achieved by adding a "penalty" to the selection of candidate vectors depending on the update length. Such a penalty is that block 14 adds a certain amount to the candidate vector alignment error. For example, the estimated vector itself is not penalized for alignment errors, a small horizontal update of length 1 is penalized by 10 units, and a large horizontal update of length 5 is 5 penalties.
0 unit penalty, length 2 vertical update is 2
Take a 0 unit penalty. If temporary candidate vectors from the previous field are also used, as described in European Patent Application EP-A 0,415,491, such temporary candidate vectors are subject to a penalty of 10 units. . These units are 1 in a block of 16 pixels with 8-bit intensity quantization.
Associated with alignment errors having a maximum of 6 × 255 = 4080 units.

In the example described above, the update length and / or penalty depends on the spatial frequency. Shorter updates are preferred in highly detailed areas. In particular, by controlling the horizontal update length and / or the penalty imposed thereon using the horizontal gradient and controlling the vertical update length and / or the penalty imposed thereon using the vertical gradient, Candidate vectors with a large update length to select have matching errors much lower than candidate vectors with a short update length. In order to determine the amount of detail, the absolute value of the difference between two adjacent pixels is determined once every eight pixels. The absolute difference thus obtained is then fed to a recursive filter to reduce noise and affect the performance of the motion estimator only when the detail area is above a certain minimum.

Boxes Di1o-Di4 in block 18
The operation within each of e is shown in more detail in FIG. In FIG. 5, under the control of the write address generated by the address counter 52, the video information is transferred to the R through the FIFO 51.
Store in AM50. The FIFO 51 is inserted to minimize the size of the RAM 50, and is provided with a delay element having a one-field delay equal to a value obtained by subtracting an offset value K described later from an integer line. Then, for each block of video information stored in RAM 50, the pixel values at four associated search locations for a given block are stored in RAM 50.
It is necessary to generate a corrected read address to appear at the output 54 of 50. For this purpose, the number of pixels per line (pixels / line) is taken into account by the adder 56, taking into account the maximum x value of the motion vector and the maximum candidate motion vector.
, And adds an offset value K equal to the sum of the selected motion vector maximum y value to cover the minimum desired offset between the write and read addresses. If the FIFO 51 is omitted, the offset value K needs to correspond to an integer field delay of the line. This sum is added to the estimated vector x by the adder 58.
Algebraically adds the component and the y component. Therefore, a read address from the estimated vector P appears on the output side of the adder 58. This address value needs to be increased as shown in part A of FIG. 3 for odd-numbered blocks and as shown in part B of FIG. 3 for even-numbered blocks. Thus, the output of summing stage 58 is provided to one input terminal of adder 60 and the other input terminal of the adder is provided to the output of multiplexer 62. This multiplexer 62
Are switched for the odd-numbered block (o) and the even-numbered block (e). For odd-numbered blocks, multiplexer 62 is a multiplexer 64
Is supplied to a second input terminal of the adder 60. These outputs are the update length L1 times the negative unit vector in the x direction and the update length L of the positive unit vector in the x direction.
Double, update length L1 of negative unit vector in y direction
Update length L of positive unit vector in double and y directions
It is twice. The unit vector in the y-direction is equal to the number of pixels per line (pixels / line).

The second input supplied to the adder 60 via the multiplexers 62 and 66 for the even-numbered blocks is the update length L2 times the unit x vector in the negative direction and the update length L1 times the unit vector in the x direction. , The update length L2 of the unit y vector in the negative direction and the update length L1 of the unit vector in the y direction.

For this reason, the output of the adder 60 is stored in the RAM 5
It becomes the corrected read address for 0. Line 5
4 is compared with the current value of line 68 in match determination stage 14 ', and its minimum difference causes the resulting vector to be passed to other processing stages 20 and delay elements 16'.
Output to

The above-described asymmetric search method can be combined with the sub-sampling shown in FIG. In particular, it is possible to improve the asymmetric search measures described above for each sub-sampled block for a transform in a given direction.

The present invention has been described with reference to a particular example in which certain steps are individually shown which can be easily combined with the particular example for clarity. One example is the addition stages 58 and 56. Also, various multiplexers can be implemented by various different means. An important aspect of the present invention is that the search plan is asymmetric, which is to select the blocks, preferably the odd and even blocks along the convergence line have different search patterns, The benefits of a stable but slow convergence system,
The benefits of rapid convergence can be combined.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a vector estimator.

FIG. 2 is an explanatory diagram showing a known method for performing estimation along individual convergence directions.

FIG. 3 is an explanatory diagram showing an asymmetric search area by the motion vector evaluation device of the present invention.

FIG. 4 is a block circuit diagram showing a configuration of an apparatus for executing an asymmetric search strategy according to the present invention.

5 is a block circuit diagram showing the configuration of the apparatus of FIG. 4 in further detail.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Low-pass filter 12 Field delay element 14 Best matching decision stage 14 'Matching decision stage 15 Terminal 16 Delay element 18 Candidate vector calculation stage 18' Candidate vector calculation stage 20 Signal processing stage 40 Multiplexer 42 Terminal 44a to 44d Line 50 RAM 51 FIFO 52 Address counter 54 Output line 56 Addition stage 58 Addition stage 60 Adder 62 Multiplexer 64 Multiplexer 66 Multiplexer 68 Line

──────────────────────────────────────────────────の Continuation of the front page (73) Patentee 590000248 Groenewoodseweg 1, 5621 BA Eindhoven, The Netherlands (72) Inventor Hendrik Veichen, Netherlands 5621 Baer Eindh-Fruhn-Frunewswech 1 (56) REFERENCE INTERNATIONAL L WORKSHOP ON HDTV (1989) [NEW ALGORITHM FOR MOTION ESTMAT ION] (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 7/01 H04N ^7/ 24-7/68

Claims (9)

(57) [Claims] 1. A pixel value of a block in a predetermined field is
A block-matched motion vector estimator for matching a pixel value of a block of a previous field shifted by x and y values constituting components of an output motion vector with respect to an associated block of the predetermined field, comprising: A delay element forming a delayed output motion vector, and at least a first element connected to the delay element and having different directions with respect to the estimated vector, thereby generating at least first and second candidate motion vectors. And a selection circuit for selecting the output motion vector from the candidate motion vectors, wherein the first increment vector has a first vector length, The second increment vector is different from the first vector length Block matching motion vector evaluation unit and having a second vector length. 2. The method according to claim 1, wherein the candidate motion vector generator further comprises a third
Are added in a third selection direction, and a fourth increment having a value different from that of the third increment is added in a fourth selection direction to generate third and fourth candidate motion vectors. The motion vector evaluation device according to claim 1, wherein the motion vector evaluation device is configured to perform the motion vector evaluation. 3. The motion vector evaluation device according to claim 2, wherein the first and third increments are equal. 4. The motion vector evaluation device according to claim 2, wherein the second and fourth increments are equal. 5. The method according to claim 5, wherein the first selected block of the block is selected by the candidate motion vector generator.
3. The motion vector evaluation device according to claim 1, wherein the direction of the second increment is reversed. 6. The method according to claim 3, wherein the third selected block of the block is selected by the candidate motion vector generator.
The motion vector evaluation device according to claim 2 or 5, wherein the direction of the fourth increment is reversed. 7. The motion vector evaluation device according to claim 5, wherein the first and second selected blocks among the blocks are alternate blocks in the convergence direction. 8. The selection circuit comprises: means for determining a matching error for each of the candidate motion vectors; and means for selecting a candidate motion vector having the lowest matching error, wherein the determining means determines the candidate motion vector. 2. The motion vector evaluation device according to claim 1, wherein a penalty is added to the matching error depending on the length of the increment of the motion vector. 9. The motion vector evaluation device according to claim 8, wherein the increment length and / or penalty depends on the spatial content of the predetermined field.