WO2005006767A1

WO2005006767A1 - Simplified motion estimation method

Info

Publication number: WO2005006767A1
Application number: PCT/IB2004/002239
Authority: WO
Inventors: Jean Gobert; Olivier Gay-Bellile
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2003-07-09
Filing date: 2004-07-05
Publication date: 2005-01-20

Abstract

The present invention relates to a method of motion estimation for a device adapted to process a sequence of frames, a frame being divided into blocks of data samples. Said motion estimation method comprises the steps of: computing (22) a residual error block from a difference between a current block (Cur) contained in a current frame and a candidate block (Can) contained in a reference frame, a motion vector (MV) defining a relative position, expressed in an integer number of data samples, of the candidate block in the reference frame compared with a block (Ref) having the same position in said frame as the current block in the current frame, computing (23) a distortion value of the candidate block from residual error block values, applying the preceding steps to a first set of candidate blocks and selecting (24) a second set of locally close candidate blocks, containing at least the candidate block of the first set that has the smallest distortion value and two other candidate blocks, estimating (25) a continuous distortion function from distortion values of the second set of candidate blocks and from components of the corresponding motions vectors, determining (26) a component with sub-sample accuracy of a final motion vector corresponding to a minimum value of the estimated distortion function.

Description

Simplified motion estimation method

FIELD OF THE INVENTION The present invention relates to a method of motion estimation for a device adapted to process a sequence of frames, a frame being divided into blocks of data samples, said motion estimation method comprising a step of computing a residual error block from a difference between a current block contained in a current frame and a candidate block contained in a reference frame, a motion vector defining a relative position, expressed in an integer number of data samples, of the candidate block in the reference frame compared with a block having the same position in said frame as the current block in the current frame, and a step of computing a distortion value of the candidate block from residual error block values. The present invention relates to a predictive block-based encoding method comprising such a motion estimation method. It also relates to the corresponding encoder. The present invention finally relates to a computer program product for implementing said motion estimation method. This invention is, for example, relevant for products embedding a digital video encoder such as, for example, camcorders, mobile phones or personal digital assistants, such apparatus comprising an embedded camera able to acquire and to encode video data before sending it.

BACKGROUND OF THE INVENTION A conventional approach for motion estimation is to perform block matching between a current block and a set of several candidate blocks according to a distortion measure, and then to select the candidate block giving the smallest distortion. Candidates are chosen thanks to a translation motion whose components can be either an integer or a non-integer number of samples. Usually, the motion estimation comprises two successive steps: the first step searches for the best match with integer motion, and the second step is a sub-sample refinement around the best match position. The sub-sample refinement is done by generating a set of re-sampled references, interpolated from the original reference data, and then matching the current block with these re-sampled references. The sub-sample refinement is a complex task, as it implies a lot of filtering and block matching operations. The patent US 6,141,382 entitled "Using estimated distortion values", by R. Krishnamurthy gives a method of simplifying the sub-sample refinement. It discloses techniques for reducing the number of candidates to be tested in the sub-sample refinement step. However, it may be necessary to go further in complexity reduction, for low-cost or low-power implementations. It may also be difficult to obtain intermediary reference picture, for example when images are 1 bit per pixel.

SUMMARY OF THE INVENTION It is an object of the invention to propose a motion estimation method, which is less complex than the one of the prior art. To this end, the motion estimation method in accordance with the invention is characterized in that it further comprises the steps of: applying the steps of computing a residual error block and of computing a distortion value to a first set of candidate blocks and selecting a second set of locally close candidate blocks, containing at least the candidate block of the first set that has the smallest distortion value and two other candidate blocks, estimating a continuous distortion function from distortion values of the second set of candidate blocks and from components of the corresponding motions vectors, determining a component with sub-sample accuracy of a final motion vector corresponding to a minimum value of the estimated distortion function. As a consequence, such a motion estimation method avoids any block matching and re-sampled candidate generation for the refinement process, said refinement process comprising the three above-listed steps. On the contrary, this sub-sample refinement is based on an analytical process. The motion estimation process is thus simplified. Moreover, thanks to the calculation of the real value of the minimum of the estimated distortion function, the sub-sample accuracy can be rounded to 1/2, 1/4, 1/8 or any other sub- sample accuracy required by the motion estimation method. The present invention relates to a predictive block-based encoding method comprising such a motion estimation method. The present invention also relates to a video encoder implementing said video encoding method. It finally relates to a computer program product comprising program instructions for implementing, when said program is executed by a processor, the motion estimation method in accordance with the invention. These and other aspects of the invention will be apparent from and will be elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention will now be described in more detail, by way of example, with reference to the accompanying drawings, wherein: Fig. 1 is a block diagram of a conventional video encoder, Fig. 2 is a block diagram showing the motion estimation method in accordance with the invention, - Fig. 3 shows a set of locally close components of motion vectors, and Fig. 4 shows an example of an estimated distortion function as a function of a component of a motion vector.

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a method of motion estimation for a device adapted to process a sequence of frames, a frame being divided into blocks of data samples. Said device is, for example, an encoder adapted to encode said sequence of frames. The present invention is more especially dedicated to the encoding of video frames. It can be used within MPEG-4 or H.264 video encoder, or any equivalent distortion-based video encoder. However, it will be apparent to a person skilled in the art that it is also applicable to the encoding of a sequence of audio frames or any other equivalent encoding. It is to be noted that the present invention is not limited to encoding but can be applied to other types of processing, such as for example, image stabilization wherein an average of the different data blocks of a video frame is computed in order to determine a global motion of said frame. Such an image stabilization process can be implemented in a camcorder, in a television receiver, or in a video decoder after the decoding of an image. The motion estimation method may be implemented in handheld devices, such as mobile phones or embedded cameras, which have limited power, and that have to deal with the encoding of video sequences.

Fig. 1 depicts a conventional video encoder for encoding an input data block IN. Said encoder comprises: a subtracter for delivering a main residual error block, a discrete cosine transform DCT unit (11) and a quantizing Q unit (12) for transforming and quantizing successively the main residual error block, a variable length coding VLC unit (13) for delivering a variable length coded data block from the quantized data block, - an inverse quantizing IQ unit (14) and inverse discrete cosine transform IDCT unit

(15) for delivering an auxiliary residual error block from the quantized data block, a motion compensation MC unit (16) for delivering a motion compensated data block to an adder and to the subtracter using a motion vector, the subtracter being adapted to subtract the motion compensated data block from the input data block, - an adder for summing the motion compensated data block and the auxiliary residual error block, a motion estimation ME unit (18) for finding, in a reference picture, a reference data block associated to the input data block, as well as its corresponding motion vector, and an external frame memory module MEM (17) to which the motion compensation and motion estimation units are coupled.

Fig. 2 is a block diagram showing the motion estimation method in accordance with the invention. Said method comprises the following steps. A conventional block matching is first performed at integer positions, using a distortion measure D. Said block matching comprises a step REB (22) of computing a residual error block from a difference between a current block Cur contained in a current frame and a candidate block Can contained in a reference frame. A motion vector MV defines a relative position, expressed in an integer number of data samples, of the candidate block Can in the reference frame compared with a block Ref having the same position in said frame as the current block in the current frame. The block matching comprises a step DC (23) of computing a distortion value of the candidate block from residual error block values. This computing step is based, for example, on the computing of the mean square error mse according to a principle known to a person skilled in the art. The computing step is based, as another example, on the computing of the mean absolute error mae. It will be apparent to a person skilled in the art that the distortion value can be computed using other equivalent calculations. For example, it can be based on a sum of an entropy h of the residual error block and on the mean square error mse. Then, the present invention avoids any block matching and re-sampled candidate generation for the refinement process. It is here described in general context of correlation of "blocks" of samples of data in a n-dimensional space. For example, n = 1 for sound samples, n = 2 for images, n = 3 for volumic data. The data are sampled at integer values of coordinates [x₀,x₁,...x_n_₁]. The refinement process of the motion estimation method in accordance with the invention comprises a step STO/SEL (24) of applying the preceding steps to a set of candidate blocks. The results are stored locally and a selection of a set of p locally close candidate blocks is done, said set containing at least the candidate block that has the smallest distortion value as given by the distortion computing step and two other candidate blocks. The p locally close candidate blocks are preferably adjacent. The motion vector of the candidate block j ( 0 < j < p ) of the set is denoted |m₀ , mj , ... m^, j and its distortion Dj . The present invention is based on the likely continuity of the distortion function D[m₀ , m, , ... m_n_, ] around its minimum. The refinement process of the motion estimation method comprises a step EST (25) of computing an estimate E[m₀,m₁,...m_n_₁] of said distortion function from the p values of distortion D. The estimate E is chosen as a continuous function of motion, defined for real values of motion vectors (so not restricted to integer values), which values are close to those of the distortion D for the p motion vectors of the set, and with the property that a final motion vector corresponding to a minimum value of the estimate E can be obtained. Lets call this final motion vector [s ₀ , s _j , ... s _n_, j . The refinement process of the motion estimation method comprises a step MVD (26) of determining this motion vector. It will be taken as the estimated motion vector with sub- sample accuracy. An example of such a determination is given in Fig. 3 with n = 2, for video images. Motion vector components are denoted x and y. A conventional block matching is performed giving the best match on integer motion (x,y), with its associated distortion Dc. The selected set contains in addition to the best match, its four neighbors, two in horizontal direction and two in vertical direction, as represented by the black dots in Fig. 3. The distortion values corresponding to left and right neighbor vectors (x - 1, y) and (x + 1, y) are denoted Dl and Dr. Depending on the type of the preliminary integer motion estimation, they may be already available. From Dl, Dr and Dc, a function E(x) is estimated as shown in Fig. 4 and the abscissa u of its minimum is computed. The chosen estimation is a "smooth" function passing by the three reference points, such as for example, a second degree polynomial. In this case the abscissa u is obtained by the formula:

Similarly, the distortion values corresponding to the top and bottom neighbor positions (x, y - 1) and (x, y + 1) are denoted Dt and Db and we obtain the ordinate v of the minimum of the function E(y) as follows: Dt -Db ^~ 2(Dt + Db - 2Dc)

The best estimated sub-pixel motion vector is then (x + u, y + v). It is to be noted that, in the above-described case, the selection of p candidates and the following estimation is performed for each of the 2 dimensions. However, it will be apparent to a person skilled in the art, that the abscissa u and v of the minimum can be computed by using a two-dimensional function E(x,y) and by determining analytically its minimum. This two-dimensional function can be based on a paraboloid, i.e. O O O 0

E(x,y) = c₀x y + c,x y + c₂xy +c₃xy + c₄x +c₅y +c₆x + c₇y + c₈ and the selected set of p candidate blocks can contain, in addition to the best match, its eight adjacent neighbors. It is also to be noted that the function E(x) can be different from a second degree polynomial. It is, as another example, a third-degree polynomial and the selected set of p candidate blocks contains in addition to the best match, three other candidate blocks. As an alternative, k candidate blocks are selected including the best match, where k is an integer, and a polynomial of degree m, where m is an integer lower than k-1 is estimated based on a regression algorithm according to a principle known to a person skilled in the art. Advantageously, the motion estimation method comprises a last step RND (27) of rounding each component of the final motion vector to 1/2, 1/4 or 1/8 nearest values, in order to be compatible with further processing of motion estimation, like in video compression, where the motion vector is used in normative motion compensated prediction. In the case of a predictive block-based encoding method the final motion vector and its associated residual error block are then coded and transmitted.

The computation of the abscissa u according to equation (1) needs a division, which is costly. The present invention proposes a tabulated implementation in order to reduce this complexity. Let us consider the following parameters: num = abs[Dl - Dr] neg = 1 if (Dl < Dr), and 0 otherwise den = (Dt + Db - 2Dc) . We test first if the numerator num is null, in which case abscissa u is zero. In the following, the numerator num is assumed to be non-null. Remarking that the denominator den is always positive, we use a base B logarithm so that equation (1) becomes: LogB[abs(u)] = LogB[num] - LogB[den] - LogB[2] (2) and abs(u) = _B ^o§B[num]-^Lo§B[den]-^L°s^Bf2] The values of LogB are obtained by table lookup, except for LogB[2] that is a constant and can be pre-computed. The logarithmic table returns values rounded to the nearest integer, so that the rest of computations can be made with integer arithmetic. The base B is chosen so that values are not too small and then rounding does not introduce large quantization errors. A second table may be used for exponentiation. The value of u is finally obtained by negating its absolute value if neg = 1. According to a first simplification of this tabulated implementation, and in order to reduce the number N of entries of the logarithmic table, we choose N and a base B that satisfy: B = 2 , N = 2 where b and n are integers, and compute the following parameter: num scalednum = • ^num_xp If the numerator num is already smaller than N, we choose num_xp = 0, else we choose the smallest value of num_xp for which the scaled numerator scalednum is smaller than N. As N is an integer power of 2, this operation can be simply done by successive tests of bits of the numerator num and a subsequent shift right operation by the value num_xp of bits. For better accuracy, a rounding constant may be added to the numerator num before the shift. Similar operations are performed on the denominator den, and equation (1) becomes: LogB[abs(u)] = LogB[scalednum] - LogB[scaledden] - LogB[2] + (num _ xp - den _ xp) x 2^b (3) The multiplication by 2^b can be implemented by a shift left of b bits. With this scheme, the logarithmic table width is determined by the largest value of its entries, i.e. by the value of the last table entry that is smaller than LogBfN) = 2b * n. The number of bits necessary to represent this number, which is the table word width, is the excess integer rounded value of b + log2(n). A second simplification can be implemented when we are not interested by the full precision of the abscissa u but by a rounded value to an integer number ru of sub-sample units. Let's take the example of 1/4 sample unit, although this is not limitative. The range interval of abs(u), being [0:1/2], the only possible values for ru are 0, 1 or 2, with the following decisions, implementing rounding: Ifabs(u) < l/8, ru = 0 If 1/8 < abs(u)< 3/8, ru = 1 If 3/8 < abs(u), ru = 2 As logarithm is a monotone growing function the tests condition can be rewritten in logarithmic domain. For example the condition tested in the first decision becomes: LogB(abs(u)) - LogB(l/8) < 0 The second term is constant and can be pre-computed. There is no more need of exponentiation table and ru is obtained by two subtractions and tests of sign bits. It is to be noted that the implementation of the division has been described in the case of a distortion function equal to a second-degree polynomial but that it stays applicable to any distortion function for which the component of the motion vector is the result of a division.

The motion estimation method in accordance with the invention can be implemented by means of items of hardware or software, or both. Said hardware or software items can be implemented in several manners, such as by means of wired electronic circuits or by means of an integrated circuit that is suitable programmed, respectively. The integrated circuit can be contained in an encoder. The integrated circuit comprises a set of instructions. Thus, said set of instructions contained, for example, in an encoder memory may cause the encoder to carry out the different steps of the motion estimation method. The set of instructions may be loaded into the programming memory by reading a data carrier such as, for example, a disk. A service provider can also make the set of instructions available via a communication network such as, for example, the Internet. Any reference sign in the following claims should not be construed as limiting the claim. It will be obvious that the use of the verb "to comprise" and its conjugations do not exclude the presence of any other steps or elements besides those defined in any claim. The word "a" or "an" preceding an element or step does not exclude the presence of a plurality of such elements or steps.

Claims

1 A method of motion estimation for a device adapted to process a sequence of frames, a frame being divided into blocks of data samples, said motion estimation method comprising the steps of: computing (22) a residual error block from a difference between a current block (Cur) contained in a current frame and a candidate block (Can) contained in a reference frame, a motion vector (MV) defining a relative position, expressed in an integer number of data samples, of the candidate block in the reference frame compared with a block (Ref) having the same position in said frame as the current block in the current frame, computing (23) a distortion value of the candidate block from residual error block values, applying the preceding steps to a first set of candidate blocks and selecting (24) a second set of locally close candidate blocks, containing at least the candidate block of the first set that has the smallest distortion value and two other candidate blocks, estimating (25) a continuous distortion function from distortion values (Dl,Dc,Dr) of the second set of candidate blocks and from components (x-l,x,x+l) of the corresponding motions vectors, determining (26) a component (u) with sub-sample accuracy of a final motion vector corresponding to a minimum value of the estimated distortion function.

2 A motion estimation method as claimed in claim 1, further comprising a step of rounding (27) the component of the final motion vector to values compatible with processing following the motion estimation.

3 A motion estimation method as claimed in claim 1, wherein the blocks of data samples are in a n-dimensional space, where n is an integer strictly higher than 1, and wherein the selection of the second set of candidate blocks and the following estimation is performed for each of the n dimensions.

4 A motion estimation method as claimed in claim 1, wherein the estimated distortion function is a second-degree polynomial function. 5 A motion estimation method as claimed in claim 1, wherein the component with sub-sample accuracy is the result of a division of a numerator by a denominator, the determining step comprising the sub-steps of: computing a logarithmic value of said component from a difference between a logarithmic value of an absolute value of the numerator and a logarithmic value of an absolute value of the denominator, the two last logarithmic values being derived from a lookup table, computing the component from a corresponding exponentiation of said difference and a sign of the division.

6 A motion estimation method as claimed in claim 5, wherein the lookup table comprises N entries, where N is a power of 2, the numerator and the denominator being divided by scaling factors, said scaling factors being a power of 2, in such a way that the scaled numerator and scaled denominator are smaller than N, the logarithmic value of the component being derived from a difference between a logarithmic value of an absolute value of the scaled numerator and a logarithmic value of an absolute value of the scaled denominator and from the scaling factors.

7 A motion estimation method as claimed in claim 1, wherein the component with sub-sample accuracy is the result of a division of a numerator by a denominator, the determining step comprising the sub-steps of: computing a logarithmic value of said component from a difference between a logarithmic value of an absolute value of the numerator and a logarithmic value of an absolute value of the denominator, the two last logarithmic values being derived from a lookup table, determining a rounded value of the component from a comparison of said difference with a predetermined value corresponding to a predetermined sub-sample accuracy.

8 A predictive block-based encoding method for encoding a sequence of frames, said encoding method comprising a motion estimation method as claimed in claim 1 for computing a motion vector to a desired accuracy, said encoding method further comprising a step of coding said motion vector and its associated residual error block. 9 An encoder for encoding a sequence of frames, said encoder comprising means for implementing the steps of the video encoding method as claimed in claim 8.

10 A computer program product comprising program instructions for implementing, when said program is executed by a processor, a motion estimation method as claimed in claim 1.