GB2236449A

GB2236449A - Motion estimation for television signals

Info

Publication number: GB2236449A
Application number: GB8921296A
Authority: GB
Inventors: Michael James Knee
Original assignee: British Broadcasting Corp
Current assignee: British Broadcasting Corp
Priority date: 1989-09-20
Filing date: 1989-09-20
Publication date: 1991-04-03
Anticipated expiration: 2009-09-20
Also published as: GB2236449B; GB8921296D0

Abstract

In apparatus for processing television signals for transmission, motion estimation is carried out by calculating 10 a correlation function C(x, y) for motion vectors (x, y) using information from signals representing a previous picture and the current picture. The best integer-pixel estimate (X, Y) for the motion vector is determined 12 from the calculated values of C(x, y). From the calculated values of C(x, y) the region relative to (X, Y) in which the best sub-integer-pixel estimate (X', Y') of the motion vector is to be found is determined. Subsequently values of the correlation functions C(x, y) for those sub-integer pixel motion vectors which are in the said region are calculated 18 and compared 18 so as to find (X', Y'). Since only the sub-integer estimates of the motion vector within the region identified from the integer-pixel estimates are calculated, the only previous-picture pixels which need to be accessed for the sub-integer pixel search are those which would be needed if a motion compensated version of the previous picture were required. Furthermore, only four streams of interpolated pixels need be generated and corresponding mean absolute error values calculated and compared. <IMAGE>

Description

M :) TICN ESTIMATION The present invention relates to methods of motion estimation for use with television signals, in particular, to methods capable of providing half-pixel accuracy and using block matching.

In a conventional approach to block matching, a block of sanples in the current frame is compared with corresponding previous frame information which has been shifted according to a possible motion vector. The motion vector giving the smallest difference between the current block and the shifted block in the previous frame is selected as the motion estimate.

The invention is defined in the claims appended hereto, to which reference should now be made.

TWO methods in accordance with the invention will now be described in detail, by way of example, with reference to the drawings, in which: Fig. 1 is a block diagram of a first motion estimator in accordance with the invention; and Fig. 2 is a block diagram of a second motion estimator in accordance with this invention.

One known method which provides half-pixel accuracy involves a full search to integer pixel accuracy followed by a local search to half-pixel accuracy using a bilinearly interpolated version of the picture. A full search is described in"Digital PicturesRepresentation and Compression" (Netravali, A. N and Haskell, B. G., New York, Plenum Press, 1988).

In one version of this method the best integer pixel estimate (X, Y) is obtained and a search is then performed over the nine half-pixel motion vectors surrounding and including (X, Y), performing bilinear interpolation on the previous picture in order to obtain the necessary intermediate pixels.

Bilinear interpolation can be defined as follows. If the values of a function f of two variables x and y are known only for integer values of x and y, then an estimate f' (X-tp, Y-tq) of

f (X+p, Y+q), where X and Y are integers and p and q are positive fractional values can be obtained by bilinear interpolation

according to the following: f' (X+p, Y+q) = (l-p) (l-q) f (X, Y) + p (l-q) f (X+1, Y) + (1-p) q f (X, Y+l) + p q f (X+1, Y+1) This method seems from past results to give the best performance of methods that avoid a full half-pixel search.

However, its hardware oonplexity is considerable because, following the integer-pixel search, nine sets of pixels from the previous frame have to be accessed, nine streams of interpolated pixels generated and the corresponding mean absolute error values calculated explicitly and compared.

The following method in accordance with the invention brings about a considerable simplification in hardware needed, particularly in the number of gates. Apparatus embodying the method is illustrated in Figure 1.

In this method, the correlation function C (x, y) is calculated by an integer pixel motion estimator 10 for motion vectors (x, y) and the best integer-pixel estimate (X, Y) is selected by comparison unit 12 as the motion vector for which C (x, y) is a minimum. Up to this point the method is conventional and well-known. However, rather than fitting some form of quadratic function to the values around C (X, Y), we propose that the values of C (x, y) around the best integer-pixel estimate (X, Y) be examined in order to determine in which quadrant, with respect to (X, Y), the half-pixel estimate should be placed. Specifically, if C (X-1, Y) < C (X+1, Y), then we deduce that XI X where X'is the half-pixel estimate. Conversely,

if C (X-1, Y) > C (X+1, Y), then X'X.

The values of C (X-1, Y) and C (X+1, Y) have, of course, already been calculated in order to find (X, Y). Y'can be found in a similar manner.

Once the appropriate quadrant has been identified, a restricted local search to half-pixel accuracy is performed over the four possible vectors in the chosen quadrant. For example, if it is established that X' X and Y' Y, then a search is made using the four vectors (X, Y), X+, Y), (X, Y- and X+Y-, which are

generated by means of bilinear interpolators 14. Correlation function generators 16 generate correlation functions for the pixel values output by the interpolators 14 and these are applied to a comparator 18 which gives the best half-integer pixel estimate.

The advantage of this is that the only previous-picture pixels which need to be accessed for the half-pixel search are those which would be needed if a motion compensated version of the previous picture were required, for example, as a motion compensated prediction. Furthermore, only four streams of interpolated pixels need be generated and corresponding mean absolute error values calculated and compared. Experiments using limited picture information suggest that this method produces results which compare favourably with methods with more complex hardware requirements.

In another known method capable of providing half-pixel accuracy, bilinear interpolation is performed on the previous picture to obtain the necessary intermediate pixels for a full search to half pixel accuracy. This constitutes, essentially, a full search to half-pixel accuracy. In a particular implementation in which motion vectors are calculated over a possible range of + 8 pixels in each direction, this requires 32 Motion Estimation Integrated Circuits (MEIC) rather than the 8 required for an integer pixel motion estimator. It also requires four versions of the previous picture to be generated, one for each possible pair of fractional values of the motion vector. Thus, as illustrated in Figure 2, four half-integer pixel values would be generated by means of bilinear interpolators 20 from the current picture input.

Integer-pixel motion estimators 22 would produce for these values correlation values, the best estimate being selected from the four correlation values by means of a comparison unit 24.

Each version would form the input to an integer-pixel motion estimator consisting of 8 MEICs, via six 1024-pixel buffer stores so 24 such buffer stores would be required. Performing the bilinear interpolation on the three stripes of previous picture information, that is, groups of eight consecutive lines, would also be costly in hardware.

We have appreciated that an alternative would be to generate four versions of the current picture. Thus, for each possible

integer motion vector (x, y), vectors (x, y), (x+, y), (x, (X'Y+1/2) and (X+1/2'+1/2) are tested using current picture signals P (i, j), P (i-j), P (i, j-J and P (i-j-) where i and j represent pixel coordinates.

Thus, half-pixel accuracy is obtained by shifting (interpolating) the current block by half-pixel shifts and comparing this shifted block with the previous frame block (shifted by the best integerpixel motion estimate). There is a slight error in the location of picture material used to test the half-pixel motion vectors, but there is also a substantial saving in hardware. Six buffer stores containing previous-picture information would be sufficient to feed all 32 MEICs and there would only be a comparatively slight increase in the buffer storage required for the input of the current-picture information.

Claims

1. A method of motion estimation for use with television signals, the method comprising ; calculating a correlation function C (x, y) for motion vectors (x, y) using information from signals representing a previous picture and the current picture; determining the best integer-pixel estimate (X, Y) for the motion vector from the calculated values of C (x, y); determining from the calculated values of C (x, y) the region relative to (X, Y) in which the best sub-integer-pixel estimate (X*, Y') of the motion vector is to be found ; and calculating and comparing values of the correlation function C (x, y) for those sub-integer-pixel motin vectors which are in the said region so as to find (X', Y').

2. A method according to claim 1 in which the said region is a quadrant.

3. A method according to claim 1 or 2 in which the best integerpixel estimate (X, Y) for the motion vector is determined by identifying the minimum value of the correlation function C (x, y).

4. A method according to claim l, 2 or 3 in which the motion vector is estimated to half-pixel accuracy; the correlation function C (x, y) being calculated and compared for half-integer-pixel motion vectors in the said region.

5. A method according to claim 4 in which C (x, y) is calculated for half-integer-pixel motion vectors using bilinear interpolation.

6. A method according to claim 1 substantially as hereinbefore described.

7. Apparatus for carrying out a method of motion estimation for use with television signals according to any of claims 1 to 6.

8. A method of motion estimation for use with television signals, the method comprising calculating a correlation function C (x, y) for sub-integer-pixel motion vectors using information from signals representing a previous picture and the current picture and determining from the calculated values of C (x, y) the best subinteger-pixel motion vector; the calculated values of C (x, y) for non-integer-pixel motion vectors being derived from a comparison of estimated information obtained from the present picture signal and information from the previous picture signal.

9. A method according to claim 8 in which the correlation function C (x, y) is calculated for half-integer-pixel motion vectors to give the motion vector to half-integer-pixel accuracy.

10. A method according to claim 9 in which information is obtained from the present picture signal by bilinear interpolation to enable non-integer-pixel values of the correlation function C (x, y) to be calculated.

11. A method according to claim 8 substantially as hereinbefore described.

12. Apparatus for carrying out a method of motion estimation for use with television signals according to any of claims 8 to 11.