Method for subpixel value interpolation
Info
 Publication number
 CA2452632C CA2452632C CA 2452632 CA2452632A CA2452632C CA 2452632 C CA2452632 C CA 2452632C CA 2452632 CA2452632 CA 2452632 CA 2452632 A CA2452632 A CA 2452632A CA 2452632 C CA2452632 C CA 2452632C
 Authority
 CA
 Grant status
 Grant
 Patent type
 Prior art keywords
 sub
 pixel
 pixels
 values
 unit
 Prior art date
 Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
 Active
Links
Classifications

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
 G06T3/00—Geometric image transformation in the plane of the image, e.g. from bitmapped to bitmapped creating a different image
 G06T3/40—Scaling the whole image or part thereof
 G06T3/4007—Interpolationbased scaling, e.g. bilinear interpolation

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
 H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
 H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
 H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
 H04N19/51—Motion estimation or motion compensation
 H04N19/523—Motion estimation or motion compensation with subpixel accuracy

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
 H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
 H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
Abstract
Description
METHOD FOR SUBPIXEL VALUE INTERPOLATION
The present invention relates to a method for subpixel value interpolation in the encoding and decoding of data. It relates particularly, but not exclusively, to encoding and decoding of digital video.
Background of the invention Digital video sequences, like ordinary motion pictures recorded on film, comprise a sequence of still images, the illusion of motion being created by displaying the images one after the other at a relatively fast frame rate, typically 15 to 30 frames per second. Because of the relatively fast frame rate, images in consecutive frames tend to be quite similar and thus contain a considerable amount of redundant information. For example, a typical scene may comprise some stationary elements, such as background scenery, and some moving areas, which may take many different forms, for example the face of a newsreader, moving traffic and so on. Alternatively, the camera recording the scene may itself be moving, in which case all elements of the image have the same kind of motion. In many cases, this means that the overall change between one video frame and the next is rather small. Of course, this depends on the nature of the movement. For example, the faster the movement, the greater the change from one frame to the next. Similarly, if a scene contains a number of moving elements, the change from one frame to the next is likely to be greater than in a scene where only one element is moving.
It should be appreciated that each frame of a raw, that is uncompressed, digital video sequence comprises a very large amount of image information.
Each frame of an uncompressed digital video sequence is formed from an array of image pixels. For example, in a commonly used digital video format, known as the Quarter Common Interchange Format (QCIF), a frame comprises an array of 176 x 144 pixels, in which case each frame has 25,344 pixels. In turn, each pixel is represented by a certain number of bits,
2 which carry information about the luminance and/or colour content of the region of the image corresponding to the pixel. Commonly, a socalled YUV
colour model is used to represent the luminance and chrominance content of the image. The luminance, or Y, component represents the intensity (brightness) of the image, while the colour content of the image is represented by two chrominance components, labelled U and V.
Colour models based on a luminance/chrominance representation of image content provide certain advantages compared with colour models that are based on a representation involving primary colours (that is Red, Green and Blue, RGB). The human visual system is more sensitive to intensity variations than it is to colour variations; YUV colour models exploit this property by using a lower spatial resolution for the chrominance components (U, V) than for the luminance component (Y). In this way the amount of information needed to code the colour information in an image can be reduced with an acceptable reduction in image quality.
The lower spatial resolution of the chrominance components is usually attained by subsampling. Typically, a block of 16x16 image pixels is represented by one block of 16x16 pixels comprising luminance information and the corresponding chrominance components are each represented by one block of 8x8 pixels representing an area of the image equivalent to that of the 16x16 pixels of the luminance component. The chrominance components are thus spatially subsampled by a factor of 2 in the x and y directions. The resulting assembly of one 16x16 pixel luminance block and two 8x8 pixel chrominance blocks is commonly referred to as a YUV
macroblock, or macroblock, for short.
A QCIF image comprises 11x9 macroblocks. If the luminance blocks and chrominance blocks are represented with 8 bit resolution (that is by numbers in the range 0 to 255), the total number of bits required per macroblock is (16x16x8)+2x(8x8x8) = 3072 bits. The number of bits needed to represent a video frame in QCIF format is thus 99x3072 = 304,128 bits.
3 This means that the amount of data required to transmit/record/display a video sequence in QCIF format, represented using a YUV colour model, at a rate of 30 frames per second, is more than 9 Mbps (million bits per second). This is an extremely high data rate and is impractical for use in video recording, transmission and display applications because of the very large storage capacity, transmission channel capacity and hardware performance required.
If video data is to be transmitted in realtime over a fixed line network such as an ISDN (Integrated Services Digital Network) or a conventional PSTN
(Public Service Telephone Network), the available data transmission bandwidth is typically of the order of 64kbits/s. In mobile videotelephony, where transmission takes place at least in part over a radio communications link, the available bandwidth can be as low as 20kbits/s. This means that a significant reduction in the amount of information used to represent video data must be achieved in order to enable transmission of digital video sequences over low bandwidth communication networks. For this reason video compression techniques have been developed which reduce the amount of information transmitted while retaining an acceptable image quality.
Video compression methods are based on reducing the redundant and perceptually irrelevant parts of video sequences. The redundancy in video sequences can be categorised into spatial, temporal and spectral redundancy. 'Spatial redundancy' is the term used to describe the correlation between neighbouring pixels within a frame. The term 'temporal redundancy' expresses the fact that the objects appearing in one frame of a sequence are likely to appear in subsequent frames, while 'spectral redundancy' refers to the correlation between different colour components of the same image.
Sufficiently efficient compression cannot usually be achieved by simply reducing the various forms of redundancy in a given sequence of images.
4 Thus, most current video encoders also reduce the quality of those parts of the video sequence which are subjectively the least important. In addition, the redundancy of the compressed video bitstream is itself reduced by means of efficient lossless encoding. Typically, this is achieved using a technique known as 'variable length coding' (VLC).
Modern video compression standards, such as ITUT recommendations H.261, H.263(+)(++), H.26L and the Motion Picture Experts Group recommendation MPEG4 make use of 'motion compensated temporal prediction'. This is a form of temporal redundancy reduction in which the content of some (often many) frames in a video sequence is 'predicted' from other frames in the sequence by tracing the motion of objects or regions of an image between frames.
Compressed images which do not make use of temporal redundancy reduction are usually called INTRAcoded or Iframes, whereas temporally predicted images are called INTERcoded or Pframes. In the case of INTER frames, the predicted (motioncompensated) image is rarely precise enough to represent the image content with sufficient quality, and therefore a spatially compressed prediction error (PE) frame is also associated with each INTER frame. Many video compression schemes can also make use of bidirectionally predicted frames, which are commonly referred to as Bpictures or Bframes. Bpictures are inserted between reference or socalled 'anchor' picture pairs (I or P frames) and are predicted from either one or both of the anchor pictures. Bpictures are not themselves used as anchor pictures, that is no other frames are predicted from them, and therefore, they can be discarded from the video sequence without causing deterioration in the quality of future pictures.
The different types of frame that occur in a typical compressed video sequence are illustrated in Figure 3 of the accompanying drawings. As can be seen from the figure, the sequence starts with an INTRA or I frame 30. In Figure 3, arrows 33 denote the 'forward' prediction process by which Pframes (labelled 34) are formed. The bidirectional prediction process by which Bframes (36) are formed is denoted by arrows 31 a and 31 b, respectively.
5 A schematic diagram of an example video coding system using motion compensated prediction is shown in Figures 1 and 2. Figure 1 illustrates an encoder 10 employing motion compensation and Figure 2 illustrates a corresponding decoder 20. The encoder 10 shown in Figure 1 comprises a Motion Field Estimation block 11, a Motion Field Coding block 12, a Motion Compensated Prediction block 13, a Prediction Error Coding block 14, a Prediction Error Decoding block 15, a Multiplexing block 16, a Frame Memory 17, and an adder 19. The decoder 20 comprises a Motion Compensated Prediction block 21, a Prediction Error Decoding block 22, a Demultiplexing block 23 and a Frame Memory 24.
The operating principle of video coders using motion compensation is to minimise the amount of information in a prediction error frame Eõ(x,y), which is the difference between a current frame 1R(x,y) being coded and a prediction frame Pn (x, y). The prediction error frame is thus:
E,, (x, y) = I,, (x,y)Pn(x,y). (1) The prediction frame Pn(x, y) is built using pixel values of a reference frame Rn (x, y) , which is generally one of the previously coded and transmitted frames, for example the frame immediately preceding the current frame and is available from the Frame Memory 17 of the encoder 10. More specifically, the prediction frame Põ(x, y) is constructed by finding socalled 'prediction pixels' in the reference frame R n (x, y) which correspond substantially with pixels in the current frame. Motion information, describing the relationship (e.g. relative location, rotation, scale etc.) between pixels in the current frame and their corresponding prediction pixels in the reference frame is derived and the prediction frame is constructed by moving the prediction
6 PCT/FI02/00729 pixels according to the motion information. In this way, the prediction frame is constructed as an approximate representation of the current frame, using pixel values in the reference frame. The prediction error frame referred to above therefore represents the difference between the approximate representation of the current frame provided by the prediction frame and the current frame itself. The basic advantage provided by video encoders that use motion compensated prediction arises from the fact that a comparatively compact description of the current frame can be obtained by representing it in terms of the motion information required to form its prediction together with the associated prediction error information in the prediction error frame.
However, due to the very large number of pixels in a frame, it is generally not efficient to transmit separate motion information for each pixel to the decoder. Instead, in most video coding schemes, the current frame is divided into larger image segments Sk and motion information relating to the segments is transmitted to the decoder. For example, motion information is typically provided for each macroblock of a frame and the same motion information is then used for all pixels within the macroblock. In some video coding standards, such as H.26L, a macroblock can be divided into smaller blocks, each smaller block being provided with its own motion information.
The motion information usually takes the form of motion vectors [Ox(x, y), Dy(x, y)] . The pair of numbers Ax(x, y) and Ay(x, y) represents the horizontal and vertical displacements of a pixel at location (x,y) in the current frame 1n (x, y) with respect to a pixel in the reference frame Rn (x, y) .
The motion vectors [Ox(x, y), Dy(x, y)] are calculated in the Motion Field Estimation block 11 and the set of motion vectors of the current frame [Ax(.), Ay(.)] is referred to as the motion vector field.
Typically, the location of a macroblock in a current video frame is specified by the (x,y) coordinate of its upper lefthand corner. Thus, in a video
7 coding scheme in which motion information is associated with each macroblock of a frame, each motion vector describes the horizontal and vertical displacement dx(x,y) and Ay(x,y) of a pixel representing the upper lefthand corner of a macroblock in the current frame 1,,(x,y) with respect to a pixel in the upper lefthand corner of a substantially corresponding block of prediction pixels in the reference frame Rõ (x, y) (as shown in Figure 4b).
Motion estimation is a computationally intensive task. Given a reference frame R,,(x, y) and, for example, a square macroblock comprising N x N
pixels in a current frame (as shown in Figure 4a), the objective of motion estimation is to find an N x N pixel block in the reference frame that matches the characteristics of the macroblock in the current picture according to some criterion. This criterion can be, for example, a sum of absolute differences (SAD) between the pixels of the macroblock in the current frame and the block of pixels in the reference frame with which it is compared. This process is known generally as 'block matching'. It should be noted that, in general, the geometry of the block to be matched and that in the reference frame do not have to be the same, as realworld objects can undergo scale changes, as well as rotation and warping. However, in current international video coding standards, only a translational motion model is used (see below) and thus fixed rectangular geometry is sufficient.
Ideally, in order to achieve the best chance of finding a match, the whole of the reference frame should be searched. However, this is impractical as it imposes too high a computational burden on the video encoder. Instead, the search region is restricted to region [ p, p] around the original location of the macroblock in the current frame, as shown in Figure 4c.
In order to reduce the amount of motion information to be transmitted from the encoder 10 to the decoder 20, the motion vector field is coded in the Motion Field Coding block 12 of the encoder 10, by representing it with a motion model. In this process, the motion vectors of image segments are re
8 expressed using certain predetermined functions or, in other words, the motion vector field is represented with a model. Almost all currently used motion vector field models are additive motion models, complying with the following general formula:
&x(x,Y) = I aif (X, Y) (2) =o Dy(x, Y) = b1g1 (x, y) (3) =o where coefficients a; and b; are called motion coefficients. The motion coefficients are transmitted to the decoder 20 (information stream 2 in Figures 1 and 2). Functions /; and g; are called motion field basis functions, and are known both to the encoder and decoder. An approximate motion vector field (Ox(x, y),Ay(x, y)) can be constructed using the coefficients and the basis functions. As the basis functions are known to (that is stored in) both the encoder 10 and the decoder 20, only the motion coefficients need to be transmitted to the encoder, thus reducing the amount of information required to represent the motion information of the frame.
The simplest motion model is the translational motion model which requires only two coefficients to describe the motion vectors of each segment. The values of motion vectors are given by:
Ax (x, y) = ao (4) Ay(x,Y) = bo This model is widely used in various international standards (ISO MPEG1, MPEG2, MPEG4, ITUT Recommendations H.261 and H.263) to describe the motion of 16x16 and 8x8 pixel blocks. Systems which use a translational
9 motion model typically perform motion estimation at full pixel resolution or some integer fraction of full pixel resolution, for example at half or one quarter pixel resolution.
The prediction frame PP(x,y) is constructed in the Motion Compensated Prediction block 13 in the encoder 10, and is given by:
I (x, y) = Rõ [x + dx(x, Y), Y + Dy(x, y)} (5) In the Prediction Error Coding block 14, the prediction error frame E,,(x,y) is typically compressed by representing it as a finite series (transform) of some 2dimensional functions. For example, a 2dimensional Discrete Cosine Transform (DCT) can be used. The transform coefficients are quantised and entropy (for example Huffman) coded before they are transmitted to the decoder (information stream 1 in Figures 1 and 2). Because of the error introduced by quantisation, this operation usually produces some degradation (loss of information) in the prediction error frame E,(x, y) . To compensate for this degradation, the encoder 10 also comprises a Prediction Error Decoding block 15, where a decoded prediction error frame En(x,y) is constructed using the transform coefficients. This locally decoded prediction error frame is added to the prediction frame P(x,y) in the adder 19 and the resulting decoded current frame I,,(x,y) is stored in the Frame Memory 17 for further use as the next reference frame Rõ+, (x, y) .
The information stream 2 carrying information about the motion vectors is combined with information about the prediction error in multiplexer 16 and an information stream 3 containing typically at least those two types of information is sent to the decoder 20.
The operation of a corresponding video decoder 20 will now be described.
The Frame Memory 24 of the decoder 20 stores a previously reconstructed reference frame Rn (x, y) . The prediction frame Pn (x, y) is constructed in the Motion Compensated Prediction block 21 of the decoder 20 according to equation 5, using received motion coefficient information and pixel values of 5 the previously reconstructed reference frame Rõ (x, y) . The transmitted transform coefficients of the prediction error frame Eõ(x, y) are used in the Prediction Error Decoding block 22 to construct the decoded prediction error frame Eõ(x,y). The pixels of the decoded current frame I,1(x,y) are then reconstructed by adding the prediction frame Põ (x, y) and the decoded
10 prediction error frame EE(x,y) :
I,, (x,y)=P,, (x,y)+E,, (x,y)=R,, [x+Ox(x,y),y+ay(x,y)] +Eõ(x,y). (6) This decoded current frame may be stored in the Frame Memory 24 as the next reference frame Rn+1 (x, y) .
In the description of motion compensated encoding and decoding of digital video presented above, the motion vector [&x(x, y), Dy(x, y)] describing the motion of a macroblock in the current frame with respect to the reference frame R, (x, y) can point to any of the pixels in the reference frame. This means that motion between frames of a digital video sequence can only be represented at a resolution which is determined by the image pixels in the frame (socalled full pixel resolution). Real motion, however, has arbitrary precision, and thus the system described above can only provide approximate modelling of the motion between successive frames of a digital video sequence. Typically, modelling of motion between video frames with full pixel resolution is not sufficiently accurate to allow efficient minimisation of the prediction error (PE) information associated with each macroblock/frame. Therefore, to enable more accurate modelling of real motion and to help reduce the amount of PE information that must be transmitted from encoder to decoder, many video coding standards, such as
11 H.263(+)(++) and H.26L, allow motion vectors to point 'in between' image pixels. In other words, the motion vectors can have 'subpixel' resolution.
Allowing motion vectors to have subpixel resolution adds to the complexity of the encoding and decoding operations that must be performed, so it is still advantageous to limit the degree of spatial resolution a motion vector may have. Thus, video coding standards, such as those previously mentioned, typically only allow motion vectors to have full, half or quarterpixel resolution.
Motion estimation with subpixel resolution is usually performed as a twostage process, as illustrated in Figure 5, for a video coding scheme which allows motion vectors to have full or halfpixel resolution. In the first step, a motion vector having fullpixel resolution is determined using any appropriate motion estimation scheme, such as the blockmatching process described in the foregoing. The resulting motion vector, having fullpixel resolution is shown in Figure 5.
In the second stage, the motion vector determined in the first stage is refined to obtain the desired halfpixel resolution. In the example illustrated in Figure 5, this is done by forming eight new search blocks of 16 x 16 pixels, the location of the topleft corner of each block being marked with an X in Figure 5. These locations are denoted as [Ax + m/2, Ay + n/2], where m and n can take the values 1, 0 and +1, but cannot be zero at the same time. As only the pixel values of original image pixels are known, the values (for example luminance and/or chrominance values) of the subpixels residing at halfpixel locations must be estimated for each of the eight new search blocks, using some form of interpolation scheme.
Having interpolated the values of the subpixels at halfpixel resolution, each of the eight search blocks is compared with the macroblock whose motion vector is being sought. As in the block matching process performed in order to determine the motion vector with full pixel resolution, the macroblock is compared with each of the eight search blocks according to some criterion,
12 for example a SAD. As a result of the comparisons, a minimum SAD value will generally be obtained. Depending on the nature of the motion in the video sequence, this minimum value may correspond to the location specified by the original motion vector (having fullpixel resolution), or it may correspond to a location having a halfpixel resolution. Thus, it is possible to determine whether a motion vector should point to a fullpixel or subpixel location and if subpixel resolution is appropriate, to determine the correct subpixel resolution motion vector. It should also be appreciated that the scheme just described can be extended to other subpixel resolutions (for example, onequarterpixel resolution) in an entirely analogous fashion.
In practice, the estimation of a subpixel value in the reference frame is performed by interpolating the value of the subpixel from surrounding pixel values. In general, interpolation of a subpixel value F(x,y) situated at a noninteger location (x, y) = (n+dx, m+Ay), can be formulated as a twodimensional operation, represented mathematically as:
K=1 L=1 F(x,y) = I Ef(k+K,I+L)F(n+k,m+l) (7) k=K l=L
where f(k,l) are filter coefficients and n and m are obtained by truncating x and y, respectively, to integer values. Typically, the filter coefficients are dependent on the x and y values and the interpolation filters are usually socalled 'separable filters', in which case subpixel value F(x,y) can be calculated as follows:
K=l K=1 F(x,y) = E f(k+K)Ef(l+K)F(n+k,m+l) (8) k=K l=K
The motion vectors are calculated in the encoder. Once the corresponding motion coefficients are transmitted to the decoder, it is a straightforward matter to interpolate the required subpixels using an interpolation method identical to that used in the encoder. In this way, a frame following a
13 reference frame in the Frame Memory 24, can be reconstructed from the reference frame and the motion vectors.
The simplest way of applying subpixel value interpolation in a video coder is to interpolate each subpixel value every time it is needed. However, this is not an efficient solution in a video encoder, because it is likely that the same subpixel value will be required several times and thus calculations to interpolate the same subpixel value will be performed multiple times. This results in an unnecessary increase of computational complexity / burden in the encoder.
An alternative approach, which limits the complexity of the encoder, is to precalculate and store all subpixel values in a memory associated with the encoder. This solution is called interpolation 'beforehand' interpolation hereafter in this document. While limiting complexity, beforehand interpolation has the disadvantage of increasing memory usage by a large margin. For example, if the motion vector accuracy is one quarter pixel in both horizontal and vertical dimensions, storing precalculated subpixel values for a complete image results in a memory usage that is 16 times that required to store the original, noninterpolated image. In addition, it involves the calculation of some subpixels which might not actually be required in calculating motion vectors in the encoder. Beforehand interpolation is also particularly inefficient in a video decoder, as the majority of precalculated subpixel values will never be required by the decoder. Thus, it is advantageous not to use precalculation in the decoder.
Socalled 'ondemand' interpolation can be used to reduce memory requirements in the encoder. For example, if the desired pixel precision is quarter pixel resolution, only subpixels at one half unit resolution are interpolated beforehand for the whole frame and stored in the memory.
Values of onequarter pixel resolution subpixels are only calculated during the motion estimation / compensation process as and when it is required. In
14 this case memory usage is only 4 times that required to store the original, noninterpolated image.
It should be noted that when beforehand interpolation is used, the interpolation process constitutes only a small fraction of the total encoder computational complexity / burden, since every pixel is interpolated just once. Therefore, in the encoder, the complexity of the interpolation process itself is not very critical when beforehand subpixel value interpolation is used. On the other hand, ondemand interpolation poses a much higher computational burden on the encoder, since subpixels may be interpolated many times. Hence the complexity of interpolation process, which may be considered in terms of the number of computational operations or operational cycles that must be performed in order to interpolate the subpixel values, becomes an important consideration.
In the decoder, the same subpixel values are used a few times at most and some are not needed at all. Therefore, in the decoder it is advantageous not to use beforehand interpolation at all, that is, it is advantageous not to precalculate any subpixel values.
Two interpolation schemes have been developed as part of the work ongoing in the ITUTelecommunications Standardization Sector, Study Group 16, Video Coding Experts Group (VCEG), Questions 6 and 15.
These approaches were proposed for incorporation into ITUT
recommendation H.26L and have been implemented in test models (TML) for the purposes of evaluation and further development. The test model corresponding to Question 15 is referred to as Test Model 5 (TML5), while that resulting from Question 6 is known as Test Model 6 (TML6). The interpolation schemes proposed in both TML5 and TML6 will now be described.
Throughout the description of the subpixel value interpolation scheme used in test model TML5, reference will be made to Figure 12a, which defines a notation for describing pixel and subpixel locations specific to TML5. A
separate notation, defined in Figure 13a, will be used in the discussion of the subpixel value interpolation scheme used in TML6. A still further notation, illustrated in Figure 14a, will be used later in the text in connection 5 with the subpixel value interpolation method according to the invention. It should be appreciated that the three different notations used in the text are intended to assist in the understanding of each interpolation method and to help distinguish differences between them. However, in all three figures, the letter A is used to denote original image pixels (full pixel resolution). More 10 specifically, the letter A represents the location of pixels in the image data representing a frame of a video sequence, the pixel values of pixels A being either received as current frame 1n(x,y) from a video source, or reconstructed and stored as a reference frame Rn (x, y) in the Frame Memory 17, 24 of the encoder 10 or the decoder 20. All other letters
15 represent subpixel locations, the values of the subpixels situated at the subpixel locations being obtained by interpolation.
Certain other terms will also be used in a consistent manner throughout the text to identify particular pixel and subpixel locations. These are as follows:
The term 'unit horizontal location' is used to describe the location of any subpixel that is constructed in a column of the original image data. Subpixels c and e in Figures 12a and 13a, as well as subpixels b and e in Figure 14a have unit horizontal locations.
The term 'unit vertical location' is used to describe any subpixel that is constructed in a row of the original image data. Subpixels b and d in Figures 12a and 13a as well as subpixels b and d in Figure 14a have unit vertical locations.
By definition, pixels A have unit horizontal and unit vertical locations.
16 The term 'half horizontal location' is used to describe the location of any subpixel that is constructed in a column that lies at half pixel resolution.
Subpixels b, c, and e shown in Figures 12a and 13a fall into this category, as do subpixels b, c and fin Figure 14a. In a similar manner, the term 'half vertical location' is used to describe the location of any subpixel that is constructed in a row that lies at halfpixel resolution, such as subpixels c and d in Figures 12a and 13a, as well as subpixels b, c and g in Figure 14a.
Furthermore, the term 'quarter horizontal location' refers to any subpixel that is constructed in a column which lies at quarterpixel resolution, such as subpixels d and e in Figure 12a, subpixels d and g in Figure 13a and subpixels d, g and h in Figure 14a. Analogously, the term 'quarter vertical location' refers to subpixels that are constructed in a row which lies at quarterpixel resolution. In Figure 12a, subpixels e and f fall into this category, as do subpixels e, f and g in Figure 13a and subpixels e, f and h in Figure 14a.
The definition of each of the terms described above is shown by 'envelopes' drawn on the corresponding figures.
It should further be noted that it is often convenient to denote a particular pixel with a twodimensional reference. In this case, the appropriate twodimensional reference can be obtained by examining the intersection of the envelopes in Figures 12a, 13a and 14a. Applying this principle, pixel d in Figure 12a, for example, has a half horizontal and half vertical location and subpixel e has a unit horizontal and quarter vertical location. In addition, and for ease of reference, subpixels that reside at half unit horizontal and unit vertical locations, unit horizontal and half unit vertical locations as well as half unit horizontal and half unit vertical locations, will be referred to as 1/2 resolution subpixels. Subpixels which reside at any quarter unit horizontal and / or quarter unit vertical location will be referred to as 1/4 resolution subpixels.
17 It should also be noted that in the descriptions of the two test models and in the detailed description of the invention itself, it will be assumed that pixels have a minimum value of 0 and a maximum value of 2"1 where n is the number of bits reserved for a pixel value. The number of bits is typically 8.
After a subpixel has been interpolated, if the value of that interpolated subpixel exceeds the value of 2"1, it is restricted to the range of [0, 2"1], i.e.
values lower than the minimum allowed value will become the minimum value (0) and values larger than the maximum will the become maximum value (2"1). This operation is called clipping.
The subpixel value interpolation scheme according to TML5 will now be described in detail with reference to Figures 12a, 12b and 12c.
1. The value for the subpixel at half unit horizontal and unit vertical location, that is '/2 resolution subpixel b in Figure 12a, is calculated using a 6tap filter. The filter interpolates a value for 1/2 resolution subpixel b based upon the values of the 6 pixels (A, to A6) situated in a row at unit horizontal locations and unit vertical locations symmetrically about b, as shown in Figure 12b, according to the formula b = (Ai  5A2 + 20A3 + 20A4  5A5 + A6 + 16) / 32. The operator / denotes division with truncation. The result is clipped to lie in the range [0, 2"1].
2. Values for the 1/2 resolution subpixels labelled c are calculated using the same six tap filter as used in step 1 and the six nearest pixels or subpixels (A or b) in the vertical direction. Referring now to Figure 12c, the filter interpolates a value for the '/2 resolution subpixel c located at unit horizontal and half vertical location based upon the values of the 6 pixels (A, to A6) situated in a column at unit horizontal locations and unit vertical locations symmetrically about c, according to the formula c = (A,  5A2 + 20A3 + 20A4  5A5 + A6 + 16) / 32. Similarly, a value for the '/2 resolution subpixel c at half horizontal and half vertical location is calculated according to c = (b,  5b2 + 20b3 + 20b4  5b5 + b6 + 16)/32.
18 Again, the operator / denotes division with truncation. The values calculated for the c subpixels are further clipped to lie in the range [0, 2 1].
At this point in the interpolation process the values of all 1/2 resolution subpixels have been calculated and the process proceeds to the calculation of 1/4 resolution subpixel values.
3. Values for the 1/4 resolution subpixels labelled d are calculated using linear interpolation and the values of the nearest pixels and / or 1/2 resolution subpixels in the horizontal direction. More specifically, values for 1/4 resolution subpixels d located at quarter horizontal and unit vertical locations, are calculated by taking the average of the immediately neighbouring pixel at unit horizontal and unit vertical location (pixel A) and the immediately neighbouring 1/2 resolution subpixel at half horizontal and unit vertical location (subpixel b), i.e.
according to d = (A + b) / 2. Values for 1/4 resolution subpixels d located at quarter horizontal and half vertical locations, are calculated by taking the average of the immediately neighbouring 1/2 resolution subpixels c which lie at unit horizontal and half vertical location and half horizontal and half vertical locations respectively, i.e. according to d = (Cl + c2)/2.
Again operator / indicates division with truncation.
4. Values for the 1/4 resolution subpixels labelled e are calculated using linear interpolation and the values of the nearest pixels and / or 1/2 resolution subpixels in the vertical direction. In particular, 1/4 resolution subpixels e at unit horizontal and quarter vertical locations are calculated by taking the average of the immediately neighbouring pixel at unit horizontal and unit vertical location (pixel A) and the immediately neighbouring subpixel at unit horizontal and half vertical location (subpixel c) according to e = (A + c) / 2. 1/4 resolution subpixels e3 at half horizontal and quarter vertical locations are calculated by taking the
19 average of the immediately neighbouring subpixel at half horizontal and unit vertical location (subpixel b) and the immediately neighbouring subpixel at half horizontal and half vertical location (subpixel c), according to e = (b + c) / 2. Furthermore, 1/4 resolution subpixels e at quarter horizontal and quarter vertical locations are calculated by taking the average of the immediately neighbouring subpixels at quarter horizontal and unit vertical location and the corresponding subpixel at quarter horizontal and half vertical location (subpixels d), according to e = (d1 +
d2) / 2. Once more, operator / indicates division with truncation.
5. The value for 1/4 resolution subpixel f is interpolated by averaging the values of the 4 closest pixels values at unit horizontal and vertical locations, according to f = (A1 + A2 + A3 + A4 + 2) / 4, where pixels A1, A2, A3 and A4 are the four nearest original pixels.
A disadvantage of TML5 is that the decoder is computationally complex.
This results from the fact that TML5 uses an approach in which interpolation of 1/4 resolution subpixel values depends upon the interpolation of 1/2 resolution subpixel values. This means that in order to interpolate the values of the 1/4 resolution subpixels, the values of the '/2 resolution subpixels from which they are determined must be calculated first. Furthermore, since the values of some of the 1/4 resolution subpixels depend upon the interpolated values obtained for other 1/4 resolution subpixels, truncation of the1/4 resolution subpixel values has a deleterious effect on the precision of some of the 1/4 resolution subpixel values. Specifically, the 1/4 resolution subpixel values are less precise than they would be if calculated from values that had not been truncated and clipped. Another disadvantage of TML5 is that it is necessary to store the values of the 'h resolution subpixels in order to interpolate the 1/4 resolution subpixel values. Therefore, excess memory is required to store a result which is not ultimately required.
The subpixel value interpolation scheme according to TML6, referred to herein as direct interpolation, will now be described. In the encoder the interpolation method according to TML6 works like the previously described TML5 interpolation method, except that maximum precision is retained throughout. This is achieved by using intermediate values which are neither rounded nor clipped. A stepbystep description of interpolation method 5 according to TML6 as applied in the encoder is given below with reference to Figures 13a, 13b and 13c.
1. The value for the subpixel at half unit horizontal and unit vertical location, that is 1/2 resolution subpixel b in Figure 13a, is obtained by first 10 calculating an intermediate value b using a six tap filter. The filter calculates b based upon the values of the 6 pixels (A1 to A6) situated in a row at unit horizontal locations and unit vertical locations symmetrically about b, as shown in Figure 13b, according to the formula b = (A1  5A2 +
20A3 + 20A4  5A5 + A6). The final value of b is then calculated as b = (b 15 + 16) / 32 and is clipped to lie in the range [0, 2"1]. As before, the operator / denotes division with truncation.
2. Values for the 1/2 resolution subpixels labelled c are obtained by first calculating intermediate values c. Referring to Figure 13c, an
20 intermediate value c for the 1/2 resolution subpixel c located at unit horizontal and half vertical location is calculated based upon the values of the 6 pixels (A, to A6) situated in a column at unit horizontal locations and unit vertical locations symmetrically about c, according to the formula c = (A,  5A2 + 20A3 + 20A4  5A5 + A6). The final value for the 1/2 resolution subpixel c located at unit horizontal and half vertical location is then calculated according to c = (c + 16) / 32. Similarly, an intermediate value c for the '/2 resolution subpixel c at half horizontal and half vertical location is calculated according to c = (b,  5b2 + 20b3 +
20b4  5b5 + b6). A final value for this '/2 resolution subpixel is then calculated according to (c + 512) / 1024. Again, the operator / denotes division with truncation and the values calculated for 1/2 resolution subpixels c are further clipped to lie in the range [0, 2"1].
21 3. Values for the 1/4 resolution subpixels labelled d are calculated as follows. Values for 1/4 resolution subpixels d located at quarter horizontal and unit vertical locations, are calculated from the value of the immediately neighbouring pixel at unit horizontal and unit vertical location (pixel A) and the intermediate value b calculated in step (1) for the immediately neighbouring 1/2 resolution subpixel at half horizontal and unit vertical location (1/2 resolution subpixel b), according to d = (32A + b + 32) / 64. Values for 1/4 resolution subpixels d located at quarter horizontal and half vertical locations, are interpolated using the intermediate values c calculated for the immediately neighbouring 1/2 resolution subpixels c which lie at unit horizontal and half vertical location and half horizontal and half vertical locations respectively, according to d = (32c, + c2 + 1024) /
2048. Again operator / indicates division with truncation and the finally obtained 1/4 resolution subpixel values d are clipped to lie in the range [0, 2"1].
4. Values for the 1/4 resolution subpixels labelled e are calculated as follows. Values for 1/4 resolution subpixels e located at unit horizontal and quarter vertical locations are calculated from the value of the immediately neighbouring pixel at unit horizontal and unit vertical location (pixel A) and the intermediate value c calculated in step (2) for the immediately neighbouring 1/2 resolution subpixel at unit horizontal and half vertical location, according to e = (32A + c+ 32) /
64. Values for 1/4 resolution subpixels e located at half horizontal and quarter vertical locations are calculated from the intermediate value b calculated in step (1) for the immediately neighbouring 1/2 resolution subpixel at half horizontal and unit vertical location and the intermediate value c calculated in step (2) for the immediately neighbouring 1/2 resolution subpixel at half horizontal and half vertical location, according to e = (32b + c + 1024) / 2048. Once more, operator /
22 indicates division with truncation and the finally obtained 1/4 resolution subpixel values e are clipped to lie in the range [0, 2"1].
5. Values for 1/4 resolution subpixels labelled g are computed using the value of the nearest original pixel A and the intermediate values of the three nearest neighbouring 1/2 resolution subpixels, according to g =
(1024A + 32b + 32c, + c2+ 2048) / 4096. As before, operator / indicates division with truncation and the finally obtained for 1/4 resolution subpixel values g are clipped to lie in the range [0, 2"1].
6. The value for 1/4 resolution subpixel f is interpolated by averaging the values of the 4 closest pixels at unit horizontal and vertical locations, according to f = (A, + A2 + A3 + A4 + 2) / 4, where the locations of pixels Ai, A2, A3 and A4 are the four nearest original pixels.
In the decoder, subpixel values can be obtained directly by applying 6tap filters in horizontal and vertical directions. In the case of 1/4 subpixel resolution, referring to Figure 13a, the filter coefficients applied to pixels and subpixels at unit vertical location are [0, 0, 64, 0, 0, 0] for a set of six pixels A, [1, 5, 52, 20, 5, 1 ] for a set of six subpixels d, [2, 10, 40, 40, 10, 2] for a set of six subpixels b, and [1, 5, 20, 52, 5, 1] for a set of six subpixels d. These filter coefficients are applied to respective sets of pixels or subpixels in the same row as the subpixel values being interpolated.
After applying the filters in the horizontal and vertical directions, interpolated value c is normalized according to c = (c + 2048) / 4096 and clipped to lie in the range [0, 2"1]. When a motion vector points to an integer pixel position in either the horizontal or vertical direction, many zero coefficients are used.
In a practical implementation of TML6, different branches are used in the software which are optimised for the different subpixel cases so that there are no multiplications by zero coefficients.
23 It should be noted that in TML6, 1/4 resolution subpixel values are obtained directly using the intermediate values referred to above and are not derived from rounded and clipped values for 1/2 resolution subpixels. Therefore, in obtaining the 1/4 resolution subpixel values, it is not necessary to calculate final values for any of the 1/2 resolution subpixels. Specifically, it is not necessary to carry out the truncation and clipping operations associated with the calculation of final values for the 1/2 resolution subpixels.
Neither is it necessary to have stored final values for 1/2 resolution subpixels for use in the calculation of the 1/4 resolution subpixel values. Therefore TML6 is computationally less complex than TML5, as fewer truncating and clipping operations are required. However, a disadvantage of TML6 is that high precision arithmetic is required both in the encoder and in the decoder. High precision interpolation requires more silicon area in ASICs and requires more computations in some CPUs. Furthermore, implementation of direct interpolation as specified in TML6 in an ondemand fashion has a high memory requirement. This is an important factor, particularly in embedded devices.
In view of the previously presented discussion, it should be appreciated that due to the different requirements of the video encoder and decoder with regard to subpixel interpolation, there exists a significant problem in developing a method of subpixel value interpolation capable of providing satisfactory performance in both the encoder and decoder. Furthermore, neither of the current test models (TML5, TML6) described in the foregoing can provide a solution that is optimum for application in both encoder and decoder.
Summary of the Invention According to a first aspect of the invention there is provided method of interpolation in video coding in which an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the
24 pixels in the columns residing at unit vertical locations, is interpolated to generate values for subpixels at fractional horizontal and vertical locations, the fractional horizontal and vertical locations being defined according to 1/2x, where x is a positive integer having a maximum value N, the method comprising:
a) when values for subpixels at 1/2N1 unit horizontal and unit vertical locations, and unit horizontal and 1/2N1 unit vertical locations are required, interpolating such values directly using weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) when values for subpixels at 1/2N1 unit horizontal and 1/2N1 unit vertical locations are required, interpolating such values directly using a choice of a first weighted sum of values for subpixels residing at 1/2N1 unit horizontal and unit vertical locations and a second weighted sum of values for subpixels residing at unit horizontal and 1/2N,1 unit vertical locations, the first and second weighted sums of values being calculated according to step (a);
and c) when a value for a subpixel situated at a 1/2N unit horizontal and 1/2N
unit vertical location is required, interpolating such a value by taking a weighted average of the value of a first subpixel or pixel situated at a 1/2Nm unit horizontal and 1/2Nn unit vertical location and the value of a second subpixel or pixel located at a 1/2Np unit horizontal and 1/2Nq unit vertical location, variables m, n, p and q taking integer values in the range 1 to N
such that the first and second subpixels or pixels are located diagonally with respect to the subpixel at 1/2N unit horizontal and 1/2N vertical location.
Preferably a first and a second weight are used in the weighted average referred to in (c), the relative magnitudes of the weights being inversely proportional to the (straightline diagonal) proximity of the first and the second subpixel or pixel to the subpixel at 1/2N unit horizontal and 1/2N
vertical location.
In a situation where the first and the second subpixel or pixel are symmetrically located with respect to (equidistant from) the subpixel at 1/2N
unit horizontal and 1/2N vertical location, the first and second weights may have equal values.
The first weighted sum of values for subpixels residing at 1/2N' unit 5 horizontal and unit vertical locations in step b) may be used when a subpixel at 1/2N' unit horizontal and 1/2N unit vertical location is required.
The second weighted sum of values for subpixels residing at unit horizontal and 1/2N' unit vertical locations in step b) may be used when a subpixel at 10 1/2N unit horizontal and 1/2N' unit vertical location is required.
In one embodiment, when values for subpixels at 1/2N unit horizontal and unit vertical locations, and 1/2N horizontal and 1/2N1 vertical locations are required, such values are interpolated by taking the average of the values of 15 a first pixel or subpixel located at a vertical location corresponding to that of the subpixel being calculated and unit horizontal location and a second pixel or subpixel located at a vertical location corresponding to that of the subpixel being calculated and 1/2N1 unit horizontal location.
20 When values for subpixels at unit horizontal and 1/2N unit vertical locations, and 1/2N.1 unit horizontal and 1/2N unit vertical locations are required, they may be interpolated by taking the average of the values of a first pixel or subpixel located at a horizontal location corresponding to that of the subpixel being calculated and unit vertical location and a second pixel or sub
25 pixel located at a horizontal location corresponding to that of the subpixel being calculated and 1/2N1 unit vertical location.
Values for subpixels at 1/2N unit horizontal and 1/2N unit vertical locations may be interpolated by taking the average of values of a pixel located at a unit horizontal and unit vertical location, and a subpixel located at a 1/2N' unit horizontal and 1/2N1 unit vertical location.
26 Values for subpixels at 1/2N unit horizontal and 1/2N unit vertical locations may be interpolated by taking the average of values of a subpixel located at a 1/2N1 unit horizontal and unit vertical location, and a subpixel located at a unit horizontal and 1/2N1 unit vertical location.
Values for half of the subpixels at 1/2N unit horizontal and 1/2N unit vertical locations may be interpolated by taking the average of a first pair of values of a subpixel located at a 1/2N1 unit horizontal and unit vertical location, and a subpixel located at a unit horizontal and 1/2N1 unit vertical location and values for the other half of the subpixels at 1/2N unit horizontal and 1/2N unit vertical locations are interpolated by taking the average of a second pair of values of a pixel located at a unit horizontal and unit vertical location, and a subpixel located at a 1/2N1 unit horizontal and 1/2N1 unit vertical location.
Values for subpixels at 1/2N unit horizontal and 1/2N unit vertical locations are alternately interpolated for one such subpixel by taking the average of a first pair of values of a subpixel located at a 1/2N1 unit horizontal and unit vertical location, and a subpixel located at a unit horizontal and 1/2N1 unit vertical location and values and for a neighbouring such subpixel by taking the average of a second pair of values of a pixel located at a unit horizontal and unit vertical location, and a subpixel located at a 1/2N.1 unit horizontal and 1/2N1 unit vertical location.
The subpixels at 1/2N unit horizontal and 1/2N unit vertical locations may be alternately interpolated in a horizontal direction.
The subpixels at 1/2N unit horizontal and 1/2N unit vertical locations may be alternately interpolated in a horizontal direction.
When values for some subpixels at 1/2N unit horizontal and 1/2N unit vertical locations are required, such values may be interpolated by taking the average of a plurality of nearest neighbouring pixels.
27 At least one of step a) and step b) interpolating subpixel values directly using weighted sums may involve the calculation of an intermediate value for the subpixel values having a dynamic range greater than the specified dynamic range.
The intermediate value for a subpixel having 1/2N1 subpixel resolution may be used the calculation of a subpixel value having 1/2N subpixel resolution.
According to a second aspect of the invention, there is provided a method of interpolation in video coding in which an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, is interpolated to generate values for subpixels at fractional horizontal and vertical locations, the method comprising:
a) when values for subpixels at half unit horizontal and unit vertical locations, and unit horizontal and half unit vertical locations are required, interpolating such values directly using weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) when values for subpixels at half unit horizontal and half unit vertical locations are required, interpolating such values directly using a weighted sum of values for subpixels residing at half unit horizontal and unit vertical locations calculated according to step (a); and c) when values for subpixels at quarter unit horizontal and quarter unit vertical locations are required, interpolating such values by taking the average of at least one pair of a first pair of values of a subpixel located at a half unit horizontal and unit vertical location, and a subpixel located at a unit horizontal and half unit vertical location and a second pair of values of a pixel located at a unit horizontal and unit vertical location, and a subpixel located at a half unit horizontal and half unit vertical location.
28 According to a third aspect of the invention, there is provided a method of interpolation in video coding in which an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, is interpolated to generate values for subpixels at fractional horizontal and vertical locations, the fractional horizontal and vertical locations being defined according to 1/2x where x is a positive integer having a maximum value N, the method comprising:
a) when values for subpixels at 1/2N1 unit horizontal and unit vertical locations, and unit horizontal and 1/2N1 unit vertical locations are required, interpolating such values directly using weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) when a value for a subpixel at a subpixel horizontal and subpixel vertical location is required, interpolating such a value directly using a choice of a first weighted sum of values for subpixels located at a vertical location corresponding to that of the subpixel being calculated and a second weighted sum of values for subpixels located at a horizontal location corresponding to that of the subpixel being calculated.
The subpixels used in the first weighted sum may be subpixels residing at 1/2N1 unit horizontal and unit vertical locations and the first weighted sum may be used to interpolate a value for a subpixel at 1/2N1 unit horizontal and 1/2N unit vertical location.
The subpixels used in the second weighted sum may be subpixels residing at unit horizontal and 1/2N1 unit vertical locations and the second weighted sum may be used to interpolate a value for a subpixel at 1/2N unit horizontal and 1/2N1 unit vertical location.
When values for subpixels at 1/2N unit horizontal and 1/2N unit vertical locations are required, they may be interpolated by taking the average of at least one pair of a first pair of values of a subpixel located at a 1/2N1 unit
29 horizontal and unit vertical location, and a subpixel located at a unit horizontal and 1/2N1 unit vertical location and a second pair of values of a pixel located at a unit horizontal and unit vertical location, and a subpixel located at a 1/2N1 unit horizontal and 1/2N1 unit vertical location.
In the foregoing, N may be equal an integer selected from a list consisting of the values 2, 3, and 4.
Subpixels at quarter unit horizontal location are to be interpreted as being subpixels having as their lefthand nearest neighbour a pixel at unit horizontal location and as their righthand nearest neighbour a subpixel at half unit horizontal location as well as subpixels having as their lefthand nearest neighbour a subpixel at half unit horizontal location and as their righthand nearest neighbour a pixel at unit horizontal location.
Correspondingly, subpixels at quarter unit vertical location are to be interpreted as being subpixels having as their upper nearest neighbour a pixel at unit vertical location and as their lower nearest neighbour a subpixel at half unit vertical location as well as subpixels having as their upper nearest neighbour a subpixel at half unit vertical location and as their lower nearest neighbour a pixel at unit vertical location.
The term dynamic range, refers to the range of values which the subpixel values and the weighted sums can take.
Preferably changing the dynamic range, whether by extending it or reducing it, means changing the number of bits which are used to represent the dynamic range.
In an embodiment of the invention, the method is applied to an image that is subdivided into a number of image blocks. Preferably each image block comprises four corners, each corner being defined by a pixel located at a unit horizontal and unit vertical location. Preferably the method is applied to each image block as the block becomes available for subpixel value interpolation. Alternatively, subpixel value interpolation according to the method of the invention is performed once all image blocks of an image have become available for subpixel value interpolation.
5 Preferably the method is used in video encoding. Preferably the method is used in video decoding.
In one embodiment of the invention, when used in encoding, the method is carried out as beforehand interpolation, in which values for all subpixels at 10 half unit locations and values for all subpixels at quarter unit locations are calculated and stored before being subsequently used in the determination of a prediction frame during motion predictive coding. In alternative embodiments, the method is carried out as a combination of beforehand and ondemand interpolation. In this case, a certain proportion or category 15 of subpixel values is calculated and stored before being used in the determination of a prediction frame and certain other subpixel values are calculated only when required during motion predictive coding.
Preferably, when the method is used in decoding, subpixels are only 20 interpolated when their need is indicated by a motion vector.
According to a fourth aspect of the invention, there is provided a video coder for coding an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in 25 the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, the video coder comprising an interpolator adapted to generate values for subpixels at fractional horizontal and vertical locations, the fractional horizontal and vertical locations being defined according to 1/2x, where x is a positive integer having a maximum
30 value N, the interpolator being adapted to:
a) interpolate values for subpixels at 1/2N 1 unit horizontal and unit vertical locations, and unit horizontal and 1/2N 1 unit vertical locations directly using
31 weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) interpolate values for subpixels at 1/2",' unit horizontal and 1/2"' unit vertical locations directly using a choice of a first weighted sum of values for subpixels residing at 1/2"' unit horizontal and unit vertical locations and a second weighted sum of values for subpixels residing at unit horizontal and 1/2",' unit vertical locations, the first and second weighted sums of values being calculated according to step (a); and c) interpolate a value for a subpixel situated at a 1/2" unit horizontal and 1/2" unit vertical location by taking a weighted average of the value of a first subpixel or pixel situated at a 1/2"""' unit horizontal and 1/2""" unit vertical location and the value of a second subpixel or pixel located at a 1/2"_p unit horizontal and 1/2",Q unit vertical location, variables m, n, p and q taking integer values in the range 1 to N such that the first and second subpixels or pixels are located diagonally with respect to the subpixel at 1/2" unit horizontal and 1/2 N vertical location.
The video coder may comprise a video encoder. It may comprise a video decoder. There may be a codec comprising both the video encoder and the video decoder.
According to a fifth aspect of the invention, there is provided a communications terminal comprising a video coder for coding an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, the video coder comprising an interpolator adapted to generate values for subpixels at fractional horizontal and vertical locations, the fractional horizontal and vertical locations being defined according to 1/2X, where x is a positive integer having a maximum value N, the interpolator being adapted to:
32 a) interpolate values for subpixels at 1/2N1 unit horizontal and unit vertical locations, and unit horizontal and 1/2N1 unit vertical locations directly using weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) interpolate values for subpixels at 1/2",1 unit horizontal and 1/2N1 unit vertical locations directly using a choice of a first weighted sum of values for subpixels residing at 1/2N 1 unit horizontal and unit vertical locations and a second weighted sum of values for subpixels residing at unit horizontal and 1/2",1 unit vertical locations, the first and second weighted sums of values being calculated according to step (a); and c) interpolate a value for a subpixel situated at a 1/2N unit horizontal and 1/2" unit vertical location by taking a weighted average of the value of a first subpixel or pixel situated at a 1/2"`" unit horizontal and 1/2"" unit vertical location and the value of a second subpixel or pixel located at a 1/2",P unit horizontal and 1/2W4 unit vertical location, variables m, n, p and q taking integer values in the range 1 to N such that the first and second subpixels or pixels are located diagonally with respect to the subpixel at 1/2N unit horizontal and 1/2N vertical location.
The communications terminal may comprise a video encoder. It may comprise a video decoder. Preferably, it comprises a video codec comprising a video encoder and a video decoder.
Preferably the communications terminal comprising a user interface, a processor and at least one of a transmitting block and a receiving block, and a video coder according to at least one of the third and fourth aspects of the invention. Preferably the processor controls the operation of the transmitting block and/or the receiving block and the video coder.
According to a sixth aspect of the invention, there is provided a telecommunications system comprising a communications terminal and a network, the telecommunications network and the communications terminal
33 being connected by a communications link over which coded video can be transmitted, the communications terminal comprising a video coder for coding for coding an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, the video coder comprising an interpolator adapted to generate values for subpixels at fractional horizontal and vertical locations, the fractional horizontal and vertical locations being defined according to 1/2x, where x is a positive integer having a maximum value N, the interpolator being adapted to:
a) interpolate values for subpixels at 1/2"1 unit horizontal and unit vertical locations, and unit horizontal and 1/2N 1 unit vertical locations directly using weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) interpolate values for subpixels at 1/2""' unit horizontal and 1/2"1 unit vertical locations directly using a choice of a first weighted sum of values for subpixels residing at 1/2N 1 unit horizontal and unit vertical locations and a second weighted sum of values for subpixels residing at unit horizontal and 1/2""' unit vertical locations, the first and second weighted sums of values being calculated according to step (a); and c) interpolate a value for a subpixel situated at a 1/2N unit horizontal and 1/2N unit vertical location by taking a weighted average of the value of a first subpixel or pixel situated at a 1/2""'" unit horizontal and 1/2"" unit vertical location and the value of a second subpixel or pixel located at a 1/2",p unit horizontal and 1/2" unit vertical location, variables m, n, p and q taking integer values in the range 1 to N such that the first and second subpixels or pixels are located diagonally with respect to the subpixel at 1/2N unit horizontal and 1/2" vertical location.
Preferably the telecommunications system is a mobile telecommunications system comprising a mobile communications terminal and a wireless network, the connection between the mobile communications terminal and
34 the wireless network being formed by a radio link. Preferably the network enables the communications terminal to communicate with other communications terminals connected to the network over communications links between the other communications terminals and the network.
According to a seventh aspect of the invention, there is provided a telecommunications system comprising a communications terminal and a network, the telecommunications network and the communications terminal being connected by a communications link over which coded video can be transmitted, the network comprising a video coder for coding for coding an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, the video coder comprising an interpolator adapted to generate values for subpixels at fractional horizontal and vertical locations, the fractional horizontal and vertical locations being defined according to 1/2', where x is a positive integer having a maximum value N, the interpolator being adapted to:
a) interpolate values for subpixels at 1/2N' unit horizontal and unit vertical locations, and unit horizontal and 1/2"' unit vertical locations directly using weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) interpolate values for subpixels at 1/2".' unit horizontal and 1/2".' unit vertical locations directly using a choice of a first weighted sum of values for subpixels residing at 1/2".' unit horizontal and unit vertical locations and a second weighted sum of values for subpixels residing at unit horizontal and 1/2N 1 unit vertical locations, the first and second weighted sums of values being calculated according to step (a); and c) interpolate a value for a subpixel situated at a 1/2" unit horizontal and 1/2" unit vertical location by taking a weighted average of the value of a first subpixel or pixel situated at a 1/2""'" unit horizontal and 1/2" unit vertical location and the value of a second subpixel or pixel located at a 1/2"p unit horizontal and 1/2"l unit vertical location, variables m, n, p and q taking integer values in the range 1 to N such that the first and second subpixels or pixels are located diagonally with respect to the subpixel at 1/2" unit horizontal and 1/2" vertical location.
According to an eighth aspect of the invention there is provided a video coder for coding an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns 10 residing at unit vertical locations, the coder comprising an interpolator adapted to generate values for subpixels at fractional horizontal and vertical locations, the resolution of the subpixels being determined by a positive integer N, the interpolator being adapted to:
a) interpolate values for subpixels at 1/2N1 unit horizontal and unit vertical 15 locations, and unit horizontal and 1/2N1 unit vertical locations directly using weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) interpolate a value for a subpixel at a subpixel horizontal and subpixel vertical location is required directly using a choice of a first weighted sum of 20 values for subpixels located at a vertical location corresponding to that of the subpixel being calculated and a second weighted sum of values for subpixels located at a horizontal location corresponding to that of the subpixel being calculated.
25 The interpolator may be further adapted to form the first weighted sum using the values of subpixels residing at 1/2N1 unit horizontal and unit vertical locations and to use the first weighted sum to interpolate a value for a subpixel at 1/2N1 unit horizontal and 1/2N unit vertical location.
30 The interpolator may be further adapted to form the second weighted sum using the values of subpixels residing at unit horizontal and 1/2N1 unit vertical locations and to use the second weighted sum to interpolate a value for a subpixel at 1/2N unit horizontal and 1/2N1 unit vertical location.
The interpolator may be further adapted to interpolate values for subpixels at 1/2N unit horizontal and 1/2N unit vertical locations by taking the average of at least one pair of a first pair of values of a subpixel located at a unit horizontal and unit vertical location, and a subpixel located at a unit horizontal and 1/2N1 unit vertical location and a second pair of values of a pixel located at a unit horizontal and unit vertical location, and a subpixel located at a 1/2N1 unit horizontal and 1/2N1 unit vertical location.
According to a ninth aspect of the invention there is provided a communications terminal comprising a video coder for coding an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, the coder comprising an interpolator adapted to generate values for subpixels at fractional horizontal and vertical locations, the resolution of the subpixels being determined by a positive integer N, the interpolator being adapted to:
a) interpolate values for subpixels at 1/2N1 unit horizontal and unit vertical locations, and unit horizontal and 1/2N1 unit vertical locations directly using weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) interpolate a value for a subpixel at a subpixel horizontal and subpixel vertical location is required directly using a choice of a first weighted sum of values for subpixels located at a vertical location corresponding to that of the subpixel being calculated and a second weighted sum of values for subpixels located at a horizontal location corresponding to that of the subpixel being calculated.
According to a tenth aspect of the invention there is provided a telecommunications system comprising a communications terminal and a network, the telecommunications network and the communications terminal being connected by a communications link over which coded video can be transmitted, the communications terminal comprising a video coder for coding an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, the coder comprising an interpolator adapted to generate values for subpixels at fractional horizontal and vertical locations, the resolution of the subpixels being determined by a positive integer N, the interpolator being adapted to:
a) interpolate values for subpixels at 1/2N1 unit horizontal and unit vertical locations, and unit horizontal and 1/2N1 unit vertical locations directly using weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) interpolate a value for a subpixel at a subpixel horizontal and subpixel vertical location is required directly using a choice of a first weighted sum of values for subpixels located at a vertical location corresponding to that of the subpixel being calculated and a second weighted sum of values for subpixels located at a horizontal location corresponding to that of the subpixel being calculated.
According to an eleventh aspect of the invention there is provided a telecommunications system comprising a communications terminal and a network, the telecommunications network and the communications terminal being connected by a communications link over which coded video can be transmitted, the network comprising a video coder for coding for coding an image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, the coder comprising an interpolator adapted to generate values for subpixels at fractional horizontal and vertical locations, the resolution of the subpixels being determined by a positive integer N, the interpolator being adapted to:
a) interpolate values for subpixels at 1/2N1 unit horizontal and unit vertical locations, and unit horizontal and 1/2"'1 unit vertical locations directly using weighted sums of pixels residing at unit horizontal and unit vertical locations;
b) interpolate a value for a subpixel at a subpixel horizontal and subpixel vertical location is required directly using a choice of a first weighted sum of values for subpixels located at a vertical location corresponding to that of the subpixel being calculated and a second weighted sum of values for subpixels located at a horizontal location corresponding to that of the subpixel being calculated.
According to a twelfth aspect of the invention there is provided a method of interpolating an image, the image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, to generate values for subpixels, a subpixel being located at at least one of a fractional horizontal and a fractional vertical location, the fractional horizontal and fractional vertical locations being representable according to the mathematical notation 1/2X, where x is a positive integer in the range 1 to N, 1 /2X representing a particular level of subpixel interpolation and N
representing a maximum level of subpixel interpolation, the method comprising:
a) interpolating values for subpixels situated at 1/2 NI unit horizontal and unit vertical locations, and for subpixels situated at unit horizontal and 1/2N1 unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations;
b) interpolating a value for a subpixel situated at a 1/2N1 unit horizontal and 1/2 N1 unit vertical location using either a first weighted sum of the interpolated values for subpixels residing at 1/2N1 unit horizontal and unit vertical locations or a second weighted sum of the interpolated values for subpixels residing at unit horizontal and 1/2 N1 unit vertical locations obtained in step a); and c) interpolating a value for a subpixel situated at a 1/2 N unit horizontal and 1/2 N
unit vertical location using either:
 a weighted average of the value of a first subpixel situated at a 1/2Nm unit horizontal and 1/2N"" unit vertical location and the value of a second subpixel situated at a 1/2Np unit horizontal and 1/2NQ unit vertical location, or 38a  a weighted average of the value of a pixel situated at a unit horizontal and unit vertical location and the value of a subpixel situated at a 1/2Nm unit horizontal and 1/2 Wn unit vertical location, variables m, n, p and q taking integer values in the range 1 to N such that the respective first and second subpixels or respective pixel and subpixel are located diagonally with respect to the subpixel at 1/2 N unit horizontal and unit vertical location being interpolated.
According to a thirteenth aspect of the invention, there is provided a method of interpolating an image, the image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, to generate values for subpixels, a subpixel being located at at least one of a fractional horizontal and a fractional vertical location, the method comprising:
a) interpolating values for subpixels at half unit horizontal and unit vertical locations, and unit horizontal and half unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations;
b) interpolating a value for a subpixel at a half unit horizontal and half unit vertical location using either a first weighted sum of the interpolated values for subpixels residing at half unit horizontal and unit vertical locations or a second weighted sum of the interpolated values for subpixels residing at unit horizontal and half unit vertical locations obtained in step a); and c) interpolating a value for a subpixel at a quarter unit horizontal and quarter unit vertical location using a weighted average of at least one pair of values, a first pair comprising the value of a subpixel situated at a half unit horizontal and unit vertical location and the value of a diagonally opposed subpixel situated at a unit horizontal and half unit vertical location, and a second pair comprising the value of a pixel located at a unit horizontal and unit vertical location and the value of a diagonally opposed subpixel situated at a half unit horizontal and half unit vertical location, the first and second 38b pairs being alternative diagonals with respect to the subpixel being interpolated.
According to a fourteenth aspect of the invention, there is provided a method of interpolating an image, the image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, to generate values for subpixels, a subpixel being located at at least one of a fractional horizontal and a fractional vertical location, the fractional horizontal and fractional vertical locations being representable according to the mathematical notation 1/2x, where x is a positive integer in the range 1 to N, 1 /2x representing a particular level of subpixel interpolation and N
representing a maximum level of subpixel interpolation, the method comprising:
a) interpolating values for subpixels situated at 1/2 N1 unit horizontal and unit vertical locations, and for subpixels situated at unit horizontal and 1/2 N1 unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations; and b) interpolating a value for a subpixel situated at a subpixel horizontal and subpixel vertical location using either a first weighted sum of interpolated values for subpixels situated at a vertical location corresponding to that of the subpixel being interpolated or a second weighted sum of interpolated values for subpixels situated at a horizontal location corresponding to that of the subpixel being interpolated.
According to a fifteenth aspect of the invention, there is provided an interpolator for interpolating an image, the image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, the interpolator being configured to generate values for subpixels, a subpixel being located at at least one of a fractional horizontal and a fractional vertical location, the fractional horizontal and fractional vertical locations being representable according to the mathematical notation 1 /2x, where x is a positive integer in the range 1 to N, 1 /2x representing a particular 38c level of subpixel interpolation and N representing a maximum level of subpixel interpolation, wherein the interpolator is configured to:
a) interpolate values for subpixels situated at 1/2 N1 unit horizontal and unit vertical locations, and for subpixels situated at unit horizontal and 1/2N1 unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations;
b) interpolate a value for a subpixel situated at a 1/2 N1 unit horizontal and 1/2 N1 unit vertical location using either a first weighted sum of the interpolated values for subpixels residing at 1/2N1 unit horizontal and unit vertical locations or a second weighted sum of the interpolated values for subpixels residing at unit horizontal and 1/2N1 unit vertical locations obtained in step a); and c) interpolate a value for a subpixel situated at a 1/2 N unit horizontal and unit vertical location using either:
 a weighted average of the value of a first subpixel situated at a 1/2Nm unit horizontal and 1/2N" unit vertical location and the value of a second subpixel situated at a 1/2 Np unit horizontal and 1/2 Nq unit vertical location, or  a weighted average of the value of a pixel situated at a unit horizontal and unit vertical location and the value of a subpixel situated at a 1/2Nm unit horizontal and 1/2 Wn unit vertical location, variables m, n, p and q taking integer values in the range 1 to N such that the respective first and second subpixels or respective pixel and subpixel are located diagonally with respect to the subpixel at 1/2 N unit horizontal and unit vertical location being interpolated.
According to a sixteenth aspect of the invention, there is provided a computer readable medium embodying a computer program for interpolating an image, the image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, to generate values for subpixels, a subpixel being located at at least one of a 38d fractional horizontal and a fractional vertical location, the fractional horizontal and fractional vertical locations being representable according to the mathematical notation 1/2x, where x is a positive integer in the range 1 to N, 1/2x representing a particular level of subpixel interpolation and N representing a maximum level of subpixel interpolation, the computer program comprising:
a) computer program code for interpolating values for subpixels situated at 1/2 N1 unit horizontal and unit vertical locations, and for subpixels situated at unit horizontal and 1/2 N1 unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations;
b) computer program code for interpolating a value for a subpixel situated at a 1/2""1 unit horizontal and 1/2N1 unit vertical location using either a first weighted sum of the interpolated values for subpixels residing at 112"' unit horizontal and unit vertical locations or a second weighted sum of the interpolated values for subpixels residing at unit horizontal and 1/2"1 unit vertical locations obtained in step a); and c) computer program code for interpolating a value for a subpixel situated at a 1/2 N unit horizontal and 1/2 N unit vertical location using either:
 a weighted average of the value of a first subpixel situated at a 1/2"m unit horizontal and 1/2" unit vertical location and the value of a second subpixel situated at a 1/2 Np unit horizontal and 1/2 Nq unit vertical location, or  a weighted average of the value of a pixel situated at a unit horizontal and unit vertical location and the value of a subpixel situated at a 1/2"m unit horizontal and 1/2"" unit vertical location, variables m, n, p and q taking integer values in the range 1 to N such that the respective first and second subpixels or respective pixel and subpixel are _ located diagonally with respect to the subpixel at 1/2 N unit horizontal and unit vertical location being interpolated.
According to a seventeenth aspect of the invention, there is provided an interpolator for subpixel value interpolation of an image, the image comprising pixels arranged in rows and columns and represented by values having a specified dynamic range, the pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, the interpolator 38e being configured to generate values for subpixels, a subpixel being located at at least one of a fractional horizontal and a fractional vertical location, fractional horizontal and fractional vertical locations being representable according to the mathematical notation 1/2x where x is a positive integer in the range 1 to N, 1/2x representing a particular level of subpixel interpolation and N representing a maximum level of subpixel interpolation, the interpolator being configured to:
a) interpolate values for subpixels situated at 1/2 N1 unit horizontal and unit vertical locations, and for subpixels situated at unit horizontal and 1/2N1 unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations; and b) interpolate a value for a subpixel situated at a subpixel horizontal and subpixel vertical location using either a first weighted sum of interpolated values for subpixels situated at a vertical location corresponding to that of the subpixel being interpolated or a second weighted sum of interpolated values for subpixels situated at a horizontal location corresponding to that of the subpixel being interpolated.
According to an eighteenth aspect of the invention, there is provided a method for subpixel value interpolation to determine values for subpixels situated within a rectangular bounded region defined by four corner pixels with no intermediate pixels between the corners, the pixels and subpixels being arranged in rows and columns, the pixel and subpixel locations being representable mathematically within the rectangular bounded region using the coordinate notation K/2N, L/2N, K
and L being positive integers having respective values between zero and 2N, N
being a positive integer greater, than one and representing a particular degree of subpixel value interpolation, the method comprising:
 interpolating a subpixel value for a subpixel having coordinates with odd values of K and L, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 1/2, 1/2, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with even values of K
and L, including zero, situated within a quadrant of the rectangular bounded 38f region, the quadrant being defined by the subpixel having coordinates 1/2, 1 /2 and the nearest neighbouring pixel;
interpolating subpixel values for subpixels having coordinates with K equal to an even value and L equal to zero and subpixels having coordinates with K equal to zero and L equal to an even value, used in the interpolation of the subpixels having coordinates with odd values of K and L, using weighted sums of the values of pixels located in rows and columns respectively; and interpolating subpixel values for subpixels having coordinates with even values of K and L, used in the interpolation of subpixel values for the subpixels having coordinates with odd values of K and L, using a choice of either a weighted sum of the values of subpixels having coordinates with K
equal to an even value and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the values of subpixels having coordinates with K equal to zero and L equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
According to a nineteenth aspect of the invention there is provided a method for quarter resolution subpixel value interpolation to determine values for subpixels situated within a rectangular bounded region defined by four corner pixels with no intermediate pixels between the corners, the pixels and subpixels being arranged in rows and columns, the pixel and subpixel locations being representable mathematically within the rectangular bounded region using the coordinate notation K/4, L/4, K and L being positive integers having respective values between zero and 4, the method comprising:
 interpolating a subpixel value for a subpixel having coordinates with K
and L equal to 1 or 3, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 2/4, 2/4, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with K and L equal to 38g zero, 2 or 4, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 2/4, 2/4 and the nearest neighbouring pixel;
interpolating subpixel values for subpixels having coordinates with K equal to 2 and L equal to zero and subpixels having coordinates with K equal to zero and L equal to 2, used in the interpolation of the subpixels having coordinates with K and L equal to 1 or 3, using weighted sums of the values of pixels located in rows and columns respectively; and interpolating a subpixel value for the subpixel having coordinates with K
and L equal to 2, used in the interpolation of subpixel values for the subpixels having coordinates with K and L equal to 1 or 3, using a choice of either a weighted sum of the values of the subpixel having coordinates with K equal to 2 and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the value of the subpixel having coordinates with K equal to zero and L equal to 2 and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
According to a twentieth aspect of the invention there is provided a method for eighth resolution subpixel value interpolation to determine values for subpixels situated within a rectangular bounded region defined by four corner pixels with no intermediate pixels between the corners, the pixels and subpixels being arranged in rows and columns, the pixel and subpixel locations being representable mathematically within the rectangular bounded region using the coordinate notation K/8, L/8, K and L being positive integers having respective values between zero and 8, the method comprising:
 interpolating a subpixel value for a subpixel having coordinates with K
and L equal to any of 1, 3, 5 or 7, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 4/8, 4/8, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with K
38h and L equal to any of zero, 2, 4, 6 or 8, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 4/8, 4/8 and the nearest neighbouring pixel;
 interpolating subpixel values for subpixels having coordinates with K
equal to any of 2, 4 or 6, and L equal to zero and subpixels having coordinates with K equal to zero and L equal to any of 2, 4 or 6, used in the interpolation of the subpixels having coordinates with K and L equal to any of 1, 3, 5 or 7, using weighted sums of the values of pixels located in rows and columns respectively; and  interpolating subpixel values for subpixels having coordinates with K and L
equal to any of 2, 4 or 6, used in the interpolation of subpixel values for the subpixels having coordinates with K and L equal to any of 1, 3, 5 or 7, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to any of 2, 4 or 6, and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions or a weighted sum of the values of subpixels having coordinates with K equal to zero and L equal to any of 2, 4 or 6 and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
According to a twentyfirst aspect of the invention there is provided an interpolator for subpixel value interpolation, the interpolator being configured to determine values for subpixels situated within a rectangular bounded region defined by four corner pixels with no intermediate pixels between the corners, the pixels and subpixels being arranged in rows and columns, the pixel and subpixel locations being representable mathematically within the rectangular bounded region using the coordinate notation K/2N, L/2N, K and L being positive integers having respective values between zero and 2N, N being a positive integer greater than one and representing a particular degree of subpixel value interpolation, the interpolator being configured to:
38i interpolate a subpixel value for a subpixel having coordinates with odd values of K and L, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 1/2, 1/2, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with even values of K
and L, including zero, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 1/2, 1/2 and the nearest neighbouring pixel;
interpolate subpixel values for subpixels having coordinates with K equal to an even value and L equal to zero and subpixels having coordinates with K equal to zero and L equal to an even value, used in the interpolation of the subpixels having coordinates with odd values of K and L, using weighted sums of the values of pixels located in rows and columns respectively; and interpolate subpixel values for subpixels having coordinates with even values of K and L, used in the interpolation of subpixel values for the subpixels having coordinates with odd values of K and L, using a choice of either a weighted sum of the values of subpixels having coordinates with K
equal to an even value and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the values of subpixels having coordinates with K equal to zero and L equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
According to a twentysecond aspect of the invention there is provided an interpolator for subpixel value interpolation, the interpolator being configured to determine values for subpixels situated within a rectangular bounded region defined by four corner pixels with no intermediate pixels between the corners, the pixels and subpixels being arranged in rows and columns, the pixel and subpixel locations being representable mathematically within the rectangular bounded region using the coordinate notation K/2N, L/2N, K and L being positive integers 38j having respective values between zero and 2N, N being a positive integer greater than one and representing a particular degree of subpixel value interpolation, the interpolator comprising:
 circuitry configured to interpolate a subpixel value for a subpixel having coordinates with odd values of K and L, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 1/2, 1/2, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with even values of K and L, including zero, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 1/2, 1 /2 and the nearest neighbouring pixel;
 circuitry configured to interpolate subpixel values for subpixels having coordinates with K equal to an even value and L equal to zero and subpixels having coordinates with K equal to zero and L equal to an even value, used in the interpolation of the subpixels having coordinates with odd values of K
and L, using weighted sums of the values of pixels located in rows and columns respectively; and  circuitry configured to interpolate subpixel values for subpixels having coordinates with even values of K and L, used in the interpolation of subpixel values for the subpixels having coordinates with odd values of K and L, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to an even value and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the values of subpixels having coordinates with K equal to zero and L equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
38k According to a twentythird aspect of the invention there is provided an interpolator for quarter resolution subpixel value interpolation, the interpolator being configured to determine values for subpixels situated within a rectangular bounded region defined by four corner pixels with no intermediate pixels between the corners, the pixels and subpixels being arranged in rows and columns, the pixel and subpixel locations being representable mathematically within the rectangular bounded region using the coordinate notation K/4, L/4, K and L
being positive integers having respective values between zero and 4, the interpolator being configured to:
 interpolate a subpixel value for a subpixel having coordinates with K and L
equal to 1 or 3, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 2/4, 2/4, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with K and L equal to zero, 2 or 4, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 2/4, 2/4 and the nearest neighbouring pixel;
 interpolate subpixel values for subpixels having coordinates with K equal to 2 and L equal to zero and subpixels having coordinates with K equal to zero and L equal to 2, used in the interpolation of the subpixels having coordinates with K and L equal to 1 or 3, using weighted sums of the values of pixels located in rows and columns respectively; and  interpolate a subpixel value for the subpixel having coordinates with K
and L equal to 2, used in the interpolation of subpixel values for the subpixels having coordinates with K and L equal to 1 or 3, using a choice of .either a weighted sum of the values of the subpixel having coordinates with K equal to 2 and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the value of the subpixel having coordinates with K equal to zero and L
equal to 2 and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
According to a twentyfourth aspect of the invention there is provided an interpolator for eighth resolution subpixel value interpolation, the interpolator being configured to determine values for subpixels situated within a rectangular bounded region defined by four corner pixels with no intermediate pixels between the corners, the pixels and subpixels being arranged in rows and columns, the pixel and subpixel locations being representable mathematically within the rectangular bounded region using the coordinate notation K/8, L/8, K and L
being positive integers having respective values between zero and 8, the interpolator being configured to:
 interpolate a subpixel value for a subpixel having coordinates with K and L
equal to any of 1, 3, 5 or 7, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 4/8, 4/8, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with K
and L equal to any of zero, 2, 4, 6 or 8, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 4/8, 4/8 and the nearest neighbouring pixel;
interpolate subpixel values for subpixels having coordinates with K equal to any of 2, 4 or 6 and L equal to zero and subpixels having coordinates with K equal to zero and L equal to any of 2, 4 or 6, used in the interpolation of the subpixels having coordinates with K and L equal to any of 1, 3, 5 or 7, using weighted sums of the values of pixels located in rows and columns respectively; and  interpolate subpixel values for subpixels having coordinates with K and L
equal to any of 2,. 4 or 6, used in the interpolation of subpixel values for the subpixels having coordinates with K and L equal to any of 1, 3, 5 or 7, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to any of 2, 4 or 6 and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions or a weighted sum of the values of subpixels having coordinates with K equal to zero and L equal to any of 2, 4 or 6 and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
38m According to a twentyfifth aspect of the invention there is provided a computer readable medium embodying a computer program for subpixel value interpolation, the computer program being configured to determine values for subpixels situated within a rectangular bounded region defined by four corner pixels with no intermediate pixels between the corners, the pixels and subpixels being arranged in rows and columns, the pixel and subpixel locations being representable mathematically within the rectangular bounded region using the coordinate notation K/2N, L/2N, K and L being positive integers having respective values between zero and 2N, N being a positive integer greater than one and representing a particular degree of subpixel value interpolation, the computer program comprising:
 computer program code for interpolating a subpixel value for a subpixel having coordinates with odd values of K and L, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 1/2, 1/2, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with even values of K and L, including zero, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 1/2, 1/2 and the nearest neighbouring pixel;
 computer program code for interpolating subpixel values for subpixels having coordinates with K equal to an even value and L equal to zero and subpixels having coordinates with K equal to zero and L equal to an even value, used in the interpolation of the subpixels having coordinates with odd values of K and L, using weighted sums of the values of pixels located in rows and columns respectively; and  computer program code for interpolating subpixel values for subpixels having coordinates with even values of K and L, used in the interpolation of subpixel values for the subpixels having coordinates with odd values of K
and L, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to an even value and L equal to zero and 38n the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the values of subpixels having coordinates with K equal to zero and L equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
According to a twentysixth aspect of the invention there is provided a network apparatus comprising an interpolator for subpixel value interpolation, the interpolator being configured to determine values for subpixels situated within a rectangular bounded region defined by four corner pixels with no intermediate pixels between the corners, the pixels and subpixels being arranged in rows and columns, the pixel and subpixel locations being representable mathematically within the rectangular bounded region using the coordinate notation K/2N, L/2N, K
and L being positive integers having respective values between zero and 2N, N
being a positive integer greater than one and representing a particular degree of subpixel value interpolation, the interpolator being configured to:
interpolate a subpixel value for a subpixel having coordinates with odd values of K and L, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 1/2, 1/2, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with even values of K
and L, including zero, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 1/2, 1 /2 and the nearest neighbouring pixel;
 interpolate subpixel values for subpixels having coordinates with K
equal to an even value and L equal to zero and subpixels having coordinates with K equal to zero and L equal to an even value, used in the interpolation of the subpixels having coordinates with odd values of K and L, using weighted sums of the values of pixels located in rows and columns respectively; and  interpolate subpixel values for subpixels having coordinates with even values of K and L, used in the interpolation of subpixel values for the subpixels having coordinates with odd values of K and L, using a choice of 38o either a weighted sum of the values of subpixels having coordinates with K
equal to an even value and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the values of subpixels having coordinates with K equal to zero and L equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions Brief Description of the Figures An embodiment of the invention will now be described by way of example only with reference to the accompanying drawings in which:
Figure 1 shows a video encoder according to the prior art;
Figure 2 shows a video decoder according to the prior art;
Figure 3 shows the types of frames used in video encoding;
Figures 4a, 4b, and 4c show steps in blockmatching;
Figure 5 illustrates the process of motion estimation to subpixel resolution;
Figure 6 shows a terminal device comprising video encoding and decoding equipment in which the method of the invention may be implemented;
Figure 7 shows a video encoder according an embodiment of the present invention;
Figure 8 shows a video decoder according to an embodiment of the present invention;
Figures 9 and 10 have not been used and any such figures should be disregarded;
Figure 11 shows a schematic diagram of a mobile telecommunications network according to an embodiment of the present invention;
Figure 12a shows a notation for describing pixel and subpixel locations specific to TML5;
Figure 12b shows interpolation of a half resolution subpixels;
Figure 12c shows interpolation of a half resolution subpixels;
Figure 13a shows a notation for describing pixel and subpixel locations specific to TML6;
Figure 13b shows interpolation of a half resolution subpixels;
Figure 13c shows interpolation of a half resolution subpixels;
Figure 14 shows a notation for describing pixel and subpixel locations specific to the invention;
Figure 14b shows interpolation of a half resolution subpixels according to the invention;
Figure 14c shows interpolation of a half resolution subpixels according to the invention;
Figure 15 shows possible choices of diagonal interpolation for subpixels;
Figure 16 shows the half resolution subpixel values required to calculate other half resolution subpixel values;
Figure 17a shows the half resolution subpixel values that must be calculated in order to interpolate values for quarter resolution subpixels in an image block using the interpolation method of TML5;
Figure 17b shows the half resolution subpixel values that must be calculated in order to interpolate values for quarter resolution subpixels in an image block using the interpolation method according to the invention;
Figures 18a shows the numbers of half resolution subpixels that must be calculated in order to obtain values for quarter resolution subpixels within an image block using the subpixel value interpolation method according to TML5;
Figure 18b shows the numbers of half resolution subpixels that must be calculated in order to obtain values for quarter resolution subpixels within an image block using the subpixel value interpolation method according to the invention;
Figure 19 shows a numbering scheme for each of the 15 subpixel positions;
Figure 20 shows nomenclature used to describe pixels, half resolution subpixels, quarter resolution subpixels and eighth resolution subpixels Figure 21a shows the diagonal direction to be used in the interpolation of each eighth resolution subpixel in an embodiment of the invention;
Figure 21b shows the diagonal direction to be used in the interpolation of each eighth resolution subpixel in another embodiment of the invention;
and Figure 22 shows nomenclature used to describe eighth resolution subpixels 5 within an image.
Detailed Description Figures 1 to 5, 12a, 12b, 12c, 13a, 13b, and 13c have been described in the 10 foregoing.
Figure 6 presents a terminal device comprising video encoding and decoding equipment which may be adapted to operate in accordance with the present invention. More precisely, the figure illustrates a multimedia 15 terminal 60 implemented according to ITUT recommendation H.324. The terminal can be regarded as a multimedia transceiver device. It includes elements that capture, encode and multiplex multimedia data streams for transmission via a communications network, as well as elements that receive, demultiplex, decode and display received multimedia content. ITU20 T recommendation H.324 defines the overall operation of the terminal and refers to other recommendations that govern the operation of its various constituent parts. This kind of multimedia terminal can be used in realtime applications such as conversational videotelephony, or non realtime applications such as the retrieval/streaming of video clips, for example from 25 a multimedia content server in the Internet.
In the context of the present invention, it should be appreciated that the H.324 terminal shown in Figure 6 is only one of a number of alternative multimedia terminal implementations suited to application of the inventive 30 method. It should also be noted that a number of alternatives exist relating to the location and implementation of the terminal equipment. As illustrated in Figure 6, the multimedia terminal may be located in communications equipment connected to a fixed line telephone network such as an analogue PSTN (Public Switched Telephone Network). In this case the multimedia terminal is equipped with a modem 71, compliant with ITUT
recommendations V.8, V.34 and optionally V.8bis. Alternatively, the multimedia terminal may be connected to an external modem. The modem enables conversion of the multiplexed digital data and control signals produced by the multimedia terminal into an analogue form suitable for transmission over the PSTN. It further enables the multimedia terminal to receive data and control signals in analogue form from the PSTN and to convert them into a digital data stream that can be demulitplexed and processed in an appropriate manner by the terminal.
An H.324 multimedia terminal may also be implemented in such a way that it can be connected directly to a digital fixed line network, such as an ISDN
(Integrated Services Digital Network). In this case the modem 71 is replaced with an ISDN usernetwork interface. In Figure 6, this ISDN usernetwork interface is represented by alternative block 72.
H.324 multimedia terminals may also be adapted for use in mobile communication applications. If used with a wireless communication link, the modem 71 can be replaced with any appropriate wireless interface, as represented by alternative block 73 in Figure 6. For example, an H.324/M
multimedia terminal can include a radio transceiver enabling connection to the current 2nd generation GSM mobile telephone network, or the proposed 3rd generation UMTS (Universal Mobile Telephone System).
It should be noted that in multimedia terminals designed for twoway communication, that is for transmission and reception of video data, it is advantageous to provide both a video encoder and video decoder implemented according to the present invention. Such an encoder and decoder pair is often implemented as a single combined functional unit, referred to as a 'codec'.
Because a video encoder according to the invention performs motion compensated video encoding to subpixel resolution using a specific interpolation scheme and a particular combination of beforehand and ondemand subpixel value interpolation, it is generally necessary for a video decoder of a receiving terminal to be implemented in a manner compatible with the encoder of the transmitting terminal which formed the compressed video data stream. Failure to ensure this compatibility may have an adverse effect on the quality of the motion compensation and the accuracy of reconstructed video frames.
A typical H.324 multimedia terminal will now be described in further detail with reference to Figure 6.
The multimedia terminal 60 includes a variety of elements referred to as 'terminal equipment'. This includes video, audio and telematic devices, denoted generically by reference numbers 61, 62 and 63, respectively. The video equipment 61 may include, for example, a video camera for capturing video images, a monitor for displaying received video content and optional video processing equipment. The audio equipment 62 typically includes a microphone, for example for capturing spoken messages, and a loudspeaker for reproducing received audio content. The audio equipment may also include additional audio processing units. The telematic equipment 63, may include a data terminal, keyboard, electronic whiteboard or a still image transceiver, such as a fax unit.
The video equipment 61 is coupled to a video codec 65. The video codec 65 comprises a video encoder and a corresponding video decoder both implemented according to the invention. Such an encoder and a decoder will be described in the following. The video codec 65 is responsible for encoding captured video data in an appropriate form for further transmission over a communications link and decoding compressed video content received from the communications network. In the example illustrated in Figure 6, the video codec is implemented according to ITUT
recommendation H.263, with appropriate modifications to implement the subpixel value interpolation method according to the invention in both the encoder and the decoder of the video codec.
Similarly, the terminal's audio equipment is coupled to an audio codec, denoted in Figure 6 by reference number 66. Like the video codec, the audio codec comprises an encoder/decoder pair. It converts audio data captured by the terminal's audio equipment into a form suitable for transmission over the communications link and transforms encoded audio data received from the network back into a form suitable for reproduction, for example on the terminal's loudspeaker. The output of the audio codec is passed to a delay block 67. This compensates for the delays introduced by the video coding process and thus ensures synchronisation of audio and video content.
The system control block 64 of the multimedia terminal controls endtonetwork signalling using an appropriate control protocol (signalling block 68) to establish a common mode of operation between a transmitting and a receiving terminal. The signalling block 68 exchanges information about the encoding and decoding capabilities of the transmitting and receiving terminals and can be used to enable the various coding modes of the video encoder. The system control block 64 also controls the use of data encryption. Information regarding the type of encryption to be used in data transmission is passed from encryption block 69 to the multiplexer/demultiplexer (MUX/DMUX unit) 70.
During data transmission from the multimedia terminal, the MUX/DMUX unit 70 combines encoded and synchronised video and audio streams with data input from the telematic equipment 63 and possible control data, to form a single bitstream. Information concerning the type of data encryption (if any) to be applied to the bitstream, provided by encryption block 69, is used to select an encryption mode. Correspondingly, when a multiplexed and possibly encrypted multimedia bitstream is being received, MUX/DMUX unit 70 is responsible for decrypting the bitstream, dividing it into its constituent multimedia components and passing those components to the appropriate codec(s) and/or terminal equipment for decoding and reproduction.
It should be noted that the functional elements of the multimedia terminal, video encoder, decoder and video codec according to the invention can be implemented as software or dedicated hardware, or a combination of the two. The video encoding and decoding methods according to the invention are particularly suited for implementation in the form of a computer program comprising machinereadable instructions for performing the functional steps of the invention. As such, the encoder and decoder according to the invention may be implemented as software code stored on a storage medium and executed in a computer, such as a personal desktop computer, in order to provide that computer with video encoding and/or decoding functionality.
If the multimedia terminal 60 is a mobile terminal, that is if it is equipped with a radio transceiver 73, it will be understood by those skilled in the art that it may also comprise additional elements. In one embodiment it comprises a user interface having a display and a keyboard, which enables operation of the multimedia terminal 60 by a user, together with necessary functional blocks including a central processing unit, such as a microprocessor, which controls the blocks responsible for different functions of the multimedia terminal, a random access memory RAM, a read only memory ROM, and a digital camera. The microprocessor's operating instructions, that is program code corresponding to the basic functions of the multimedia terminal 60, is stored in the readonly memory ROM and can be executed as required by the microprocessor, for example under control of the user. In accordance with the program code, the microprocessor uses the radio transceiver 73 to form a connection with a mobile communication network, enabling the multimedia terminal 60 to transmit information to and receive information from the mobile communication network over a radio path.
The microprocessor monitors the state of the user interface and controls the digital camera. In response to a user command, the microprocessor instructs the camera to record digital images into the RAM. Once an image is captured, or alternatively during the capturing process, the 5 microprocessor segments the image into image segments (for example macroblocks) and uses the encoder to perform motion compensated encoding for the segments in order to generate a compressed image sequence as explained in the foregoing description. A user may command the multimedia terminal 60 to display the captured images on its display or 10 to send the compressed image sequence using the radio transceiver 73 to another multimedia terminal, a video telephone connected to a fixed line network (PSTN) or some other telecommunications device. In a preferred embodiment, transmission of image data is started as soon as the first segment is encoded so that the recipient can start a corresponding 15 decoding process with a minimum delay.
Figure 11 is a schematic diagram of a mobile telecommunications network according to an embodiment of the invention. Multimedia terminals MS are in communication with base stations BTS by means of a radio link. The 20 base stations BTS are further connected, through a socalled Abis interface, to a base station controller BSC, which controls and manages several base stations.
The entity formed by a number of base stations BTS (typically, by a few 25 tens of base stations) and a single base station controller BSC, controlling the base stations, is called a base station subsystem BSS. Particularly, the base station controller BSC manages radio communication channels and handovers. The base station controller BSC is also connected, through a socalled A interface, to a mobile services switching centre MSC, which co30 ordinates the formation of connections to and from mobile stations. A
further connection is made, through the mobile service switching centre MSC, to outside the mobile communications network. Outside the mobile communications network there may further reside other network(s) connected to the mobile communications network by gateway(s) GTW, for example the Internet or a Public Switched Telephone Network (PSTN). In such an external network, or within the telecommunications network, there may be located video decoding or encoding stations, such as computers PC. In an embodiment of the invention, the mobile telecommunications network comprises a video server VSRVR to provide video data to a MS
subscribing to such a service. The video data is compressed using the motion compensated video compression method as described in the foregoing. The video server may function as a gateway to an online video source or it may comprise previously recorded video clips. Typical videotelephony applications may involve, for example, two mobile stations or one mobile station MS and a videotelephone connected to the PSTN, a PC connected to the Internet or a H.261 compatible terminal connected either to the Internet or to the PSTN.
Figure 7 shows a video encoder 700 according to an embodiment the invention. Figure 8 shows a video decoder 800 according to an embodiment the invention.
The encoder 700 comprises an input 701 for receiving a video signal from a camera or other video source (not shown). It further comprises a DCT
transformer 705, a quantiser 706, an inverse quantiser 709, an inverse DCT
transformer 710, combiners 712 and 716, a beforehand subpixel interpolation block 730, a frame store 740 and an ondemand subpixel interpolation block 750, implemented in combination with motion estimation block 760. The encoder also comprises a motion field coding block 770 and a motion compensated prediction block 780. Switches 702 and 714 are operated cooperatively by a control manager 720 to switch the encoder between an INTRAmode of video encoding and an INTERmode of video encoding. The encoder 700 also comprises a multiplexer unit (MUX/DMUX) 790 to form a single bitstream from the various types of information produced by the encoder 700 for further transmission to a remote receiving terminal, or for example for storage on a mass storage medium such as a computer hard drive (not shown).
It should be noted that the presence and implementations of beforehand subpixel interpolation block 730 and ondemand subpixel value interpolation block 750 in the encoder architecture depend on the way in which the subpixel interpolation method according to the invention is applied. In embodiments of the invention in which beforehand subpixel value interpolation is not performed, encoder 700 does not comprise beforehand subpixel value interpolation block 730. In other embodiments of the invention, only beforehand subpixel interpolation is performed and thus the encoder does not include ondemand subpixel value interpolation block 750. In embodiments in which both beforehand and ondemand subpixel value interpolation are performed, both blocks 730 and 750 are present in the encoder 700.
Operation of the encoder 700 according to the invention will now be described in detail. In the description, it will be assumed that each frame of uncompressed video, received from the video source at the input 701, is received and processed on a macroblockbymacroblock basis, preferably in rasterscan order. It will further be assumed that when the encoding of a new video sequence starts, the first frame of the sequence is encoded in INTRAmode. Subsequently, the encoder is programmed to code each frame in INTERformat, unless one of the following conditions is met: 1) it is judged that the current frame being coded is so dissimilar from the reference frame used in its prediction that excessive prediction error information is produced; 2) a predefined INTRA frame repetition interval has expired; or 3) feedback is received from a receiving terminal indicating a request for a frame to be coded in INTRA format.
The occurrence of condition 1) is detected by monitoring the output of the combiner 716. The combiner 716 forms a difference between the current macroblock of the frame being coded and its prediction, produced in the motion compensated prediction block 780. If a measure of this difference (for example a sum of absolute differences of pixel values) exceeds a predetermined threshold, the combiner 716 informs the control manager 720 via a control line 717 and the control manager 720 operates the switches 702 and 714 so as to switch the encoder 700 into INTRA coding mode. Occurrence of condition 2) is monitored by means of a timer or frame counter implemented in the control manager 720, in such a way that if the timer expires, or the frame counter reaches a predetermined number of frames, the control manager 720 operates the switches 702 and 714 to switch the encoder into INTRA coding mode. Condition 3) is triggered if the control manager 720 receives a feedback signal from, for example, a receiving terminal, via control line 718 indicating that an INTRA frame refresh is required by the receiving terminal. Such a condition might arise, for example, if a previously transmitted frame were badly corrupted by interference during its transmission, rendering it impossible to decode at the receiver. In this situation, the receiver would issue a request for the next frame to be encoded in INTRA format, thus reinitialising the coding sequence.
It will further be assumed that the encoder and decoder are implemented in such a way as to allow the determination of motion vectors with a spatial resolution of up to quarterpixel resolution. As will be seen in the following, finer levels of resolution are also possible.
Operation of the encoder 700 in INTRA coding mode will now be described.
In INTRAmode, the control manager 720 operates the switch 702 to accept video input from input line 719. The video signal input is received macroblock by macroblock from input 701 via the input line 719 and each macroblock of original image pixels is transformed into DCT coefficients by the DCT transformer 705. The DCT coefficients are then passed to the quantiser 706, where they are quantised using a quantisation parameter QP. Selection of the quantisation parameter QP is controlled by the control manager 720 via control line 722. Each DCT transformed and quantised macroblock that makes up the INTRA coded image information 723 of the frame is passed from the quantiser 706 to the MUX/DMUX 790. The MUX/DMUX 790 combines the INTRA coded image information with possible control information (for example header data, quantisation parameter information, error correction data etc.) to form a single bitstream of coded image information 725. Variable length coding (VLC) is used to reduce redundancy of the compressed video bitstream, as is known to those skilled in the art.
A locally decoded picture is formed in the encoder 700 by passing the data output by the quantiser 706 through inverse quantiser 709 and applying an inverse DCT transform 710 to the inversequantised data. The resulting data is then input to the combiner 712. In INTRA mode, switch 714 is set so that the input to the combiner 712 via the switch 714 is set to zero. In this way, the operation performed by the combiner 712 is equivalent to passing the decoded image data formed by the inverse quantiser 709 and the inverse DCT transform 710 unaltered.
In embodiments of the invention in which beforehand subpixel value interpolation is performed, the output from combiner 712 is applied to beforehand subpixel interpolation block 730. The input to the beforehand subpixel value interpolation block 730 takes the form of decoded image blocks. In the beforehand subpixel value interpolation block 730, each decoded macroblock is subjected to subpixel interpolation in such a way that a predetermined subset of subpixel resolution subpixel values is calculated according to the interpolation method of the invention and is stored together with the decoded pixel values in frame store 740.
In embodiments in which beforehand subpixel interpolation is not performed, beforehand subpixel interpolation block is not present in the encoder architecture and the output from combiner 712, comprising decoded image blocks, is applied directly to frame store 740.
As subsequent macroblocks of the current frame are received and undergo the previously described coding and decoding steps in blocks 705, 706, 709, 710, 712, a decoded version of the INTRA frameis built up in the frame store 740. When the last macroblock of the current frame has been INTRA
5 coded and subsequently decoded, the frame store 740 contains a completely decoded frame, available for use as a prediction reference frame in coding a subsequently received video frame in INTER format. In embodiments of the invention in which beforehand subpixel value interpolation is performed, the reference frame held in frame store 740 is at 10 least partially interpolated to subpixel resolution.
Operation of the encoder 700 in INTER coding mode will now be described.
In INTER coding mode, the control manager 720 operates switch 702 to receive its input from line 721, which comprises the output of the combiner 15 716. The combiner 716 forms prediction error information representing the difference between the current macroblock of the frame being coded and its prediction, produced in the motion compensated prediction block 780. The prediction error information is DCT transformed in block 705 and quantised in block 706 to form a macroblock of DCT transformed and quantised 20 prediction error information. Each macroblock of DCT transformed and quantised prediction error information is passed from the quantiser 706 to the MUX/DMUX unit 790. The MUX/DMUX unit 790 combines the prediction error information 723 with motion coefficients 724 (described in the following) and control information (for example header data, quantisation 25 parameter information, error correction data etc.) to form a single bitstream of coded image information, 725.
Locally decoded prediction error information for the each macroblock of the INTER coded frame is then formed in the encoder 700 by passing the 30 encoded prediction error information 723 output by the quantiser 706 through the inverse quantiser 709 and applying an inverse DCT transform in block 710. The resulting locally decoded macroblock of prediction error information is then input to combiner 712. In INTERmode, switch 714 is set so that the combiner 712 also receives motion predicted macroblocks for the current INTER frame, produced in the motion compensated prediction block 780. The combiner 712 combines these two pieces of information to produce reconstructed image blocks for the current INTER frame.
As described above when considering INTRA coded frames, in embodiments of the invention in which beforehand subpixel value interpolation is performed, the output from combiner 712 is applied to the beforehand subpixel interpolation block 730. Thus, the input to the beforehand subpixel value interpolation block 730 in INTER coding mode also takes the form of decoded image blocks. In the beforehand subpixel value interpolation block 730, each decoded macroblock is subjected to subpixel interpolation in such a way that a predetermined subset of subpixel values is calculated according to the interpolation method of the invention and is stored together with the decoded pixel values in frame store 740. In embodiments in which beforehand subpixel interpolation is not performed, beforehand subpixel interpolation block is not present in the encoder architecture and the output from combiner 712, comprising decoded image blocks, is applied directly to frame store 740.
As subsequent macroblocks of the video signal are received from the video source and undergo the previously described coding and decoding steps in blocks 705, 706, 709, 710, 712, a decoded version of the INTER frame is built up in the frame store 740. When the last macroblock of the frame has been INTER coded and subsequently decoded, the frame store 740 contains a completely decoded frame, available for use as a prediction reference frame in encoding a subsequently received video frame in INTER
format. In embodiments of the invention in which beforehand subpixel value interpolation is performed, the reference frame held in frame store 740 is at least partially interpolated to subpixel resolution.
Formation of a prediction for a macroblock of the current frame will now be described.
Any frame encoded in INTER format requires a reference frame for motion compensated prediction. This means, inter alia, that when encoding a video sequence, the first frame to be encoded, whether it is the first frame in the sequence, or some other frame, must be encoded in INTRA format. This, in turn, means that when the video encoder 700 is switched into INTER coding mode by control manager 720, a complete reference frame, formed by locally decoding a previously encoded frame, is already available in the frame store 740 of the encoder. In general, the reference frame is formed by locally decoding either an INTRA coded frame or an INTER coded frame.
The first step in forming a prediction for a macroblock of the current frame is performed by motion estimation block 760. The motion estimation block 760 receives the current macroblock of the frame being coded via line 727 and performs a block matching operation in order to identify a region in the reference frame which corresponds substantially with the current macroblock. According to the invention, the blockmatching process is performed to subpixel resolution in a manner that depends on the implementation of the encoder 700 and the degree of beforehand subpixel interpolation performed. However, the basic principle behind the blockmatching process is similar in all cases. Specifically, motion estimation block 760 performs blockmatching by calculating difference values (e.g.
sums of absolute differences) representing the difference in pixel values between the macroblock of the current frame under examination and candidate bestmatching regions of pixels / subpixels in the reference frame. A difference value is produced for all possible offsets (e.g. quarteror one eighth subpixel precision x, y displacements) between the macroblock of the current frame and candidate test region within a predefined search region of the reference frame and motion estimation block 760 determines the smallest calculated difference value. The offset between the macroblock in the current frame and the candidate test region of pixel values / subpixel values in the reference frame that yields the smallest difference value defines the motion vector for the macroblock in question. In certain embodiments of the invention, an initial estimate for the motion vector having unit pixel precision is first determined and then refined to a finer level of subpixel precision, as described in the foregoing.
In embodiments of the encoder in which beforehand subpixel value interpolation is not performed, all subpixel values required in the block matching process are calculated in ondemand subpixel value interpolation block 750. Motion estimation block 760 controls ondemand subpixel value interpolation block 750 to calculate each subpixel value needed in the blockmatching process in an ondemand fashion, as and when it is required. In this case, motion estimation block 760 may be implemented so as to perform blockmatching as a onestep process, in which case a motion vector with the desired subpixel resolution is sought directly, or it may be implemented so as to perform blockmatching as a two step process. If the twostep process is adopted, the first step may comprise a search for e.g. a full or halfpixel resolution motion vector and the second step is performed in order to refine the the motion vector to the desired subpixel resolution.
As block matching is an exhaustive process, in which blocks of n x m pixels in the current frame are compared onebyone with blocks of n x m pixels or subpixels in the interpolated reference frame, it should be appreciated that a subpixel calculated in an ondemand fashion by the ondemand pixel interpolation block 750 may need to be calculated multiple times as successive difference values are determined. In a video encoder, this approach is not the most efficient possible in terms of computational complexity / burden.
In embodiments of the encoder which use only beforehand subpixel value interpolation, blockmatching may be performed as a one step process, as all subpixel values of the reference frame required to determine a motion vector with the desired subpixel resolution are calculated beforehand in block 730 and stored in frame store 740. Thus, they are directly available for use in the blockmatching process and can be retrieved as required from frame store 740 by motion estimation block 760. However, even in the case where all subpixel values are available from frame store 740, it is still more computationally efficient to perform blockmatching as a twostep process, as fewer difference calculations are required. It should be appreciated that while full beforehand subpixel value interpolation reduces computational complexity in the encoder, it is not the most efficient approach in terms of memory consumption.
In embodiments of the encoder in which both beforehand and ondemand subpixel value interpolation are used, motion estimation block 760 is implemented in such a way that it can retrieve subpixel values previously calculated in beforehand subpixel value interpolation block 730 and stored in frame store 740 and further control ondemand subpixel value interpolation block 750 to calculate any additional subpixel values that may be required. The blockmatching process may be performed as a onestep or a twostep process. If a twostep implementation is used, beforehand calculated subpixel values retrieved from frame store 740 may be used in the first step of the process and the second step may be implemented so as to use subpixel values calculated by ondemand subpixel value interpolation block 750. In this case, certain subpixel values used in the second step of the block matching process may need to be calculated multiple times as successive comparisons are made, but the number of such duplicate calculations is significantly less than if beforehand subpixel value calculation is not used. Furthermore, memory consumption is reduced with respect to embodiments in which only beforehand subpixel value interpolation is used.
Once the motion estimation block 760 has produced a motion vector for the macroblock of the current frame under examination, it outputs the motion vector to the motion field coding block 770. Motion field coding block 770 then approximates the motion vector received from motion estimation block 760 using a motion model. The motion model generally comprises a set of basis functions. More specifically, the motion field coding block 770 represents the motion vector as a set of coefficient values (known as motion coefficients) which, when multiplied by the basis functions, form an approximation of the motion vector. The motion coefficients 724 are passed from motion field coding block 770 to motion compensated prediction block 780. Motion compensated prediction block 780 also receives the pixel / sub5 pixel values of the bestmatching candidate test region of the reference frame identified by motion estimation block 760. In Figure 7, these values are shown to be passed via line 729 from ondemand subpixel interpolation block 750. In alternative embodiments of the invention, the pixel values in question are provided from the motion estimation block 760 itself.
Using the approximate representation of the motion vector generated by motion field coding block 770 and the pixel / subpixel values of the bestmatching candidate test region, motion compensated prediction block 780 produces a macroblock of predicted pixel values. The macroblock of predicted pixel values represents a prediction for the pixel values of the current macroblock generated from the interpolated reference frame. The macroblock of predicted pixel values is passed to the combiner 716 where it is subtracted from the new current frame in order to produce prediction error information 723 for the macroblock, as described in the foregoing.
The motion coefficients 724 formed by motion field coding block are also passed to the MUX/DMUX unit 790, where they are combined with prediction error information 723 for the macroblock in question and possible control information from control manager 720 to form an encoded video stream 725 for transmission to a receiving terminal.
Operation of a video decoder 800 according to the invention will now be described. Referring to Figure 8, the decoder 800 comprises a demultiplexing unit (MUX/DMUX) 810, which receives the encoded video stream 725 from the encoder 700 and demultiplexes it, an inverse quantiser 820, an inverse DCT transformer 830, a motion compensated prediction block 840, a frame store 850, a combiner 860, a control manager 870, an output 880, beforehand subpixel value interpolation block 845 and ondemand subpixel interpolation block 890 associated with the motion compensated prediction block 840. In practice the control manager 870 of the decoder 800 and the control manager 720 of the encoder 700 may be the same processor. This may be the case if the encoder 700 and decoder 800 are part of the same video codec.
Figure 8 shows an embodiment in which a combination of beforehand and ondemand subpixel value interpolation is used in the decoder. In other embodiments, only beforehand subpixel value interpolation is used, in which case decoder 800 does not include ondemand subpixel value interpolation block 890. In a preferred embodiment of the invention, no beforehand subpixel value interpolation is used in the decoder and therefore beforehand subpixel value interpolation block 845 is omitted from the decoder architecture. If both beforehand and ondemand subpixel value interpolation are performed, the decoder comprises both blocks 845 and 890.
The control manager 870 controls the operation of the decoder 800 in response to whether an INTRA or an INTER frame is being decoded. An INTRA / INTER trigger control signal, which causes the decoder to switch between decoding modes is derived, for example, from picture type information provided in the header portion of each compressed video frame received from the encoder. The INTRA / INTER trigger control signal is passed to control manager 870 via control line 815, together with other video codec control signals demultiplexed from the encoded video stream 725 by the MUX/DMUX unit 810.
When an INTRA frame is decoded, the encoded video stream 725 is demultiplexed into INTRA coded macroblocks and control information. No motion vectors are included in the encoded video stream 725 for an INTRA
coded frame. The decoding process is performed macroblockbymacroblock. When the encoded information 723 for a macroblock is extracted from video stream 725 by MUX/DMUX unit 810, it is passed to inverse quantiser 820. The control manager controls inverse quantiser 820 to apply a suitable level of inverse quantisation to the macroblock of encoded information, according to control information provided in video stream 725. The inverse quantised macroblock is then inversely transformed in the inverse DCT transformer 830 to form a decoded block of image information. Control manager 870 controls combiner 860 to prevent any reference information being used in the decoding of the INTRA coded macroblock. The decoded block of image information is passed to the video output 880 of the decoder.
In embodiments of the decoder which employ beforehand subpixel value interpolation, the decoded block of image information (i.e. pixel values) produced as a result of the inverse quantisation and inverse transform operations performed in blocks 820 and 830 is passed to beforehand subpixel value interpolation block 845. Here, subpixel value interpolation is performed according to the method of the invention, the degree of beforehand subpixel value interpolation applied being determined by the details of the decoder implementation. In embodiments of the invention in which ondemand subpixel value interpolation is not performed, beforehand subpixel value interpolation block 845 interpolates all subpixelvalues. In embodiments that use a combination of beforehand and ondemand subpixel value interpolation, beforehand subpixel value interpolation block 845 interpolates a certain subset of subpixel values. This may comprise, for example, all subpixels at halfpixel locations, or a combination of subpixels at halfpixel and one quarterpixel locations. In any case, after beforehand subpixel value interpolation, the interpolated subpixel values are stored in frame store 850, together with the original decoded pixel values. As subsequent macroblocks are decoded, beforehand interpolated and stored, a decoded frame, at least partially interpolated to subpixel resolution is progressively assembled in the frame store 850 and becomes available for use as a reference frame for motion compensated prediction.
In embodiments of the decoder which do not employ beforehand subpixel value interpolation, the decoded block of image information (i.e. pixel values) produced as a result of the inverse quantisation and inverse transform operations performed on the macroblock in blocks 820 and 830 is passed directly to frame store 850. As subsequent macroblocks are decoded and stored, a decoded frame, having unit pixel resolution is progressively assembled in the frame store 850 and becomes available for use as a reference frame for motion compensated prediction.
When an INTER frame is decoded, the encoded video stream 725 is demultiplexed into encoded prediction error information 723 for each macroblock of the frame, associated motion coefficients 724 and control information. Again, the decoding process is performed macroblockbymacroblock. When the encoded prediction error information 723 for a macroblock is extracted from the video stream 725 by MUX/DMUX unit 810, it is passed to inverse quantiser 820. Control manager 870 controls inverse quantiser 820 to apply a suitable level of inverse quantisation to the macroblock of encoded prediction error information, according to control information received in video stream 725. The inverse quantised macroblock of prediction error information is then inversely transformed in the inverse DCT transformer 830 to yield decoded prediction error information for the macroblock.
The motion coefficients 724 associated with the macroblock in question are extracted from the video stream 725 by MUX/DMUX unit 810 and passed to motion compensated prediction block 840, which reconstructs a motion vector for the macroblock using the same motion model as that used to encode the INTERcoded macroblock in encoder 700. The reconstructed motion vector approximates the motion vector originally determined by motion estimation block 760 of the encoder. The motion compensated prediction block 840 of the decoder uses the reconstructed motion vector to identify the location of a block of pixel / subpixel values in a prediction reference frame stored in frame store 850. The reference frame may be, for example, a previously decoded INTRA frame, or a previously decoded INTER frame. In either case, the block of pixel / subpixel values indicated by the reconstructed motion vector, represents the prediction of the macroblock in question.
The reconstructed motion vector may point to any pixel or subpixel. If the motion vector indicates that the prediction for the current macroblock is formed from pixel values (i.e. the values of pixels at unit pixel locations), these can simply be retrieved from frame store 850, as the values in question are obtained directly during the decoding of each frame. If the motion vector indicates that the prediction for the current macroblock is formed from subpixel values, these must either be retrieved from frame store 850, or calculated in ondemand subpixel interpolation block 890.
Whether subpixel values must be calculated, or can simply be retrieved from the frame store, depends on the degree of beforehand subpixel value interpolation used in the decoder.
In embodiments of the decoder that do not employ beforehand subpixel value interpolation, the required subpixel values are all calculated in ondemand subpixel value interpolation block 890. On the other hand, in embodiments in which all subpixel values are interpolated beforehand, motion compensated prediction block 840 can retrieve the required subpixel values directly from the frame store 850. In embodiments that use a combination beforehand and ondemand subpixel value interpolation, the action required in order to obtain the required subpixel values depends on which subpixel values are interpolated beforehand. Taking as an example an embodiment in which all subpixel values at halfpixel locations are calculated beforehand, it is evident that if a reconstructed motion vector for a macroblock points to a pixel at unit location or a subpixel at halfpixel location, all the pixel or subpixel values required to form the prediction for the macroblock are present in the frame store 850 and can be retrieved from there by motion compensated prediction block 840. If, however, the motion vector indicates a subpixel at a quarterpixel location, the subpixels required to form the prediction for the macroblock are not present in frame store 850 and are therefore calculated in ondemand subpixel value interpolation block 890. In this case, ondemand subpixel value interpolation block 890 retrieves any pixel or subpixel required to perform 5 the interpolation from frame store 850 and applies the interpolation method described below. Subpixel values calculated in ondemand subpixel value interpolation block 890 are passed to motion compensated prediction block 840.
10 Once a prediction for a macroblock has been obtained, the prediction (that is, a macroblock of predicted pixel values) is passed from motion compensated prediction block 840 to combiner 860 where it combined with the decoded prediction error information for the macroblock to form a reconstructed image block which, in turn, is passed to the video output 880 15 of the decoder.
It should be appreciated that in practical implementations of encoder 700 and decoder 800, the extent to which frames are beforehand subpixel interpolated, and thus the amount of ondemand subpixel value 20 interpolation that is performed, can be chosen according to, or dictated by, the hardware implementation of the video encoder 700, or the environment in which it is intended to be used. For example, if the memory available to the video encoder is limited, or memory must be reserved for other functions, it is appropriate to limit the amount of beforehand subpixel value 25 interpolation that is performed. In other cases, where the microprocessor performing the video encoding operation has limited processing capacity, e.g. the number of operations per second that can be executed is comparatively low, it is more appropriate to restrict the amount of ondemand subpixel value interpolation that is performed. In a mobile 30 communications environment, for example, when video encoding and decoding functionality is incorporated in a mobile telephone or similar wireless terminal for communication with a mobile telephone network, both memory and processing power may be limited. In this case a combination of beforehand and ondemand subpixel value interpolation may be the best choice to obtain an efficient implementation in the video encoder. In video decoder 800, use of beforehand subpixel value is generally not preferred, as it typically results in the calculation of many subpixel values that are not actually used in the decoding process. However, it should be appreciated that although different amounts of beforehand and ondemand interpolation can be used in the encoder and decoder in order to optimise the operation of each, both encoder and decoder can be implemented so as to use the same division between beforehand and ondemand subpixel value interpolation.
Although the foregoing description does not describe the construction of bidirectionally predicted frames (Bframes) in the encoder 700 and the decoder 800, it should be understood that in embodiments of the invention, such a capability may be provided. Provision of such capability is considered within the ability of one skilled in the art.
An encoder 700 or a decoder 800 according to the invention can be realised using hardware or software, or using a suitable combination of both. An encoder or decoder implemented in software may be, for example, a separate program or a software building block that can be used by various programs. In the above description and in the drawings, the functional blocks are represented as separate units, but the functionality of these blocks can be implemented, for example, in one software program unit.
The encoder 700 and decoder 800 can further be combined in order to form a video codec having both encoding and decoding functionality. In addition to being implemented in a multimedia terminal, such a codec may also be implemented in a network. A codec according to the invention may be a computer program or a computer program element, or it may be implemented at least partly using hardware.
The subpixel interpolation method used in the encoder 700 and decoder 800 according to the invention now be described in detail. The method will first be introduced at a general conceptual level and then two preferred embodiments will be described. In the first preferred embodiment, subpixel value interpolation is performed to 1/4 pixel resolution and in the second the method is extended to y pixel resolution.
It should be noted that interpolation must produce identical values in the encoder and the decoder, but its implementation should be optimized for both entities separately. For example, in an encoder according to the first embodiment of the invention in which subpixel value interpolation is preformed to 1/4 pixel resolution, it is most efficient to calculate 'h resolution pixels beforehand and to calculate values for 1/4 resolution subpixels in an ondemand fashion, only when they are needed during motion estimation.
This has the effect of limiting memory usage while keeping the computational complexity / burden at an acceptable level. In the decoder, on the other hand, it is advantageous not to precalculate any of the subpixels. Therefore, it should be appreciated that a preferred embodiment of the decoder does not include beforehand subpixel value interpolation block 845 and all subpixel value interpolation is performed in ondemand subpixel value interpolation block 890.
In the description of the interpolation method provided below, references are made to the pixel positions depicted in Figure 14a. In this figure pixels labelled A represent original pixels (that is, pixels residing at unit horizontal and vertical locations). Pixels labelled with other letters represent subpixels that are to be interpolated. The description that follows will adhere to the previously introduced conventions regarding the description of pixel and subpixel locations.
Next, the steps required to interpolate all subpixel positions are described:
Values for the 1/2 resolution subpixels labelled b are obtained by first calculating an intermediate value b using a Kth order filter, according to:
K
b=E x, A, (9) where x, is a vector of filter coefficients, A, is a corresponding vector of original pixel values A situated at unit horizontal and unit vertical locations, and K is an integer which defines the order of the filter. Thus, equation 9 can be reexpressed as:
b=x1A,+X2A2+X3A3+....+XK1AKI+XKAK (10) The values of the filter coefficients x; and the order of the filter K may vary from embodiment to embodiment. Equally, different coefficient values may be used in the calculation of different subpixels within an embodiment. In other embodiments, the values of filter coefficients x; and the order of the filter may depend on which of the 1/2 resolution b subpixels is being interpolated. Pixels A; are disposed symmetrically with respect to the 1/2 resolution subpixel b being interpolated and are the closest neighbours of that subpixel. In the case of the 1/2 resolution subpixel b situated at half horizontal and unit vertical location, pixels A; are disposed horizontally with respect to b (as shown in Figure 14b). If the 1/2 resolution subpixel b situated at unit horizontal and half vertical location is being interpolated, pixels A; are disposed vertically with respect to b (as shown in Figure 14c).
A final value for 1/2 resolution subpixel b is calculated by dividing intermediate value b by a constant scale,, truncating it to obtain an integer number and clipping the result to lie in the range [0, 2"1]. In alternative embodiments of the invention rounding may be performed instead of truncation. Preferably, constant scale, is chosen to be equal to the sum of filter coefficients x;.
A value for the 1/2 resolution subpixel labelled c is also obtained by first calculating an intermediate value c using an Mth order filter, according to:
M
c=j y1bi X11) where y; is a vector of filter coefficients, b; is a corresponding vector of intermediate values b; in the horizontal or vertical direction. i.e.:
c=y,bl+y2b2+y3b3+.... +yM1bM1+YMbM (12) The values of the filter coefficients y; and the order of the filter M may vary from embodiment to embodiment. Equally, different coefficient values may be used in the calculation of different subpixels within an embodiment.
Preferably, the b values are intermediate values for 1/2 resolution subpixels b which are disposed symmetrically with respect to 1/2 resolution subpixel c and are the closest neighbours of subpixel c. In an embodiment of the invention, the '/2 resolution subpixels b are disposed horizontally with respect to subpixel c, in an alternative embodiment they are disposed vertically with respect to subpixel c.
A final value of 1/2 resolution subpixel c is computed by dividing intermediate value c by a constant scale2, truncating it to obtain an integer number and clipping the result to lie in the range [0, 2n1]. In alternative embodiments of the invention rounding may be performed instead of truncation. Preferably, constant scale2 is equal to scale, * scale,.
It should be noted that the use of intermediate values b in the horizontal direction leads to the same result as using intermediate values b in the vertical direction.
There are two alternatives for interpolating values for the 1/4 resolution subpixels labelled h. Both involve linear interpolation along a diagonal line linking '/2 resolution subpixels neighbouring the 1/4 resolution subpixel h being interpolated. In a first embodiment, a value for subpixel h is calculated by averaging the values of the two 1/2 resolution subpixels b closest to subpixel h. In a second embodiment, a value for subpixel h is calculated by averaging the values of the closest pixel A and the closest 1/2 5 resolution subpixel c. It should be appreciated that this provides the possibility of using different combinations of diagonal interpolations to determine the values for subpixels h within the confines of different groups of 4 image pixels A. However, it should also be realised that the same combination should be used in both the encoder and the decoder in order to 10 produce identical interpolation results. Figure 15 depicts 4 possible choices of diagonal interpolation for subpixels h in adjacent groups of 4 pixels within an image. Simulations in the TML environment have verified that both embodiments result in similar compression efficiency. The second embodiment has higher complexity, since calculation of subpixel c requires 15 calculation of several intermediate values. Therefore the first embodiment is preferred.
Values for 1/4 resolution subpixels labelled d and g are calculated from the values of their nearest horizontal neighbours using linear interpolation. In 20 other words, a value for 1/4 resolution subpixel d is obtained by averaging values of its nearest horizontal neighbours, original image pixel A and 1/2 resolution subpixel b. Similarly, a value for 1/4 resolution subpixel g is obtained by taking the average of its two nearest horizontal neighbours, 1/2 resolution subpixels b and c.
Values fort/4 resolution subpixels labelled e, f and i are calculated from the values of their nearest neighbours in the vertical direction using linear interpolation. More specifically, a value for 1/4 resolution subpixel e is obtained by averaging the values of its two nearest vertical neighbours, original image pixel A and 1/2 resolution subpixel b. Similarly, a value for resolution subpixel f is obtained by taking the average of its two nearest vertical neighbours, 1/2 resolution subpixels b and c. In an embodiment of the invention, a value for 1/4 resolution subpixel i is obtained in manner identical to that just described in connection with 1/4 resolution subpixel f.
However, in an alternative embodiment of the invention, and in common with the H.26 test models TML5 and TML6 previously described, 1/4 resolution subpixel i is determined using the values of the four closest original image pixels, according to (A,+A2+A3+A4+2) / 4.
It should also be noted that in all cases where an average involving pixel and / or subpixel values is determined, the average may be formed in any appropriate manner. For example, the value for 1/4 resolution subpixel d can be defined as d = (A + b) / 2 or as d = (A + b + 1) / 2. The addition of 1 to the sum of values for pixel A and '/2 resolution subpixel b has the effect of causing any rounding or truncation operation subsequently applied to round or truncate the value for d to the next highest integer value. This is true for any sum of integer values and may be applied to any of the averaging operations performed according to the method of the invention in order to control rounding or truncation effects.
It should be noted that the subpixel value interpolation method according to the invention provides advantages over each of TML5 and TML6.
In contrast to TML5, in which the values of some of the 1/4 resolution subpixels depend on previously interpolated values obtained for other 1/4 resolution subpixels, in the method according to the invention, all 1/4 resolution subpixels are calculated from original image pixels or '/2 resolution subpixel positions using linear interpolation. Thus, the reduction in precision of those 1/4 resolution subpixel values which occurs in TML5 due to the intermediate truncation and clipping of the other 1/4 resolution subpixels from which they are calculated, does not take place in the method according to the invention. In particular, referring to Figure 14a, 1/4 resolution subpixels h (and subpixel i in one embodiment of the invention) are interpolated diagonally in order to reduce dependency on other 1/4pixels. Furthermore, in the method according to the invention, the number of calculations (and therefore the number of processor cycles) required to obtain a value for those 1/4 resolution subpixels in the decoder is reduced compared with TML5. Additionally, the calculation of any 1/4 resolution subpixel value requires a number of calculations which is substantially similar to the number of calculations required to determine any other 1/4 resolution subpixel value. More specifically, in a situation where the required 1/2 resolution subpixel values are already available, e.g. they have been calculated beforehand, the number of calculations required to interpolate a 1/4 resolution subpixel value from the precalculated 1/2 resolution subpixel values is the same as the number of calculations required to calculate any other 1/4 resolution subpixel value from the available '/2 resolution subpixel values.
In comparison with TML6, the method according to the invention does not require high precision arithmetic to be used in the calculation of all subpixels. Specifically, as all of the 1/4 resolution subpixel values are calculated from original image pixels or 1/2 resolution subpixel values using linear interpolation, lower precision arithmetic can be used in their interpolation.
Consequently, in hardware implementations of the inventive method, for example in an ASIC (Application Specific Integrated Circuit), the use of lower precision arithmetic reduces the number of components (e.g. gates) that must be devoted to the calculation of 1/4 resolution subpixel values.
This, in turn, reduces the overall area of silicon that must be dedicated to the interpolation function. As the majority of subpixels are, in fact, 1/4 resolution subpixels (12 out of the 15 subpixels illustrated in Figure 14a), the advantage provided by the invention in this respect is particularly significant. In software implementations, where subpixel interpolation is performed using the standard instruction set of a general purpose CPU
(Central Processor Unit) or using a DSP (Digital Signal Processor), a reduction in the precision of the arithmetic required generally leads to an increase in the speed at which the calculations can be performed. This is particularly advantageous in 'low cost' implementations, in which it is desirable to use a general purpose CPU rather than any form of ASIC.
The method according to the invention provides still further advantages compared with TML5. As mentioned previously, in the decoder only 1 out of the 15 subpixel positions is required at any given time, namely that which is indicated by received motion vector information. Therefore, it is advantageous if the value of a subpixel in any subpixel location can be calculated with the minimum number of steps that result in a correctly interpolated value. The method according to the invention provides this capability. As mentioned in the detailed description provided above, '/2 resolution subpixel c can be interpolated by filtering in either the vertical or the horizontal direction, the same value being obtained for c regardless of whether horizontal or vertical filtering is used. The decoder can therefore take advantage of this property when calculating values for 1/4 resolution subpixels f and g in such a way as to minimise the number of operations required in order to obtain the required values. For example, if the decoder requires a value for 1/4 resolution subpixel f, 1/2 resolution subpixel c should interpolated in the vertical direction. If a value is required for 1/4 resolution subpixel g, it is advantageous to interpolate a value for c in the horizontal direction. Thus, in general, it can be said that the method according to the invention provides flexibility in the way in which values are derived for certain 1/4 resolution subpixels. No such flexibility is provided in TML5.
Two specific embodiments will now be described in detail. The first represents a preferred embodiment for calculating subpixels with up to 1/4 pixel resolution, while in the second, the method according to the invention is extended to the calculation of values for subpixels having up toy pixel resolution. For both embodiments a comparison is provided between the computational complexity / burden resulting from use of the method according to the invention and that which would result from use of the interpolation methods according to TML5 and TML6 in equivalent circumstances.
The preferred embodiment for interpolating subpixels at % pixel resolution will be described with reference to Figures 14a, 14b and 14c. In the following, it will be assumed that all image pixels and final interpolated values for subpixels are represented with 8bits.
Calculation of 1/2 resolution subpixels at i) half unit horizontal and unit vertical locations and ii) unit horizontal and half unit vertical locations.
1. A value for the subpixel at half unit horizontal and unit vertical location, that is 1/2 resolution subpixel b in Figure 14a, is obtained by first calculating intermediate value b = (A,  5A2 + 20A3 + 20A4  5A5 + A6) using the values of the six pixels (A, to A6) which are situated at unit horizontal and unit vertical locations in either the row or the column of pixels containing b and which are disposed symmetrically about b, as shown in Figures 14b and 14c. A final value for 1/2 resolution subpixel b is calculated as (b + 16) / 32 where the operator / denotes division with truncation. The result is clipped to lie in the range [0, 255].
Calculation of 1/2 resolution subpixels at half unit horizontal and half unit vertical locations.
2. A value for the subpixel at half unit horizontal and half unit vertical location, that is 1/2 resolution subpixel c in Figure 14a, is calculated as c = (b,  5b2 + 20b3 + 20b4  5b5 + b6 + 512) / 1024 using the intermediate values b for the six closest 1/2 resolution subpixels which are situated in either the row or the column of subpixels containing c and which are disposed symmetrically about c. Again, operator / denotes division with truncation and the result is clipped to lie in the range [0, 255]. As previously explained, using intermediate values b for 1/2 resolution subpixels b in the horizontal direction leads to the same result as using intermediate values b for 1/2 resolution subpixels b in the vertical direction. Thus, in an encoder according to the invention, the direction for interpolating 1/2 resolution subpixels b can be chosen according to a preferred mode of implementation. In a decoder according to the invention, the direction for interpolating subpixels b is chosen according to which, if any, 1/4 resolution subpixels will be interpolated using the result obtained for 1/2 resolution subpixel c.
Calculation of 1/4 resolution subpixels at i) quarter unit horizontal and unit vertical locations; ii) quarter unit horizontal and half unit vertical locations; iii) unit horizontal and quarter unit vertical locations; and iv) half unit horizontal 5 and quarter unit vertical locations.
3. Values for 1/4 resolution subpixels d, situated at quarter unit horizontal and unit vertical locations are calculated according to d = (A + b) / 2 using the nearest original image pixel A and the closest 1/2 resolution subpixel b in the horizontal direction. Similarly, values for 1/4 resolution 10 subpixels g, situated at quarter unit horizontal and half unit vertical locations are calculated according to g = (b + c) / 2 using the two nearest 1/2 resolution subpixels in the horizontal direction. In a similar manner, values for 1/4 resolution subpixels e, situated at unit horizontal and quarter unit vertical locations, are calculated according e = (A + b) / 2 15 using the nearest original image pixel A and the closest 1/2 resolution subpixel b in the vertical direction. Values for 1/4 resolution subpixels f, situated at half unit horizontal and quarter unit vertical locations, are determined from f = (b + c) / 2 using the two nearest 1/2 resolution subpixel in the vertical direction. In all cases, operator / denotes division with 20 truncation.
Calculation of 1/4 resolution subpixels at quarter unit horizontal and quarter unit vertical locations.
4. Values for 1/4 resolution subpixels h, situated at quarter unit horizontal 25 and quarter unit vertical locations are calculated according to h = (b1 +
b2) / 2, using the two nearest 1/2 resolution subpixels b in the diagonal direction. Again, operator / denotes division with truncation.
5. A value for the 1/4 resolution subpixel labeled i is computed from i = (A1 +
30 A2 + A3 + A4 +2 ) / 4 using the four nearest original pixels A. Once more, operator / denotes division with truncation.
An analysis of the computational complexity of the first preferred embodiment of the invention will now be presented.
In the encoder, it is likely that the same subpixel values will be calculated multiple times. Therefore, and as previously explained, the complexity of the encoder can be reduced by precalculating all subpixel values and storing them in memory. However, this solution increases memory usage by a large margin. In the preferred embodiment of the invention, in which motion vector accuracy is 1/4 pixel resolution in both the horizontal and vertical dimensions, storing precalculated subpixel values for the whole image requires 16 times the memory required to store the original, noninterpolated image. To reduce memory usage, all 1/2 resolution subpixels can be interpolated beforehand and 1/4 resolution subpixels can be calculated ondemand, that is, only when they are needed. According to the method of the invention, ondemand interpolation of values for 1/4 resolution subpixels only requires linear interpolation from 1/2 resolution subpixels. Four times the original picture memory is required to store the precalculated 1/2 resolution subpixels since only 8 bits are necessary to represent them.
However, if the same strategy of precalculating all 1/2 resolution subpixels using beforehand interpolation is used in conjunction with the direct interpolation scheme of TML6, the memory requirements increase to 9 times the memory required to store the original noninterpolated image. This results from the fact that a larger number of bits is required to store the high precision intermediate values associated with each 1/2 resolution subpixel in TML6. In addition, the complexity of subpixel interpolation during motion estimation is higher in TML6, since scaling and clipping has to be performed for every '/2 and'/4 subpixel location.
In the following, the complexity of the subpixel value interpolation method according to the invention, when applied in a video decoder, is compared with that of the interpolation schemes used in TML5 and TML6. Throughout the analysis which follows, it is assumed that in each method the interpolation of any subpixel value is performed using only the minimum number of steps required to obtain a correctly interpolated value. It is further assumed that each method is implemented in a block based manner, that is, intermediate values common for all the subpixels to be interpolated in a particular N x M block are calculated only once. An illustrative example is provided in Figure 16. Referring to Figure 16, it can be seen that in order to calculate a 4 x 4 block of 1/2 resolution subpixels c, a 9 x 4 block of 1/2 resolution subpixels b is first calculated.
Compared with the subpixel value interpolation method used in TML5, the method according to the invention has a lower computational complexity for the following reasons:
1. Unlike the subpixel value interpolation scheme used in TML5, according to the method of the invention, a value for 1/2 resolution subpixel c can be obtained by filtering in either the vertical or the horizontal direction. Thus, in order to reduce the number of operations, 1/2 resolution subpixel c can be interpolated in the vertical direction if a value for 1/4 resolution subpixel f is required and in the horizontal direction if a value for 1/4 resolution subpixel pixel g is required. As an example, Figure 17 shows all the 1/2 resolution subpixel values that must be calculated in order to interpolate values for 1/4 resolution subpixels g in an image block defined by 4 x 4 original image pixels using the interpolation method of TML5 (Figure 17a) and using the method according to the invention (Figure 17b). In this example, the subpixel value interpolation method according to TML5 requires a total of 88 1/2 resolution subpixels to be interpolated, while the method according to the invention requires the calculation of 721/2 resolution subpixels. As can be seen from Figure 17b, according to the invention, 1/2 resolution subpixels c are interpolated in the horizontal direction in order to reduce the number of calculations required.
2. According to the method of the invention, 1/4 resolution subpixel h is calculated by linear interpolation from its two closest neighbouring 1/2 resolution subpixels in the diagonal direction. The respective numbers of 1/2 resolution subpixels that must be calculated in order to obtain values for 1/4 resolution subpixels h within a 4 x 4 block of original image pixels using the subpixel value interpolation method according to TML5 and the method according to the invention are shown in Figures 18(a) and 18(b), respectively. Using the method according to TML5 it is necessary to interpolate a total of 56 1/2 resolution subpixels, while according to the method of the invention it is necessary to interpolate 40 '/2 resolution subpixels.
Table 1 summarizes the decoder complexities of the three subpixel value interpolation methods considered here, that according to TML5, the direct interpolation method used in TML6 and the method according to the invention. Complexity is measured in terms of the number of 6tap filtering and linear interpolation operations performed. It is assumed that Interpolation of 1/4 resolution subpixel i is calculated according to i = (Ai + A2 + A3 + A4 +2 ) / 4 which is bilinear interpolation and effectively comprises two linear interpolation operations. The operations needed to interpolate subpixel values with one 4 x 4 block of original image pixels are listed for each of the 15 subpixel positions which, for convenience of reference, are numbered according to the scheme shown in Figure 19. Referring to Figure 19, location 1 is the location of an original image pixel A and locations 2 to16 are subpixel locations. Location 16 is the location of 1/4 resolution subpixel i. In order to compute the average number of operations it has been assumed that the probability of a motion vector pointing to each subpixel position is the same. The average complexity is therefore the average of the 15 sums calculated for each subpixel location and the single fullpixel location.
TML5 TML6 Inventive Method Location linear. 6tap linear. 6tap linear. 6tap 3,9 0 16 0 16 0 16 2,4,5,13 16 16 0 16 16 16 7,15 16 52 0 52 16 52 10,12 16 68 0 52 16 52 6, 8,14 48 68 0 52 16 32 Average 19 37 2 32 13 28.25 Table 1: Complexity of 1/4 resolution subpixel interpolation in TML5, TML6 and the method according to the invention.
It can be seen from Table 1 that the method according to the invention requires fewer 6tap filter operations than the subpixel value interpolation method according to TML6 and only a few additional linear interpolation operations. Since 6tap filter operations are much more complex than linear interpolation operations the complexity of the two methods is similar. The subpixel value interpolation method according to TML5 has a considerably higher complexity.
The preferred embodiment for interpolating subpixels with up toy pixel resolution will now be described with reference to Figures 20, 21 and 22.
Figure 20 presents the nomenclature used to describe pixels, 1/2 resolution subpixels, 1/4 resolution subpixels and y resolution subpixels in this extended application of the method according to the invention.
1. Values for the 1/2 resolution and 1/4 resolution subpixels labelled b1, b2 and b3 in Figure 20 are obtained by first calculating intermediate values b'= (3A1 + 12A2  37A3 + 229A4 + 71 A5  21 A6 + 6A7  A8); b2 = (3A1 +
12A2  39A3 + 158A4 + 158A5  39A6 + 12A7  3A8); and b3 = (A1 + 6A2 21 A3 + 71 A4 + 229A5  37A6 + 13A7  3A8) using the values of the eight nearest image pixels (A1 to A8) situated at unit horizontal and unit vertical locations in either the row or the column containing b1, b2 and b3 and disposed symmetrically about 1/2 resolution subpixel b2. The asymmetries in the filter coefficients used to obtain intermediate values b' and b3 account for the fact that pixels A, to A8 are not symmetrically located with respect to 1/4 resolution subpixels b' and b3. Final values for subpixels b', i=1, 2, 3 are calculated according to b' = (b'+ 128) / 256 where the operator / denotes division with truncation. The result is 5 clipped to lie in the range [0, 255].
2. Values for the 1/2 resolution and 1/4 resolution subpixels labelled c'', i, j =
1, 2, 3, are calculated according to c'' = (3 b1' + 12 b2  37 b3 j+ 229 b4 +
71 b5  21 bs + 6 b;  be + 32768) / 65536, c2' _ (3 b,' + 12 b2 39 b3 +
10 158 b4 + 158 b5  39 bs + 12 b;  3 b' + 32768) / 65536 and c3' _ ( b; +
6 b2  21 b3 + 71 b4 + 229 bs  37 bs + 13 bb  3 ba + 32768) / 65536 using the intermediate values b', b2 and b3 calculated for the eight closest subpixels (b,' to b8') in the vertical direction, subpixels b' being situated in the column comprising the 1/2 resolution and 1/4 resolution sub15 pixels c'' being interpolated and disposed symmetrically about the '/2 resolution subpixel c2'. The asymmetries in the filter coefficients used to obtain values for subpixels c'' and c3' account for the fact that subpixels b,' to b8' are not symmetrically located with respect to 1/4 resolution subpixels c'' and c3'. Once more, operator / denotes division with truncation.
20 Before the interpolated values for subpixels c'' are stored in the frame memory they are clipped to lie in the range [0, 255]. In an alternative embodiment of the invention, 1/2 resolution and 1/4 resolution subpixels c'' are calculated using in an analogous manner using intermediate values b', b2 and b3 in the horizontal direction.
3. Values for X resolution subpixels labelled d are calculated using linear interpolation from the values of their closest neighbouring image pixel, 1/2 resolution or 1/4 resolution subpixels in the horizontal or vertical direction. For example, upper leftmost j8 resolution subpixel d is calculated according to d = (A + b' + 1) / 2. As before, operator /
indicates division with truncation.
4. Values for 's resolution subpixels labelled e and f are calculated using linear interpolation from the values of image pixels, 1/2 resolution or 1/4 resolution subpixels in the diagonal direction. For example, referring to Figure 20, upper leftmost pixel y resolution subpixel e is calculated according to e = (b' + b'+ 1) / 2. The diagonal direction to be used in the interpolation of each j8 resolution subpixel in a first preferred embodiment of the invention, hereinafter referred to as 'preferred method 1', is indicated in Figure 21(a). Values for 3s resolution subpixels labelled g are calculated according to g = (A + 3c22 + 3) / 4. As always, operator / denotes division with truncation. In an alternative embodiment of the invention, hereinafter referred to as 'preferred method 2', computational complexity is further reduced by interpolating Y8 resolution subpixels f using linear interpolation from 1/2 resolution subpixels b2, that is, according to the relationship f = (3b2 + b2 + 2) / 4.
The b2 subpixel which is closer to f is multiplied by 3. The diagonal interpolation scheme used in this alternative embodiment of the invention is depicted in Figure 21(b). In further alternative embodiments, different diagonal interpolation schemes can be envisaged.
It should be noted that in all cases where an average involving pixel and / or subpixel values is used in the determination of 's resolution subpixels, the average may be formed in any appropriate manner. The addition of 1 to the sum of values used in calculating such an average has the effect of causing any rounding or truncation operation subsequently applied to round or truncate the average in question to the next highest integer value. In alternative embodiments of the invention, the addition of 1 is not used.
As in the case of subpixel value interpolation to 1/4 pixel resolution previously described, memory requirements in the encoder can be reduced by precalculating only a part of the subpixel values to be interpolated. In the case of subpixel value interpolation to j8 pixel resolution, it is advantageous to calculate all 1/2 resolution and '/4 resolution subpixels beforehand and to compute values for y resolution subpixels in an ondemand fashion, only when they are required. When this approach is taken, both the interpolation method according to TML5 and that according to the invention require 16 times the original picture memory to store the '/2 resolution and 1/4 resolution subpixel values. However, if the direct interpolation method according to TML6 is used in the same way, intermediate values for the '/2 resolution and 1/4 pixel resolution subpixels must be stored. These intermediate values are represented with 32bit precision and this results in a memory requirement 64 times that for the original, noninterpolated image.
In the following, the complexity of the subpixel value interpolation method according to the invention, when applied in a video decoder to calculate values for subpixels at up toy pixel resolution, is compared with that of the interpolation schemes used in TML5 and TML6. As in the equivalent analysis for'/4 pixel resolution subpixel value interpolation described above, it is assumed that in each method the interpolation of any subpixel value is performed using only the minimum number of steps required to obtain a correctly interpolated value. It is also assumed that each method is implemented in a block based manner, such that intermediate values common for all the subpixels to be interpolated in a particular N x M block are calculated only once.
Table 2 summarizes complexities of the three interpolation methods.
Complexity is measured in terms of the number of 8tap filter and linear interpolation operations performed in each method. The table presents the number of operations required to interpolate each of the 63 y resolution subpixels within one 4 x 4 block of original image pixels, each subpixel location being identified with a corresponding number, as illustrated in Figure 22. In Figure 22, location 1 is the location of an original image pixel and locations 2 to 64 are subpixel locations. When computing the average number of operations, it has been assumed that the probability of a motion vector pointing to each subpixel position is the same. The average complexity is thus the average of the 63 sums calculated for each subpixel location and the single fullpixel location.
TML5 TML6 Preferred Preferred Method 1 Method 2 Location linea 8tap linea 8tap linera 8tap linear 8tap r. r.
3,5,7,17,33,49 0 16 0 16 0 16 0 16 19,21,23,35,37,39, 0 60 0 60 0 60 0 60 51,53,55 2,8,9,57 16 16 0 16 16 16 16 16 4,6,25,41 16 32 0 16 16 32 16 32 10,16,58,64 32 76 0 60 16 32 16 32 11,13,15,59,61,63 16 60 0 60 16 60 16 60 18,24,34,40,50,56 16 76 0 60 16 60 16 60 12,14,60,62 32 120 0 60 16 32 16 32 26,32,42,48 32 108 0 60 16 32 16 32 20,22,36,38,52,54 16 120 0 60 16 76 16 76 27,29,31,43,45,47 16 76 0 60 16 76 16 76 28,30,44,46 32 152 0 60 16 60 16 60 Average 64 290.25 0 197.75 48 214.75 48 192.75 Table 2: Complexity of y resolution subpixel interpolation in TML5, TML6 and the method according to the invention. (Results shown separately for Preferred Method 1 and Preferred Method 2).
As can be seen from Table 2, the number of 8tap filtering operations performed according to preferred methods 1 and 2 are, respectively, 26%
and 34% lower than the number of 8tap filtering operation performed in the subpixel value interpolation method of TML5. The number of linear operations is 25% lower, in both preferred method 1 and preferred method 2, compared with TML5, but this improvement is of lesser importance compared to the reduction in 8tap filtering operations. It can further be seen that the direct interpolation method used in TML6 has a complexity comparable that of both preferred methods 1 and 2 when used to interpolate values fort' resolution subpixels.
In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention. While a number of preferred embodiments of the invention have been described in detail, it should be apparent that many modifications and variations thereto are possible, all of which fall within the true spirit and scope of the invention.
Claims (149)
a) interpolating values for subpixels situated at 1/2N1 unit horizontal and unit vertical locations, and for subpixels situated at unit horizontal and 1/2N1 unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations;
b) interpolating a value for a subpixel situated at a 1/2N1 unit horizontal and 1/2N1 unit vertical location using either a first weighted sum of the interpolated values for subpixels residing at 1/2N1 unit horizontal and unit vertical locations or a second weighted sum of the interpolated values for subpixels residing at unit horizontal and 1/2N1 unit vertical locations obtained in step a); and c) interpolating a value for a subpixel situated at a 1/2N unit horizontal and 1/2N unit vertical location using either:
 a weighted average of the value of a first subpixel situated at a 1/2Nm unit horizontal and 1/2Nn unit vertical location and the value of a second subpixel situated at a 1/2Np unit horizontal and 1/2Nq unit vertical location, or  a weighted average of the value of a pixel situated at a unit horizontal and unit vertical location and the value of a subpixel situated at a 1/2Nm unit horizontal and 1/2Nn unit vertical location, variables m, n, p and q taking integer values in the range 1 to N such that the respective first and second subpixels or respective pixel and subpixel are located diagonally with respect to the subpixel at 1/2N unit horizontal and unit vertical location being interpolated.
unit horizontal and 1/2N unit vertical location being interpolated, the first and second weights have equal values.
unit vertical location is interpolated.
a) interpolating values for subpixels at half unit horizontal and unit vertical locations, and unit horizontal and half unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations;
b) interpolating a value for a subpixel at a half unit horizontal and half unit vertical location using either a first weighted sum of the interpolated values for subpixels residing at half unit horizontal and unit vertical locations or a second weighted sum of the interpolated values for subpixels residing at unit horizontal and half unit vertical locations obtained in step a); and c) interpolating a value for a subpixel at a quarter unit horizontal and quarter unit vertical location using a weighted average of at least one pair of values, a first pair comprising the value of a subpixel situated at a half unit horizontal and unit vertical location and the value of a diagonally opposed subpixel situated at a unit horizontal and half unit vertical location, and a second pair comprising the value of a pixel located at a unit horizontal and unit vertical location and the value of a diagonally opposed subpixel situated at a half unit horizontal and half unit vertical location, the first and second pairs being alternative diagonals with respect to the subpixel being interpolated.
a) interpolating values for subpixels situated at 1/2N1 unit horizontal and unit vertical locations, and for subpixels situated at unit horizontal and 1/2N1 unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations; and b) interpolating a value for a subpixel situated at a subpixel horizontal and subpixel vertical location using either a first weighted sum of interpolated values for subpixels situated at a vertical location corresponding to that of the subpixel being interpolated or a second weighted sum of interpolated values for subpixels situated at a horizontal location corresponding to that of the subpixel being interpolated.
a) interpolate values for subpixels situated at 1/2N1 unit horizontal and unit vertical locations, and for subpixels situated at unit horizontal and 1/2N1 unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations;
b) interpolate a value for a subpixel situated at a 1/2N1 unit horizontal and 1/2N1 unit vertical location using a choice of either a first weighted sum of the interpolated values for subpixels residing at 1/2N1 unit horizontal and unit vertical locations or a second weighted sum of the interpolated values for subpixels residing at unit horizontal and 1/2N1 unit vertical locations obtained in step a); and c) interpolate a value for a subpixel situated at a 1/2N unit horizontal and 1/2N unit vertical location using either:
 a weighted average of the value of a first subpixel situated at a 1/2Nm unit horizontal and 1/2Nn unit vertical location and the value of a second subpixel situated at a 1/2Np unit horizontal and 1/2Nq unit vertical location, or  a weighted average of the value of a pixel situated at a unit horizontal and unit vertical location and the value of a subpixel situated at a 1/2Nm unit horizontal and 1/2Nn unit vertical location, variables m, n, p and q taking integer values in the range 1 to N such that the respective first and second subpixels or respective pixel and subpixel are located diagonally with respect to the subpixel at 1/2N unit horizontal and unit vertical location being interpolated.
a) computer program code for interpolating values for subpixels situated at 1/2N1 unit horizontal and unit vertical locations, and for subpixels situated at unit horizontal and 1/2N1 unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations;
b) computer program code for interpolating a value for a subpixel situated at a 1/2N1 unit horizontal and 1/2N1 unit vertical location using either a first weighted sum of the interpolated values for subpixels residing at 1/2N1 unit horizontal and unit vertical locations or a second weighted sum of the interpolated values for subpixels residing at unit horizontal and 1/2N1 unit vertical locations obtained in step a); and c) computer program code for interpolating a value for a subpixel situated at a 1/2N unit horizontal and 1/2N unit vertical location using either:
 a weighted average of the value of a first subpixel situated at a 1/2Nm unit horizontal and 1/2Nn unit vertical location and the value of a second subpixel situated at a 1/2Np unit horizontal and 1/2Nq unit vertical location, or  a weighted average of the value of a pixel situated at a unit horizontal and unit vertical location and the value of a subpixel situated at a 1/2Nm unit horizontal and 1/2Nn unit vertical location, variables m, n, p and q taking integer values in the range 1 to N such that the respective first and second subpixels or respective pixel and subpixel are located diagonally with respect to the subpixel at 1/2N unit horizontal and unit vertical location being interpolated.
a) interpolate values for subpixels situated at 1/2N1 unit horizontal and unit vertical locations, and for subpixels situated at unit horizontal and 1/2N1 unit vertical locations using weighted sums of pixels residing at respective unit horizontal and unit vertical locations; and b) interpolate a value for a subpixel situated at a subpixel horizontal and subpixel vertical location using either a first weighted sum of interpolated values for subpixels situated at a vertical location corresponding to that of the subpixel being interpolated or a second weighted sum of interpolated values for subpixels situated at a horizontal location corresponding to that of the subpixel being interpolated.
 interpolating a subpixel value for a subpixel having coordinates with odd values of K and L, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 1/2, 1/2, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with even values of K and L, including zero, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 1/2, 1/2 and the nearest neighbouring pixel;
 interpolating subpixel values for subpixels having coordinates with K
equal to an even value and L equal to zero and subpixels having coordinates with K equal to zero and L equal to an even value, used in the interpolation of the subpixels having coordinates with odd values of K
and L, using weighted sums of the values of pixels located in rows and columns respectively; and  interpolating subpixel values for subpixels having coordinates with even values of K and L, used in the interpolation of subpixel values for the subpixels having coordinates with odd values of K and L, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to an even value and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the values of subpixels having coordinates with K equal to zero and L equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
equal to 2N and L equal to zero, and a subpixel having coordinates with K
equal to 2N  2 and L equal to zero.
equal to 2N and L equal to 2N, and a subpixel having coordinates with K
equal to 2N 2 and L equal to 2N
equal to the even value O  1 and a second subpixel having coordinates with K
equal to E and L equal to the even value O + 1.
equal to zero and L equal to 2N, and a subpixel having coordinates with K equal to zero and L equal to 2N  2.
1, by taking an average of the value of a pixel having coordinates with K
equal to 2N and L equal to 2N, and a subpixel having coordinates with K equal to and L equal to 2N  2.
 interpolating a subpixel value for a subpixel having coordinates with K
and L equal to 1 or 3, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 2/4, 2/4, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with K and L
equal to zero, 2 or 4, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 2/4, 2/4 and the nearest neighbouring pixel;
 interpolating subpixel values for subpixels having coordinates with K
equal to 2 and L equal to zero and subpixels having coordinates with K
equal to zero and L equal to 2, used in the interpolation of the subpixels having coordinates with K and L equal to 1 or 3, using weighted sums of the values of pixels located in rows and columns respectively; and  interpolating a subpixel value for the subpixel having coordinates with K
and L equal to 2, used in the interpolation of subpixel values for the subpixels having coordinates with K and L equal to 1 or 3, using a choice of either a weighted sum of the values of the subpixel having coordinates with K equal to 2 and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the value of the subpixel having coordinates with K equal to zero and L equal to 2 and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
 interpolating a subpixel value for a subpixel having coordinates with K
and L equal to any of 1, 3, 5 or 7, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 4/8, 4/8, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with K and L equal to any of zero, 2, 4, 6 or 8, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 4/8, 4/8 and the nearest neighbouring pixel;
 interpolating subpixel values for subpixels having coordinates with K
equal to any of 2, 4 or 6, and L equal to zero and subpixels having coordinates with K equal to zero and L equal to any of 2, 4 or 6, used in the interpolation of the subpixels having coordinates with K and L equal to any of 1, 3, 5 or 7, using weighted sums of the values of pixels located in rows and columns respectively; and  interpolating subpixel values for subpixels having coordinates with K
and L equal to any of 2, 4 or 6, used in the interpolation of subpixel values for the subpixels having coordinates with K and L equal to any of 1, 3, 5 or 7, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to any of 2, 4 or 6, and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions or a weighted sum of the values of subpixels having coordinates with K equal to zero and L
equal to any of 2, 4 or 6 and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
N being a positive integer greater than one and representing a particular degree of subpixel value interpolation, the interpolator being configured to:
 interpolate a subpixel value for a subpixel having coordinates with odd values of K and L, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 1/2, 1/2, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with even values of K and L, including zero, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 1/2, 1/2 and the nearest neighbouring pixel;
 interpolate subpixel values for subpixels having coordinates with K
equal to an even value and L equal to zero and subpixels having coordinates with K equal to zero and L equal to an even value, used in the interpolation of the subpixels having coordinates with odd values of K
and L, using weighted sums of the values of pixels located in rows and columns respectively; and  interpolate subpixel values for subpixels having coordinates with even values of K and L, used in the interpolation of subpixel values for the subpixels having coordinates with odd values of K and L, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to an even value and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the values of subpixels having coordinates with K equal to zero and L equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
whose value is being interpolated.
equal to an odd value is also required.
equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions when the value of a subpixel having coordinates with K equal to an odd value and L equal to an even value is also required.
and L.
and L.
N being a positive integer greater than one and representing a particular degree of subpixel value interpolation, the interpolator comprising:
 circuitry configured to interpolate a subpixel value for a subpixel having coordinates with odd values of K and L, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 1/2, 1/2, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with even values of K and L, including zero, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 1/2, 1/2 and the nearest neighbouring pixel;
 circuitry configured to interpolate subpixel values for subpixels having coordinates with K equal to an even value and L equal to zero and subpixels having coordinates with K equal to zero and L equal to an even value, used in the interpolation of the subpixels having coordinates with odd values of K and L, using weighted sums of the values of pixels located in rows and columns respectively; and  circuitry configured to interpolate subpixel values for subpixels having coordinates with even values of K and L, used in the interpolation of subpixel values for the subpixels having coordinates with odd values of K
and L, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to an even value and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the values of subpixels having coordinates with K equal to zero and L
equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
 interpolate a subpixel value for a subpixel having coordinates with K
and L equal to 1 or 3, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 2/4, 2/4, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with K and L
equal to zero, 2 or 4, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 2/4, 2/4 and the nearest neighbouring pixel;
 interpolate subpixel values for subpixels having coordinates with K
equal to 2 and L equal to zero and subpixels having coordinates with K
equal to zero and L equal to 2, used in the interpolation of the subpixels having coordinates with K and L equal to 1 or 3, using weighted sums of the values of pixels located in rows and columns respectively; and  interpolate a subpixel value for the subpixel having coordinates with K
and L equal to 2, used in the interpolation of subpixel values for the subpixels having coordinates with K and L equal to 1 or 3, using a choice of either a weighted sum of the values of the subpixel having coordinates with K equal to 2 and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the value of the subpixel having coordinates with K equal to zero and L equal to 2 and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
 interpolate a subpixel value for a subpixel having coordinates with K
and L equal to any of 1, 3, 5 or 7, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 4/8, 4/8, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with K and L equal to any of zero, 2, 4, 6 or 8, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 4/8, 4/8 and the nearest neighbouring pixel;
 interpolate subpixel values for subpixels having coordinates with K
equal to any of 2, 4 or 6 and L equal to zero and subpixels having coordinates with K equal to zero and L equal to any of 2, 4 or 6, used in the interpolation of the subpixels having coordinates with K and L equal to any of 1, 3, 5 or 7, using weighted sums of the values of pixels located in rows and columns respectively; and  interpolate subpixel values for subpixels having coordinates with K and L equal to any of 2, 4 or 6, used in the interpolation of subpixel values for the subpixels having coordinates with K and L equal to any of 1, 3, 5 or 7, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to any of 2, 4 or 6 and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions or a weighted sum of the values of subpixels having coordinates with K equal to zero and L
equal to any of 2, 4 or 6 and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
 computer program code for interpolating a subpixel value for a subpixel having coordinates with odd values of K and L, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 1/2, 1/2, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with even values of K and L, including zero, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 1/2, 1/2 and the nearest neighbouring pixel;
 computer program code for interpolating subpixel values for subpixels having coordinates with K equal to an even value and L equal to zero and subpixels having coordinates with K equal to zero and L equal to an even value, used in the interpolation of the subpixels having coordinates with odd values of K and L, using weighted sums of the values of pixels located in rows and columns respectively; and  computer program code for interpolating subpixel values for subpixels having coordinates with even values of K and L, used in the interpolation of subpixel values for the subpixels having coordinates with odd values of K and L, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to an even value and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the values of subpixels having coordinates with K equal to zero and L
equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
 interpolate a subpixel value for a subpixel having coordinates with odd values of K and L, according to a choice of either a weighted average of the value of a nearestneighbouring pixel and the value of the subpixel situated at coordinates 1/2, 1/2, or a weighted average of the values of a pair of diagonallyopposed subpixels having coordinates with even values of K and L, including zero, situated within a quadrant of the rectangular bounded region, the quadrant being defined by the subpixel having coordinates 1/2, 1/2 and the nearest neighbouring pixel;
 interpolate subpixel values for subpixels having coordinates with K
equal to an even value and L equal to zero and subpixels having coordinates with K equal to zero and L equal to an even value, used in the interpolation of the subpixels having coordinates with odd values of K
and L, using weighted sums of the values of pixels located in rows and columns respectively; and interpolate subpixel values for subpixels having coordinates with even values of K and L, used in the interpolation of subpixel values for the subpixels having coordinates with odd values of K and L, using a choice of either a weighted sum of the values of subpixels having coordinates with K equal to an even value and L equal to zero and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions, or a weighted sum of the values of subpixels having coordinates with K equal to zero and L equal to an even value and the values of subpixels having corresponding coordinates in immediately adjacent rectangular bounded regions.
Priority Applications (3)
Application Number  Priority Date  Filing Date  Title 

US09954608 US6950469B2 (en)  20010917  20010917  Method for subpixel value interpolation 
US09/954,608  20010917  
PCT/FI2002/000729 WO2003026296A1 (en)  20010917  20020911  Method for subpixel value interpolation 
Publications (2)
Publication Number  Publication Date 

CA2452632A1 true CA2452632A1 (en)  20030327 
CA2452632C true CA2452632C (en)  20130423 
Family
ID=25495681
Family Applications (1)
Application Number  Title  Priority Date  Filing Date 

CA 2452632 Active CA2452632C (en)  20010917  20020911  Method for subpixel value interpolation 
Country Status (9)
Country  Link 

US (3)  US6950469B2 (en) 
JP (3)  JP4698947B2 (en) 
KR (3)  KR20080007276A (en) 
CN (2)  CN101232622B (en) 
CA (1)  CA2452632C (en) 
EP (1)  EP1433316B9 (en) 
ES (1)  ES2540583T7 (en) 
RU (2)  RU2317654C2 (en) 
WO (1)  WO2003026296A1 (en) 
Families Citing this family (184)
Publication number  Priority date  Publication date  Assignee  Title 

US7266150B2 (en)  20010711  20070904  Dolby Laboratories, Inc.  Interpolation of video compression frames 
US6950469B2 (en) *  20010917  20050927  Nokia Corporation  Method for subpixel value interpolation 
KR100557911B1 (en) *  20010918  20060310  마쯔시다덴기산교 가부시키가이샤  Image encoding method and image decoding method 
US7630566B2 (en) *  20010925  20091208  Broadcom Corporation  Method and apparatus for improved estimation and compensation in digital video compression and decompression 
US20030059089A1 (en) *  20010925  20030327  Quinlan James E.  Block matching at the fractional pixel level for motion estimation 
US7181070B2 (en) *  20011030  20070220  Altera Corporation  Methods and apparatus for multiple stage video decoding 
US20030099403A1 (en) *  20011113  20030529  Trusight Ltd.  Image compression with dynamic programming 
CN101448162B (en) *  20011217  20130102  微软公司  Method for processing video image 
KR100924850B1 (en) *  20020124  20091102  가부시키가이샤 히타치세이사쿠쇼  Moving picture signal coding method and decoding method 
US8175159B2 (en) *  20020124  20120508  Hitachi, Ltd.  Moving picture signal coding method, decoding method, coding apparatus, and decoding apparatus 
US7003035B2 (en) *  20020125  20060221  Microsoft Corporation  Video coding methods and apparatuses 
US7110459B2 (en)  20020410  20060919  Microsoft Corporation  Approximate bicubic filter 
US7116831B2 (en)  20020410  20061003  Microsoft Corporation  Chrominance motion vector rounding 
US7305034B2 (en) *  20020410  20071204  Microsoft Corporation  Rounding control for multistage interpolation 
US7620109B2 (en) *  20020410  20091117  Microsoft Corporation  Subpixel interpolation in motion estimation and compensation 
US7154952B2 (en)  20020719  20061226  Microsoft Corporation  Timestampindependent motion vector prediction for predictive (P) and bidirectionally predictive (B) pictures 
US20040001546A1 (en)  20020603  20040101  Alexandros Tourapis  Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multipicture reference motion compensation 
KR100472476B1 (en) *  20020831  20050310  삼성전자주식회사  Interpolation apparatus and method for moving vector compensation 
US7400774B2 (en) *  20020906  20080715  The Regents Of The University Of California  Encoding and decoding of digital data using cues derivable at a decoder 
US7231090B2 (en) *  20021029  20070612  Winbond Electronics Corp.  Method for performing motion estimation with WalshHadamard transform (WHT) 
US7408988B2 (en) *  20021220  20080805  Lsi Corporation  Motion estimation engine with parallel interpolation and search hardware 
US7212676B2 (en)  20021230  20070501  Intel Corporation  Match MSB digital image compression 
JP3997171B2 (en) *  20030327  20071024  株式会社エヌ・ティ・ティ・ドコモ  Moving picture coding apparatus, moving picture coding method, the moving picture encoding program, video decoding device, video decoding method and video decoding program 
US9330060B1 (en)  20030415  20160503  Nvidia Corporation  Method and device for encoding and decoding video image data 
US8660182B2 (en) *  20030609  20140225  Nvidia Corporation  MPEG motion estimation based on dual start points 
US7738554B2 (en)  20030718  20100615  Microsoft Corporation  DC coefficient signaling at small quantization step sizes 
US7426308B2 (en) *  20030718  20080916  Microsoft Corporation  Intraframe and interframe interlace coding and decoding 
US20050013498A1 (en)  20030718  20050120  Microsoft Corporation  Coding of motion vector information 
US8064520B2 (en)  20030907  20111122  Microsoft Corporation  Advanced bidirectional predictive coding of interlaced video 
US7724827B2 (en) *  20030907  20100525  Microsoft Corporation  Multilayer run level encoding and decoding 
US7317839B2 (en) *  20030907  20080108  Microsoft Corporation  Chroma motion vector derivation for interlaced forwardpredicted fields 
US7567617B2 (en)  20030907  20090728  Microsoft Corporation  Predicting motion vectors for fields of forwardpredicted interlaced video frames 
US7253374B2 (en) *  20030915  20070807  General Motors Corporation  Sheettotube welded structure and method 
JP4412672B2 (en) *  20031128  20100210  タンベルグ テレコム エーエス  How to modify the interpolated pixel value 
WO2005055615A1 (en) *  20031205  20050616  Tandberg Telecom As  Improved calculation of interpolated pixel values 
EP1578137A2 (en) *  20040317  20050921  Matsushita Electric Industrial Co., Ltd.  Moving picture coding apparatus with multistep interpolation process 
JP4419062B2 (en) *  20040329  20100224  ソニー株式会社  An image processing apparatus and method, recording medium, and program 
CN1939065A (en) *  20040331  20070328  皇家飞利浦电子股份有限公司  Motion estimation and segmentation for video data 
WO2005104564A1 (en) *  20040421  20051103  Matsushita Electric Industrial Co., Ltd.  Motion compensating apparatus 
KR100605105B1 (en) *  20040528  20060726  삼성전자주식회사  Apparatus of Image Interpolation 
US7565020B2 (en) *  20040703  20090721  Microsoft Corp.  System and method for image coding employing a hybrid directional prediction and wavelet lifting 
US20060088104A1 (en) *  20041027  20060427  Stephen Molloy  Noninteger pixel sharing for video encoding 
US7792192B2 (en) *  20041119  20100907  Analog Devices, Inc.  System and method for subpixel interpolation in motion vector estimation 
JP4277793B2 (en) *  20041217  20090610  ソニー株式会社  The image processing apparatus, the encoding apparatus and their methods 
US7668455B2 (en) *  20041220  20100223  Fujifilm Corporation  Image capturing apparatus, image capturing method, reproducing apparatus, reproducing method and program 
US7653132B2 (en)  20041221  20100126  Stmicroelectronics, Inc.  Method and system for fast implementation of subpixel interpolation 
US20060165162A1 (en) *  20050124  20060727  RenWei Chiang  Method and system for reducing the bandwidth access in video encoding 
JP4736456B2 (en) *  20050215  20110727  株式会社日立製作所  Scanning line interpolation apparatus, the image display device, a video signal processing device 
US8175168B2 (en) *  20050318  20120508  Sharp Laboratories Of America, Inc.  Methods and systems for picture upsampling 
CN101147399B (en) *  20050406  20111130  汤姆森许可贸易公司  Enhancement layer video data encoding method and apparatus 
JP4081103B2 (en) *  20050511  20080423  株式会社東芝  Moving picture coding apparatus 
US7526419B2 (en) *  20050524  20090428  International Business Machines Corporation  Methods for reconstructing data from simulation models 
US20090119454A1 (en) *  20050728  20090507  Stephen John Brooks  Method and Apparatus for Video Motion Process Optimization Using a Hierarchical Cache 
US20070040837A1 (en) *  20050819  20070222  Seok Jin W  Motion vector estimation method and continuous picture generation method based on convexity property of sub pixel 
KR100623036B1 (en) *  20050922  20060905  삼익전자공업 주식회사  Electric signboard system improved resolution with dynamic interpolation scanning 
CN1859576A (en) *  20051011  20061108  华为技术有限公司  Top sampling method and its system for space layered coding video image 
US8265151B1 (en) *  20051214  20120911  Ambarella Taiwan Ltd.  Mode decision using approximate 1/2 pel interpolation 
US8731071B1 (en)  20051215  20140520  Nvidia Corporation  System for performing finite input response (FIR) filtering in motion estimation 
US20070146242A1 (en) *  20051222  20070628  Eastman Kodak Company  High resolution display for monochrome images with color highlighting 
CN1794821A (en) *  20060111  20060628  浙江大学  Method and device of interpolation in grading video compression 
JP4677351B2 (en) *  20060217  20110427  キヤノン株式会社  Motion compensator, a motion compensation processing method, computer program and storage medium 
US8724702B1 (en) *  20060329  20140513  Nvidia Corporation  Methods and systems for motion estimation used in video coding 
WO2007114368A1 (en) *  20060330  20071011  Kabushiki Kaisha Toshiba  Image coding apparatus and method, and image decoding apparatus and method 
WO2007116551A1 (en) *  20060330  20071018  Kabushiki Kaisha Toshiba  Image coding apparatus and image coding method, and image decoding apparatus and image decoding method 
US8208553B2 (en) *  20060504  20120626  Altera Corporation  Methods and apparatus for quarterpel refinement in a SIMD array processor 
JP4682384B2 (en) *  20060711  20110511  株式会社メガチップス  Quarterpixel luminance for motion prediction mechanism, combined luminance for motion prediction mechanism, and combinations luminance and chrominance motion prediction mechanism 
US8155454B2 (en) *  20060720  20120410  Qualcomm Incorporated  Method and apparatus for encoder assisted postprocessing 
US8253752B2 (en) *  20060720  20120828  Qualcomm Incorporated  Method and apparatus for encoder assisted preprocessing 
US8660380B2 (en)  20060825  20140225  Nvidia Corporation  Method and system for performing twodimensional transform on data value array with reduced power consumption 
KR100804451B1 (en) *  20060925  20080220  광운대학교 산학협력단  1/4 quarter pixel interpolation method for imaging process and processor thereof 
US9307122B2 (en) *  20060927  20160405  Core Wireless Licensing S.A.R.L.  Method, apparatus, and computer program product for providing motion estimation for video encoding 
KR100827093B1 (en) *  20061013  20080502  삼성전자주식회사  Method for video encoding and apparatus for the same 
KR100800761B1 (en) *  20061019  20080201  삼성전자주식회사  Apparatus and method of interpolationing chroma signal for minimization of calculation load 
KR101354659B1 (en) *  20061108  20140128  삼성전자주식회사  Method and apparatus for motion compensation supporting multicodec 
KR100874949B1 (en)  20061115  20081219  삼성전자주식회사  Singleinstruction multipledata processor and memory array architecture for it 
JP4753204B2 (en) *  20061117  20110824  株式会社ソニー・コンピュータエンタテインメント  Coding processing apparatus and the encoding processing method 
US8411709B1 (en)  20061127  20130402  Marvell International Ltd.  Use of previously buffered state information to decode in an hybrid automatic repeat request (HARQ) transmission mode 
JP2008165381A (en) *  20061227  20080717  Ricoh Co Ltd  Image processing device and image processing method 
KR101411315B1 (en) *  20070122  20140626  삼성전자주식회사  Method and apparatus for intra/inter prediction 
US8296662B2 (en) *  20070205  20121023  Brother Kogyo Kabushiki Kaisha  Image display device 
CN101658044B (en)  20070409  20121114  诺基亚公司  High accuracy motion vectors for video coding with low encoder and decoder complexity 
US8756482B2 (en)  20070525  20140617  Nvidia Corporation  Efficient encoding/decoding of a sequence of data frames 
US9118927B2 (en)  20070613  20150825  Nvidia Corporation  Subpixel interpolation and its application in motion compensated encoding of a video signal 
KR101380615B1 (en) *  20070628  20140414  삼성전자주식회사  Method and apparatus for improving dynamic range of images 
US8254455B2 (en)  20070630  20120828  Microsoft Corporation  Computing collocated macroblock information for direct mode macroblocks 
US8509567B2 (en)  20070709  20130813  Analog Devices, Inc.  Half pixel interpolator for video motion estimation accelerator 
US8873625B2 (en)  20070718  20141028  Nvidia Corporation  Enhanced compression in representing nonframeedge blocks of image frames 
CN100501771C (en)  20070727  20090617  西南林学院  Digital image subpixel decomposition and enhancement method 
KR101396365B1 (en) *  20070828  20140530  삼성전자주식회사  Method and apparatus for spatiotemporal motion estimation and motion compensation of video 
KR100909390B1 (en) *  20070918  20090724  한국과학기술원  Fast motion compensation apparatus and method 
JP4461165B2 (en) *  20070926  20100512  株式会社東芝  An image processing apparatus, method and program 
JP4900175B2 (en) *  20071004  20120321  セイコーエプソン株式会社  An image processing apparatus and method, and program 
RU2010117612A (en) *  20071005  20111110  Нокиа Корпорейшн (Fi)  Encoding video directed adaptive interpolation filter 
US8416861B2 (en)  20071014  20130409  Nokia Corporation  Fixedpoint implementation of an adaptive image filter with high coding efficiency 
US8897393B1 (en)  20071016  20141125  Marvell International Ltd.  Protected codebook selection at receiver for transmit beamforming 
US8542725B1 (en)  20071114  20130924  Marvell International Ltd.  Decision feedback equalization for signals having unequally distributed patterns 
US8200032B2 (en) *  20071206  20120612  Mstar Semiconductor, Inc.  Image processing method and related apparatus for performing image processing operation according to image blocks in horizontal direction 
KR101456487B1 (en) *  20080304  20141031  삼성전자주식회사  Method and apparatus for encoding and decoding using subpixel motion prediction 
US8565325B1 (en)  20080318  20131022  Marvell International Ltd.  Wireless device communication in the 60GHz band 
US8462842B2 (en)  20080410  20130611  Qualcomm, Incorporated  Symmetry for interpolation filtering of subpixel positions in video coding 
US20090257499A1 (en) *  20080410  20091015  Qualcomm Incorporated  Advanced interpolation techniques for motion compensation in video coding 
US8804831B2 (en)  20080410  20140812  Qualcomm Incorporated  Offsets at subpixel resolution 
US9077971B2 (en)  20080410  20150707  Qualcomm Incorporated  Interpolationlike filtering of integerpixel positions in video coding 
US8705622B2 (en)  20080410  20140422  Qualcomm Incorporated  Interpolation filter support for subpixel resolution in video coding 
RU2505938C2 (en) *  20080410  20140127  Квэлкомм Инкорпорейтед  Distortionbased interpolation depending on transmission rate for video coding based on fixed filter or adaptive filter 
US8831086B2 (en) *  20080410  20140909  Qualcomm Incorporated  Prediction techniques for interpolation in video coding 
US8971412B2 (en) *  20080410  20150303  Qualcomm Incorporated  Advanced interpolation techniques for motion compensation in video coding 
CN102084658B (en) *  20080701  20130605  法国电信公司  Method and device for encoding images using improved prediction, and corresponding decoding method and device, signal and computer software 
US8811484B2 (en) *  20080707  20140819  Qualcomm Incorporated  Video encoding by filter selection 
RU2509355C2 (en) *  20080708  20140310  Нортек Интернэшнл (Пти) Лимитед  Apparatus and method of classifying movement of objects in monitoring zone 
JP2010028220A (en) *  20080715  20100204  Sony Corp  Motion vector detecting device, motion vector detecting method, image encoding device, and program 
US8498342B1 (en)  20080729  20130730  Marvell International Ltd.  Deblocking filtering 
KR101638206B1 (en) *  20080729  20160708  오렌지  Method for updating an encoder by filter interpolation 
US8761261B1 (en) *  20080729  20140624  Marvell International Ltd.  Encoding using motion vectors 
WO2010017166A8 (en)  20080804  20100923  Dolby Laboratories Licensing Corporation  Overlapped block disparity estimation and compensation architecture 
US8345533B1 (en)  20080818  20130101  Marvell International Ltd.  Frame synchronization techniques 
US8750378B2 (en) *  20080923  20140610  Qualcomm Incorporated  Offset calculation in switched interpolation filters 
US8131056B2 (en) *  20080930  20120306  International Business Machines Corporation  Constructing variability maps by correlating offstate leakage emission images to layout information 
US8681893B1 (en)  20081008  20140325  Marvell International Ltd.  Generating pulses using a lookup table 
US8666181B2 (en)  20081210  20140304  Nvidia Corporation  Adaptive multiple engine image motion detection system and method 
US20100166076A1 (en) *  20081230  20100701  Tandberg Telecom As  Method, apparatus, and computer readable medium for calculating run and level representations of quantized transform coefficients representing pixel values included in a block of a video picture 
US20100165078A1 (en) *  20081230  20100701  Sensio Technologies Inc.  Image compression using checkerboard mosaic for luminance and chrominance color space images 
JP2010161747A (en) *  20090109  20100722  Toshiba Corp  Apparatus and method for generating subpixel, and motion compensating apparatus 
US8189666B2 (en)  20090202  20120529  Microsoft Corporation  Local picture identifier and computation of colocated information 
JP5580541B2 (en) *  20090306  20140827  パナソニック株式会社  The video decoding apparatus and method 
US8520771B1 (en)  20090429  20130827  Marvell International Ltd.  WCDMA modulation 
US7991245B2 (en) *  20090529  20110802  Putman Matthew C  Increasing image resolution method employing known background and specimen 
JP2011030184A (en) *  20090701  20110210  Sony Corp  Image processing apparatus, and image processing method 
JP5325745B2 (en) *  20091102  20131023  株式会社ソニー・コンピュータエンタテインメント  Moving image processing program, apparatus and method, an imaging apparatus equipped with a moving image processing apparatus 
KR101601848B1 (en) *  20091201  20160310  에스케이 텔레콤주식회사  Inter prediction frame generating apparatus and method for generating, and there is a reference frame interpolation apparatus and method for use in 
US8406537B2 (en) *  20091217  20130326  General Electric Company  Computed tomography system with data compression and transfer 
EP2514210A4 (en)  20091217  20140319  Ericsson Telefon Ab L M  Method and arrangement for video coding 
US20110200108A1 (en) *  20100218  20110818  Qualcomm Incorporated  Chrominance high precision motion filtering for motion interpolation 
KR20110112187A (en) *  20100405  20111012  삼성전자주식회사  Method and apparatus for video encoding, and method and apparatus for video decoding 
KR101682147B1 (en) *  20100405  20161205  삼성전자주식회사  Method and apparatus for interpolation based on transform and inverse transform 
US9219921B2 (en)  20100412  20151222  Qualcomm Incorporated  Mixed tap filters 
KR20170122302A (en) *  20100413  20171103  지이 비디오 컴프레션, 엘엘씨  Inheritance in sample array multitree subdivision 
KR101598510B1 (en)  20100413  20160302  지이 비디오 컴프레션, 엘엘씨  Video coding using multitree sub  divisions of images 
US8963996B2 (en) *  20100505  20150224  Samsung Electronics Co., Ltd.  Communication of stereoscopic threedimensional (3D) video information including an uncompressed eye view video frames 
CN102402781B (en) *  20100913  20140514  慧荣科技股份有限公司  Method and relevant device for generating highquality enlarged image 
US8472755B2 (en)  20100507  20130625  Silicon Motion Inc.  Method for generating a high quality upscaled image, and associated device 
CN102986212B (en)  20100507  20151125  日本电信电话株式会社  Moving image encoding control method, moving image encoding apparatus 
EP2571268A4 (en) *  20100512  20151216  Nippon Telegraph & Telephone  Video coding control method, video coding device and video coding program 
US8447105B2 (en) *  20100607  20130521  Microsoft Corporation  Data driven interpolation using geodesic affinity 
US9317896B2 (en) *  20100709  20160419  Samsung Electronics Co., Ltd.  Image interpolation method and apparatus 
US8817771B1 (en)  20100716  20140826  Marvell International Ltd.  Method and apparatus for detecting a boundary of a data frame in a communication network 
US20120027081A1 (en) *  20100730  20120202  Cisco Technology Inc.  Method, system, and computer readable medium for implementing runlevel coding 
US20120063515A1 (en) *  20100909  20120315  Qualcomm Incorporated  Efficient Coding of Video Parameters for Weighted Motion Compensated Prediction in Video Coding 
US20120147967A1 (en) *  20101210  20120614  Qualcomm Incorporated  Adaptive support for interpolating values of subpixels for video coding 
US9445126B2 (en)  20110105  20160913  Qualcomm Incorporated  Video filtering using a combination of onedimensional switched filter and onedimensional adaptive filter 
US8797391B2 (en) *  20110114  20140805  Himax Media Solutions, Inc.  Stereo image displaying method 
CN102595118B (en) *  20110114  20150408  华为技术有限公司  Prediction method and predictor in encoding and decoding 
US9049454B2 (en)  20110119  20150602  Google Technology Holdings Llc.  High efficiency low complexity interpolation filters 
US20120224639A1 (en) *  20110303  20120906  General Instrument Corporation  Method for interpolating half pixels and quarter pixels 
US9313519B2 (en)  20110311  20160412  Google Technology Holdings LLC  Interpolation filter selection using prediction unit (PU) size 
JP5768491B2 (en) *  20110517  20150826  ソニー株式会社  An image processing apparatus and method, program, and recording medium 
WO2012178178A3 (en) *  20110624  20130221  General Instrument Corporation  Selection of phase offsets for interpolation filters for motion compensation 
KR20170070262A (en) *  20110624  20170621  가부시키가이샤 엔.티.티.도코모  Method and apparatus for motion compensation prediction 
RU2595262C2 (en) *  20110628  20160827  Самсунг Электроникс Ко., Лтд.  Method for image interpolation using asymmetric interpolation filter and device therefor 
CN102857752B (en) *  20110701  20160330  华为技术有限公司  A pixel prediction method and apparatus 
WO2013006573A1 (en)  20110701  20130110  General Instrument Corporation  Joint subpixel interpolation filter for temporal prediction 
US9129411B2 (en) *  20110721  20150908  Luca Rossato  Upsampling in a tiered signal quality hierarchy 
US9137433B2 (en)  20110919  20150915  Michael Mojaver  Super resolution binary imaging and tracking system 
US20130070091A1 (en) *  20110919  20130321  Michael Mojaver  Super resolution imaging and tracking system 
RU2473124C1 (en) *  20110923  20130120  Общество С Ограниченной Ответственностью "Виси Рус"  Method of detecting pornography on digital images (versions) 
EP2595382A3 (en)  20111121  20140820  BlackBerry Limited  Methods And Devices For Encoding And Decoding Transform Domain Filters 
US9325991B2 (en)  20120411  20160426  Qualcomm Incorporated  Motion vector rounding 
US8819525B1 (en)  20120614  20140826  Google Inc.  Error concealment guided robustness 
US9041834B2 (en) *  20120919  20150526  Ziilabs Inc., Ltd.  Systems and methods for reducing noise in video streams 
JP5697649B2 (en) *  20121127  20150408  京セラドキュメントソリューションズ株式会社  Image processing apparatus 
JP5730274B2 (en) *  20121127  20150603  京セラドキュメントソリューションズ株式会社  Image processing apparatus 
US9432690B2 (en) *  20130130  20160830  Ati Technologies Ulc  Apparatus and method for video processing 
US9225979B1 (en) *  20130130  20151229  Google Inc.  Remote access encoding 
US20140269943A1 (en) *  20130312  20140918  Tandent Vision Science, Inc.  Selective perceptual masking via downsampling in the spatial and temporal domains using intrinsic images for use in data compression 
US20140267916A1 (en) *  20130312  20140918  Tandent Vision Science, Inc.  Selective perceptual masking via scale separation in the spatial and temporal domains using intrinsic images for use in data compression 
CA2900488C (en) *  20130501  20180102  Lg Electronics Inc.  Apparatus and method of transmitting and receiving signal 
US9390354B2 (en) *  20130806  20160712  Canon Kabushiki Kaisha  Method for printing an upscaled image 
US9774881B2 (en)  20140108  20170926  Microsoft Technology Licensing, Llc  Representing motion vectors in an encoded bitstream 
US9749642B2 (en)  20140108  20170829  Microsoft Technology Licensing, Llc  Selection of motion vector precision 
CN103793917B (en) *  20140224  20170201  哈尔滨工程大学  One binding two kinds of interpolation algorithms remote sensing image subpixel positioning method 
US20160191946A1 (en) *  20141231  20160630  Microsoft Technology Licensing, Llc  Computationally efficient motion estimation 
US20160261882A1 (en) *  20150306  20160908  Qualcomm Incorporated  Method and apparatus for low complexity quarter pel generation in motion search 
US20160293085A1 (en) *  20150402  20161006  Apple Inc.  Electronic Device With Image Processor to Reduce Color Motion Blur 
US9542732B2 (en) *  20150403  20170110  Cognex Corporation  Efficient image transformation 
Family Cites Families (33)
Publication number  Priority date  Publication date  Assignee  Title 

GB8713454D0 (en)  19870609  19870715  Sony Corp  Television standards converters 
US4816913A (en) *  19871116  19890328  Technology, Inc., 64  Pixel interpolation circuitry as for a video signal processor 
US4937666A (en) *  19891204  19900626  Bell Communications Research, Inc.  Circuit implementation of block matching algorithm with fractional precision 
GB2249906B (en) *  19901115  19940427  Sony Broadcast & Communication  Motion compensated interpolation of images 
JP2861462B2 (en) *  19910412  19990224  ソニー株式会社  Motion vector detecting device 
US5337088A (en)  19910418  19940809  Matsushita Electric Industrial Co. Ltd.  Method of correcting an image signal decoded in block units 
US5430811A (en) *  19911225  19950704  Matsushita Electric Industrial Co., Ltd.  Method for interpolating missing pixels and an apparatus employing the method 
US5594813A (en) *  19920219  19970114  Integrated Information Technology, Inc.  Programmable architecture and methods for motion estimation 
JP2636622B2 (en) *  19920313  19970730  松下電器産業株式会社  Encoding apparatus and decoding apparatus of the coding method and decoding method and a video signal of the video signal 
US5461423A (en) *  19920529  19951024  Sony Corporation  Apparatus for generating a motion vector with halfpixel precision for use in compressing a digital motion picture signal 
JP2723199B2 (en)  19920603  19980309  シャープ株式会社  A tracking servo pullin circuit apparatus for an optical disc player 
EP0576290B1 (en)  19920625  19990120  Sony Corporation  Picture signal coding and decoding 
JPH06197334A (en) *  19920703  19940715  Sony Corp  Picture signal coding method, picture signal decoding method, picture signal coder, picture signal decoder and picture signal recording medium 
KR970000761B1 (en) *  19921007  19970118  배순훈  Mini highdefinition television 
DE69424909D1 (en) *  19930409  20000720  Sony Corp  Method and apparatus for image coding 
JP2967014B2 (en) *  19930524  19991025  キヤノン株式会社  Image processing apparatus 
JPH08510880A (en) *  19930601  19961112  トムソン マルチメディア ソシエテ アノニム  Method and apparatus for motion compensated interpolation 
US5684538A (en)  19940818  19971104  Hitachi, Ltd.  System and method for performing video coding/decoding using motion compensation 
JP3392564B2 (en) *  19950227  20030331  三洋電機株式会社  Singlechip color video camera 
JPH09102954A (en) *  19951004  19970415  Matsushita Electric Ind Co Ltd  Method for calculating picture element value of block from one or two predictive blocks 
US5991463A (en) *  19951108  19991123  Genesis Microchip Inc.  Source data interpolation method and apparatus 
KR100192270B1 (en) *  19960203  19990615  구자홍  The video decoding circuit in hdtv 
KR100226684B1 (en) *  19960322  19991015  전주범  A half pel motion estimator 
JP3224514B2 (en) *  19960821  20011029  シャープ株式会社  Moving picture coding apparatus and a moving picture decoding apparatus 
RU2131172C1 (en) *  19961210  19990527  Полыковский Андрей Маркович  Interpolation method for compressing tv signal 
DE19730305A1 (en)  19970715  19990121  Bosch Gmbh Robert  A method for generating an improved video signal during the motion estimation of picture sequences, in particular of a predicted signal for moving pictures with motioncompensating prediction 
DE19746214A1 (en) *  19971021  19990422  Bosch Gmbh Robert  Movement compensated prediction method for moving image sequences 
US6122017A (en) *  19980122  20000919  HewlettPackard Company  Method for providing motioncompensated multifield enhancement of still images from video 
WO1999056247A1 (en)  19980429  19991104  Koninklijke Philips Electronics N.V.  Image interpolation 
US6252576B1 (en) *  19980806  20010626  InSystem Design, Inc.  Hardwareefficient system for hybridbilinear image scaling 
EP1083752A1 (en) *  19990908  20010314  STMicroelectronics S.r.l.  Video decoder with reduced memory 
JP4599672B2 (en)  19991221  20101215  株式会社ニコン  Interpolation processing apparatus and recording medium recording an interpolation processing program 
US6950469B2 (en) *  20010917  20050927  Nokia Corporation  Method for subpixel value interpolation 
Also Published As
Publication number  Publication date  Type 

CN1537384A (en)  20041013  application 
US7280599B2 (en)  20071009  grant 
ES2540583T7 (en)  20170710  grant 
ES2540583T3 (en)  20150710  grant 
RU2477575C2 (en)  20130310  grant 
US20030112864A1 (en)  20030619  application 
RU2317654C2 (en)  20080220  grant 
US6950469B2 (en)  20050927  grant 
EP1433316B1 (en)  20150429  grant 
KR20040036902A (en)  20040503  application 
EP1433316B9 (en)  20170412  grant 
US8036273B2 (en)  20111011  grant 
CN1331353C (en)  20070808  grant 
KR20110115181A (en)  20111020  application 
KR101176903B1 (en)  20120830  grant 
US20050220353A1 (en)  20051006  application 
JP2005503734A (en)  20050203  application 
RU2004101290A (en)  20050620  application 
JP2011101411A (en)  20110519  application 
KR100972850B1 (en)  20100728  grant 
JP5502765B2 (en)  20140528  grant 
EP1433316A1 (en)  20040630  application 
JP4700704B2 (en)  20110615  grant 
KR20080007276A (en)  20080117  application 
CN101232622B (en)  20110413  grant 
JP2008187727A (en)  20080814  application 
CA2452632A1 (en)  20030327  application 
EP1433316B3 (en)  20170118  grant 
CN101232622A (en)  20080730  application 
JP4698947B2 (en)  20110608  grant 
WO2003026296A1 (en)  20030327  application 
US20080069203A1 (en)  20080320  application 
RU2007133925A (en)  20090320  application 
Similar Documents
Publication  Publication Date  Title 

US7567617B2 (en)  Predicting motion vectors for fields of forwardpredicted interlaced video frames  
Cote et al.  H. 263+: Video coding at low bit rates  
US6735249B1 (en)  Apparatus, and associated method, for forming a compressed motion vector field utilizing predictive motion coding  
US20030156646A1 (en)  Multiresolution motion estimation and compensation  
US20040213470A1 (en)  Image processing apparatus and method  
US7263125B2 (en)  Method and device for indicating quantizer parameters in a video coding system  
US7324595B2 (en)  Method and/or apparatus for reducing the complexity of nonreference frame encoding using selective reconstruction  
US20050013500A1 (en)  Intelligent differential quantization of video coding  
US20070098067A1 (en)  Method and apparatus for video encoding/decoding  
US20050129122A1 (en)  Computational reduction in motion estimation based on lower bound of cost function  
US20030202594A1 (en)  Method for coding motion in a video sequence  
US20080037642A1 (en)  Motion Compensation Prediction Method and Motion Compensation Prediction Apparatus  
US20070206872A1 (en)  Method of and apparatus for video intraprediction encoding/decoding  
US20060222074A1 (en)  Method and system for motion estimation in a video encoder  
EP2262267A1 (en)  Filter coefficient coding scheme for video coding  
US20070217512A1 (en)  Picture Encoding Method and Picture Encoding Device  
US20040062307A1 (en)  Method and system for selecting interpolation filter type in video coding  
US20070047649A1 (en)  Method for coding with motion compensated prediction  
US20120076203A1 (en)  Video encoding device, video decoding device, video encoding method, and video decoding method  
US20050013497A1 (en)  Intraframe and interframe interlace coding and decoding  
US6711211B1 (en)  Method for encoding and decoding video information, a motion compensated video encoder and a corresponding decoder  
US20120033736A1 (en)  Image processing device and method  
US20060083310A1 (en)  Adaptive overlapped block matching for accurate motion compensation  
JP2005039842A (en)  Video coding/decoding device for colored image and its method  
US7379501B2 (en)  Differential coding of interpolation filters 
Legal Events
Date  Code  Title  Description 

EEER  Examination request 