WO2006006609A1 - Motion compensation method - Google Patents

Motion compensation method Download PDF

Info

Publication number
WO2006006609A1
WO2006006609A1 PCT/JP2005/012873 JP2005012873W WO2006006609A1 WO 2006006609 A1 WO2006006609 A1 WO 2006006609A1 JP 2005012873 W JP2005012873 W JP 2005012873W WO 2006006609 A1 WO2006006609 A1 WO 2006006609A1
Authority
WO
WIPO (PCT)
Prior art keywords
sub
pixels
pixel values
motion compensation
reference picture
Prior art date
Application number
PCT/JP2005/012873
Other languages
French (fr)
Inventor
Steffen Wittmann
Thomas Wedi
Satoshi Kondo
Original Assignee
Matsushita Electric Industrial Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Matsushita Electric Industrial Co., Ltd. filed Critical Matsushita Electric Industrial Co., Ltd.
Priority to CN200580006213.4A priority Critical patent/CN1926875B/en
Priority to US10/590,524 priority patent/US20070133687A1/en
Priority to EP05760112A priority patent/EP1766992A1/en
Priority to JP2007521057A priority patent/JP2008507190A/en
Publication of WO2006006609A1 publication Critical patent/WO2006006609A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy

Definitions

  • the present invention relates to a motion compensation method for interpolating sub-pixels into a reference picture and for performing motion compensation based on the interpolated reference picture.
  • Moving pictures are being adopted in an increasingly number of applications ranging form video telephony and video conferencing to DVD and digital television.
  • moving pictures are transmitted, a substantial amount of data has to be sent through conventional transmission channels of a limited available frequency bandwidth.
  • In order to transmit the digital data through the limited channel bandwidth it is inevitable to compress or reduce the volume of the transmission data.
  • Temporal dependencies between blocks of subsequent frames are determined by employing a motion estimation and compensation technique. For any given block, a search is performed in previously coded and transmitted frames to determine a motion vector which will be used by the coding apparatus and the decoding apparatus to predict the image data of a block.
  • the shown video coding apparatus generally denoted with reference numeral 900 includes: a transform/quantization unit 920 to output quantized transform coefficients QC by transforming spatial image data to the frequency domain and quantizing the transformed image data; an entropy coding unit 990 for performing entropy coding (variable length coding) of the quantized transform coefficients QC and outputting the bit stream BS; and a video buffer (not shown) for adopting the compressed video data having a variable bit rate to a transmission channel which may have a fixed bit rate.
  • the coding apparatus shown in FIG. 1 employs a DPCM (Differential Pulse Code Modulation) by only transmitting differentials between subsequent fields or frames.
  • a subtractor 910 obtains these differentials by receiving the video data to be coded as an input signal IS and subtracting the previous image indicated by a prediction signal PS therefrom. The previous image is obtained by decoding the previously coded image. This is accomplished by a decoding apparatus which is incorporated into video coding apparatus 900. The decoding apparatus performs the coding steps in a reverse manner.
  • the decoding apparatus includes: an inverse quantization/transform unit 930, and an adder 935 for adding the decoded differential (differential decoding signal DDS) to the previously decoded picture (prediction signal PS) in order to produce the image as will be obtained on the decoding side.
  • a current frame or field is predicted from image data of a previous frame or field based on an estimation of the motion between the current and the previous images.
  • estimated motion may be described in terms of 2-dimensional motion vectors representing the displacement of pixels between the previous and the current images.
  • motion estimation is performed on a block-by-block basis. An example of the division of the current image into plurality of blocks is illustrated in FIG. 2.
  • a block of a current frame is compared with blocks in previous frames until a best match is determined. Based on the comparison results, an inter-frame displacement vector for the whole block can be estimated for the current frame.
  • a motion estimation unit 970 is incorporated into the coding apparatus together with the corresponding motion compensation unit 960 included into the decoding path.
  • the video coding apparatus 900 of FIG. 1 performs operations as follows.
  • a given video image indicated by an input signal IS is divided into a number of small blocks, usually denoted as "macro blocks".
  • video image shown in FIG. 2 is divided into a plurality of macro blocks, each of which usually having a size of 16 x 16 pixels.
  • I-picture When coding the video data of an image by only reducing spatial redundancies within the image, the resulting frame is referred to as an I-picture.
  • I-pictures are typically coded by directly applying the transform to the macro blocks of a frame. I-pictures are large in size as no temporal information is exploited to reduce the amount of data.
  • a prediction coding between subsequent fields or frames is performed based on motion estimation and compensation.
  • a selected reference frame in motion estimation is a previously coded frame
  • the frame to be coded is referred to as a P-picture.
  • the frame to be coded is referred to as B-picture.
  • the motion compensation has been described to be based on a 16 x 16 macro block, motion estimation and compensation can be performed using a number of different block sizes. Individual motion vectors may be determined for blocks having 4 x 4, 4 x 8, 8 x 4, 8 x 8, 8 x 16, 16 x 8, or 16 x 16 pixels. The provision of small motion compensation blocks improves the ability to handle fine motion details.
  • the motion compensation operation Based on the results of the motion estimation operation, the motion compensation operation provides a prediction based on the determined motion vector.
  • the information contained in a prediction error block resulting from the predicted block is then transformed into transform coefficients in transform/quantization unit 920.
  • transform/quantization unit 920 Generally, a 2-dimensional DCT (Discrete Cosine Transform) is employed.
  • the resulting transform coefficients are quantized and finally entropy coded (VLC) in entropy coding unit 990.
  • VLC entropy coded
  • a decoding apparatus receives the transmitted bit stream BS of compressed video data and reproduces a sequence of coded video images based on the received data.
  • the configuration of the decoding apparatus corresponds to that of the decoding apparatus included in the coding apparatus shown in FIG. 1. A detailed description of the configuration of the decoding apparatus is therefore omitted.
  • a sub-pixel accuracy of reference frames is widely used.
  • 1/2 sub-pixel accuracy motion compensation is used in the MPEG-2 format.
  • Patent Literature EP 1073276 In order to further increase the motion vector accuracy and coding efficiency, a 1/3 and a 1/6 sub-pixel vector accuracies have been proposed in Patent Literature EP 1073276.
  • the motion vector accuracy and coding efficiency can further be increased by applying interpolation filters in motion estimation and compensation yielding 1/8 sub-pixel displacements.
  • interpolation filters in motion estimation and compensation yielding 1/8 sub-pixel displacements.
  • a sub-pixel resolution requires high computation complexity, in particular, calculation registers having a length of up to 25 bits.
  • Such a complex implementation may be based on a 2-step approach.
  • a 1/4 sub-pixel image employing an 8-tap filter is calculated.
  • a 1/8 sub-pixel is obtained based on the 1/4 sub-pixel image by employing a bilinear filtering.
  • the filtering operation for generating the image with the 1/4 sub-pixel accuracy includes the steps of horizontal and subsequent vertical filtering.
  • the horizontal interpolation may be performed based on the following Equations (1) to (3):
  • /* 1 -3- ⁇ 4 +12-£ 4 -37-C 4 +229- J D 4 +71-E 4 -21-F 4 +6-G 4 -l-H 4 ...(l)
  • h 2 -3 -A 4 +12 B 4 -39 -C 4 +158 -D 4 +158 -E 4 -39 -F 4 +12 -G 4 -3 -H 4 ...(2)
  • /2 3 -l- ⁇ 4 +6-£ 4 -21-C 4 +71-/J> 4 +229-E 4 -37-F 4 +12-G 4 -3-H 4 ...(3)
  • hi to h 3 denote the 1/4 sub-pixel values and A x -H x represent the original full-pel pixel values, namely, the pixels from the original image.
  • Coefficients applied to the above A x -H x are set in a way that the signal processing is performed preventing the occurrence of imaging by upsampling, in other words, unnecessary high frequency components generated through interpolation are eliminated.
  • the horizontal filtering is illustrated in FIG. 3. Eight-tap filtering is performed based on the pixel values of the original pixels
  • Equations (4) to (6) having coefficients which correspond to those of the above described horizontal filter are employed.
  • v, -3 - D 1 + 12 - D 2 - 37 - D 3 + 229 - D 4 + 71 - D 5 - 21 - D 6 + 6 - D 1 -I - D s ...(4)
  • v 2 -3 • D x + 12 • D 2 - 39 • D 3 + 158 • D, + 158 • D 5 - 39 • D 6 + 12 • D 1 - 3 • Z) 8 ...
  • v. -l D, + 6 D 2 - 21 ⁇ D 3 + 71 • D 1 + 229 • D 5 - 37 • D 6 + 12 • D 1 - 3 • D 8 ... (6)
  • V 1 to V 3 refer to the calculated vertical
  • D 1 , D 2 , D 3 , D 4 , D 5 , D 6 , D 7 .and D 8 represent the full-pel resolution pixels , namely, the pixel values of the original pixels 210.
  • coefficients applied to D x are set in a way that the signal processing is performed preventing the occurrence of imaging by upsampling, in other words, unnecessary high frequency components generated through interpolation are eliminated.
  • the resulting pixel values have a length of up to 25 bits.
  • the calculation results are downshifted and rounded as illustrated.
  • An example case of pixel value V 1 is shown by the following Equation (7) :
  • V 1 represents the pixel value resulting from the horizontal and vertical filtering
  • V 1 ' represents the downshifted pixel value.
  • the downshifted pixel values are further clipped to the range of 0 to 255.
  • the vertical filtering is illustrated in FIG. 4.
  • the pixel values of the pixels 230 obtained during vertical filtering complete the sub-pixel array illustrated by way of filtering example between original pixels D 4 , D5, E 4 and E5.
  • a 1/8 sub-pixel frame is calculated by applying a bilinear filtering to the 1/4 sub-pixel resolution.
  • a bilinear filtering is applied in two steps and is illustrated by way of examples in FIG. 5 and FIG. 6.
  • FIG. 5 illustrates the application of a horizontal and vertical filtering.
  • a mean value is calculated from the respective neighbouring pixel values in order to obtain an intermediate pixel value of a 1/8 sub-pixel resolution.
  • Equation (8) can be applied. Note that " »l " in Equation (8) represents 1-bit downshifting.
  • the remaining 1/8 sub-pixel values to be interpolated are calculated by diagonal filtering as illustrated in FIG. 6. It is a particular advantage of this approach that, in the bilinear filtering, the number of sub-pixel values stemming from multiple filtering is minimized as much as possible. For this purpose, it is preferable that only those pixel values, of the interpolated pixels, that are directly derived from original pixel values 210 are taken into account. In other words, those derived pixel values are the pixel values of the interpolated pixels located between the original pixels.
  • All intermediate pixel values can be calculated therefrom, in other words, from the pixel values of the original pixels 210 and the intermediate pixel values derived from the original pixel values, when additionally taking center pixel 240 of the sub-pixel array into account.
  • the calculation operation for the additional 1/8 sub-pixel values is based on two of the 1/4 sub-pixel resolution values, respectively.
  • the individual pixel values taken into account for the calculation of an intermediate pixel value are illustrated in FIG. 6 by respective arrows. Each of the arrows shows two pixel values of pixels based on which each intermediate pixel value of the two is calculated.
  • Equations (9) and (10) are employed :
  • D and G represent new intermediate pixel values as illustrated in FIG. 6, and E, F, ⁇ and I represent the pixel values obtained from the 1/4 resolution image.
  • the additional values of "1" and "2" in the above equations only serve for correctly rounding the calculation results.
  • the above-described conventional motion compensation method requires to record a long operation value of 25 bits in the filtering process for 1/4 sub-pixel interpolation. This causes a particular disadvantage of such an interpolation approach that long registers are needed resulting in high hardware complexity and computational effort.
  • An object of the present invention is to provide a motion compensation method for reducing operational workload and simplifying a hardware configuration.
  • the motion compensation method of the present invention includes: interpolating sub-pixels in a reference picture; and performing motion compensation based on the interpolated reference picture, in the method, the interpolating including : a first calculation step of calculating base values which are bases of sub-pixel values of the sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a first rounding step of deriving the sub-pixel values of the sub-pixels by rounding the base values calculated in the first calculation step instead of directly using the base values in calculating sub-pixel values of other sub-pixels; and the performing of motion compensation includes performing motion compensation based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values.
  • base values of sub-pixels that have been calculated are directly used in calculating sub-pixel values of other sub-pixels.
  • the base values of sub-pixels that have been calculated in the first calculation step are rounded in stead of being directly used in calculating the sub-pixel values of other sub-pixel values. Therefore, even in the case where the sub-pixel values of the other sub-pixels are calculated using the base values rounded, the number of bits to be used in the calculation can be more reduced than in the conventional way. As a result, it becomes possible to reduce the operational workload and to simplify the hardware configuration.
  • the first calculation step may include calculating base values of sub-pixels to be interpolated in a first direction
  • the first rounding step may include deriving sub-pixel values of the sub-pixels to be interpolated in the first direction by rounding the base values calculated in the first calculation step.
  • the interpolation may further include: a second calculation step of calculating, using the sub-pixel values of the sub-pixels derived in the first rounding step, base values of sub-pixels to be interpolated in a second direction that is different from the first direction; and a second rounding step of deriving the sub-pixel values of the sub-pixels to be interpolated in the second direction by rounding the base values calculated in the second calculation step.
  • the number of bits to be used in the calculation can be reduced down to 16 bits from, for example, 25 bits needed in a conventional way.
  • the first calculation step may include calculating the base values of three a-fourths sub-pixels using the following equations when eight pixel values of pixels arrayed in the first direction are represented as A, B, C, D, E, F, G and H respectively and the three a-fourths sub-pixel values are represented as hi, h 2 and h 3 respectively:
  • ⁇ 1 -l - ⁇ + 3 - 5-10 -C + 59-I> + 18- £-6- F + l - G -0 - /- ⁇ ;
  • h 2 - ⁇ - A + 4 - B-IO -C + 39- D + 39-E-W- F + 4-G - ⁇ - H; and
  • h i -0-A + ⁇ -B-6-C + ⁇ 8- D + 59 E- ⁇ 0- F + 3 -G - ⁇ - H.
  • V 2 -3 ⁇ Z ) 1 + 12 • Z ) 2 - 39 -Z) 3 +158 -D 4 +158 -Z ) 5 -39 -Z ) 6 +12 -Z) 7 -3- Z ) 8 ;and
  • V 3 -1-Z ) 1 +6- D 2 -21 -Z ) 3 +71 D 4 +229-D 5 -37 -D 6 +12 -Z) 7 -3 -Z) 8 .
  • the coefficients used in calculating sub-pixel values of sub-pixels are smaller than the conventional coefficients. This makes it possible to further reduce the number of bits to be used in calculating the sub-pixel values.
  • the motion compensation method may further include a bilinear filtering of raising a sub-pixel accuracy by applying bilinear filtering to the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values.
  • the increase in sub-pixel accuracy makes it possible to prevent picture quality from deteriorating during the picture coding processing and the picture decoding processing.
  • the present invention can be realized as a motion compensation method, a motion estimation method, a moving picture coding method and a moving picture decoding method using the motion compensation method, a program causing a computer to execute these steps of the respective methods, a recording medium for storing the program, and an apparatus for performing operations according to these methods.
  • FIG. 1 is a block diagram showing the structure of a moving picture coding apparatus
  • FIG. 2 is an illustration of how a video image is divided into blocks
  • FIG. 3 is an illustration of horizontal filtering for calculating a 1/4 sub-pixel accuracy in the horizontal direction ;
  • FIG. 4 is an illustration of vertical filtering for calculating a 1/4 sub-pixel accuracy in the vertical direction
  • FIG. 5 is an illustration of horizontal and vertical filtering for calculating a 1/8 sub-pixel accuracy
  • FIG. 6 is an illustration of bilinear filtering in the diagonal direction for calculating a 1/8 sub-pixel accuracy
  • FIG. 7 is a block diagram showing the configuration of a moving picture coding apparatus in the embodiment of the present invention.
  • FIG. 8 is a flow chart showing the motion compensation operation performed by the moving picture coding apparatus in the embodiment.
  • FIG. 9 is a comparison graph illustrating the difference between a coding result of a first image in the present invention and a coding result of another image obtained using a conventional method
  • FIG. 10 is a comparison graph illustrating the difference between a coding result of a second image in the present invention and a coding result of another image obtained using a conventional method ;
  • FIG. 11 is a block diagram showing the structure of a moving picture decoding apparatus in the embodiment of the present invention.
  • FIG. 12 is an illustration of an interpolation method concerning the variation of the embodiment.
  • a moving picture coding apparatus and a moving picture decoding apparatus in the embodiment of the present invention will be described below with reference to figures.
  • the coding efficiency is increased by applying motion estimation and motion compensation in predictive coding.
  • the motion estimation and compensation can be improved by reducing the differential remaining between the image data to be coded and the predictive image data.
  • a 1/8 sub-pixel motion vector accuracy can further improve the coding efficiency.
  • the present invention achieves an improved motion estimation and compensation without increasing the hardware complexity and the computational effort accordingly. This is because the present invention enables to only employ a 16-bit accuracy of intermediate calculation results for this purpose.
  • FIG. 7 is a block diagram showing the configuration of the moving picture coding apparatus in this embodiment.
  • This moving picture coding apparatus 100 includes: a substractor 110; a transform/quantization unit 120; an inverse quantization/inverse transform unit 130; an adder 135; a deblocking filter 137; a memory 140; a 16-bit operation interpolation filter 150; a motion compensation/prediction unit 160; a motion estimation unit 170; and an entropy coding unit 190.
  • the subtractor 110 subtracts a prediction signal PS from an input signal IS indicating a moving picture and outputs the differential to the transform/quantization unit 120.
  • the transform/quantization unit 120 obtains the differential from the subtractor 110 and performs coding processing of frequency transform (such as DCT transform) and quantization using the differential. After that, the transform/quantization unit 120 outputs the quantized transform coefficient QC that is the processing result to the entropy coding unit 190 and the inverse quantization/inverse transform unit 130.
  • frequency transform such as DCT transform
  • the inverse quantization/inverse transform unit 130 performs decoding processing of inverse quantization and inverse DCT transform using the quantized transform coefficient QC outputted from the transform/quantization unit 120. After that the inverse quantization/inverse transform unit 130 outputs the differential decoding signal DDS that is the processing result to the adder 135.
  • the adder 135 adds the differential decoding signal DDS to the prediction signal PS obtained from the motion compensation prediction unit 160, and outputs the picture obtained as the result to the deblocking filter 137.
  • the deblocking filter 137 removes the block distortion of the picture outputted from the adder 135, and stores the picture with no block distortion in the memory 140 as a reference picture.
  • the 16-bit operation interpolation filter 150 extracts a reference picture from the memory 140 and performs 1/8 sub-pixel interpolation of the reference picture.
  • the motion estimation unit 170 estimates a motion vector based on the picture indicated by the input signal IS and the reference picture on which 1/8 sub-pixel interpolation has been performed using the 16-bit operation interpolation filter 150. After that, the motion estimation unit 170 outputs the motion data MD indicating the detected motion vector to the motion compensation/prediction unit 160 and the entropy coding unit 190.
  • the motion compensation/prediction unit 160 performs motion compensation based on the motion vector indicated by the motion data MD and the reference picture on which 1/8 sub-pixel interpolation has been performed using the 16-bit operation interpolation filter 150. In this way, the motion compensation/prediction unit 160 predicts the current picture indicated by the input signal IS and outputs the prediction signal PS indicating the prediction picture to the subtractor 110.
  • the entropy coding unit 190 performs entropy coding of the quantized transform coefficients QC outputted by the transform/quantization unit 120 and the motion data MD outputted by the motion estimation unit 170, and outputs the result as a bit stream BS.
  • the moving picture coding apparatus 100 in the embodiment like this has a feature of including a 16-bit operation interpolation filter 150.
  • the motion compensation method in this embodiment has a feature that motion compensation is performed using the 1/8 sub-pixel interpolation by this 16-bit operation interpolation filter 150.
  • the respective functional units other than the 16-bit operation interpolation filter 150 have the same functions as the respective functional units included in the above-described conventional moving picture coding apparatus.
  • the 16-bit operation interpolation filter 150 calculates a 1/4 sub-pixel value using a method other than a conventional method, and then calculates 1/8 sub-pixel value using the 1/4 sub-pixel value like in the case of the conventional method.
  • the method how this 16-bit operation interpolation filter 150 calculates 1/4 sub-pixel value will be described in detail.
  • a two-step procedure is employed for obtaining the 1/8 pixel accuracy. In a first stage including two interpolation steps, a horizontal and a vertical filtering is subsequently employed.
  • hi to hi 3 represent the 1/4 sub-pixel values to be interpolated, and A x -H x represent the original full-pel pixel values.
  • the respective coefficients of A x -H x in this embodiment are set so that unnecessary high frequency components generated through interpolation are eliminated like in the conventional method. More specifically, the coefficients are set smaller than the conventional coefficients under the condition that picture quality does not deteriorate in the coding and decoding processing. In other words, the respective coefficients in this embodiment are set smaller in proportion to the respective coefficients of the conventional Equations (1) to (3). After completing the horizontal filtering, the calculated values are rounded by being downshifted. For example, the intermediate value of hi is rounded using the following Equation (14).
  • hi represents the interpolated pixel value resulting from horizontal filtering
  • hi' represents the respectively downshifted pixel value.
  • a corresponding processing is applied to all of the interpolated pixel values resulting from horizontal filtering. Note that "»6" in the Equation (14) represents 6-bit downshifting.
  • the horizontally increased sub-pixel accuracy is also obtained in the vertical direction.
  • a vertical filtering is applied.
  • the previously performed downshift operation provides that none of the intermediate calculations exceeds a 16-bit accuracy in the vertical filtering step.
  • the vertical filtering is performed by employing the filter coefficients as shown in the following Equations (15) to (17) which correspond to Equations (11) to (13) in the case of the horizontal filtering:
  • v, - ⁇ -D v _ 3+ 3-D v _ 2 -l0-D v _ ]+ 59-D v+ ⁇ S-D v+ ,-6-D v+2 + ⁇ -D v+3 -0-D v+ ,..X ⁇ 5)
  • v 2 -l D v _ 3 +4-D v _ 2 -10- J D v _,+39-Z) v +39-/J> v+1 -10-Z ) v+2 +4-A +3 -I- ⁇ +4 ...(16)
  • V 3 -0-/ ⁇ _ 3 +l-£U-6-/ ⁇ _ 1+ 18.Z ) v +59-A +1 -IO- ⁇ +2 +3- D v+ ,-hD v+ ,...( ⁇ 7)
  • vi to V 3 refer to the 1/4 sub-pixel values in the vertical direction and D v-3 , D v -2, D v -i, D v , D v+ i, D v+ 2, D v+ 3 and D v+4 , represent the full-pel pixels in the vertical direction.
  • the full-pel pixels are pixels 210 and 220 from FIG.3.
  • the respective coefficients of D x (D v _ 3 to D v+4 ) in this embodiment are set smaller in proportion to the respective coefficients of the conventional Equations (4) to (6) like in the case of the respective coefficients of the above A x -H x .
  • Equation (18) which is illustrated as an example case OfV 1 only:
  • the finally obtained sub-pixel values with a 1/4 sub-pixel accuracy are clipped in order to fall within a range between 0 and 255.
  • the obtained 1/4 sub-pixel values are subjected to a bilinear filtering as it has been described above in connection with FIG. 5 and FIG. 6 in order to obtain a 1/8 sub-pixel resolution.
  • ⁇ 2 -l -0 + 4-255 + (-10)-0 + 39-255 + 39-255 + (-10)-0 + 4-255 -l -0...(19)
  • this embodiment can eliminate the necessity of performing the calculation over 16 bits in the calculation processing of 1/4 sub-pixel values.
  • the result of the downshift operation is clipped to the range of 0 to 255.
  • the required pixel accuracy for the largest possible values during the filtering operation does not exceed 16-bits.
  • the above example only relates to the 1/4 sub-pixel resolution calculation.
  • the bilinear filtering for generating a 1/8 sub-pixel resolution only requires a maximum accuracy of 10-bits.
  • FIG. 8 is a flow chart showing the motion compensation operation performed by the moving picture coding apparatus 100 in the embodiment.
  • the 16-bit operation interpolation filter 150 of the moving picture coding apparatus 100 calculates 1/4 sub-pixel values
  • the 16-bit operation interpolation filter 150 performs downshifting of the pixel values obtained in Step 100, and rounds the pixel values (Step 102).
  • the 16-bit operation interpolation filter 150 calculates
  • Step 104 1/4 sub-pixel values in the vertical direction using the pixel values rounded in Step 102.
  • the 16-bit operation interpolation filter 150 performs downshifting of the pixel values obtained in Step 104 and rounds the pixel values (Step 106).
  • Step 100 to Step 106 Through the operation of Step 100 to Step 106 like this, 1/4 sub-pixels of the reference picture are interpolated in the horizontal direction and the vertical direction.
  • the 16-bit operation interpolation filter 150 calculates 1/8 sub-pixels by performing bilinear filtering using the interpolated 1/4 sub-pixels like in the conventional case, in other words, the 16-bit operation interpolation filter 150 raises the pixel accuracy of the reference picture from 1/4 sub-pixel accuracy to 1/8 sub-pixel accuracy (Step 108).
  • Step 100 to Step 108 performed by the 16-bit operation interpolation filter 150 like this a reference picture with interpolated 1/8 sub-pixel values is generated.
  • the motion compensation/prediction unit 160 performs motion compensation using the reference picture with interpolated 1/8 sub-pixels and outputs the prediction signal PS indicating the result (Step 110).
  • the algorithm of the present invention has been implemented into the H. 264/MPEG encoder processing software (JM4.2).
  • JM4.2 H. 264/MPEG encoder processing software
  • the calculation results are illustrated in FIG. 9 and FIG. 10 by rate distortion curves indicating the impact on the perceived picture quality. Both figures differ only because different image sequences are employed as examples.
  • the rate distortion curves of FIG. 9 and FIG. 10 are shown over the bit rate on the X-axis and the peak signal to noise ratio (PSNR) on the Y-axis representing a measure for the introduced distortions.
  • PSNR peak signal to noise ratio
  • FIG. 9 and FIG. 10 demonstrate that the 16-bit implementation of a 1/8 sub-pixel filter (1/8-pel 16 bit) does not result in an image quality degradation compared to the conventional JM4.2 algorithm (1/8-pel 25-bit) although the JM4.2 algorithm requires longer registers.
  • the approach of the present invention actually performs better than 1/4 sub-pixel 20-bit coding (1/4-pel 20 bit).
  • FIG. 11 is a block diagram showing the configuration of a moving picture decoding apparatus in the embodiment of the present invention.
  • This moving picture decoding apparatus 300 includes: an entropy decoding unit 310; an inverse quantization/inverse transform unit 320 ; an adder 330 ; a deblocking filter 340; a memory 350 and a motion compensation unit 360.
  • the entropy decoding unit 310 obtains a bit stream BS outputted by the moving picture coding apparatus 100 and performs entropy decoding processing of the bit stream . As the result, the entropy decoding unit 310 outputs the quantized transform coefficients QC to the inverse quantization/inverse transform unit 320 and outputs the motion data MD indicating the motion vector to the motion compensation unit 360.
  • the inverse quantization/inverse transform unit 320 performs decoding processing of inverse quantization and inverse DCT transform using the quantized transform coefficients QC. After that, the inverse quantization/inverse transform unit 320 outputs the differential decoding signal DDS that is the result of the processing to the adder 330.
  • the adder 330 adds the differential decoding signal DDS to the prediction signal PS obtained from the motion compensation unit 360, and outputs the resulting picture to the deblocking filter 340.
  • the deblocking filter 340 eliminates the block distortion of the picture outputted from the adder 330, and stores the picture with no block distortion to the memory 350.
  • the decoded picture is extracted from the memory 350 as the output signal OS.
  • the motion compensation unit 360 includes : a 16-bit operation interpolation filter 361 for extracting the picture stored in the memory 350 as a reference picture and performing 1/8 sub-pixel interpolation of the reference picture; and a motion compensation prediction unit 361 for predicting the current picture.
  • This motion compensation prediction unit 361 performs motion compensation based on the motion vector indicated by the motion data M D and the reference picture on which 1/8 sub-pixel interpolation is performed using a 16-bit operation interpolation filter 361. In this way, the motion compensation/prediction unit 361 predicts the current picture and outputs the prediction signal PS indicating the prediction picture to the adder 330.
  • the moving picture decoding apparatus 300 like this also has a feature of including a 16-bit operation interpolation filter 361 like in the case of the moving picture coding apparatus 100.
  • This 16-bit operation interpolation filter 361 has the same function as the 16-bit operation interpolation filter 150 of the moving picture coding apparatus 100. Therefore, with this moving picture decoding apparatus 300, it is possible to reduce operation workload and simplify a hardware configuration without using pixel values exceeding 16 bits in the process of calculating the pixel values.
  • the present invention provides an improved motion estimation and compensation by only employing a simplified hardware configuration and less computational effort. This is achieved by employing particular filter coefficients and additional downshift operations when obtaining a 1/4 sub-pixel resolution image. Accordingly, a more efficient coding and decoding with a simpler hardware configuration can be achieved.
  • a two-step interpolation is performed in the following way: 1/4 sub-pixel values are interpolated in the horizontal direction; and then other 1/4 sub-pixel values are interpolated in the vertical direction.
  • a single-step interpolation is performed instead of the two-step interpolation in this variation, the single-step interpolation being able to achieve the same effect as the effect obtained through both the interpolation in the horizontal direction and the vertical direction.
  • the 16-bit operation interpolation filter 150 of this variation has a function as a two-dimensional filter.
  • FIG. 12 is an illustration of an interpolation method concerning the variation of the embodiment.
  • the motion compensation method concerning the present invention provides the following two effects that: operation workload can be reduced; and a hardware configuration can be simplified.
  • the motion compensation method can be applied for a moving picture coding apparatus for coding a moving picture, a moving picture decoding apparatus for decoding the coded moving picture, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

A motion compensation method for achieving the reduction of operation workload and the simplification of a hardware configuration includes: interpolation having (i) a first calculation step (S100) of calculating base values of sub-pixel values by multiplying coefficients with pixel values of pixels included in a reference picture and (ii) a first rounding step (S102) of deriving sub-pixel values of sub-pixels by rounding the base values calculated in the first calculation step (S100) in stead of directly using the base values in calculating other sub-pixel values; and motion compensation (S110) of the reference picture that has interpolated sub-pixels with the derived sub-pixel values.

Description

DESCRIPTION
MOTION COMPENSATION METHOD
Technical Field The present invention relates to a motion compensation method for interpolating sub-pixels into a reference picture and for performing motion compensation based on the interpolated reference picture.
Background Art
Moving pictures are being adopted in an increasingly number of applications ranging form video telephony and video conferencing to DVD and digital television. When moving pictures are transmitted, a substantial amount of data has to be sent through conventional transmission channels of a limited available frequency bandwidth. In order to transmit the digital data through the limited channel bandwidth, it is inevitable to compress or reduce the volume of the transmission data.
In order to enable inter-operability between systems designed by different manufactures of any given application, video-coding standards have been developed for compressing the amount of video data. The coding approach underlying most of these standards consist of the following main steps:
(1) Dividing each video frame into blocks of pixels so that processing of the video frame can be conducted at a block level;
(2) Reducing spatial redundancies within a video frame by subjecting video data of each block to transform, quantization and entropy coding;
(3) Exploiting temporal dependencies between blocks of subsequent frames in order to only transmit differentials between subsequent frames.
Temporal dependencies between blocks of subsequent frames are determined by employing a motion estimation and compensation technique. For any given block, a search is performed in previously coded and transmitted frames to determine a motion vector which will be used by the coding apparatus and the decoding apparatus to predict the image data of a block.
An example configuration of a video coding apparatus is illustrated in FIG. 1. The shown video coding apparatus generally denoted with reference numeral 900 includes: a transform/quantization unit 920 to output quantized transform coefficients QC by transforming spatial image data to the frequency domain and quantizing the transformed image data; an entropy coding unit 990 for performing entropy coding (variable length coding) of the quantized transform coefficients QC and outputting the bit stream BS; and a video buffer (not shown) for adopting the compressed video data having a variable bit rate to a transmission channel which may have a fixed bit rate.
The coding apparatus shown in FIG. 1 employs a DPCM (Differential Pulse Code Modulation) by only transmitting differentials between subsequent fields or frames. A subtractor 910 obtains these differentials by receiving the video data to be coded as an input signal IS and subtracting the previous image indicated by a prediction signal PS therefrom. The previous image is obtained by decoding the previously coded image. This is accomplished by a decoding apparatus which is incorporated into video coding apparatus 900. The decoding apparatus performs the coding steps in a reverse manner. More specifically, the decoding apparatus includes: an inverse quantization/transform unit 930, and an adder 935 for adding the decoded differential (differential decoding signal DDS) to the previously decoded picture (prediction signal PS) in order to produce the image as will be obtained on the decoding side.
In motion compensated DPCM, a current frame or field is predicted from image data of a previous frame or field based on an estimation of the motion between the current and the previous images. Such estimated motion may be described in terms of 2-dimensional motion vectors representing the displacement of pixels between the previous and the current images. Usually, motion estimation is performed on a block-by-block basis. An example of the division of the current image into plurality of blocks is illustrated in FIG. 2.
During motion estimation, a block of a current frame is compared with blocks in previous frames until a best match is determined. Based on the comparison results, an inter-frame displacement vector for the whole block can be estimated for the current frame. For this purpose, a motion estimation unit 970 is incorporated into the coding apparatus together with the corresponding motion compensation unit 960 included into the decoding path.
The video coding apparatus 900 of FIG. 1 performs operations as follows. A given video image indicated by an input signal IS is divided into a number of small blocks, usually denoted as "macro blocks". For example, video image shown in FIG. 2 is divided into a plurality of macro blocks, each of which usually having a size of 16 x 16 pixels.
When coding the video data of an image by only reducing spatial redundancies within the image, the resulting frame is referred to as an I-picture. I-pictures are typically coded by directly applying the transform to the macro blocks of a frame. I-pictures are large in size as no temporal information is exploited to reduce the amount of data.
In order to take advantage of temporal redundancies that exist between successive images, a prediction coding between subsequent fields or frames is performed based on motion estimation and compensation. When a selected reference frame in motion estimation is a previously coded frame, the frame to be coded is referred to as a P-picture. In case both, a previously coded frame and a future frame are chosen as reference frames, the frame to be coded is referred to as B-picture. Although the motion compensation has been described to be based on a 16 x 16 macro block, motion estimation and compensation can be performed using a number of different block sizes. Individual motion vectors may be determined for blocks having 4 x 4, 4 x 8, 8 x 4, 8 x 8, 8 x 16, 16 x 8, or 16 x 16 pixels. The provision of small motion compensation blocks improves the ability to handle fine motion details.
Based on the results of the motion estimation operation, the motion compensation operation provides a prediction based on the determined motion vector. The information contained in a prediction error block resulting from the predicted block is then transformed into transform coefficients in transform/quantization unit 920. Generally, a 2-dimensional DCT (Discrete Cosine Transform) is employed. The resulting transform coefficients are quantized and finally entropy coded (VLC) in entropy coding unit 990.
A decoding apparatus receives the transmitted bit stream BS of compressed video data and reproduces a sequence of coded video images based on the received data. The configuration of the decoding apparatus corresponds to that of the decoding apparatus included in the coding apparatus shown in FIG. 1. A detailed description of the configuration of the decoding apparatus is therefore omitted.
In order to improve the accuracy of motion compensation, a sub-pixel accuracy of reference frames is widely used. For example, 1/2 sub-pixel accuracy motion compensation is used in the MPEG-2 format.
In order to further increase the motion vector accuracy and coding efficiency, a 1/3 and a 1/6 sub-pixel vector accuracies have been proposed in Patent Literature EP 1073276.
The motion vector accuracy and coding efficiency can further be increased by applying interpolation filters in motion estimation and compensation yielding 1/8 sub-pixel displacements. However, such a sub-pixel resolution requires high computation complexity, in particular, calculation registers having a length of up to 25 bits.
Such a complex implementation may be based on a 2-step approach. In the first step a 1/4 sub-pixel image employing an 8-tap filter is calculated. In second step a 1/8 sub-pixel is obtained based on the 1/4 sub-pixel image by employing a bilinear filtering.
The filtering operation for generating the image with the 1/4 sub-pixel accuracy includes the steps of horizontal and subsequent vertical filtering. The horizontal interpolation may be performed based on the following Equations (1) to (3):
/*1=-3-Λ4+12-£4-37-C4+229-JD4+71-E4-21-F4+6-G4-l-H4...(l) h2 =-3 -A4 +12 B4 -39 -C4 +158 -D4 +158 -E4 -39 -F4 +12 -G4 -3 -H4...(2) /23=-l-Λ4+6-£4-21-C4+71-/J>4+229-E4-37-F4+12-G4-3-H4...(3)
In the above equation, hi to h3 denote the 1/4 sub-pixel values and Ax-Hx represent the original full-pel pixel values, namely, the pixels from the original image.
Coefficients applied to the above Ax-Hx are set in a way that the signal processing is performed preventing the occurrence of imaging by upsampling, in other words, unnecessary high frequency components generated through interpolation are eliminated.
The horizontal filtering is illustrated in FIG. 3. Eight-tap filtering is performed based on the pixel values of the original pixels
210 and the pixel values of the three intermediate pixels 220 are calculated in order to obtain a 1/4 sub-pixel accuracy in the horizontal direction. After the horizontal filtering has been completed, the resulting image data having a full-pel pixel accuracy in the vertical direction and a 1/4 sub-pixel accuracy in the horizontal direction are subjected to vertical filtering. For this purpose, the following Equations (4) to (6) having coefficients which correspond to those of the above described horizontal filter are employed.
v, = -3 - D1 + 12 - D2 - 37 - D3 + 229 - D4 + 71 - D5 - 21 - D6 + 6 - D1 -I - Ds ...(4) v2 = -3 • Dx + 12 • D2 - 39 • D3 + 158 • D, + 158 • D5 - 39 • D6 + 12 • D1 - 3 • Z)8 ... (5) v. = -l D, + 6 D2 - 21 ■ D3 + 71 • D1 + 229 • D5 - 37 • D6 + 12 • D1 - 3 • D8 ... (6)
In the above equations, V1 to V3 refer to the calculated vertical
1/4 sub-pixel values and D1, D2, D3, D4, D5, D6, D7.and D8 represent the full-pel resolution pixels , namely, the pixel values of the original pixels 210.
Like in the case described above, coefficients applied to Dx are set in a way that the signal processing is performed preventing the occurrence of imaging by upsampling, in other words, unnecessary high frequency components generated through interpolation are eliminated.
The resulting pixel values have a length of up to 25 bits. In order to obtain image data in each of the pixel values fall into a predefined range of allowable pixel values, the calculation results are downshifted and rounded as illustrated. An example case of pixel value V1 is shown by the following Equation (7) :
Figure imgf000008_0001
Here, V1 represents the pixel value resulting from the horizontal and vertical filtering, while V1' represents the downshifted pixel value. The downshifted pixel values are further clipped to the range of 0 to 255.
The vertical filtering is illustrated in FIG. 4. The pixel values of the pixels 230 obtained during vertical filtering complete the sub-pixel array illustrated by way of filtering example between original pixels D4, D5, E4 and E5.
After having the 1/4 sub-pixel image completed, a 1/8 sub-pixel frame is calculated by applying a bilinear filtering to the 1/4 sub-pixel resolution. In this manner, intermediate pixels are generated between each of the 1/4 resolution pixels. A bilinear filtering is applied in two steps and is illustrated by way of examples in FIG. 5 and FIG. 6. Starting from the 1/4 sub-pixel resolution, FIG. 5 illustrates the application of a horizontal and vertical filtering. For this purpose, a mean value is calculated from the respective neighbouring pixel values in order to obtain an intermediate pixel value of a 1/8 sub-pixel resolution. When employing a binary representation for this processing, the following Equation (8) can be applied. Note that " »l " in Equation (8) represents 1-bit downshifting.
_4 = (5 + C + l)» l...(8)
The remaining 1/8 sub-pixel values to be interpolated are calculated by diagonal filtering as illustrated in FIG. 6. It is a particular advantage of this approach that, in the bilinear filtering, the number of sub-pixel values stemming from multiple filtering is minimized as much as possible. For this purpose, it is preferable that only those pixel values, of the interpolated pixels, that are directly derived from original pixel values 210 are taken into account. In other words, those derived pixel values are the pixel values of the interpolated pixels located between the original pixels.
All intermediate pixel values can be calculated therefrom, in other words, from the pixel values of the original pixels 210 and the intermediate pixel values derived from the original pixel values, when additionally taking center pixel 240 of the sub-pixel array into account. The calculation operation for the additional 1/8 sub-pixel values is based on two of the 1/4 sub-pixel resolution values, respectively. The individual pixel values taken into account for the calculation of an intermediate pixel value are illustrated in FIG. 6 by respective arrows. Each of the arrows shows two pixel values of pixels based on which each intermediate pixel value of the two is calculated. Depending on the distance of the pixels to be taken into account for interpolation, the following Equations (9) and (10) are employed :
D = (E + F + \) » \...(9) G = (3H + / + 2)» 1...(1O)
In the above equations, D and G represent new intermediate pixel values as illustrated in FIG. 6, and E, F, Η and I represent the pixel values obtained from the 1/4 resolution image. The additional values of "1" and "2" in the above equations only serve for correctly rounding the calculation results. However, the above-described conventional motion compensation method requires to record a long operation value of 25 bits in the filtering process for 1/4 sub-pixel interpolation. This causes a particular disadvantage of such an interpolation approach that long registers are needed resulting in high hardware complexity and computational effort.
The present invention is conceived in view of this drawback. An object of the present invention is to provide a motion compensation method for reducing operational workload and simplifying a hardware configuration.
Disclosure of Invention In order to achieve the above-described object, the motion compensation method of the present invention includes: interpolating sub-pixels in a reference picture; and performing motion compensation based on the interpolated reference picture, in the method, the interpolating including : a first calculation step of calculating base values which are bases of sub-pixel values of the sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a first rounding step of deriving the sub-pixel values of the sub-pixels by rounding the base values calculated in the first calculation step instead of directly using the base values in calculating sub-pixel values of other sub-pixels; and the performing of motion compensation includes performing motion compensation based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values.
For example, in the conventional method, base values of sub-pixels that have been calculated are directly used in calculating sub-pixel values of other sub-pixels. However, in the present invention, the base values of sub-pixels that have been calculated in the first calculation step are rounded in stead of being directly used in calculating the sub-pixel values of other sub-pixel values. Therefore, even in the case where the sub-pixel values of the other sub-pixels are calculated using the base values rounded, the number of bits to be used in the calculation can be more reduced than in the conventional way. As a result, it becomes possible to reduce the operational workload and to simplify the hardware configuration.
Also, in a first aspect of the present invention, in the motion compensation method, the first calculation step may include calculating base values of sub-pixels to be interpolated in a first direction, and the first rounding step may include deriving sub-pixel values of the sub-pixels to be interpolated in the first direction by rounding the base values calculated in the first calculation step. At this time, in a second aspect of the present invention, in the motion compensation method, the interpolation may further include: a second calculation step of calculating, using the sub-pixel values of the sub-pixels derived in the first rounding step, base values of sub-pixels to be interpolated in a second direction that is different from the first direction; and a second rounding step of deriving the sub-pixel values of the sub-pixels to be interpolated in the second direction by rounding the base values calculated in the second calculation step.
In this way, in the process of calculating sub-pixel values of sub-pixels to be interpolated in the first direction and in the second direction, the number of bits to be used in the calculation can be reduced down to 16 bits from, for example, 25 bits needed in a conventional way.
Also, a fourth aspect of the present invention, in the motion compensation method, the first calculation step may include calculating the base values of three a-fourths sub-pixels using the following equations when eight pixel values of pixels arrayed in the first direction are represented as A, B, C, D, E, F, G and H respectively and the three a-fourths sub-pixel values are represented as hi, h2 and h3 respectively:
Λ1 = -l -^ + 3 - 5-10 -C + 59-I> + 18- £-6- F + l - G -0 - /-τ; h2 = -\ - A + 4 - B-IO -C + 39- D + 39-E-W- F + 4-G -\ - H; and hi = -0-A + \ -B-6-C + \8- D + 59 E-\0- F + 3 -G -\ - H.
Here, in a fifth aspect of the present invention, in the motion compensation method, the second calculation step may include calculating the base values of three a-fourths sub-pixels using the following equations when eight pixel values of pixels arrayed in the second direction are represented as Di, D2, D3, D4, D5, D6, D7 and D8 respectively and the three a-fourths sub-pixel values are represented as vi, v2 and V3 respectively: V1= -3-Z) 1 +12 -D2 -37- Z) 3 +229- Z)4 +71 D5 -21 -D6 +6- Z)7 -1-Z)8;
V2 = -3 ■ Z) 1 + 12 • Z) 2 - 39 -Z)3 +158 -D4 +158 -Z) 5 -39 -Z) 6 +12 -Z)7 -3- Z) 8 ;and
V3 = -1-Z) 1 +6- D2 -21 -Z) 3 +71 D4 +229-D5 -37 -D6 +12 -Z)7 -3 -Z)8.
In this way, the coefficients used in calculating sub-pixel values of sub-pixels are smaller than the conventional coefficients. This makes it possible to further reduce the number of bits to be used in calculating the sub-pixel values.
Also, in the fourth aspect of the present invention, the motion compensation method may further include a bilinear filtering of raising a sub-pixel accuracy by applying bilinear filtering to the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values.
In this way, the increase in sub-pixel accuracy makes it possible to prevent picture quality from deteriorating during the picture coding processing and the picture decoding processing.
Note that the present invention can be realized as a motion compensation method, a motion estimation method, a moving picture coding method and a moving picture decoding method using the motion compensation method, a program causing a computer to execute these steps of the respective methods, a recording medium for storing the program, and an apparatus for performing operations according to these methods.
Further Information about Technical Background to this Application
The disclosure of EP Application No.04016437.8 filed on July 13th, 2004 including specification, drawings and claims is incorporated herein by reference in its entirety.
Brief Description of Drawings
These and other objects, advantages and features of the invention will become apparent from the following description thereof taken in conjunction with the accompanying drawings that illustrate a specific embodiment of the invention. In the Drawings:
FIG. 1 is a block diagram showing the structure of a moving picture coding apparatus; FIG. 2 is an illustration of how a video image is divided into blocks;
FIG. 3 is an illustration of horizontal filtering for calculating a 1/4 sub-pixel accuracy in the horizontal direction ;
FIG. 4 is an illustration of vertical filtering for calculating a 1/4 sub-pixel accuracy in the vertical direction;
FIG. 5 is an illustration of horizontal and vertical filtering for calculating a 1/8 sub-pixel accuracy;
FIG. 6 is an illustration of bilinear filtering in the diagonal direction for calculating a 1/8 sub-pixel accuracy; FIG. 7 is a block diagram showing the configuration of a moving picture coding apparatus in the embodiment of the present invention;
FIG. 8 is a flow chart showing the motion compensation operation performed by the moving picture coding apparatus in the embodiment;
FIG. 9 is a comparison graph illustrating the difference between a coding result of a first image in the present invention and a coding result of another image obtained using a conventional method; FIG. 10 is a comparison graph illustrating the difference between a coding result of a second image in the present invention and a coding result of another image obtained using a conventional method ;
FIG. 11 is a block diagram showing the structure of a moving picture decoding apparatus in the embodiment of the present invention; and
FIG. 12 is an illustration of an interpolation method concerning the variation of the embodiment.
Best Mode for Carrying Out the Invention
A moving picture coding apparatus and a moving picture decoding apparatus in the embodiment of the present invention will be described below with reference to figures.
In video coding, the coding efficiency is increased by applying motion estimation and motion compensation in predictive coding. The motion estimation and compensation can be improved by reducing the differential remaining between the image data to be coded and the predictive image data. In particular, a 1/8 sub-pixel motion vector accuracy can further improve the coding efficiency.
The present invention achieves an improved motion estimation and compensation without increasing the hardware complexity and the computational effort accordingly. This is because the present invention enables to only employ a 16-bit accuracy of intermediate calculation results for this purpose.
FIG. 7 is a block diagram showing the configuration of the moving picture coding apparatus in this embodiment. This moving picture coding apparatus 100 includes: a substractor 110; a transform/quantization unit 120; an inverse quantization/inverse transform unit 130; an adder 135; a deblocking filter 137; a memory 140; a 16-bit operation interpolation filter 150; a motion compensation/prediction unit 160; a motion estimation unit 170; and an entropy coding unit 190.
The subtractor 110 subtracts a prediction signal PS from an input signal IS indicating a moving picture and outputs the differential to the transform/quantization unit 120.
The transform/quantization unit 120 obtains the differential from the subtractor 110 and performs coding processing of frequency transform (such as DCT transform) and quantization using the differential. After that, the transform/quantization unit 120 outputs the quantized transform coefficient QC that is the processing result to the entropy coding unit 190 and the inverse quantization/inverse transform unit 130.
The inverse quantization/inverse transform unit 130 performs decoding processing of inverse quantization and inverse DCT transform using the quantized transform coefficient QC outputted from the transform/quantization unit 120. After that the inverse quantization/inverse transform unit 130 outputs the differential decoding signal DDS that is the processing result to the adder 135. The adder 135 adds the differential decoding signal DDS to the prediction signal PS obtained from the motion compensation prediction unit 160, and outputs the picture obtained as the result to the deblocking filter 137.
The deblocking filter 137 removes the block distortion of the picture outputted from the adder 135, and stores the picture with no block distortion in the memory 140 as a reference picture.
The 16-bit operation interpolation filter 150 extracts a reference picture from the memory 140 and performs 1/8 sub-pixel interpolation of the reference picture. The motion estimation unit 170 estimates a motion vector based on the picture indicated by the input signal IS and the reference picture on which 1/8 sub-pixel interpolation has been performed using the 16-bit operation interpolation filter 150. After that, the motion estimation unit 170 outputs the motion data MD indicating the detected motion vector to the motion compensation/prediction unit 160 and the entropy coding unit 190.
The motion compensation/prediction unit 160 performs motion compensation based on the motion vector indicated by the motion data MD and the reference picture on which 1/8 sub-pixel interpolation has been performed using the 16-bit operation interpolation filter 150. In this way, the motion compensation/prediction unit 160 predicts the current picture indicated by the input signal IS and outputs the prediction signal PS indicating the prediction picture to the subtractor 110.
The entropy coding unit 190 performs entropy coding of the quantized transform coefficients QC outputted by the transform/quantization unit 120 and the motion data MD outputted by the motion estimation unit 170, and outputs the result as a bit stream BS.
The moving picture coding apparatus 100 in the embodiment like this has a feature of including a 16-bit operation interpolation filter 150. In other words, the motion compensation method in this embodiment has a feature that motion compensation is performed using the 1/8 sub-pixel interpolation by this 16-bit operation interpolation filter 150.
Note that, in the moving picture coding apparatus 100 in this embodiment, the respective functional units other than the 16-bit operation interpolation filter 150 have the same functions as the respective functional units included in the above-described conventional moving picture coding apparatus.
The 16-bit operation interpolation filter 150 calculates a 1/4 sub-pixel value using a method other than a conventional method, and then calculates 1/8 sub-pixel value using the 1/4 sub-pixel value like in the case of the conventional method. The method how this 16-bit operation interpolation filter 150 calculates 1/4 sub-pixel value will be described in detail. A two-step procedure is employed for obtaining the 1/8 pixel accuracy. In a first stage including two interpolation steps, a horizontal and a vertical filtering is subsequently employed. For interpolating 1/4 sub-pixel values in the horizontal direction, the following Equations ( 11) to ( 13) are applied : h, =-\-Ah +3- Bh -10-Ch +59- Dh+U-Eh -6- Fh+\-Gh-0-Hh..X\\) h2=-\-Ah+4-Bh-\0-Ch+39-Dh+39-Eh-\0-Fh+4-Gh-l-Hh..Xl2) h3=-0-Ah+\-Bh-6-Ch+18-Dh+59-Eh-\0-Fh+3-Gh-\-Hh..X\3)
In the above equations, hi to hi3 represent the 1/4 sub-pixel values to be interpolated, and Ax-Hx represent the original full-pel pixel values.
Here, the respective coefficients of Ax-Hx in this embodiment are set so that unnecessary high frequency components generated through interpolation are eliminated like in the conventional method. More specifically, the coefficients are set smaller than the conventional coefficients under the condition that picture quality does not deteriorate in the coding and decoding processing. In other words, the respective coefficients in this embodiment are set smaller in proportion to the respective coefficients of the conventional Equations (1) to (3). After completing the horizontal filtering, the calculated values are rounded by being downshifted. For example, the intermediate value of hi is rounded using the following Equation (14).
Figure imgf000018_0001
Here, hi represents the interpolated pixel value resulting from horizontal filtering, and hi' represents the respectively downshifted pixel value. A corresponding processing is applied to all of the interpolated pixel values resulting from horizontal filtering. Note that "»6" in the Equation (14) represents 6-bit downshifting.
In the second step of the first stage, the horizontally increased sub-pixel accuracy is also obtained in the vertical direction. For this purpose, a vertical filtering is applied. The previously performed downshift operation provides that none of the intermediate calculations exceeds a 16-bit accuracy in the vertical filtering step. The vertical filtering is performed by employing the filter coefficients as shown in the following Equations (15) to (17) which correspond to Equations (11) to (13) in the case of the horizontal filtering:
v,=-\-Dv_3+3-Dv_2-l0-Dv_]+59-Dv+\S-Dv+,-6-Dv+2+\-Dv+3-0-Dv+,..X\5) v2 =-l Dv_3+4-Dv_2-10-JDv_,+39-Z)v+39-/J>v+1-10-Z) v+2 +4-A+3-I-^+4...(16) V3 =-0-/Λ_3+l-£U-6-/Λ_1+18.Z) v +59-A+1-IO-^+2 +3- Dv+,-hDv+,...(\7)
Here, vi to V3 refer to the 1/4 sub-pixel values in the vertical direction and Dv-3, Dv-2, Dv-i, Dv, Dv+i, Dv+2, Dv+3 and Dv+4, represent the full-pel pixels in the vertical direction. In other words, the full-pel pixels are pixels 210 and 220 from FIG.3.
Here, the respective coefficients of Dx (Dv_3 to Dv+4) in this embodiment are set smaller in proportion to the respective coefficients of the conventional Equations (4) to (6) like in the case of the respective coefficients of the above Ax-Hx.
The calculation results from the vertical filtering, namely, pixel values 230, are subjected to downshifting by applying the following Equation (18) which is illustrated as an example case OfV1 only:
vΗvi+f]>>6-(i8)
A rounding during the downshift operation is achieved by adding the value 26/2 = 64/2 to the interpolated pixel value. Although, the above description firstly applies a horizontal filtering and secondly a vertical filtering together with respective downshift operations, a skilled person in the art is aware that the horizontal and vertical operations may be exchanged to achieve the same result. Thus the vertical filtering may be performed before a horizontal filtering.
The finally obtained sub-pixel values with a 1/4 sub-pixel accuracy are clipped in order to fall within a range between 0 and 255. The obtained 1/4 sub-pixel values are subjected to a bilinear filtering as it has been described above in connection with FIG. 5 and FIG. 6 in order to obtain a 1/8 sub-pixel resolution.
The following example demonstrates that the processing of the present invention does not require any registers for intermediate pixel values exceeding a 16-bit accuracy.
Assuming that a pixel value falls in the range between 0 and 255, the largest possible values during a horizontal 8-tap filtering may occur when employing the following Equation (19) for calculating intermediate pixel value h2:
Λ2 = -l -0 + 4-255 + (-10)-0 + 39-255 + 39-255 + (-10)-0 + 4-255 -l -0...(19) h2 = 21930 < 32768 = 215 => \5bit + \bit(sign)... (20)
In this way, this embodiment can eliminate the necessity of performing the calculation over 16 bits in the calculation processing of 1/4 sub-pixel values.
The resulting pixel value is downshifted as indicated by the following Equation (21) :
[ 21930 + — | » 6 = 343...(21) I 2 )
The result of the downshift operation is clipped to the range of 0 to 255.
As demonstrated above, the required pixel accuracy for the largest possible values during the filtering operation does not exceed 16-bits. Although the above operation example has only been calculated for the horizontal direction, corresponding coefficients are used for the vertical filtering and, thus, identical advantages are applied to the vertical filtering.
The above example only relates to the 1/4 sub-pixel resolution calculation. The bilinear filtering for generating a 1/8 sub-pixel resolution only requires a maximum accuracy of 10-bits.
Thus, a maximum accuracy of 16-bits is sufficient for performing all calculations of the present invention. Accordingly, the motion estimation, motion compensation and the coding and decoding of data moving picture can be improved in a simple manner.
FIG. 8 is a flow chart showing the motion compensation operation performed by the moving picture coding apparatus 100 in the embodiment.
First, the 16-bit operation interpolation filter 150 of the moving picture coding apparatus 100 calculates 1/4 sub-pixel values
(base values which are bases of sub-pixel values) of the reference picture extracted from the memory 140 in the horizontal direction
(SlOO). After that, the 16-bit operation interpolation filter 150 performs downshifting of the pixel values obtained in Step 100, and rounds the pixel values (Step 102).
Next, the 16-bit operation interpolation filter 150 calculates
1/4 sub-pixel values in the vertical direction using the pixel values rounded in Step 102 (Step 104). After that, the 16-bit operation interpolation filter 150 performs downshifting of the pixel values obtained in Step 104 and rounds the pixel values (Step 106).
Through the operation of Step 100 to Step 106 like this, 1/4 sub-pixels of the reference picture are interpolated in the horizontal direction and the vertical direction.
When 1/4 sub-pixels are interpolated, the 16-bit operation interpolation filter 150 calculates 1/8 sub-pixels by performing bilinear filtering using the interpolated 1/4 sub-pixels like in the conventional case, in other words, the 16-bit operation interpolation filter 150 raises the pixel accuracy of the reference picture from 1/4 sub-pixel accuracy to 1/8 sub-pixel accuracy (Step 108).
Through Step 100 to Step 108 performed by the 16-bit operation interpolation filter 150 like this, a reference picture with interpolated 1/8 sub-pixel values is generated.
After that, the motion compensation/prediction unit 160 performs motion compensation using the reference picture with interpolated 1/8 sub-pixels and outputs the prediction signal PS indicating the result (Step 110). For demonstrating that similar results compared to conventional interpolation implementations can be achieved when applying the present invention, the algorithm of the present invention has been implemented into the H. 264/MPEG encoder processing software (JM4.2). The calculation results are illustrated in FIG. 9 and FIG. 10 by rate distortion curves indicating the impact on the perceived picture quality. Both figures differ only because different image sequences are employed as examples.
The rate distortion curves of FIG. 9 and FIG. 10 are shown over the bit rate on the X-axis and the peak signal to noise ratio (PSNR) on the Y-axis representing a measure for the introduced distortions.
FIG. 9 and FIG. 10 demonstrate that the 16-bit implementation of a 1/8 sub-pixel filter (1/8-pel 16 bit) does not result in an image quality degradation compared to the conventional JM4.2 algorithm (1/8-pel 25-bit) although the JM4.2 algorithm requires longer registers. In addition, the approach of the present invention actually performs better than 1/4 sub-pixel 20-bit coding (1/4-pel 20 bit).
FIG. 11 is a block diagram showing the configuration of a moving picture decoding apparatus in the embodiment of the present invention.
This moving picture decoding apparatus 300 includes: an entropy decoding unit 310; an inverse quantization/inverse transform unit 320 ; an adder 330 ; a deblocking filter 340; a memory 350 and a motion compensation unit 360.
The entropy decoding unit 310 obtains a bit stream BS outputted by the moving picture coding apparatus 100 and performs entropy decoding processing of the bit stream . As the result, the entropy decoding unit 310 outputs the quantized transform coefficients QC to the inverse quantization/inverse transform unit 320 and outputs the motion data MD indicating the motion vector to the motion compensation unit 360.
The inverse quantization/inverse transform unit 320 performs decoding processing of inverse quantization and inverse DCT transform using the quantized transform coefficients QC. After that, the inverse quantization/inverse transform unit 320 outputs the differential decoding signal DDS that is the result of the processing to the adder 330.
The adder 330 adds the differential decoding signal DDS to the prediction signal PS obtained from the motion compensation unit 360, and outputs the resulting picture to the deblocking filter 340. The deblocking filter 340 eliminates the block distortion of the picture outputted from the adder 330, and stores the picture with no block distortion to the memory 350. The decoded picture is extracted from the memory 350 as the output signal OS.
The motion compensation unit 360 includes : a 16-bit operation interpolation filter 361 for extracting the picture stored in the memory 350 as a reference picture and performing 1/8 sub-pixel interpolation of the reference picture; and a motion compensation prediction unit 361 for predicting the current picture. This motion compensation prediction unit 361 performs motion compensation based on the motion vector indicated by the motion data M D and the reference picture on which 1/8 sub-pixel interpolation is performed using a 16-bit operation interpolation filter 361. In this way, the motion compensation/prediction unit 361 predicts the current picture and outputs the prediction signal PS indicating the prediction picture to the adder 330.
The moving picture decoding apparatus 300 like this also has a feature of including a 16-bit operation interpolation filter 361 like in the case of the moving picture coding apparatus 100. This 16-bit operation interpolation filter 361 has the same function as the 16-bit operation interpolation filter 150 of the moving picture coding apparatus 100. Therefore, with this moving picture decoding apparatus 300, it is possible to reduce operation workload and simplify a hardware configuration without using pixel values exceeding 16 bits in the process of calculating the pixel values.
Summarizing, the present invention provides an improved motion estimation and compensation by only employing a simplified hardware configuration and less computational effort. This is achieved by employing particular filter coefficients and additional downshift operations when obtaining a 1/4 sub-pixel resolution image. Accordingly, a more efficient coding and decoding with a simpler hardware configuration can be achieved.
(Variation)
Here, an variation of the method for interpolating 1/4 sub-pixel values in the embodiment will be described below.
In the above-described embodiment, a two-step interpolation is performed in the following way: 1/4 sub-pixel values are interpolated in the horizontal direction; and then other 1/4 sub-pixel values are interpolated in the vertical direction. However, a single-step interpolation is performed instead of the two-step interpolation in this variation, the single-step interpolation being able to achieve the same effect as the effect obtained through both the interpolation in the horizontal direction and the vertical direction. In other words, the 16-bit operation interpolation filter 150 of this variation has a function as a two-dimensional filter.
FIG. 12 is an illustration of an interpolation method concerning the variation of the embodiment.
In this FIG. 12, white circles show pixels of full pixel unit that are present in a reference picture, and the pixel values of the pixels that are present in the horizontal position h and the vertical position v are represented as Ph, v- Also, the number of taps of the two-dimensional filter is 36 (6 taps in both the horizontal direction and the vertical direction). In this case, the 16-bit operation interpolation filter 150 calculates pixel values PhV, ij (i, j = 0 to 3, excluding "i = 0 and j = 0") of sub-pixels to be interpolated using the following Equation (22). Here, QJ (m, n) is a filter coefficients (m, n = -2 to 3) and generally vary depending on the position (i, j) of the pixel to be interpolated. After that, the sub-pixel values calculated in this way are downshifted.
Figure imgf000025_0001
In this variation like the case of the conventional example, those calculated sub-pixel values are always rounded and not used for calculating the pixel values of other sub-pixels. Thus, it is possible to reduce the number of bits necessary for the calculation process of sub-pixels. Although only an exemplary embodiment of this invention has been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiment without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention. Industrial Applicability
The motion compensation method concerning the present invention provides the following two effects that: operation workload can be reduced; and a hardware configuration can be simplified. For example, the motion compensation method can be applied for a moving picture coding apparatus for coding a moving picture, a moving picture decoding apparatus for decoding the coded moving picture, and the like.

Claims

1. A motion compensation method comprising : interpolating sub-pixels in a reference picture; and performing motion compensation based on the interpolated reference picture, wherein said interpolating includes: a first calculation step of calculating base values which are bases of sub-pixel values of the sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a first rounding step of deriving the sub-pixel values of the sub-pixels by rounding the base values calculated in said first calculation step instead of directly using the base values in calculating sub-pixel values of other sub-pixels, and said performing of motion compensation includes performing motion compensation based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values.
2. The motion compensation method according to Claim 1 , wherein said first calculation step includes calculating base values of sub-pixels to be interpolated in a first direction, and said first rounding step includes deriving sub-pixel values of the sub-pixels to be interpolated in the first direction by rounding the base values calculated in said first calculation step.
3. The motion compensation method according to Claim 2, wherein said interpolation further includes : a second calculation step of calculating, using the sub-pixel values of the sub-pixels derived in said first rounding step, base values of sub-pixels to be interpolated in a second direction that is different from the first direction; and a second rounding step of deriving the sub-pixel values of the sub-pixels to be interpolated in the second direction by rounding the base values calculated in said second calculation step.
4. The motion compensation method according to Claim 3, wherein said first calculation step includes calculating three base values of a-fourths sub-pixels that are arrayed in the second direction, and said second calculation step includes calculating three base values of a-fourths sub-pixels that are arrayed in the second direction.
5. The motion compensation method according to Claim 4, wherein said first calculation step includes calculating the base values of three a-fourths sub-pixels using the following equations when eight pixel values of pixels arrayed in the first direction are represented as A, B, C, D, E, F, G and H respectively and the three a-fourths sub-pixel values are represented as hi, hi2 and hi3 respectively: hi = -l - A + 3 -B-\0 -C + 59 D + n- E- 6-F + \ -G -0-H; h2 = -I - A + 4-B-\0-C + 39- D + 39- E -\0 - F + 4 G -\ - H; and hJ = -0- A + \ - B-6-C + n -D + 59 E-\0- F + 3 -G -\ -H.
6. The motion compensation method according to Claim 5, wherein said second calculation step includes calculating the base values of three a-fourths sub-pixels using the following equations when eight pixel values of pixels arrayed in the second direction are represented as Di, D2, D3, D4, D5, D6, D7 and D8 respectively and the three a-fourths sub-pixel values are represented as Vi, V2 and V3 respectively:
V1 = -3 - D1 + 12 - D2 - 37 - D3 + 229 - D4 + 71 - D5 - 21 - D6 + 6 - D7 - 1 - D8 ; v2 = -3 • D1 + 12 • D2 - 39 • D3 + 158 • D4 + 158 ■ D5 - 39 • Z) 6 + 12 • D1 - 3 ■ D8 ;α/κ/
V3 = - 1 • D1 + 6 • D2 - 21 ■ D3 + 71 • D4 + 229 • D5 - 37 • D6 + 12 • D7 - 3 • D8 .
7. The motion compensation method according to Claim 6, wherein said first calculation step includes calculating base values of the sub-pixels to be interpolated in a horizontal direction, the horizontal direction being determined as the first direction, and said second calculation step includes calculating base values of the sub-pixels to be interpolated in a vertical direction, the vertical direction being determined as the second direction.
8. The motion compensation method according to Claim 4, further comprising a bilinear filtering of raising a sub-pixel accuracy by applying bilinear filtering to the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values.
9. The motion compensation method according to Claim 8, wherein said bilinear filtering includes raising the sub-pixel accuracy of the reference picture from a a-fourths sub-pixel accuracy to an a-eighths sub-pixel accuracy.
10. The motion compensation method according to Claim 1, wherein said first rounding step includes rounding the base values of the sub-pixels by means of downshifting.
11. The motion compensation method according to Claim 1, wherein said first calculation step includes calculating base values of sub-pixels that should be arrayed in a horizontal direction and in a vertical direction by multiplying coefficients with pixel values of pixels included in the reference picture.
12. A motion estimation method comprising : interpolating sub-pixels in a reference picture; and performing motion estimation based on the interpolated reference picture, wherein said interpolating includes: a calculation step of calculating base values which are bases of sub-pixel values of the sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a rounding step of deriving the sub-pixel values of the sub-pixels by rounding the base values calculated in said calculation step instead of directly using the base values in calculating sub-pixel values of other sub-pixels, and said performing of motion estimation includes performing motion estimation based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values.
13. A moving picture coding method comprising : obtaining a picture to be coded ; interpolating sub-pixels in a reference picture; performing motion compensation based on the interpolated reference picture; and coding a picture based on the reference picture, wherein said interpolating includes : a calculation step of calculating base values which are bases of sub-pixel values of the sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a rounding step of deriving the sub-pixel values of the sub-pixels by rounding the base values calculated in said calculation step instead of directly using the base values in calculating sub-pixel values of other sub-pixels, and said performing of motion compensation includes performing motion compensation of the picture based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values, and said coding includes coding a differential between the picture to be coded that has been obtained in said picture obtaining and the reference picture of which motion compensation has been performed in said performing of motion compensation .
14. A moving picture decoding method comprising : obtaining a differential picture that is a resultant from coding the differential between a picture and another picture; interpolating sub-pixels in a reference picture; performing motion compensation based on the interpolated reference picture; and decoding a coded picture based on a reference picture wherein said interpolating includes : a calculation step of calculating base values which are bases of sub-pixel values of sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a rounding step of deriving the sub-pixel values of the sub-pixels by rounding the base values calculated in said calculation step instead of directly using the base values in calculating sub-pixel values of other sub-pixels, said performing of motion compensation includes performing motion compensation of the picture based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values, and said decoding includes decoding the differential picture obtained in said differential picture obtaining and adding the decoded differential picture to the reference picture of which motion compensation has been performed in said performing of motion compensation.
15. A motion compensation apparatus comprising : an interpolation unit operable to interpolate sub-pixels in a reference picture; and a motion compensation unit operable to perform motion compensation based on the interpolated reference picture, wherein said interpolation unit includes: a calculation unit operable to calculate base values which are bases of sub-pixel values of the sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a rounding unit operable to derive the sub-pixel values of the sub-pixels by rounding the base values calculated by said calculation unit instead of directly using the base values in calculating sub-pixel values of other sub-pixels, and said motion compensation unit is operable to perform motion compensation of the picture based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values.
16. A motion estimation apparatus comprising : an interpolation unit operable to interpolate pixels in a reference picture; and an motion estimation unit operable to perform motion compensation based on the interpolated reference picture, wherein said interpolation unit includes: a calculation unit operable to calculate base values which are bases of sub-pixel values of sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a rounding unit operable to derive the sub-pixel values of the sub-pixels by rounding the base values calculated by said calculation unit instead of directly using the base values in calculating sub-pixel values of other sub-pixels, and said motion estimation unit is operable to perform motion estimation based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values.
17. A moving picture coding apparatus comprising : a picture obtainment unit operable to obtain the picture to be coded; an interpolation unit operable to interpolate sub-pixels in a reference picture; a motion compensation unit operable to perform motion compensation based on the interpolated reference picture; and a coding unit operable to code a picture based on a reference picture, wherein said interpolation unit includes: a calculation unit operable to calculate base values which are bases of sub-pixel values of the sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a rounding unit operable to derive the sub-pixel values of the sub-pixels by rounding the base values calculated by said calculation sub-unit instead of directly using the base values in calculating pixel values of other sub-pixels, said motion compensation unit is operable to perform motion compensation of the picture based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values, and said coding unit is operable to code a differential between the picture to be coded that has been obtained by said picture obtainment unit and the reference picture of which motion compensation has been performed by said motion compensation unit.
18. A moving picture decoding apparatus comprising : a differential picture obtainment unit operable to obtain a differential picture that is a resultant from coding the differential between a picture and another picture; an interpolation unit operable to interpolate sub-pixels in a reference picture; a motion compensation unit operable to perform motion compensation based on the interpolated reference picture; and a decoding unit operable to decode a coded picture based on a reference picture, wherein said interpolation unit includes : a calculation unit operable to calculate base values which are bases of sub-pixel values of sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a rounding unit operable to derive the sub-pixel values of the sub-pixels by rounding the base values calculated by said calculation unit instead of directly using the base values in calculating sub-pixel values of other sub-pixels, said motion compensation unit is operable to perform motion compensation of the picture based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values, and said decoding unit is operable to decode the differential picture obtained by said differential picture obtainment unit and operable to add the decoded differential picture to the reference picture of which motion compensation has been performed by said motion compensation unit.
19. A motion compensation program for causing a computer to execute interpolating sub-pixels in a reference picture and performing motion compensation based on the interpolated reference picture, wherein said interpolating includes : a calculation step of calculating base values which are bases of sub-pixel values of the sub-pixels by multiplying coefficients with pixel values of pixels included in the reference picture; and a rounding step of rounding the base values of the sub-pixel values calculated in said calculation step instead of directly using the base values in calculating sub-pixel values of other sub-pixels, and said performing of motion compensation includes performing motion compensation of the picture based on the reference picture having the interpolated sub-pixels with the correspondingly derived sub-pixel values.
PCT/JP2005/012873 2004-07-13 2005-07-06 Motion compensation method WO2006006609A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN200580006213.4A CN1926875B (en) 2004-07-13 2005-07-06 Motion compensation method
US10/590,524 US20070133687A1 (en) 2004-07-13 2005-07-06 Motion compensation method
EP05760112A EP1766992A1 (en) 2004-07-13 2005-07-06 Motion compensation method
JP2007521057A JP2008507190A (en) 2004-07-13 2005-07-06 Motion compensation method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP04016437.8 2004-07-13
EP04016437A EP1617672A1 (en) 2004-07-13 2004-07-13 Motion estimator/compensator including a 16-bit 1/8 pel interpolation filter

Publications (1)

Publication Number Publication Date
WO2006006609A1 true WO2006006609A1 (en) 2006-01-19

Family

ID=34925736

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2005/012873 WO2006006609A1 (en) 2004-07-13 2005-07-06 Motion compensation method

Country Status (5)

Country Link
US (1) US20070133687A1 (en)
EP (2) EP1617672A1 (en)
JP (1) JP2008507190A (en)
CN (2) CN101945290A (en)
WO (1) WO2006006609A1 (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010516082A (en) * 2007-01-09 2010-05-13 テレフオンアクチーボラゲット エル エム エリクソン（パブル） Adaptive filter expression
JP2011505756A (en) * 2007-11-30 2011-02-24 ドルビー・ラボラトリーズ・ライセンシング・コーポレーション Temporal image detection
JP2011517239A (en) * 2008-04-10 2011-05-26 クゥアルコム・インコーポレイテッド Advanced interpolation techniques for motion compensation in video coding
US20120224098A1 (en) * 2011-03-04 2012-09-06 General Instrument Corporation Method and system for interpolating fractional video pixels
WO2012118932A1 (en) * 2011-03-03 2012-09-07 General Instrument Corporation Method for interpolating half pixels and quarter pixels
WO2012177301A1 (en) * 2011-06-22 2012-12-27 General Instrument Corporation Fractional pixel interpolation filter for video compression
JP5323252B2 (en) * 2010-03-30 2013-10-23 株式会社東芝 Video encoding method, decoding method, encoding device, and decoding device
JP2014531153A (en) * 2011-09-14 2014-11-20 サムスン エレクトロニクスカンパニー リミテッド Video encoding method, video decoding method and apparatus
US9264725B2 (en) 2011-06-24 2016-02-16 Google Inc. Selection of phase offsets for interpolation filters for motion compensation
US9313519B2 (en) 2011-03-11 2016-04-12 Google Technology Holdings LLC Interpolation filter selection using prediction unit (PU) size
US9319711B2 (en) 2011-07-01 2016-04-19 Google Technology Holdings LLC Joint sub-pixel interpolation filter for temporal prediction
US10009622B1 (en) 2015-12-15 2018-06-26 Google Llc Video coding with degradation of residuals
US10440388B2 (en) 2008-04-10 2019-10-08 Qualcomm Incorporated Rate-distortion defined interpolation for video coding based on fixed filter or adaptive filter

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9071847B2 (en) 2004-10-06 2015-06-30 Microsoft Technology Licensing, Llc Variable coding resolution in video codec
US8243820B2 (en) 2004-10-06 2012-08-14 Microsoft Corporation Decoding variable coded resolution video with native range/resolution post-processing operation
US7956930B2 (en) 2006-01-06 2011-06-07 Microsoft Corporation Resampling and picture resizing operations for multi-resolution video coding and decoding
KR100899730B1 (en) * 2006-10-20 2009-05-27 삼성전자주식회사 Method Chroma Interpolation of Image Encoder and Decoder
EP2092752A2 (en) * 2006-12-01 2009-08-26 France Telecom Adaptive interpolation method and system for motion compensated predictive video coding and decoding
US8107571B2 (en) 2007-03-20 2012-01-31 Microsoft Corporation Parameterized filters and signaling techniques
KR101365444B1 (en) * 2007-11-19 2014-02-21 삼성전자주식회사 Method and apparatus for encoding/decoding moving image efficiently through adjusting a resolution of image
KR101386891B1 (en) * 2007-12-13 2014-04-18 삼성전자주식회사 Method and apparatus for interpolating image
RU2477577C2 (en) * 2008-04-10 2013-03-10 Квэлкомм Инкорпорейтед Interpolation-like filtering of integer pixel positions in video coding
US8705622B2 (en) 2008-04-10 2014-04-22 Qualcomm Incorporated Interpolation filter support for sub-pixel resolution in video coding
US20090257499A1 (en) * 2008-04-10 2009-10-15 Qualcomm Incorporated Advanced interpolation techniques for motion compensation in video coding
US8804831B2 (en) 2008-04-10 2014-08-12 Qualcomm Incorporated Offsets at sub-pixel resolution
US9077971B2 (en) 2008-04-10 2015-07-07 Qualcomm Incorporated Interpolation-like filtering of integer-pixel positions in video coding
US8831086B2 (en) 2008-04-10 2014-09-09 Qualcomm Incorporated Prediction techniques for interpolation in video coding
EP2141927A1 (en) * 2008-07-03 2010-01-06 Panasonic Corporation Filters for video coding
US8788555B2 (en) * 2008-07-29 2014-07-22 Orange Method for updating an encoder by filter interpolation
US8548041B2 (en) * 2008-09-25 2013-10-01 Mediatek Inc. Adaptive filter
US9078007B2 (en) * 2008-10-03 2015-07-07 Qualcomm Incorporated Digital video coding with interpolation filters and offsets
US9219921B2 (en) * 2010-04-12 2015-12-22 Qualcomm Incorporated Mixed tap filters
US9049454B2 (en) 2011-01-19 2015-06-02 Google Technology Holdings Llc. High efficiency low complexity interpolation filters
ES2869948T3 (en) * 2011-06-24 2021-10-26 Ntt Docomo Inc Method and apparatus for prediction with motion compensation
US9325991B2 (en) 2012-04-11 2016-04-26 Qualcomm Incorporated Motion vector rounding
JP5612177B2 (en) * 2013-07-17 2014-10-22 株式会社東芝 Video encoding method, decoding method, encoding device, and decoding device
WO2020071672A1 (en) * 2018-10-02 2020-04-09 엘지전자 주식회사 Method for compressing motion vector and apparatus therefor

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5353026A (en) * 1992-12-15 1994-10-04 Analog Devices, Inc. Fir filter with quantized coefficients and coefficient quantization method
EP1359763A2 (en) * 2002-04-10 2003-11-05 Microsoft Corporation Approximate bicubic filter

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6968008B1 (en) * 1999-07-27 2005-11-22 Sharp Laboratories Of America, Inc. Methods for motion estimation with adaptive motion accuracy
CN1205818C (en) * 2000-04-11 2005-06-08 皇家菲利浦电子有限公司 Video encoding and decoding method
KR100791666B1 (en) * 2000-04-14 2008-01-03 지멘스 악티엔게젤샤프트 Method and device for storing and processing image information of temporally successive images
JP3936335B2 (en) * 2001-11-30 2007-06-27 株式会社エヌ・ティ・ティ・ドコモ Moving picture encoding apparatus, moving picture decoding apparatus, moving picture encoding method, moving picture decoding method, program, and computer-readable recording medium storing program
US7116831B2 (en) * 2002-04-10 2006-10-03 Microsoft Corporation Chrominance motion vector rounding
US7305034B2 (en) * 2002-04-10 2007-12-04 Microsoft Corporation Rounding control for multi-stage interpolation
US7620109B2 (en) * 2002-04-10 2009-11-17 Microsoft Corporation Sub-pixel interpolation in motion estimation and compensation
JP4724351B2 (en) * 2002-07-15 2011-07-13 三菱電機株式会社 Image encoding apparatus, image encoding method, image decoding apparatus, image decoding method, and communication apparatus

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5353026A (en) * 1992-12-15 1994-10-04 Analog Devices, Inc. Fir filter with quantized coefficients and coefficient quantization method
EP1359763A2 (en) * 2002-04-10 2003-11-05 Microsoft Corporation Approximate bicubic filter

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
"H.26L TEST MODEL LONG TERM NUMBER 8 (TML-8) DRAFT0", ITU-T TELECOMMUNICATION STANDARIZATION SECTOR OF ITU, GENEVA, CH, 2 April 2001 (2001-04-02), pages 1 - 54, XP001089814 *
ALVAREZ J R: "Discrepancies in Documentation and Implementation of Sub-pel interpolation in TML-8 (Draft 0), VCEG-N71", ITU - TELECOMMUNICATIONS STANDARDIZATION SECTOR, STUDY GROUP 16 QUESTION 6, VIDEO CODING EXPERTS GROUP VCEG, 24 September 2001 (2001-09-24), XP002301983 *
OPPENHEIM A V ET AL: "EFFECTS OF FINITE REGISTER LENGTH IN DIGITAL FILTERING AND THE FASTFOURIER TRANSFORM", PROCEEDINGS OF THE IEEE, IEEE. NEW YORK, US, vol. 60, no. 8, August 1972 (1972-08-01), pages 957 - 976, XP000971821, ISSN: 0018-9219 *
WAN Y ET AL: "DESIGN OF N-D DIGITAL FILTERS WITH FINITE WORDLENGTH COEFFICIENTS", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, IEEE INC. NEW YORK, US, vol. 36, no. 3, 1 March 1989 (1989-03-01), pages 375 - 382, XP000110738 *
WEDI T: "Results of core experiment on Adaptive Motion Accuracy (AMA) with 1/2, 1/4 and 1/8-pel accuracy", ITU - TELECOMMUNICATIONS STANDARDIZATION SECTOR, STUDY GROUP 16, VIDEO CODING EXPERTS GROUP QUESTION 15, DOCUMENT Q15J15, 16 May 2000 (2000-05-16), XP002301984 *

Cited By (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010516082A (en) * 2007-01-09 2010-05-13 テレフオンアクチーボラゲット エル エム エリクソン（パブル） Adaptive filter expression
JP2011505756A (en) * 2007-11-30 2011-02-24 ドルビー・ラボラトリーズ・ライセンシング・コーポレーション Temporal image detection
JP2011517239A (en) * 2008-04-10 2011-05-26 クゥアルコム・インコーポレイテッド Advanced interpolation techniques for motion compensation in video coding
US8971412B2 (en) 2008-04-10 2015-03-03 Qualcomm Incorporated Advanced interpolation techniques for motion compensation in video coding
US11683519B2 (en) 2008-04-10 2023-06-20 Qualcomm Incorporated Rate-distortion defined interpolation for video coding based on fixed filter or adaptive filter
US10440388B2 (en) 2008-04-10 2019-10-08 Qualcomm Incorporated Rate-distortion defined interpolation for video coding based on fixed filter or adaptive filter
JP2013243720A (en) * 2008-04-10 2013-12-05 Qualcomm Inc Advanced interpolation techniques for motion compensation in video coding
JP5323252B2 (en) * 2010-03-30 2013-10-23 株式会社東芝 Video encoding method, decoding method, encoding device, and decoding device
US9100650B2 (en) 2010-03-30 2015-08-04 Kabushiki Kaisha Toshiba Video encoding method, decoding method, and apparatus
WO2012118932A1 (en) * 2011-03-03 2012-09-07 General Instrument Corporation Method for interpolating half pixels and quarter pixels
US8565558B2 (en) 2011-03-04 2013-10-22 Motorola Mobility Llc Method and system for interpolating fractional video pixels
US20130216156A1 (en) * 2011-03-04 2013-08-22 General Instrument Corporation Method and system for interpolating fractional video pixels
US8437581B2 (en) 2011-03-04 2013-05-07 General Instrument Corporation Method and system for interpolating fractional video pixels
WO2012121965A1 (en) * 2011-03-04 2012-09-13 General Instrument Corporation Method and system for interpolating fractional video pixels
US20120224098A1 (en) * 2011-03-04 2012-09-06 General Instrument Corporation Method and system for interpolating fractional video pixels
US9313519B2 (en) 2011-03-11 2016-04-12 Google Technology Holdings LLC Interpolation filter selection using prediction unit (PU) size
US9036706B2 (en) 2011-06-22 2015-05-19 Google Inc. Fractional pixel interpolation filter for video compression
WO2012177301A1 (en) * 2011-06-22 2012-12-27 General Instrument Corporation Fractional pixel interpolation filter for video compression
US9264725B2 (en) 2011-06-24 2016-02-16 Google Inc. Selection of phase offsets for interpolation filters for motion compensation
US9319711B2 (en) 2011-07-01 2016-04-19 Google Technology Holdings LLC Joint sub-pixel interpolation filter for temporal prediction
US9538187B2 (en) 2011-09-14 2017-01-03 Samsung Electronics Co., Ltd. Method and device for encoding and decoding video
US9538188B2 (en) 2011-09-14 2017-01-03 Samsung Electronics Co., Ltd. Method and device for encoding and decoding video
US9538184B2 (en) 2011-09-14 2017-01-03 Samsung Electronics Co., Ltd. Method and device for encoding and decoding video
US9544600B2 (en) 2011-09-14 2017-01-10 Samsung Electronics Co., Ltd. Method and device for encoding and decoding video
US9578332B2 (en) 2011-09-14 2017-02-21 Samsung Electronics Co., Ltd. Method and device for encoding and decoding video
JP2017063468A (en) * 2011-09-14 2017-03-30 サムスン エレクトロニクス カンパニー リミテッド Video encoding method, video decoding method and device
JP2014531153A (en) * 2011-09-14 2014-11-20 サムスン エレクトロニクスカンパニー リミテッド Video encoding method, video decoding method and apparatus
US10009622B1 (en) 2015-12-15 2018-06-26 Google Llc Video coding with degradation of residuals

Also Published As

Publication number Publication date
CN101945290A (en) 2011-01-12
EP1617672A1 (en) 2006-01-18
US20070133687A1 (en) 2007-06-14
CN1926875B (en) 2010-09-01
CN1926875A (en) 2007-03-07
EP1766992A1 (en) 2007-03-28
JP2008507190A (en) 2008-03-06

Similar Documents

Publication Publication Date Title
EP1766992A1 (en) Motion compensation method
EP1797722B1 (en) Adaptive overlapped block matching for accurate motion compensation
US8325805B2 (en) Video encoding/decoding apparatus and method for color image
US8244048B2 (en) Method and apparatus for image encoding and image decoding
US20100135398A1 (en) Method for determining filter coefficient of two-dimensional adaptive interpolation filter
US20110176614A1 (en) Image processing device and method, and program
JP5579936B2 (en) Optimized deblocking filter
JP5623640B2 (en) Video coding using high resolution reference frames
CN101964863B (en) Self-adaptive time-space domain video image denoising method
US7958177B2 (en) Method of parallelly filtering input data words to obtain final output data words containing packed half-pel pixels
US20100086048A1 (en) System and Method for Video Image Processing
US7450639B2 (en) Advanced noise estimation method and apparatus based on motion compensation, and method and apparatus to encode a video using the same
JP2007515115A (en) Improvement of calculation method of interpolated pixel value
KR19980017213A (en) Image Decoding System with Compensation Function for Degraded Image
KR20050084396A (en) Digital filter with spatial scalability
EP2953365A1 (en) Moving image coding device and method
US6061401A (en) Method and apparatus for selectively encoding/decoding a video signal
JP4161676B2 (en) Signal processing apparatus, method and program thereof
JP2006180173A (en) Device and method for encoding dynamic image, and device and method for decoding dynamic image
KR100734876B1 (en) A motion vector estimation method and continuous picture generation method based on convexity property of sub pixel
EP0734179A2 (en) Method and apparatus for selectively encoding/decoding a video signal
JPH09261661A (en) Method for forming bidirectional coding picture from two reference pictures
KR0174444B1 (en) Motion compensated apparatus for very low speed transmission
JP2003235047A (en) Encoding and decoding method for moving picture
JPH11317957A (en) Moving picture encoding and decoding method

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

ENP Entry into the national phase

Ref document number: 2007521057

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2005760112

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2007133687

Country of ref document: US

Ref document number: 10590524

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 200580006213.4

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

WWP Wipo information: published in national office

Ref document number: 2005760112

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 10590524

Country of ref document: US