WO2010058895A2 - Apparatus and method for encoding/decoding a video signal - Google Patents

Apparatus and method for encoding/decoding a video signal Download PDF

Info

Publication number
WO2010058895A2
WO2010058895A2 PCT/KR2009/004576 KR2009004576W WO2010058895A2 WO 2010058895 A2 WO2010058895 A2 WO 2010058895A2 KR 2009004576 W KR2009004576 W KR 2009004576W WO 2010058895 A2 WO2010058895 A2 WO 2010058895A2
Authority
WO
WIPO (PCT)
Prior art keywords
image
template
filter coefficient
unit
motion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2009/004576
Other languages
French (fr)
Other versions
WO2010058895A3 (en
Inventor
Il-Hong Shin
Jung-Won Kang
Jeong-Ju Yoo
Jin-Woo Hong
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Electronics and Telecommunications Research Institute ETRI
Original Assignee
Electronics and Telecommunications Research Institute ETRI
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Electronics and Telecommunications Research Institute ETRI filed Critical Electronics and Telecommunications Research Institute ETRI
Priority to US13/130,905 priority Critical patent/US20110228854A1/en
Publication of WO2010058895A2 publication Critical patent/WO2010058895A2/en
Anticipated expiration legal-status Critical
Publication of WO2010058895A3 publication Critical patent/WO2010058895A3/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/573—Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • the present invention relates to an apparatus and method for encoding and decoding a video signal, and more particularly, to an apparatus and method for encoding and decoding a video signal that may estimate a motion using a template and may perform an adaptive motion compensation using the template, when decoding or encoding a video signal.
  • H.264 indicates a video compression standard that may provide a significantly high compression rate in comparison to MPEG-4 or H.263+ corresponding to a previous video compression standard.
  • H.264 is also referred to as H.264/advanced video coding (AVC).
  • AVC advanced video coding
  • paid or free services are being provided using H.264/AVC.
  • the free services may include, for example, a digital multimedia broadcasting (DMB) service.
  • the paid services may include communication provider services, for example, June of SKT and Fimm of KTF, and a satellite DMB service.
  • various types of H.264/AVC codec based services may be provided to users.
  • H.264 may provide video data of a high quality corresponding to a DVD level at a speed less than 1 Mbps. Accordingly, in the case of a high-speed Internet access through a radio, a satellite, or an asymmetric digital subscriber line (ADSL), H.264 may provide perfect motion picture video data of 30 frames per second (fps).
  • a current MPEG-2 based broadcasting system is also changed to H.264/AVC-based broadcasting system
  • a broadcasting service with a further enhanced high definition, for example, about twice greater than MPEG-2, using a bandwidth 19.39 Mbps that is currently used by a current high definition broadcasting service.
  • the number of channels may increase by about twice greater than the current number of channels.
  • a block including 16 x 16 pixels indicates a single macro block.
  • the macro block may be divided into various sizes of blocks.
  • a macro block mode may be determined using an intra prediction in a pixel area, various types of block sizes, a multi-reference screen, and a ratio-distortion optimization scheme. Through this, in comparison to an existing motion picture compression standard, it is possible to save bits of about 50 % with respect to the same screen quality.
  • JVT Joint Video Team
  • KTA manufacturing key technology area
  • a filter coefficient for 1/2 or 1/4 pixel motion compensation is fixed.
  • a non-separable adaptive interpolation filtering scheme is currently applied.
  • the non-separable adaptive interpolation filtering scheme may obtain an adaptive filter coefficient according to a pixel 1, 1/2, or 1/4 of a motion vector that is selected in a current image and a motion-compensated image generated after estimating a motion.
  • the non-separable adaptive interpolation filtering scheme may transmit the adaptive filter coefficient to a decoding unit.
  • the decoding unit may perform a motion compensation calculation and filtering according to the filter coefficient and then generate a reference image frame.
  • a transmission may be performed by assuming a 6 x 6 two-dimensional (2D) filter with respect to a single frame screen.
  • the above scheme generally considers the entire single frame as a target. Therefore, when the above scheme is applied to a significantly variable image, the efficiency may be small.
  • An aspect of the present invention provides an apparatus and method for encoding and decoding a video signal.
  • Another aspect of the present invention also provides an apparatus and method for encoding and decoding a video signal that may estimate a motion using a template and may adaptively compensate for the motion using the template, when decoding or encoding a video signal.
  • Another aspect of the present invention also provides an apparatus and method for encoding and decoding a video signal that may estimate a motion using a template, calculate a motion compensation filter coefficient using the template, filter a predicted image using the filter coefficient, and perform an adaptive motion compensation, when encoding or decoding a video signal.
  • an apparatus for encoding a video signal including: a reference image storage unit to store a reference image; a template motion estimation unit to estimate a motion between an input image and the reference image using a template, and to generate motion estimation information; an adaptive motion compensation unit to generate a predicted image using the motion estimation information and the reference image, to calculate a filter coefficient with respect to the predicted image using the template, and to filter the predicted image using the filter coefficient; a transformation and quantization unit to perform integer-transformation and quantization for a differential image that is a difference between the input image and the filtered predicted image; and an entropy encoding unit to entropy encode the quantized differential image and to output an image stream.
  • an apparatus for decoding a video signal including: an entropy decoding unit to entropy decode an input image stream; an inverse-transformation and inverse-quantization unit to perform inverse-quantization and inverse-transformation for the decoded input image stream and to recover a differential image; a de-blocking filter to de-blocking filter an image where a predicted image and the differential image are added, and to generate an output image; a reference image storage unit to store the output image as a reference image for the predicted image; a template motion estimation unit to estimate a motion between recent reference images using a template, and to generate motion estimation information; and an adaptive motion compensation unit to generate a subsequent predicted image using the motion estimation information and the reference image, to calculate a filter coefficient, and to filter the subsequent predicted image using the filter coefficient.
  • a method of encoding a video signal including: verifying a template and estimating a motion of the template using an input image to be encoded and a reference image to generate motion estimation information, when the input image is received; generating a predicted image using the motion estimation information and the reference image; calculating a filter coefficient with respect to the predicted image using the template; filtering the predicted image using the filter coefficient; calculating a difference between the input image and the filtered predicted image to generate a differential image; performing integer-transformation and quantization for the differential image; and entropy encoding the quantized differential image to output an image stream.
  • a method of decoding a video signal including: entropy decoding an input image stream, when the input image stream is received; performing inverse-quantization and inverse-transformation for the decoded input image stream to recover a differential image; de-blocking filtering an image where a predicted image and the differential image are added, to generate an output image; storing the output image as a reference image; verifying a template and estimating a motion of the template using recent reference images to generate motion estimation information; generating a subsequent predicted image using the motion estimation information and the reference image; calculating a filter coefficient with respect to the subsequent predicted image using the template; and filtering the subsequent predicted image using the filter coefficient.
  • an apparatus and method for encoding and decoding a video signal may estimate a motion using a template, calculate a motion compensation filter coefficient using the template, perform filtering using the filter coefficient, and perform an adaptive motion compensation, when encoding or decoding a video signal. Since the motion is estimated using the template, there is no need to transmit a motion vector. Also, the adaptive motion compensation may be performed using the calculated filter coefficient.
  • FIG. 1 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment of the present invention
  • FIG. 2 is a diagram illustrating a configuration of a template with respect to a current macro block in an encoding and decoding apparatus according to an embodiment of the present invention
  • FIG. 3 is a block diagram illustrating a configuration of an adaptive motion compensation unit in an encoding and decoding apparatus according to an embodiment of the present invention
  • FIG. 4 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment of the present invention.
  • FIG. 5 is a flowchart illustrating a method of encoding an image in an encoding apparatus according to an embodiment of the present invention.
  • FIG. 6 is a flowchart illustrating a method of decoding an image stream in a decoding apparatus according to an embodiment of the present invention.
  • an apparatus and method for encoding and decoding a video signal may estimate a motion using a template, calculate a motion compensation filter coefficient using the template, perform filtering using the filter coefficient, and perform an adaptive motion compensation, when encoding or decoding a video signal.
  • FIG. 1 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment of the present invention.
  • the encoding apparatus may include an encoding control unit 110, a transformation and quantization unit 120, an entropy encoding unit 130, an inverse-transformation and inverse-quantization unit 140, a de-blocking filter 150, a reference image storage unit 160, a template motion estimation unit 170, an adaptive motion compensation unit 180, and an intra prediction unit 190.
  • a prediction process may be performed for the input image by the intra prediction unit 190 via the encoding control unit 110.
  • a predicted image signal may be transformed and be quantized by the transformation and quantization unit 120 and be output via the entropy encoding unit 130.
  • the image signal may be stored in the reference image storage unit 160 via the inverse-transformation and inverse-quantization unit 140 and the de-blocking unit 150, and be used for a prediction process of a subsequent frame.
  • the template motion estimation unit 170 may verify a template, estimate a motion between the input image and a reference image stored in the reference image storage unit 160, using the template, and provide motion estimation information using the template to the adaptive motion compensation unit 180.
  • the motion estimation information using the template denotes information associated with the estimated motion of the image by searching for the reference image in a most similar region to a template region that is pre-set in the input image.
  • the template will be described with reference to FIG. 2.
  • FIG. 2 is a diagram illustrating a configuration of a template with respect to a current macro block 200 in an encoding and decoding apparatus according to an embodiment of the present invention.
  • the encoding and decoding apparatus may include a left macro block 212, an upper-left macro block 214, an upper macro block 216, and an upper-right macro block 218 based on the current macro block 200.
  • the adaptive motion compensation unit 180 may generate a predicted image using motion estimation information and a reference image stored in the reference image storage unit 160, calculate a filter coefficient with respect to the predicated image using a template, and filter the predicted image using the filter coefficient.
  • the adaptive motion compensation unit 180 will be described in detail with reference to FIG. 3.
  • FIG. 3 is a block diagram illustrating a configuration of the adaptive motion compensation unit 180 in an encoding and decoding apparatus according to an embodiment of the present invention.
  • the adaptive motion compensation unit 180 may include a motion compensation unit 310, a motion compensation filter coefficient calculation unit 320, and a filtering unit 330.
  • the motion compensation unit 310 may generate a predicted image using motion estimation information and a reference image stored in the reference image storage unit 160 of FIG. 1, and may provide the predicted mage to the motion compensation filter coefficient calculation unit 320.
  • the motion compensation filter coefficient calculation unit 320 may calculate a filter coefficient with respect to the predicted image, using a template based on a current macro block.
  • the calculation scheme may include, for example, a least square estimation scheme as given by the following Equation 1:
  • B indicates a current macro block as a matrix
  • H indicates a convolution kernel of a filter as the matrix
  • T i denotes a macro block of a reference image corresponding to a macro block of an input image.
  • the motion compensation filter coefficient may be calculated by obtaining H that may minimize the above Equation 1.
  • the filtering unit 330 may filter the predicted image using the calculated motion compensation filter coefficient.
  • the transformation and quantization unit 120 may receive a differential image that is a difference between the input image and the filtered predicted image, perform an integer-transformation and quantization for the differential image, and provide the quantized differential image to the entropy encoding unit 130 and the inverse-transformation and inverse-quantization unit 140.
  • the entropy encoding unit 130 may entropy encode the quantized differential image to output an image stream.
  • the inverse-transformation and inverse-quantization unit 140 may recover the differential image by performing inverse-transformation and inverse-quantization for the quantized differential image.
  • the reference image storage unit 160 may store the reference image that is generated by adding the differential image and the input image.
  • FIG. 4 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment of the present invention.
  • the decoding apparatus may include an entropy decoding unit 410, an inverse-transformation and inverse-quantization unit 420, a de-blocking filter 430, a reference image storage unit 440, a template motion estimation unit 450, and an adaptive motion compensation unit 460.
  • the entropy decoding unit 410 may entropy decode an input image stream.
  • the inverse-transformation and inverse-quantization unit 420 may perform inverse-quantization and inverse-transformation for the decoded input image stream to recover a differential image.
  • the de-blocking filter 430 denotes a filter that may smooth a boundary error of a block occurring in block-based coding.
  • the de-blocking unit 430 may de-blocking filter an image where a predicted image and the differential image are added and may generate an output image.
  • the reference image storage unit 440 may store the output image as a reference image for a subsequent predicted image.
  • the template motion estimation unit 450 may verify a template, estimate a motion between recent reference images stored in the reference image storage unit 440, using the template, and provide motion estimation information using the template to the adaptive motion compensation unit 460.
  • the adaptive motion compensation unit 460 may generate the predicted image using motion estimation information and a reference image stored in the reference image storage unit 440, calculate a filter coefficient with respect to the predicted image using the template, and filter the predicted image using the filter coefficient.
  • a detailed configuration of the adaptive motion compensation unit 460 will be the same as the configuration of FIG. 3 and thus further detailed description related thereto will be omitted here.
  • FIG. 5 is a flowchart illustrating a method of encoding an image in an encoding apparatus according to an embodiment of the present invention.
  • the encoding apparatus may verify a template and estimate a motion of the template using the input image and a reference image in operation 502.
  • the encoding apparatus may generate a predicted image using motion estimation information using the template and the reference image.
  • the encoding apparatus may calculate a filter coefficient with respect to the predicted image using the template.
  • the encoding apparatus may filter the predicted image using the filter coefficient.
  • the encoding apparatus may calculate a difference between a current input image and the filtered predicted image to generate a differential image.
  • the encoding apparatus may perform integer-transformation and quantization for the differential image.
  • the encoding apparatus may entropy encode the quantized differential image to output an image stream.
  • the encoding apparatus may perform inverse-transformation and inverse-quantization for the quantized differential image to recover the differential image in operation 516.
  • the encoding apparatus may generate a reference image for a subsequent motion prediction and compensation by adding the differential image and the current input image and may store the generated reference image.
  • FIG. 6 is a flowchart illustrating a method of decoding an image stream in a decoding apparatus according to an embodiment of the present invention.
  • the decoding apparatus may entropy decode the input image stream in operation 602.
  • the decoding apparatus may perform inverse-quantization and inverse-transformation for the decoded input image stream to recover a differential image.
  • the decoding apparatus may de-blocking filter an image where a predicted image and the differential image are added, to generate an output image and may output the output image.
  • the decoding apparatus may store the output image as a reference image.
  • the decoding apparatus may verify a template and estimate a motion using the template based on recent reference images.
  • the decoding apparatus may generate a predicted image for a subsequent decoding process using the motion estimation information using the template and the reference image.
  • the decoding apparatus may calculate a filtering coefficient with respect to the predicted image using the template.
  • the decoding apparatus may filter the predicted image using the filter coefficient.
  • the exemplary embodiments of the present invention include computer-readable media including program instructions to implement various operations embodied by a computer.
  • the media may also include, alone or in combination with the program instructions, data files, data structures, tables, and the like.
  • the described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described embodiments of the present invention, or vice versa.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Provided is an apparatus and method for encoding and decoding a video signal. A video signal encoding apparatus may include: a reference image storage unit to store a reference image; a template motion estimation unit to estimate a motion between an input image and the reference image using a template, and to generate motion estimation information; an adaptive motion compensation unit to generate a predicted image using the motion estimation information and the reference image, to calculate a filter coefficient with respect to the predicted image using the template, and to filter the predicted image using the filter coefficient; a transformation and quantization unit to perform integer-transformation and quantization for a differential image that is a difference between the input image and the filtered predicted image; and an entropy encoding unit to entropy encode the quantized differential image and to output an image stream.

Description

APPARATUS AND METHOD FOR ENCODING/DECODING A VIDEO SIGNAL
The present invention relates to an apparatus and method for encoding and decoding a video signal, and more particularly, to an apparatus and method for encoding and decoding a video signal that may estimate a motion using a template and may perform an adaptive motion compensation using the template, when decoding or encoding a video signal.
H.264 indicates a video compression standard that may provide a significantly high compression rate in comparison to MPEG-4 or H.263+ corresponding to a previous video compression standard. H.264 is also referred to as H.264/advanced video coding (AVC). Currently, various types of paid or free services are being provided using H.264/AVC. The free services may include, for example, a digital multimedia broadcasting (DMB) service. The paid services may include communication provider services, for example, June of SKT and Fimm of KTF, and a satellite DMB service. Also, various types of H.264/AVC codec based services may be provided to users.
H.264 may provide video data of a high quality corresponding to a DVD level at a speed less than 1 Mbps. Accordingly, in the case of a high-speed Internet access through a radio, a satellite, or an asymmetric digital subscriber line (ADSL), H.264 may provide perfect motion picture video data of 30 frames per second (fps).
Also, when a current MPEG-2 based broadcasting system is also changed to H.264/AVC-based broadcasting system, it is possible to provide a broadcasting service with a further enhanced high definition, for example, about twice greater than MPEG-2, using a bandwidth 19.39 Mbps that is currently used by a current high definition broadcasting service. When increasing a number of channels, the number of channels may increase by about twice greater than the current number of channels.
In H.264, a block including 16 x 16 pixels indicates a single macro block. The macro block may be divided into various sizes of blocks. In order to more accurately perform a motion prediction and compensation, a macro block mode may be determined using an intra prediction in a pixel area, various types of block sizes, a multi-reference screen, and a ratio-distortion optimization scheme. Through this, in comparison to an existing motion picture compression standard, it is possible to save bits of about 50 % with respect to the same screen quality.
Although H.264 is a highly effective video coding scheme, there is an increasing need for a new video codec with a further enhanced compression ratio. Accordingly, the Joint Video Team (JVT) is on manufacturing key technology area (KTA) reference software and verifying a new idea.
In H.264, a filter coefficient for 1/2 or 1/4 pixel motion compensation is fixed. However, in order to enhance a performance of the H.264 codec, a non-separable adaptive interpolation filtering scheme is currently applied. The non-separable adaptive interpolation filtering scheme may obtain an adaptive filter coefficient according to a pixel 1, 1/2, or 1/4 of a motion vector that is selected in a current image and a motion-compensated image generated after estimating a motion. The non-separable adaptive interpolation filtering scheme may transmit the adaptive filter coefficient to a decoding unit. The decoding unit may perform a motion compensation calculation and filtering according to the filter coefficient and then generate a reference image frame. Currently, a transmission may be performed by assuming a 6 x 6 two-dimensional (2D) filter with respect to a single frame screen. The above scheme generally considers the entire single frame as a target. Therefore, when the above scheme is applied to a significantly variable image, the efficiency may be small.
An aspect of the present invention provides an apparatus and method for encoding and decoding a video signal.
Another aspect of the present invention also provides an apparatus and method for encoding and decoding a video signal that may estimate a motion using a template and may adaptively compensate for the motion using the template, when decoding or encoding a video signal.
Another aspect of the present invention also provides an apparatus and method for encoding and decoding a video signal that may estimate a motion using a template, calculate a motion compensation filter coefficient using the template, filter a predicted image using the filter coefficient, and perform an adaptive motion compensation, when encoding or decoding a video signal.
According to an aspect of the present invention, there is provided an apparatus for encoding a video signal, the apparatus including: a reference image storage unit to store a reference image; a template motion estimation unit to estimate a motion between an input image and the reference image using a template, and to generate motion estimation information; an adaptive motion compensation unit to generate a predicted image using the motion estimation information and the reference image, to calculate a filter coefficient with respect to the predicted image using the template, and to filter the predicted image using the filter coefficient; a transformation and quantization unit to perform integer-transformation and quantization for a differential image that is a difference between the input image and the filtered predicted image; and an entropy encoding unit to entropy encode the quantized differential image and to output an image stream.
According to another aspect of the present invention, there is provided an apparatus for decoding a video signal, the apparatus including: an entropy decoding unit to entropy decode an input image stream; an inverse-transformation and inverse-quantization unit to perform inverse-quantization and inverse-transformation for the decoded input image stream and to recover a differential image; a de-blocking filter to de-blocking filter an image where a predicted image and the differential image are added, and to generate an output image; a reference image storage unit to store the output image as a reference image for the predicted image; a template motion estimation unit to estimate a motion between recent reference images using a template, and to generate motion estimation information; and an adaptive motion compensation unit to generate a subsequent predicted image using the motion estimation information and the reference image, to calculate a filter coefficient, and to filter the subsequent predicted image using the filter coefficient.
According to still another aspect of the present invention, there is provided a method of encoding a video signal, the method including: verifying a template and estimating a motion of the template using an input image to be encoded and a reference image to generate motion estimation information, when the input image is received; generating a predicted image using the motion estimation information and the reference image; calculating a filter coefficient with respect to the predicted image using the template; filtering the predicted image using the filter coefficient; calculating a difference between the input image and the filtered predicted image to generate a differential image; performing integer-transformation and quantization for the differential image; and entropy encoding the quantized differential image to output an image stream.
According to yet another aspect of the present invention, there is provided a method of decoding a video signal, the method including: entropy decoding an input image stream, when the input image stream is received; performing inverse-quantization and inverse-transformation for the decoded input image stream to recover a differential image; de-blocking filtering an image where a predicted image and the differential image are added, to generate an output image; storing the output image as a reference image; verifying a template and estimating a motion of the template using recent reference images to generate motion estimation information; generating a subsequent predicted image using the motion estimation information and the reference image; calculating a filter coefficient with respect to the subsequent predicted image using the template; and filtering the subsequent predicted image using the filter coefficient.
According to embodiments of the present invention, there is provided an apparatus and method for encoding and decoding a video signal that may estimate a motion using a template, calculate a motion compensation filter coefficient using the template, perform filtering using the filter coefficient, and perform an adaptive motion compensation, when encoding or decoding a video signal. Since the motion is estimated using the template, there is no need to transmit a motion vector. Also, the adaptive motion compensation may be performed using the calculated filter coefficient.
FIG. 1 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a configuration of a template with respect to a current macro block in an encoding and decoding apparatus according to an embodiment of the present invention;
FIG. 3 is a block diagram illustrating a configuration of an adaptive motion compensation unit in an encoding and decoding apparatus according to an embodiment of the present invention;
FIG. 4 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a method of encoding an image in an encoding apparatus according to an embodiment of the present invention; and
FIG. 6 is a flowchart illustrating a method of decoding an image stream in a decoding apparatus according to an embodiment of the present invention.
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments are described below in order to explain the present invention by referring to the figures. When it is determined detailed description related to a related known function or configuration they may make the purpose of the present invention unnecessarily ambiguous in describing the present invention, the detailed description will be omitted here.
Provided is an apparatus and method for encoding and decoding a video signal that may estimate a motion using a template, calculate a motion compensation filter coefficient using the template, perform filtering using the filter coefficient, and perform an adaptive motion compensation, when encoding or decoding a video signal.
FIG. 1 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment of the present invention.
Referring to FIG. 1, the encoding apparatus may include an encoding control unit 110, a transformation and quantization unit 120, an entropy encoding unit 130, an inverse-transformation and inverse-quantization unit 140, a de-blocking filter 150, a reference image storage unit 160, a template motion estimation unit 170, an adaptive motion compensation unit 180, and an intra prediction unit 190.
When an input image is processed as an intra frame, a prediction process may be performed for the input image by the intra prediction unit 190 via the encoding control unit 110. Next, a predicted image signal may be transformed and be quantized by the transformation and quantization unit 120 and be output via the entropy encoding unit 130. Also, the image signal may be stored in the reference image storage unit 160 via the inverse-transformation and inverse-quantization unit 140 and the de-blocking unit 150, and be used for a prediction process of a subsequent frame.
According to an embodiment of the present invention, when the input image is processed as an inter frame, the template motion estimation unit 170 may verify a template, estimate a motion between the input image and a reference image stored in the reference image storage unit 160, using the template, and provide motion estimation information using the template to the adaptive motion compensation unit 180. Here, the motion estimation information using the template denotes information associated with the estimated motion of the image by searching for the reference image in a most similar region to a template region that is pre-set in the input image. Hereinafter, the template will be described with reference to FIG. 2.
FIG. 2 is a diagram illustrating a configuration of a template with respect to a current macro block 200 in an encoding and decoding apparatus according to an embodiment of the present invention.
The encoding and decoding apparatus may include a left macro block 212, an upper-left macro block 214, an upper macro block 216, and an upper-right macro block 218 based on the current macro block 200.
The adaptive motion compensation unit 180 may generate a predicted image using motion estimation information and a reference image stored in the reference image storage unit 160, calculate a filter coefficient with respect to the predicated image using a template, and filter the predicted image using the filter coefficient. Hereinafter, the adaptive motion compensation unit 180 will be described in detail with reference to FIG. 3.
FIG. 3 is a block diagram illustrating a configuration of the adaptive motion compensation unit 180 in an encoding and decoding apparatus according to an embodiment of the present invention. Referring to FIG. 3, the adaptive motion compensation unit 180 may include a motion compensation unit 310, a motion compensation filter coefficient calculation unit 320, and a filtering unit 330.
The motion compensation unit 310 may generate a predicted image using motion estimation information and a reference image stored in the reference image storage unit 160 of FIG. 1, and may provide the predicted mage to the motion compensation filter coefficient calculation unit 320.
The motion compensation filter coefficient calculation unit 320 may calculate a filter coefficient with respect to the predicted image, using a template based on a current macro block. The calculation scheme may include, for example, a least square estimation scheme as given by the following Equation 1:
[Equation 1]
Figure PCTKR2009004576-appb-I000001
.
Here, B indicates a current macro block as a matrix, H indicates a convolution kernel of a filter as the matrix, and Ti denotes a macro block of a reference image corresponding to a macro block of an input image.
The motion compensation filter coefficient may be calculated by obtaining H that may minimize the above Equation 1. The filtering unit 330 may filter the predicted image using the calculated motion compensation filter coefficient.
Referring again to FIG. 1, the transformation and quantization unit 120 may receive a differential image that is a difference between the input image and the filtered predicted image, perform an integer-transformation and quantization for the differential image, and provide the quantized differential image to the entropy encoding unit 130 and the inverse-transformation and inverse-quantization unit 140. The entropy encoding unit 130 may entropy encode the quantized differential image to output an image stream.
The inverse-transformation and inverse-quantization unit 140 may recover the differential image by performing inverse-transformation and inverse-quantization for the quantized differential image. The reference image storage unit 160 may store the reference image that is generated by adding the differential image and the input image.
FIG. 4 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment of the present invention. Referring to FIG. 4, the decoding apparatus may include an entropy decoding unit 410, an inverse-transformation and inverse-quantization unit 420, a de-blocking filter 430, a reference image storage unit 440, a template motion estimation unit 450, and an adaptive motion compensation unit 460.
The entropy decoding unit 410 may entropy decode an input image stream. The inverse-transformation and inverse-quantization unit 420 may perform inverse-quantization and inverse-transformation for the decoded input image stream to recover a differential image.
The de-blocking filter 430 denotes a filter that may smooth a boundary error of a block occurring in block-based coding. The de-blocking unit 430 may de-blocking filter an image where a predicted image and the differential image are added and may generate an output image.
The reference image storage unit 440 may store the output image as a reference image for a subsequent predicted image.
The template motion estimation unit 450 may verify a template, estimate a motion between recent reference images stored in the reference image storage unit 440, using the template, and provide motion estimation information using the template to the adaptive motion compensation unit 460.
The adaptive motion compensation unit 460 may generate the predicted image using motion estimation information and a reference image stored in the reference image storage unit 440, calculate a filter coefficient with respect to the predicted image using the template, and filter the predicted image using the filter coefficient. A detailed configuration of the adaptive motion compensation unit 460 will be the same as the configuration of FIG. 3 and thus further detailed description related thereto will be omitted here.
Hereinafter, a method of encoding and decoding a video signal according to an embodiment of the present invention will be described with reference to FIGS. 5 and 6.
FIG. 5 is a flowchart illustrating a method of encoding an image in an encoding apparatus according to an embodiment of the present invention.
Referring to FIG. 5, when an input image to be encoded is received in operation 500, the encoding apparatus may verify a template and estimate a motion of the template using the input image and a reference image in operation 502. In operation 504, the encoding apparatus may generate a predicted image using motion estimation information using the template and the reference image. In operation 506, the encoding apparatus may calculate a filter coefficient with respect to the predicted image using the template. In operation 508, the encoding apparatus may filter the predicted image using the filter coefficient.
In operation 510, the encoding apparatus may calculate a difference between a current input image and the filtered predicted image to generate a differential image. In operation 512, the encoding apparatus may perform integer-transformation and quantization for the differential image. In operation 514, the encoding apparatus may entropy encode the quantized differential image to output an image stream.
Also, for an encoding process of a subsequent input image after operation 510, the encoding apparatus may perform inverse-transformation and inverse-quantization for the quantized differential image to recover the differential image in operation 516. In operation 518, the encoding apparatus may generate a reference image for a subsequent motion prediction and compensation by adding the differential image and the current input image and may store the generated reference image.
FIG. 6 is a flowchart illustrating a method of decoding an image stream in a decoding apparatus according to an embodiment of the present invention.
Referring to FIG. 6, when an input image stream is received in operation 600, the decoding apparatus may entropy decode the input image stream in operation 602. In operation 604, the decoding apparatus may perform inverse-quantization and inverse-transformation for the decoded input image stream to recover a differential image. In operation 606, the decoding apparatus may de-blocking filter an image where a predicted image and the differential image are added, to generate an output image and may output the output image.
In operation 608, the decoding apparatus may store the output image as a reference image. In operation 610, the decoding apparatus may verify a template and estimate a motion using the template based on recent reference images.
In operation 612, the decoding apparatus may generate a predicted image for a subsequent decoding process using the motion estimation information using the template and the reference image. In operation 614, the decoding apparatus may calculate a filtering coefficient with respect to the predicted image using the template. In operation 616, the decoding apparatus may filter the predicted image using the filter coefficient.
The exemplary embodiments of the present invention include computer-readable media including program instructions to implement various operations embodied by a computer. The media may also include, alone or in combination with the program instructions, data files, data structures, tables, and the like. The described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described embodiments of the present invention, or vice versa.
Although a few embodiments of the present invention have been shown and described, the present invention is not limited to the described embodiments. Instead, it would be appreciated by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims (15)

  1. An apparatus for encoding a video signal, the apparatus comprising:
    a reference image storage unit to store a reference image;
    a template motion estimation unit to estimate a motion between an input image and the reference image using a template, and to generate motion estimation information;
    an adaptive motion compensation unit to generate a predicted image using the motion estimation information and the reference image, to calculate a filter coefficient with respect to the predicted image using the template, and to filter the predicted image using the filter coefficient;
    a transformation and quantization unit to perform integer-transformation and quantization for a differential image that is a difference between the input image and the filtered predicted image; and
    an entropy encoding unit to entropy encode the quantized differential image and to output an image stream.
  2. The apparatus of claim 1, wherein the template is a macro block that is located in any one of an upper side, a left side, an upper-left side, and an upper-right side of a current macro block.
  3. The apparatus of claim 1, wherein the adaptive motion compensation unit comprises:
    a motion compensation unit to generate the predicted image using the motion estimation information and the reference image;
    a motion compensation filter coefficient calculation unit to calculate the filter coefficient with respect to the predicted image using the template; and
    a filtering unit to filter the predicted image using the filter coefficient.
  4. The apparatus of claim 3, wherein the motion compensation filter coefficient calculation unit calculates the filter coefficient using a least square estimation scheme.
  5. An apparatus for decoding a video signal, the apparatus comprising:
    an entropy decoding unit to entropy decode an input image stream;
    an inverse-transformation and inverse-quantization unit to perform inverse-quantization and inverse-transformation for the decoded input image stream and to recover a differential image;
    a de-blocking filter to de-blocking filter an image where a predicted image and the differential image are added, and to generate an output image;
    a reference image storage unit to store the output image as a reference image for the predicted image;
    a template motion estimation unit to estimate a motion between recent reference images using a template, and to generate motion estimation information; and
    an adaptive motion compensation unit to generate a subsequent predicted image using the motion estimation information and the reference image, to calculate a filter coefficient, and to filter the subsequent predicted image using the filter coefficient.
  6. The apparatus of claim 5, wherein the template is a macro block that is located in any one of an upper side, a left side, an upper-left side, and an upper-right side of a current macro block.
  7. The apparatus of claim 5, wherein the adaptive motion compensation unit comprises:
    a motion compensation unit to generate the subsequent predicted image using the motion estimation information and the reference image;
    a motion compensation filter coefficient calculation unit to calculate the filter coefficient with respect to the subsequent predicted image using the template; and
    a filtering unit to filter the subsequent predicted image using the filter coefficient.
  8. The apparatus of claim 7, wherein the motion compensation filter coefficient calculation unit calculates the filter coefficient using a least square estimation scheme.
  9. A method of encoding a video signal, the method comprising:
    verifying a template and estimating a motion of the template using an input image to be encoded and a reference image to generate motion estimation information, when the input image is received;
    generating a predicted image using the motion estimation information and the reference image;
    calculating a filter coefficient with respect to the predicted image using the template;
    filtering the predicted image using the filter coefficient;
    calculating a difference between the input image and the filtered predicted image to generate a differential image;
    performing integer-transformation and quantization for the differential image; and
    entropy encoding the quantized differential image to output an image stream.
  10. The method of claim 9, further comprising:
    generating a reference image for a subsequent motion compensation by adding the differential image and the input image, to store the generated reference image.
  11. The method of claim 9, wherein the template is a macro block that is located in any one of an upper side, a left side, an upper-left side, and an upper-right side of a current macro block.
  12. The method of claim 9, wherein the filter coefficient is calculated using a least square estimation scheme.
  13. A method of decoding a video signal, the method comprising:
    entropy decoding an input image stream, when the input image stream is received;
    performing inverse-quantization and inverse-transformation for the decoded input image stream to recover a differential image;
    de-blocking filtering an image where a predicted image and the differential image are added, to generate an output image;
    storing the output image as a reference image;
    verifying a template and estimating a motion of the template using recent reference images to generate motion estimation information;
    generating a subsequent predicted image using the motion estimation information and the reference image;
    calculating a filter coefficient with respect to the subsequent predicted image using the template; and
    filtering the subsequent predicted image using the filter coefficient.
  14. The method of claim 13, wherein the template is a macro block that is located in any one of an upper side, a left side, an upper-left side, and an upper-right side of a current macro block.
  15. The method of claim 13, wherein the filter coefficient is calculated using a least square estimation scheme.
PCT/KR2009/004576 2008-11-24 2009-08-17 Apparatus and method for encoding/decoding a video signal Ceased WO2010058895A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/130,905 US20110228854A1 (en) 2008-11-24 2009-08-17 Apparatus and method for encoding/decoding a video signal

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2008-0116931 2008-11-24
KR1020080116931A KR101098739B1 (en) 2008-11-24 2008-11-24 Apparatus and method for encoding/decoding a video signal

Publications (2)

Publication Number Publication Date
WO2010058895A2 true WO2010058895A2 (en) 2010-05-27
WO2010058895A3 WO2010058895A3 (en) 2013-02-28

Family

ID=42198613

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2009/004576 Ceased WO2010058895A2 (en) 2008-11-24 2009-08-17 Apparatus and method for encoding/decoding a video signal

Country Status (3)

Country Link
US (1) US20110228854A1 (en)
KR (1) KR101098739B1 (en)
WO (1) WO2010058895A2 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2617199A4 (en) * 2010-09-14 2014-02-19 Blackberry Ltd METHODS AND DEVICES FOR COMPRESSING DATA WITH ADAPTIVE FILTERING IN THE TRANSFORMATION FIELD
CN107257488A (en) * 2011-06-21 2017-10-17 英迪股份有限公司 Method and device based on quadtree coding and self-adaption of decoding quantization parameter

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
PL400344A1 (en) * 2012-08-13 2014-02-17 Politechnika Poznanska Method for determining the the motion vector predictor
WO2016091122A1 (en) * 2014-12-11 2016-06-16 Mediatek Inc. Method and apparatus of intra prediction in video coding
US10419755B2 (en) 2016-05-16 2019-09-17 Qualcomm Incorporated Confusion of multiple filters in adaptive loop filtering in video coding
CN106954070B (en) * 2017-04-28 2023-04-11 河南工程学院 Sliding pixel block integer DCT core matrix transformation motion compensator and method
US11102503B2 (en) 2019-01-07 2021-08-24 Electronics And Telecommunications Research Institute Motion information prediction method and apparatus for distortion due to projection formation conversion
CN117395397A (en) 2019-06-04 2024-01-12 北京字节跳动网络技术有限公司 Motion candidate list construction using neighboring block information
KR20220016839A (en) 2019-06-04 2022-02-10 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 Motion candidate list with geometric segmentation mode coding
CN114128258B (en) 2019-07-14 2023-12-22 北京字节跳动网络技术有限公司 Limitations of transform block size in video codecs
WO2021057996A1 (en) 2019-09-28 2021-04-01 Beijing Bytedance Network Technology Co., Ltd. Geometric partitioning mode in video coding
US12101477B2 (en) 2019-11-22 2024-09-24 Electronics And Telecommunications Research Institute Video processing method and device using resolution of reference picture, and recording medium
WO2021180022A1 (en) 2020-03-07 2021-09-16 Beijing Bytedance Network Technology Co., Ltd. Handling of transform skip mode in video coding
WO2022037700A1 (en) 2020-08-21 2022-02-24 Beijing Bytedance Network Technology Co., Ltd. Coding mode dependent selection of transform skip mode
CN116601953A (en) 2020-11-24 2023-08-15 抖音视界有限公司 Position-dependent coefficient reordering in codec video
US12114009B2 (en) * 2021-09-22 2024-10-08 Tencent America LLC Method and apparatus for adaptive reordering for reference frames
WO2024039910A1 (en) * 2022-08-19 2024-02-22 Beijing Dajia Internet Information Technology Co., Ltd. Method and apparatus for adaptive motion compensated filtering

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6188795B1 (en) * 1996-06-13 2001-02-13 Dublin City University Data compression
JPH10191360A (en) 1996-08-22 1998-07-21 Cirrus Logic Inc Method for obtaining motion estimation vector and method for compressing moving image data using motion estimation vector
US7599435B2 (en) * 2004-01-30 2009-10-06 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Video frame encoding and decoding
JP2007043651A (en) * 2005-07-05 2007-02-15 Ntt Docomo Inc Moving picture coding apparatus, moving picture coding method, moving picture coding program, moving picture decoding apparatus, moving picture decoding method, and moving picture decoding program
US20070171987A1 (en) * 2006-01-20 2007-07-26 Nokia Corporation Method for optical flow field estimation using adaptive Filting
JP2010524383A (en) * 2007-04-09 2010-07-15 エルジー エレクトロニクス インコーポレイティド Video signal processing method and apparatus
EP2048886A1 (en) * 2007-10-11 2009-04-15 Panasonic Corporation Coding of adaptive interpolation filter coefficients

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2617199A4 (en) * 2010-09-14 2014-02-19 Blackberry Ltd METHODS AND DEVICES FOR COMPRESSING DATA WITH ADAPTIVE FILTERING IN THE TRANSFORMATION FIELD
CN107257488A (en) * 2011-06-21 2017-10-17 英迪股份有限公司 Method and device based on quadtree coding and self-adaption of decoding quantization parameter
CN107257488B (en) * 2011-06-21 2020-04-17 英迪股份有限公司 Method and device for coding and decoding adaptive quantization parameter based on quad-tree
USRE49330E1 (en) 2011-06-21 2022-12-06 Dolby Laboratories Licensing Corporation Method and apparatus for adaptively encoding and decoding a quantization parameter based on a quadtree structure

Also Published As

Publication number Publication date
WO2010058895A3 (en) 2013-02-28
US20110228854A1 (en) 2011-09-22
KR20100058201A (en) 2010-06-03
KR101098739B1 (en) 2011-12-23

Similar Documents

Publication Publication Date Title
WO2010058895A2 (en) Apparatus and method for encoding/decoding a video signal
US20220408111A1 (en) Image encoding method using a skip mode, and a device using the method
TWI902491B (en) Image encoding device, image decoding device, and non-transitory storage medium
CN109155863B (en) Encoding device, decoding device, encoding method, and decoding method
RU2606303C2 (en) Decoding device and decoding method, and encoding device and encoding method
AU2011312795B2 (en) Optimized deblocking filters
JP7783943B2 (en) Bitstream transmitting device, transmitting method, receiving device and receiving method
CN113170143A (en) Corresponding derivation method for boundary strength of encoder, decoder and deblocking filter
WO2011016678A2 (en) Apparatus and method for deblocking filtering image data and video decoding apparatus and method using the same
CN109479135B (en) Encoding device, decoding device, encoding method, and decoding method
US20090201991A1 (en) Method for intra prediction coding of image data
CN113508592A (en) Encoder, decoder and corresponding inter-frame prediction method
WO2012051164A1 (en) Internal bit depth increase in deblocking filters and ordered dither
JP6039178B2 (en) Image encoding apparatus, image decoding apparatus, method and program thereof
TW202541500A (en) Coding devices, decoding devices, and non-transitory computer-readable media
JP7653911B2 (en) Encoding device, decoding device, encoding method, and decoding method
CN115804094A (en) Encoding device, decoding device, encoding method, and decoding method
CN112136326A (en) Encoding device, decoding device, encoding method, and decoding method
CN121887993A (en) Encoding device, decoding device, encoding method, decoding method, and storage medium
EP4226621A1 (en) Motion coding using a geometrical model for video compression
US20110194602A1 (en) Method and apparatus for sub-pixel interpolation
KR20140124919A (en) A method for adaptive illuminance compensation based on object and an apparatus using it
TW202110185A (en) Encoding device, decoding device, encoding method, and decoding method
EP3754983B1 (en) Early intra coding decision
JP6232117B2 (en) Image encoding method, image decoding method, and recording medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09827660

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 13130905

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 09827660

Country of ref document: EP

Kind code of ref document: A2