EP1938598A2 - Encoding and decoding of a video signal - Google Patents

Encoding and decoding of a video signal

Info

Publication number
EP1938598A2
EP1938598A2 EP06816113A EP06816113A EP1938598A2 EP 1938598 A2 EP1938598 A2 EP 1938598A2 EP 06816113 A EP06816113 A EP 06816113A EP 06816113 A EP06816113 A EP 06816113A EP 1938598 A2 EP1938598 A2 EP 1938598A2
Authority
EP
European Patent Office
Prior art keywords
adaptive
image area
motion
wavelet transform
frames
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP06816113A
Other languages
German (de)
French (fr)
Other versions
EP1938598A4 (en
Inventor
Ebroul Izquierdo
Marta Mrak
Nikola Sprljan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Motorola Mobility LLC
Original Assignee
Motorola Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Motorola Inc filed Critical Motorola Inc
Publication of EP1938598A2 publication Critical patent/EP1938598A2/en
Publication of EP1938598A4 publication Critical patent/EP1938598A4/en
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H04N19/619—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding the transform being operated outside the prediction loop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
    • H04N19/635—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets characterised by filter definition or implementation details

Definitions

  • the invention relates to encoding and/or decoding of a video signal and in particular to encoding and/or decoding using a wavelet transform.
  • the redundancy of information between consecutive frames is often used to reduce the coding rate.
  • adjacent frames in a video sequence comprise similar images and it is therefore possible to represent information of a frame using data from already encoded frames.
  • motion compensation techniques are used to remove temporal redundancy to produce so-called temporal frames.
  • Motion models (such as well known block-based models) are used in this context as parameters that drive motion compensation.
  • an improved system for video encoding would be advantageous and in particular a system allowing increased flexibility, improved wavelet transform encoding/decoding, improved adaptation, facilitated implementation, reduced computational requirements, reduced data rate and/or improved performance would be advantageous.
  • the Invention seeks to preferably mitigate, alleviate or eliminate one or more of the above mentioned disadvantages singly or in any combination.
  • an encoder for encoding a video signal comprising: means for generating motion compensated video frames for the video signal; wavelet means for applying an adaptive spatial wavelet transform to the motion compensated video frames to generate spatially transformed frames; means for generating encoded video data for the video signal in response to the spatially transformed frames; and adaptive means for determining adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames.
  • the invention may allow an improved video encoding.
  • An improved wavelet transformation may be used which more closely reflects the characteristics of the motion compensated video frames thereby allowing a more efficient encoding resulting in improved quality to data rate ratio.
  • the invention may allow adaptation of an adaptive spatial wavelet transform at the encoder (and decoder) without resulting in the necessity for explicit data for the adaptation to be distributed with the encoded video data. Thus, a reduced data rate may be achieved. Also, the adaptation may be based on processing which has already been performed for other purposes resulting in a facilitated and reduced complexity implementation.
  • the invention may provide a better wavelet domain representation taking into consideration both spatial and temporal characteristics of the video signal.
  • the improved wavelet domain representation may allow a more efficient compression of the video signal.
  • the adaptive parameters may comprise one or more adaptive parameters which are used to modify the operation of the adaptive spatial wavelet transform.
  • the motion data may comprise ' identifications of one or more motion compensated image areas in one or more of the motion compensated video frames.
  • the adaptive parameters comprise wavelet transform coefficients .
  • the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the ' second image area.
  • the wavelet transform may be adapted to provide a more efficient wavelet representation of higher frequency components around image area transitions which tend to have in increased concentration of such components.
  • the feature may allow a better encoding of detail around an image area transition.
  • the adaptive means is arranged to determine the first image area and the second image area as image areas having different motion characteristics.
  • This may provide efficient encoding and/or facilitated/ reduced complexity encoding.
  • different motion characteristics tend to indicate that the image areas belong to different objects in the image and thus to the existence of a sharp image transition (e.g. an object edge) comprising high frequency components and high levels of detail.
  • the first image area is a motion compensated image area and the second image area is a non-motion compensated image area.
  • the adaptation of the wavelet transform may be achieved without necessitating that additional adaptation data is included in the encoded video signal.
  • the first image area is an inter-coded image area and the second image area is an intra-coded image area.
  • An inter-coded image area is an image area encoded relative to an image area in another frame.
  • An intra-coded image area is an image area which is not encoded relative to an image area in any other frame.
  • the feature may provide efficient encoding and/or facilitated/ reduced complexity encoding.
  • the adaptation of the wavelet transform may be achieved without necessitating that additional adaptation data is included in the encoded video signal.
  • the adaptive parameters comprise wavelet transform coefficients for picture elements of the first frame adjacent to the boundary.
  • This may provide efficient encoding and/or facilitated/ reduced complexity encoding. It may allow a particularly efficient and/or facilitated adaptation of the adaptive spatial wavelet transform.
  • the adaptive parameters do not comprise wavelet transform coefficients for picture elements of the first frame not adjacent to the boundary.
  • This may provide efficient encoding and/or facilitated/ reduced complexity encoding. It may allow a particularly efficient and/or facilitated adaptation of the adaptive spatial wavelet transform.
  • the wavelet means is arranged to apply the adaptive spatial wavelet transform by applying an adaptive wavelet lifting operation.
  • the adaptive parameters comprise coefficient values for multiple levels of the adaptive wavelet lifting operation.
  • the boundary may be the same for each of the multiple levels.
  • the adaptive parameters comprise different coefficient values for different levels of the multiple levels.
  • the wavelet means is arranged to apply the adaptive spatial wavelet transform to transformed spatial subband data.
  • the encoder further comprises means for generating an encoded data signal for the video signal comprising the encoded video data and the motion data.
  • the encoded data signal is a self consistent signal comprising all the data which is required to decode the video signal.
  • the encoded data signal may thus be transmitted, broadcast or distributed to one or more decoders .
  • the encoder is arranged to not include data of the adaptive parameters in the encoded data signal.
  • This may provide for a more efficient encoded data signal and may in particular reduce the data rate for a given encoded video quality.
  • a decoder for decoding an encoded video signal, the decoder comprising: means for receiving the encoded video signal; means for generating spatially transformed frames from the encoded video signal; wavelet means for applying an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames; means for generating motion data for the motion compensated video frames; and adaptive means for determining adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
  • the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the second image area.
  • the adaptive means is arranged to determine the first and second image areas as image areas having different motion characteristics .
  • a method of encoding a video signal comprising: generating motion compensated video frames for the video signal; applying an adaptive spatial wavelet transform to the motion compensated video frames to generate spatially transformed frames; generating encoded video data for the video signal in response to the spatially transformed frames; and determining adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames.
  • a method of decoding an encoded video signal comprising: receiving the encoded video signal; generating spatially transformed frames from the encoded video signal; applying an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames; generating motion data for the motion compensated video frames; and determining adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
  • a computer program product enabling the carrying out of a method as described above.
  • FIG. 1 is an illustration of an encoder for encoding a video signal in accordance with some embodiments of the invention
  • FIG. 2 illustrates an example of a lifting operation in accordance with some embodiments of the invention.
  • FIG. 3 illustrates a decoder in accordance with some embodiments of the invention.
  • FIG. 1 is an illustration of an encoder for encoding a video signal in accordance with some embodiments of the invention.
  • the encoder 100 comprises a video frame source 101 which generates video frames from a video signal to be encoded.
  • the video frame source 101 can be a receiver for receiving a digitised video signal already comprising suitable video frames from an external source, or that it can for example be an analog video signal which is then digitised and structured in suitable frames as will be well known to the person skilled in the art.
  • the video frame source 101 is coupled to a motion estimation processor 103 which performs motion estimation and compensation on the video frames to generate motion compensated video frames.
  • a motion estimation processor 103 which performs motion estimation and compensation on the video frames to generate motion compensated video frames.
  • Various methods and algorithms for motion estimation and compensation of video frames is well known to the person skilled in the art and is for brevity not described in detail herein.
  • the motion estimation processor 103 simply determines the movement of motion estimation blocks of a fixed size in different video frames.
  • the block size can be 16 x 16 pixels (picture elements) . If the video frames comprise images with a moving object, a given block of one image can frequently be found in a subsequent image at an (often slightly) different position. An efficient video encoding can then be obtained by merely encoding the displacement of the original block rather than the actual image data of the block in the subsequent frame. This displacement is known as the motion vector. Improved quality can be obtained by determining the differential between the blocks of the different frames and encoding this residual or error signal. It will be appreciated that more advanced motion compensation is 5 possible using for example further degrees of freedom in the movement (such as rotation) or differently shaped motion compensation blocks (possibly adaptive) . Indeed, advanced two or three dimensional motion estimation models for different objects can be used.
  • a simple approach comprises a brute force search of subsequent frame (s) to detect any blocks that closely correspond to a block of a previous frame in order
  • motion estimation comprises evaluating a simple or complex object motion model to determine locations in subsequent frames.
  • the motion estimation processor 103 is coupled to a 20 wavelet processor 105.
  • the wavelet processor 105 implements an adaptive spatial wavelet transform which is applied the compensated video frames to generate spatially transformed frames.
  • wavelet processing may in many situations 25 provide a more efficient representation than conventional Fourier transforms.
  • the motion estimation processor 103 provides temporal frames data to the wavelet processor 30 105, on which the adaptive spatial wavelet transform is applied.
  • the output of the motion estimation processor 103 is the spatio-temporal subbands of the source video frames .
  • the wavelet processor 105 is coupled to an encoding processor 107 which is fed the motion compensated video frames (in the form of spatio-temporal subband data) .
  • the encoding processor 107 is arranged to generate encoded video data for the transformed video frames. Specifically, the encoding processor 107 performs quantisation, scaling and encoding of the data values of the transformed video frames as will be known to the person skilled in the art.
  • the incoming processor 107 can proceed to generate a complete encoded video signal which comprises all the information required to decode the video signal.
  • the encoding processor 107 includes the motion data used by the motion estimation processor 103 for the motion compensation.
  • the encoding processor 107 is coupled to a transmit interface 109 which is fed the encoded video signal and which is arranged to transmit the signal to one and more decoders .
  • the transmit interface 109 for example can be an interface to the Internet or e.g. a radio transceiver, such as cellular radio transceiver, arranged to transmit the encoded video signal over an air interface of a wireless communication system.
  • the applied wavelet transform is an adaptive wavelet transform. Specifically, some or all of the coefficients of the wavelet transform can be adapted to the specific characteristics of the individual video frames.
  • the adaptation of the wavelet transform is performed in response to motion data generated by the motion estimation processor 103 and used for the motion compensation of the motion compensated video frames.
  • the encoder 100 comprises an adaptation processor 111 which is arranged to determine adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames .
  • the adaptation processor 111 and the wavelet processor 105 is applied to one frame at a time. In the following, the processing of one frame will be described in more detail.
  • the adaptation processor 111 comprises an image area processor 113 which is arranged to identify different image areas in the video frame in response to the motion data. Specifically, the image area processor 113 divides the video frame into image areas for which motion compensation has been performed and image areas for which no motion compensation has been performed. The different image areas can easily be determined by the image area processor 113 as the motion data generated by the motion estimation processor 103 explicitly defines areas that are motion compensated. This information is already required by a decoder in order to apply the corresponding motion compensation at the decoding end. In the example, motion compensation is performed relative to video data in other video frames. Thus, a motion compensated area will be an inter-coded image area, meaning that it is encoded relative to video data in other frames. At the same time, non motion compensated image areas will be intra-coded image areas meaning that they will be encoded without reference to video data in other video frames.
  • the adaptation may be based only on inter/intra image areas, only on motion compensated/non-motion compensated image areas or on both inter/intra and motion compensated/non-motion compensated image areas .
  • the image area processor 113 thus divides the video frame into inter and intra areas. This creates a boundary between the intra areas and the inter areas and the wavelet transformation is adapted for these boundaries .
  • the adaptation processor 111 further comprises a parameter processor 115 which determines suitable adaptive parameters (may in some embodiments be a single adaptive parameter) for the wavelet transformation. Specifically, the parameter processor 115 receives information from the image area processor 113 identifying the inter and intra image areas (for example by an identification of the boundary between these) . The parameter processor 115 proceeds to determine adaptive parameters which are suited for the specific image area characteristics of the video frame .
  • the parameter processor 115 determines coefficients for the wavelet transformation taking into account the identified different areas of the video frame.
  • the parameter processor 115 of FIG. 1 selects a fixed set of wavelet coefficients for wavelet operations on pixels within each image area. However, for pixels adjacent to the boundaries between intra and inter image areas, different coefficients are applied.
  • the adaptation coefficients are selected such that the wavelet coefficients for pixels adjacent to a boundary are better suited for sharp transitions or edges in the image. Thus, the coefficients are selected such that the prediction and correlation for pixels of different image areas is substantially reduced.
  • image areas having different motion compensation characteristics are transformed as independent (or at least less dependent) objects.
  • motion information to determine the adaptation parameters for the wavelet transformation of a temporal frame
  • a spatially transformed frame which is much better suited for further coding is achieved.
  • the adaptation is based on information which is already- required for decoding, no additional data is required to be transmitted thus allowing a reduced data rate for a given encoded video quality.
  • the application of an adaptive spatial wavelet transform results in the energy introduced into low pass subbands by wavelet filtering over edges between intra and inter areas to be reduced. Specifically, intra-coded areas in high pass frames possess different properties than the other areas of motion-compensated frames.
  • the non-adaptive spatial energy compaction fails to concentrate most of the energy into low pass frames, leaving high-amplitude "edge" coefficients in high-frequency subbands.
  • the described approach thus allows the adaptation of the wavelet transform to be based on local properties of a high-pass temporal frame.
  • the wavelet processor 105 specifically implements the wavelet transform by a lifting algorithm.
  • the adaptation processor 111 is arranged to reduce the effect of the linear prediction of the lifting implementation on the adaptation boundaries between the inter- and intra-coded image areas of a frame. As a consequence of reducing this prediction between intra and inter coded areas, intra classified areas are transformed as (more) independent objects.
  • FIG. 2 illustrates an example of a lifting operation in accordance with some embodiments of the invention.
  • a 1-dimensional wavelet transform lifting algorithm is illustrated.
  • a non-adaptive lifting algorithm (resulting in a fixed transform) is applied to pixels (or input coefficients) that are not next to any adaptation boundary (or adaptation border) .
  • the wavelet transform within each area is the same whether the area is an inter area or an intra area and the same lifting coefficients are used.
  • the coefficients are changed to adapt the lifting algorithm.
  • the detected adaptation boundaries are used for choosing the suitable lifting algorithm coefficients during the transform of each frame pixel .
  • a series of lifting steps are applied to the pixels using adapted lifting coefficients or weights between pixels that belong to different image areas. Specifically, the absolute values of the coefficients are reduced between pixels of different image areas relative to coefficients applied to pixels within the same image area.
  • the lifting step that is performed on a pixel in one area, the neighbouring pixel from a different area is taken with the zero weight, while the other pixel from the same area is taken with the weight of 2.
  • the coupling between pixels from areas of different types is reduced or completely cancelled (illustrated by dashed lines in FIG. 2) .
  • the wavelet processor 105 uses the same adaptation boundary for all levels (or steps) of the lifting operation. Similarly, the same coefficients may be used at different levels of the lifting operation, or in some embodiments the coefficients may be varied between different levels.
  • the wavelet processor 105 of the encoder 100 of FIG. 1 applies the adaptive transform to the original high-pass frame and to each low-pass subband produced by one level of spatial transform.
  • the adaptive spatial wavelet transform can be applied separately to each colour component (e.g. Y, U, V), as typically the same motion information is used for all components.
  • the adaptation of the adaptive spatial wavelet transform of the encoder 100 is based on motion data which is already included in the encoded video signal in order to allow a decoder to perform correct decoding of the motion compensated data. Accordingly, the encoding processor 107 includes only the motion data in the encoded video signal and does not include any other side information related to the adaptation of the adaptive spatial wavelet transform.
  • FIG. 3 illustrates a decoder 300 in accordance with some embodiments of the invention.
  • the decoder 300 is operable to decode the signal received from the encoder 100 of FIG. 1. Specifically, the decoder 300 is arranged perform the ⁇ inverse operations of the encoder 100 and in particular it is arranged perform an inverse adaptive spatial wavelet transform in response to motion data.
  • the decoder 300 comprises a video data receiver 301 which receives the encoded video signal from the encoder 100.
  • the video data receiver 301 is coupled to a motion data extractor 303 which extracts motion data from the encoded video signal.
  • the motion data extractor 303 furthermore generates spatially transformed frames from the encoded video signal.
  • the motion data extractor 303 is fed to an inverse wavelet processor 305 which applies an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames.
  • the motion compensated video frames are fed to a decoding processor 307 coupled to the inverse wavelet processor 305.
  • the decoding processor 307 is arranged to generate a decoded video signal from the motion compensated video frames.
  • the decoder 300 furthermore comprises a decoder adaptation processor 309 which is coupled to the motion data extractor 303 and the inverse wavelet processor 305.
  • the decoder adaptation processor 309 is arranged to determine adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
  • the decoder adaptation processor 309 comprises a decode image area processor 311 which divides a given video frame into motion compensated and non-motion compensated image areas (corresponding to inter/intra image areas) .
  • the decoder adaptation processor 309 furthermore comprises a decode parameter processor 313 which determines the adaptive parameters in response to the identified inter/intra image areas.
  • the decode parameter processor 313 operates similarly to the parameter processor 115 of the encoder 100 and determines adaptive wavelet coefficients for pixels adjacent a boundary between the intra/intra image areas. These coefficients are fed to the inverse wavelet processor 305 and are used for the inverse adaptive spatial wavelet transform.
  • the operation of the adaptive spatial wavelet transform of the encoder 100 is effectively inversed using only motion data already included in the encoded signal for other purposes.
  • motion information drives the inverse spatial transform in the same way as at the encoder 100.
  • the temporal subbands are synthesised using the same adaptation boundaries that define the applied adaptive lifting. Therefore invertibility of the adaptive transform is ensured.
  • the proposed adaptive transform can be used in scalable video coding scenarios for all combinations of spatial, temporal and quality scalabilities .
  • the invention can be implemented in any suitable form including hardware, software, firmware or any combination of these.
  • the invention may optionally be implemented at least partly as computer software running on one or more data processors and/or digital signal processors.
  • the elements and components of an embodiment of the invention may be physically, functionally and logically implemented in any suitable way. Indeed the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the invention may be implemented in a single unit or may be physically and functionally distributed between different units and processors .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

An encoder (100) comprises a motion estimation processor (103) which generates motion compensated video frames for a video signal. The motion compensated video frames are fed to a wavelet processor (105) which applies an adaptive spatial wavelet transform to the frames to generate spatially transformed frames. An encoding processor (107) generates encoded video data for the video signal in response to the spatially transformed frames. The encoder (100) further comprises an adaptation processor (111) which determines adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames. The adaptive parameters may be wavelet coefficients for the adaptive spatial wavelet transform. The decoder (300) performs the inverse operation and determines adaptive parameters for an inverse adaptive spatial wavelet transform based on received motion data. The use of motion information for adaptation of the adaptive spatial wavelet transforms allows improved video quality to data rate ratio without necessitating additional side data.

Description

ENCODING AND DECODING OF A VIDEO SIGNAL
Field of the invention
The invention relates to encoding and/or decoding of a video signal and in particular to encoding and/or decoding using a wavelet transform.
Background of the Invention
As the distribution of digital content becomes increasingly popular the importance of efficient encoding of content signals becomes increasingly significant. For example, a number of advanced techniques and standards for encoding and decoding of video signals have been developed including the well known MPEG (Motion Picture Expert Group) standards. Furthermore, research is continuously being undertaken to develop further improved video encoding techniques. In recent years, there has been significant research into the application of wavelet transforms to video signals. In wavelet video encoding, wavelets were initially used as a replacement for the two dimensional transform, and later were extended to the temporal axis, thus yielding full three dimensional (spatio-temporal) wavelet coding schemes. The wavelet transform in the temporal direction can be performed either without or with motion compensated prediction, the latter case leading to a higher compression ratio. The key techniques for efficient video coding are based on temporal and spatial decorrelation of video frames. The application of a wavelet transform in the temporal and spatial domains provides high coding efficiency and allows an embedded and scalable representation of a video bitstream.
In video coding, the redundancy of information between consecutive frames is often used to reduce the coding rate. Typically, adjacent frames in a video sequence comprise similar images and it is therefore possible to represent information of a frame using data from already encoded frames. Specifically, motion compensation techniques are used to remove temporal redundancy to produce so-called temporal frames. Motion models (such as well known block-based models) are used in this context as parameters that drive motion compensation.
It has been proposed to use adaptive wavelet transforms for encoding of images rather than to use a fixed set of basis functions. This can achieve high energy compaction and thus an efficient encoding. However, there are also a number of disadvantages associated with currently known adaptive wavelet transform encoding techniques.
Specifically, current techniques tend to focus on individual images thereby not fully exploiting information of temporal characteristics of a video signal. Furthermore, most known techniques using adaptive wavelet transforms are based on an explicit identification of different areas within an image, adapting the wavelet transform to these areas and including data identifying the areas in the encoding data. Although, this may allow an adaptive wavelet transform encoding, it adds additional side data which must be included in the data stream thereby increasing the data rate. Also, known techniques tend to lead to suboptimal encoding performance in many situations and in particular the adaptation of the wavelet transform to the characteristics of the images tend to be suboptimal .
Hence, an improved system for video encoding would be advantageous and in particular a system allowing increased flexibility, improved wavelet transform encoding/decoding, improved adaptation, facilitated implementation, reduced computational requirements, reduced data rate and/or improved performance would be advantageous.
Summary of the Invention
Accordingly, the Invention seeks to preferably mitigate, alleviate or eliminate one or more of the above mentioned disadvantages singly or in any combination.
According to a first aspect of the invention there is provided an encoder for encoding a video signal, the encoder comprising: means for generating motion compensated video frames for the video signal; wavelet means for applying an adaptive spatial wavelet transform to the motion compensated video frames to generate spatially transformed frames; means for generating encoded video data for the video signal in response to the spatially transformed frames; and adaptive means for determining adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames.
The invention may allow an improved video encoding. An improved wavelet transformation may be used which more closely reflects the characteristics of the motion compensated video frames thereby allowing a more efficient encoding resulting in improved quality to data rate ratio. The invention may allow adaptation of an adaptive spatial wavelet transform at the encoder (and decoder) without resulting in the necessity for explicit data for the adaptation to be distributed with the encoded video data. Thus, a reduced data rate may be achieved. Also, the adaptation may be based on processing which has already been performed for other purposes resulting in a facilitated and reduced complexity implementation.
The invention may provide a better wavelet domain representation taking into consideration both spatial and temporal characteristics of the video signal. The improved wavelet domain representation may allow a more efficient compression of the video signal.
The adaptive parameters may comprise one or more adaptive parameters which are used to modify the operation of the adaptive spatial wavelet transform. The motion data may comprise' identifications of one or more motion compensated image areas in one or more of the motion compensated video frames. According to an optional feature of the invention, the adaptive parameters comprise wavelet transform coefficients .
This may allow a practical, easy to implement and/or efficient adaptation of the adaptive spatial wavelet transform.
According to an optional feature of the invention, the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the' second image area.
This may allow improved video encoding and may in particular provide an improved quality to data rate ratio. For example, the wavelet transform may be adapted to provide a more efficient wavelet representation of higher frequency components around image area transitions which tend to have in increased concentration of such components. Specifically, the feature may allow a better encoding of detail around an image area transition.
According to an optional feature of the invention, the adaptive means is arranged to determine the first image area and the second image area as image areas having different motion characteristics.
This may provide efficient encoding and/or facilitated/ reduced complexity encoding. In particular, different motion characteristics tend to indicate that the image areas belong to different objects in the image and thus to the existence of a sharp image transition (e.g. an object edge) comprising high frequency components and high levels of detail.
According to an optional feature of the invention, the first image area is a motion compensated image area and the second image area is a non-motion compensated image area.
This may provide efficient encoding and/or facilitated/ reduced complexity encoding. In particular, as information identifying the motion compensated areas is typically included in the encoded video signal, the adaptation of the wavelet transform may be achieved without necessitating that additional adaptation data is included in the encoded video signal.
According to an optional feature of the invention, the first image area is an inter-coded image area and the second image area is an intra-coded image area.
An inter-coded image area is an image area encoded relative to an image area in another frame. An intra-coded image area is an image area which is not encoded relative to an image area in any other frame. The feature may provide efficient encoding and/or facilitated/ reduced complexity encoding. In particular, as information of which areas are inter- or intra-coded areas are typically included in the encoded video signal, the adaptation of the wavelet transform may be achieved without necessitating that additional adaptation data is included in the encoded video signal.
According to an optional feature of the invention, the adaptive parameters comprise wavelet transform coefficients for picture elements of the first frame adjacent to the boundary.
This may provide efficient encoding and/or facilitated/ reduced complexity encoding. It may allow a particularly efficient and/or facilitated adaptation of the adaptive spatial wavelet transform.
According to an optional feature of the invention, the adaptive parameters do not comprise wavelet transform coefficients for picture elements of the first frame not adjacent to the boundary.
This may provide efficient encoding and/or facilitated/ reduced complexity encoding. It may allow a particularly efficient and/or facilitated adaptation of the adaptive spatial wavelet transform.
According to an optional feature of the invention, the wavelet means is arranged to apply the adaptive spatial wavelet transform by applying an adaptive wavelet lifting operation.
This may allow a particularly efficient implementation which may easily be adapted in response to the motion data. According to an optional feature of the invention, the adaptive parameters comprise coefficient values for multiple levels of the adaptive wavelet lifting operation.
This may allow particularly efficient adaptation which is easy to implement. The boundary may be the same for each of the multiple levels.
According to an optional feature of the invention, the adaptive parameters comprise different coefficient values for different levels of the multiple levels.
This may allow an improved and more flexible adaptation and thus improved encoding.
According to an optional feature of the invention, the wavelet means is arranged to apply the adaptive spatial wavelet transform to transformed spatial subband data.
This may provide efficient encoding and/or facilitated/ reduced complexity encoding.
According to an optional feature of the invention, the encoder further comprises means for generating an encoded data signal for the video signal comprising the encoded video data and the motion data.
The encoded data signal is a self consistent signal comprising all the data which is required to decode the video signal. The encoded data signal may thus be transmitted, broadcast or distributed to one or more decoders . According to an optional feature of the invention, the encoder is arranged to not include data of the adaptive parameters in the encoded data signal.
This may provide for a more efficient encoded data signal and may in particular reduce the data rate for a given encoded video quality.
According to another aspect of the invention, there is provided a decoder for decoding an encoded video signal, the decoder comprising: means for receiving the encoded video signal; means for generating spatially transformed frames from the encoded video signal; wavelet means for applying an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames; means for generating motion data for the motion compensated video frames; and adaptive means for determining adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
It will be appreciated that the comments and advantages provided for the encoder tend to apply equally to the decoder.
According to an optional feature of the invention, the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the second image area.
According to an optional feature of the invention, the adaptive means is arranged to determine the first and second image areas as image areas having different motion characteristics .
According to another aspect of the invention, there is provided a method of encoding a video signal comprising: generating motion compensated video frames for the video signal; applying an adaptive spatial wavelet transform to the motion compensated video frames to generate spatially transformed frames; generating encoded video data for the video signal in response to the spatially transformed frames; and determining adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames.
According to another aspect of the invention, there is provided a method of decoding an encoded video signal, the method comprising: receiving the encoded video signal; generating spatially transformed frames from the encoded video signal; applying an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames; generating motion data for the motion compensated video frames; and determining adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data. According to another aspect of the invention, there is provided a computer program product enabling the carrying out of a method as described above.
These and other aspects, features and advantages of the invention will be apparent from and elucidated with reference to the embodiment (s) described hereinafter.
Brief Description of the Drawings
Embodiments of the invention will be described, by way of example only, with reference to the drawings, in which
FIG. 1 is an illustration of an encoder for encoding a video signal in accordance with some embodiments of the invention;
FIG. 2 illustrates an example of a lifting operation in accordance with some embodiments of the invention; and
FIG. 3 illustrates a decoder in accordance with some embodiments of the invention.
Detailed Description of Some Embodiments of the Invention
FIG. 1 is an illustration of an encoder for encoding a video signal in accordance with some embodiments of the invention. In the example, the encoder 100 comprises a video frame source 101 which generates video frames from a video signal to be encoded. It will be appreciated that the video frame source 101 can be a receiver for receiving a digitised video signal already comprising suitable video frames from an external source, or that it can for example be an analog video signal which is then digitised and structured in suitable frames as will be well known to the person skilled in the art.
The video frame source 101 is coupled to a motion estimation processor 103 which performs motion estimation and compensation on the video frames to generate motion compensated video frames. Various methods and algorithms for motion estimation and compensation of video frames is well known to the person skilled in the art and is for brevity not described in detail herein.
In the specific example, the motion estimation processor 103 simply determines the movement of motion estimation blocks of a fixed size in different video frames. For example, the block size can be 16 x 16 pixels (picture elements) . If the video frames comprise images with a moving object, a given block of one image can frequently be found in a subsequent image at an (often slightly) different position. An efficient video encoding can then be obtained by merely encoding the displacement of the original block rather than the actual image data of the block in the subsequent frame. This displacement is known as the motion vector. Improved quality can be obtained by determining the differential between the blocks of the different frames and encoding this residual or error signal. It will be appreciated that more advanced motion compensation is 5 possible using for example further degrees of freedom in the movement (such as rotation) or differently shaped motion compensation blocks (possibly adaptive) . Indeed, advanced two or three dimensional motion estimation models for different objects can be used.
10
There are many different approaches known for motion estimation. A simple approach comprises a brute force search of subsequent frame (s) to detect any blocks that closely correspond to a block of a previous frame in order
15 to allow motion compensation. Typically, motion estimation comprises evaluating a simple or complex object motion model to determine locations in subsequent frames.
The motion estimation processor 103 is coupled to a 20 wavelet processor 105. The wavelet processor 105 implements an adaptive spatial wavelet transform which is applied the compensated video frames to generate spatially transformed frames. As will be known to the person skilled in the art, wavelet processing may in many situations 25 provide a more efficient representation than conventional Fourier transforms.
In the example of FIG. 1, the motion estimation processor 103 provides temporal frames data to the wavelet processor 30 105, on which the adaptive spatial wavelet transform is applied. The output of the motion estimation processor 103 is the spatio-temporal subbands of the source video frames .
The wavelet processor 105 is coupled to an encoding processor 107 which is fed the motion compensated video frames (in the form of spatio-temporal subband data) . The encoding processor 107 is arranged to generate encoded video data for the transformed video frames. Specifically, the encoding processor 107 performs quantisation, scaling and encoding of the data values of the transformed video frames as will be known to the person skilled in the art. The incoming processor 107 can proceed to generate a complete encoded video signal which comprises all the information required to decode the video signal. Specifically the encoding processor 107 includes the motion data used by the motion estimation processor 103 for the motion compensation.
The encoding processor 107 is coupled to a transmit interface 109 which is fed the encoded video signal and which is arranged to transmit the signal to one and more decoders . It will be appreciated that the transmit interface 109 for example can be an interface to the Internet or e.g. a radio transceiver, such as cellular radio transceiver, arranged to transmit the encoded video signal over an air interface of a wireless communication system.
In the encoder 100 of FIG. 1, the applied wavelet transform is an adaptive wavelet transform. Specifically, some or all of the coefficients of the wavelet transform can be adapted to the specific characteristics of the individual video frames. In the encoder 100 the adaptation of the wavelet transform is performed in response to motion data generated by the motion estimation processor 103 and used for the motion compensation of the motion compensated video frames.
Accordingly, the encoder 100 comprises an adaptation processor 111 which is arranged to determine adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames .
In the example, the adaptation processor 111 and the wavelet processor 105 is applied to one frame at a time. In the following, the processing of one frame will be described in more detail.
The adaptation processor 111 comprises an image area processor 113 which is arranged to identify different image areas in the video frame in response to the motion data. Specifically, the image area processor 113 divides the video frame into image areas for which motion compensation has been performed and image areas for which no motion compensation has been performed. The different image areas can easily be determined by the image area processor 113 as the motion data generated by the motion estimation processor 103 explicitly defines areas that are motion compensated. This information is already required by a decoder in order to apply the corresponding motion compensation at the decoding end. In the example, motion compensation is performed relative to video data in other video frames. Thus, a motion compensated area will be an inter-coded image area, meaning that it is encoded relative to video data in other frames. At the same time, non motion compensated image areas will be intra-coded image areas meaning that they will be encoded without reference to video data in other video frames.
It will be appreciated that in some embodiments there may not be a one-to-one correspondence between inter/intra image areas and motion compensated/non-motion compensated image areas. In such embodiments, the adaptation may be based only on inter/intra image areas, only on motion compensated/non-motion compensated image areas or on both inter/intra and motion compensated/non-motion compensated image areas .
In a specific example, the image area processor 113 thus divides the video frame into inter and intra areas. This creates a boundary between the intra areas and the inter areas and the wavelet transformation is adapted for these boundaries .
The adaptation processor 111 further comprises a parameter processor 115 which determines suitable adaptive parameters (may in some embodiments be a single adaptive parameter) for the wavelet transformation. Specifically, the parameter processor 115 receives information from the image area processor 113 identifying the inter and intra image areas (for example by an identification of the boundary between these) . The parameter processor 115 proceeds to determine adaptive parameters which are suited for the specific image area characteristics of the video frame .
In the example, the parameter processor 115 determines coefficients for the wavelet transformation taking into account the identified different areas of the video frame. The parameter processor 115 of FIG. 1 selects a fixed set of wavelet coefficients for wavelet operations on pixels within each image area. However, for pixels adjacent to the boundaries between intra and inter image areas, different coefficients are applied.
The adaptation coefficients are selected such that the wavelet coefficients for pixels adjacent to a boundary are better suited for sharp transitions or edges in the image. Thus, the coefficients are selected such that the prediction and correlation for pixels of different image areas is substantially reduced.
Thus, in the encoder 100, image areas having different motion compensation characteristics are transformed as independent (or at least less dependent) objects. By using motion information to determine the adaptation parameters for the wavelet transformation of a temporal frame, a spatially transformed frame which is much better suited for further coding is achieved. Furthermore, as the adaptation is based on information which is already- required for decoding, no additional data is required to be transmitted thus allowing a reduced data rate for a given encoded video quality. The application of an adaptive spatial wavelet transform results in the energy introduced into low pass subbands by wavelet filtering over edges between intra and inter areas to be reduced. Specifically, intra-coded areas in high pass frames possess different properties than the other areas of motion-compensated frames. Since the boundaries between intra and inter areas can typically be considered sharp edges, the non-adaptive spatial energy compaction fails to concentrate most of the energy into low pass frames, leaving high-amplitude "edge" coefficients in high-frequency subbands. The described approach thus allows the adaptation of the wavelet transform to be based on local properties of a high-pass temporal frame.
In the encoder 100, the wavelet processor 105 specifically implements the wavelet transform by a lifting algorithm. The adaptation processor 111 is arranged to reduce the effect of the linear prediction of the lifting implementation on the adaptation boundaries between the inter- and intra-coded image areas of a frame. As a consequence of reducing this prediction between intra and inter coded areas, intra classified areas are transformed as (more) independent objects.
FIG. 2 illustrates an example of a lifting operation in accordance with some embodiments of the invention. In the example, a 1-dimensional wavelet transform lifting algorithm is illustrated.
In the lifting operation of FIG. 2, a non-adaptive lifting algorithm (resulting in a fixed transform) is applied to pixels (or input coefficients) that are not next to any adaptation boundary (or adaptation border) . Thus, the wavelet transform within each area is the same whether the area is an inter area or an intra area and the same lifting coefficients are used. However, around the adaptation boundaries, the coefficients are changed to adapt the lifting algorithm. The detected adaptation boundaries are used for choosing the suitable lifting algorithm coefficients during the transform of each frame pixel .
As illustrated in FIG. 2, a series of lifting steps (prediction and update) are applied to the pixels using adapted lifting coefficients or weights between pixels that belong to different image areas. Specifically, the absolute values of the coefficients are reduced between pixels of different image areas relative to coefficients applied to pixels within the same image area. In some embodiments, for an intra/inter transition, the lifting step that is performed on a pixel in one area, the neighbouring pixel from a different area is taken with the zero weight, while the other pixel from the same area is taken with the weight of 2. As a result of adaptation, the coupling between pixels from areas of different types is reduced or completely cancelled (illustrated by dashed lines in FIG. 2) .
As illustrated in FIG. 2, the wavelet processor 105 uses the same adaptation boundary for all levels (or steps) of the lifting operation. Similarly, the same coefficients may be used at different levels of the lifting operation, or in some embodiments the coefficients may be varied between different levels. The wavelet processor 105 of the encoder 100 of FIG. 1 applies the adaptive transform to the original high-pass frame and to each low-pass subband produced by one level of spatial transform.
Furthermore, for colour video signals, the adaptive spatial wavelet transform can be applied separately to each colour component (e.g. Y, U, V), as typically the same motion information is used for all components.
It will be appreciated that the adaptation of the adaptive spatial wavelet transform of the encoder 100 is based on motion data which is already included in the encoded video signal in order to allow a decoder to perform correct decoding of the motion compensated data. Accordingly, the encoding processor 107 includes only the motion data in the encoded video signal and does not include any other side information related to the adaptation of the adaptive spatial wavelet transform.
FIG. 3 illustrates a decoder 300 in accordance with some embodiments of the invention. The decoder 300 is operable to decode the signal received from the encoder 100 of FIG. 1. Specifically, the decoder 300 is arranged perform the ^inverse operations of the encoder 100 and in particular it is arranged perform an inverse adaptive spatial wavelet transform in response to motion data.
The decoder 300 comprises a video data receiver 301 which receives the encoded video signal from the encoder 100. The video data receiver 301 is coupled to a motion data extractor 303 which extracts motion data from the encoded video signal. The motion data extractor 303 furthermore generates spatially transformed frames from the encoded video signal.
The motion data extractor 303 is fed to an inverse wavelet processor 305 which applies an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames.
The motion compensated video frames are fed to a decoding processor 307 coupled to the inverse wavelet processor 305. The decoding processor 307 is arranged to generate a decoded video signal from the motion compensated video frames.
The decoder 300 furthermore comprises a decoder adaptation processor 309 which is coupled to the motion data extractor 303 and the inverse wavelet processor 305. The decoder adaptation processor 309 is arranged to determine adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
Specifically, the decoder adaptation processor 309 comprises a decode image area processor 311 which divides a given video frame into motion compensated and non-motion compensated image areas (corresponding to inter/intra image areas) .
The decoder adaptation processor 309 furthermore comprises a decode parameter processor 313 which determines the adaptive parameters in response to the identified inter/intra image areas. Specifically, the decode parameter processor 313 operates similarly to the parameter processor 115 of the encoder 100 and determines adaptive wavelet coefficients for pixels adjacent a boundary between the intra/intra image areas. These coefficients are fed to the inverse wavelet processor 305 and are used for the inverse adaptive spatial wavelet transform.
Thus, the operation of the adaptive spatial wavelet transform of the encoder 100 is effectively inversed using only motion data already included in the encoded signal for other purposes.
Hence, at the decoder 300, motion information drives the inverse spatial transform in the same way as at the encoder 100. The temporal subbands are synthesised using the same adaptation boundaries that define the applied adaptive lifting. Therefore invertibility of the adaptive transform is ensured. Moreover, the proposed adaptive transform can be used in scalable video coding scenarios for all combinations of spatial, temporal and quality scalabilities .
It will be appreciated that the above description for clarity has described embodiments of the invention with reference to different functional units and processors. However, it will be apparent that any suitable distribution of functionality between different functional units or processors may be used without detracting from the invention. For example, functionality illustrated to be performed by separate processors or controllers may be performed by the same processor or controllers. Hence, references to specific functional units are only to be seen as references to suitable means for providing the described functionality rather than indicative of a strict logical or physical structure or organization.
The invention can be implemented in any suitable form including hardware, software, firmware or any combination of these. The invention may optionally be implemented at least partly as computer software running on one or more data processors and/or digital signal processors. The elements and components of an embodiment of the invention may be physically, functionally and logically implemented in any suitable way. Indeed the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the invention may be implemented in a single unit or may be physically and functionally distributed between different units and processors .
Although the present invention has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Rather, the scope of the present invention is limited only by the accompanying claims. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in accordance with the invention. In the claims, the term comprising does not exclude the presence of other elements or steps. Furthermore, although individually listed, a plurality of means, elements or method steps may be implemented by e.g. a single unit or processor. Additionally, although individual features may be included in different claims, these may possibly be advantageously combined, and the inclusion in different claims does not imply that a combination of features is not feasible and/or advantageous. Also the inclusion of a feature in one category of claims does not imply a limitation to this category but rather indicates that the feature is equally applicable to other claim categories as appropriate. Furthermore, the order of features in the claims does not imply any specific order in which the features must be worked and in particular the order of individual steps in a method claim does not imply that the steps must be performed in this order. Rather, the steps may be performed in any suitable order.

Claims

1. An encoder for encoding a video signal, the encoder comprising: means for generating motion compensated video frames for the video signal; wavelet means for applying an adaptive spatial wavelet transform to the motion compensated video frames to generate spatially transformed frames; means for generating encoded video data for the video signal in response to the spatially transformed frames; and adaptive means for determining adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames.
2. The encoder of claim 1 wherein the adaptive parameters comprise wavelet transform coefficients.
3. The encoder of claim 2 wherein the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the second image area.
4. The encoder of claim 3 wherein the adaptive means is arranged to determine the first image area and the second image area as image areas having different motion characteristics .
5. The encoder of claim 4 wherein the first image area is a motion compensated image area and the second image area is a non-motion compensated image area.
6. The encoder of the claims 3 wherein the first image area is an inter-coded image area and the second image area is an intra-coded image area.
7. The encoder of claims 3 wherein the adaptive parameters comprise wavelet transform coefficients for picture elements of the first frame adjacent to the boundary.
8. The encoder of claim 3 wherein the wavelet means is arranged to apply the adaptive spatial wavelet transform by aPPlying an adaptive wavelet lifting operation.
9. The encoder of claim 8 wherein the adaptive parameters comprise coefficient values for multiple levels of the adaptive wavelet lifting operation.
10. The encoder of claim 9 wherein the adaptive parameters comprise different coefficient values for different levels of the multiple levels.
11. A decoder for decoding an encoded video signal, the decoder comprising: means for receiving the encoded video signal; means for generating spatially transformed frames from the encoded video signal; wavelet means for applying an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames; means for generating motion data for the motion compensated video frames; and adaptive means for determining adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
12. The decoder of claim 11 wherein the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the second image area.
EP06816113A 2005-10-10 2006-10-03 Encoding and decoding of a video signal Withdrawn EP1938598A4 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB0520557A GB2431064A (en) 2005-10-10 2005-10-10 Encoding a video signal using a motion-dependent adaptive spatial wavelet transform
PCT/US2006/038624 WO2007044333A2 (en) 2005-10-10 2006-10-03 Encoding and decoding of a video signal

Publications (2)

Publication Number Publication Date
EP1938598A2 true EP1938598A2 (en) 2008-07-02
EP1938598A4 EP1938598A4 (en) 2011-06-01

Family

ID=35430100

Family Applications (1)

Application Number Title Priority Date Filing Date
EP06816113A Withdrawn EP1938598A4 (en) 2005-10-10 2006-10-03 Encoding and decoding of a video signal

Country Status (4)

Country Link
EP (1) EP1938598A4 (en)
CN (1) CN101322404A (en)
GB (1) GB2431064A (en)
WO (1) WO2007044333A2 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112616058B (en) * 2019-10-03 2024-07-12 腾讯美国有限责任公司 Video encoding or decoding method, apparatus, computer device, and storage medium

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5495292A (en) * 1993-09-03 1996-02-27 Gte Laboratories Incorporated Inter-frame wavelet transform coder for color video compression
WO1997017797A2 (en) * 1995-10-25 1997-05-15 Sarnoff Corporation Apparatus and method for quadtree based variable block size motion estimation
JP4254017B2 (en) * 2000-03-10 2009-04-15 ソニー株式会社 Image coding apparatus and method
US7023922B1 (en) * 2000-06-21 2006-04-04 Microsoft Corporation Video coding system and method using 3-D discrete wavelet transform and entropy coding with motion information
US7991051B2 (en) * 2003-11-21 2011-08-02 Electronics And Telecommunications Research Institute Interframe wavelet coding apparatus and method capable of adjusting computational complexity
US7580461B2 (en) * 2004-02-27 2009-08-25 Microsoft Corporation Barbell lifting for wavelet coding

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
CLAYPOOLE R L JR ET AL: "Adaptive wavelet transforms via lifting", ACOUSTICS, SPEECH AND SIGNAL PROCESSING, 1998. PROCEEDINGS OF THE 1998 IEEE INTERNATIONAL CONFERENCE ON SEATTLE, WA, USA 12-15 MAY 1998, NEW YORK, NY, USA,IEEE, US, vol. 3, 12 May 1998 (1998-05-12), pages 1513-1516, XP010279965, DOI: DOI:10.1109/ICASSP.1998.681737 ISBN: 978-0-7803-4428-0 *
NIKOLA SPRLJAN ET AL: "Motion Driven Adaptation of Spatial Wavelet Transform", ITU STUDY GROUP 16 - VIDEO CODING EXPERTS GROUP -ISO/IEC MPEG & ITU-T VCEG(ISO/IEC JTC1/SC29/WG11 AND ITU-T SG16 Q6), XX, XX, no. M12633, 15 October 2005 (2005-10-15), XP030041303, *
RUSERT T ET AL: "Enhanced interframe wavelet video coding considering the interrelation of spatio-temporal transform and motion compensation", SIGNAL PROCESSING. IMAGE COMMUNICATION, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 19, no. 7, 1 August 2004 (2004-08-01) , pages 617-635, XP004524456, ISSN: 0923-5965, DOI: DOI:10.1016/J.IMAGE.2004.05.005 *
See also references of WO2007044333A2 *
ZHEN LI ET AL: "Wavelet video coding via a spatially adaptive lifting structure", PROCEEDINGS OF INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP'03) 6-10 APRIL 2003 HONG KONG, CHINA; [IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP)], 2003 IEEE INTERNATIONAL CONFERENCE, vol. 3, 6 April 2003 (2003-04-06), pages III_93-III_96, XP010639017, DOI: DOI:10.1109/ICASSP.2003.1199115 ISBN: 978-0-7803-7663-2 *

Also Published As

Publication number Publication date
WO2007044333A3 (en) 2007-12-21
GB2431064A (en) 2007-04-11
WO2007044333A2 (en) 2007-04-19
EP1938598A4 (en) 2011-06-01
CN101322404A (en) 2008-12-10
GB0520557D0 (en) 2005-11-16

Similar Documents

Publication Publication Date Title
KR100802203B1 (en) Bitstream Control Post-Processing Filtering Method and Computer-readable Media
US7120197B2 (en) Motion compensation loop with filtering
KR100957322B1 (en) Encoder Auxiliary Frame Rate Up Conversion Using Various Motion Models
KR100643819B1 (en) Parameterization for Fading Correction
US20150350667A1 (en) Methods and systems for motion vector derivation at a video decoder
EP1404136A1 (en) Image encoder, image decoder, image encoding method, and image decoding method
KR100578432B1 (en) Signaling for fading correction
KR100578433B1 (en) Fading Estimation / Calibration
US20040076231A1 (en) Method and apparatus for efficient video processing
WO2012033971A1 (en) Recovering a pruned version of a picture in a video sequence for example - based data pruning using intra- frame patch similarity
CN1720729A (en) Decoder apparatus and method for smoothing artifacts generated during error concealment
CN110199521A (en) Low complex degree hybrid domain for damaging Video coding cooperates with in-loop filter
CN104937937A (en) Video coding method and related video coding apparatus using at least assessed visual quality
EP1639829A2 (en) Optical flow estimation method
WO2008054799A2 (en) Spatial sparsity induced temporal prediction for video compression
WO2014117048A1 (en) Spatially adaptive video coding
US12526409B2 (en) Inter-frame prediction method, coder, decoder, and storage medium
JP5746193B2 (en) Efficient adaptive filtering method and apparatus for video encoding and decoding
JPH06292173A (en) Video signal encoding / decoding method
CN117750020A (en) Method, system, equipment and storage medium for learning video coding
JP4943586B2 (en) Method and apparatus for efficient video processing
CN101841705A (en) Video lossless compression method based on adaptive template
WO2006046834A1 (en) Method for coding and decoding moving image
WO2007044333A2 (en) Encoding and decoding of a video signal
WO2006074043A2 (en) Method and apparatus for providing motion estimation with weight prediction

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20080407

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK RS

RBV Designated contracting states (corrected)

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: MOTOROLA MOBILITY, INC.

A4 Supplementary search report drawn up and despatched

Effective date: 20110502

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 7/50 20060101AFI20110426BHEP

17Q First examination report despatched

Effective date: 20110906

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: MOTOROLA MOBILITY LLC

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20140910

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230520