WO2010051846A1 - Algorithme de sélection de blocs rapide pour le codage vidéo utilisant une transformation variable dans l'espace - Google Patents

Algorithme de sélection de blocs rapide pour le codage vidéo utilisant une transformation variable dans l'espace Download PDF

Info

Publication number
WO2010051846A1
WO2010051846A1 PCT/EP2008/065088 EP2008065088W WO2010051846A1 WO 2010051846 A1 WO2010051846 A1 WO 2010051846A1 EP 2008065088 W EP2008065088 W EP 2008065088W WO 2010051846 A1 WO2010051846 A1 WO 2010051846A1
Authority
WO
WIPO (PCT)
Prior art keywords
candidate pixel
block
pixel block
candidate
blocks
Prior art date
Application number
PCT/EP2008/065088
Other languages
English (en)
Inventor
Cixun Zhang
Kemal Ugur
Jani Lainema
Original Assignee
Nokia Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Corporation filed Critical Nokia Corporation
Priority to PCT/EP2008/065088 priority Critical patent/WO2010051846A1/fr
Publication of WO2010051846A1 publication Critical patent/WO2010051846A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/19—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding using optimisation based on Lagrange multipliers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H04N19/463—Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop

Definitions

  • the present invention relates to apparatus for coding and decoding and specifically but not only for coding and decoding of image and video signals
  • a video codec comprises an encoder which transforms input video into a compressed representation suitable for storage and/or transmission and a decoder than can uncompress the compressed video representation back into a viewable form.
  • the encoder discards some information in the original video sequence in order to represent the video in a more compact form, for example at a lower bit rate.
  • Typical video codecs for example International Telephone Union - Technical Board (ITU-T) H.263 and H.264 coding standards, encode video information in two phases.
  • pixel values in a certain picture area or "block" are predicted.
  • These pixel values can be predicted, for example, by motion compensation mechanisms, which involve finding and indicating an area in one of the previously encoded video frames (or a later coded video frame) that corresponds closely to the block being coded.
  • pixel values can be predicted by spatial mechanisms which involve finding and indicating a spatial region relationship.
  • the second phase is one of coding the error between the predicted block of pixels and the original block of pixels. This is typically accomplished by transforming the difference in pixel values using a specified transform. This transform is typically a Discrete Cosine Transform (DCT) or a variant thereof. After transforming the difference, the transformed difference is quantized and entropy encoded. By varying the fidelity of the quantisation process, the encoder can control the balance between the accuracy of the pixei representation, (in other words, the quality of the picture) and the size of the resulting encoded video representation (in other words, the file size or transmission bit rate).
  • DCT Discrete Cosine Transform
  • the decoder reconstructs the output video by applying a prediction mechanism similar to that used by the encoder in order to form a predicted representation of the pixel blocks (using the motion or spatial information created by the encoder and stored in the compressed representation of the image) and prediction error decoding (the inverse operation of the prediction error coding to recover the quantised prediction signal in the spatial domain).
  • the decoder After applying pixel prediction and error decoding processes the decoder combines the prediction and the prediction error signals (the pixei values) to form the output video frame.
  • the decoder may also apply additional filtering processes in order to improve the quality of the output video before passing it for display and/or storing as a prediction reference for the forthcoming frames in the video sequence.
  • the motion information is indicated by motion vectors associated with each motion compensated image block.
  • Each of these motion vectors represents the displacement of the image block in the picture to be coded (in the encoder) or decoded (at the decoder) and the prediction source block in one of the previously coded or decoded images (or pictures).
  • motion vectors are typically coded differentially with respect to block specific predicted motion vector.
  • the predicted motion vectors are created in a predefined way, for example by calculating the median of the encoded or decoded motion vectors of the adjacent blocks.
  • Typical video encoders utilise the Lagrangian cost function to find optimal coding modes, for example the desired macro block mode and associated motion vectors. This type of cost function uses a weighting factor or ⁇ to tie together the exact or estimated image distortion due to lossy coding methods and the exact or estimated amount of information required to represent the pixel values in an image area.
  • C the Lagrangian cost to be minimised
  • D the image distortion (in other words the mean-squared error) with the mode and motion vectors currently considered
  • R the number of bits needed to represent the required data to reconstruct the image block in the decoder (including the amount of data to represent the candidate motion vectors).
  • codecs encode the residual signal typically using a MxN DCT transform.
  • edge detail within these MxN macro blocks prevents the basis functions of the transform from being able to exploit any correlation in the residual signal and may produce a lower coding efficiency.
  • alignment of the block size used for transform coding of the prediction error to the block size used for motion compensation occurs.
  • edges occur within a block, produce a sub-optimal coding efficiency.
  • the spatially varying transform improves the coding efficiency of the video coders and allows the video codec to use transformation coding over a region within the residual macro block where the spatial position of the region may vary.
  • This technique however has the problem that a significantly increased amount of encoding complexity is required as the encoder needs to search for the best position for the spatially varying transform (SVT) block among a number of candidate positions.
  • This additional encoding complexity is problematic particularly in real time encoding and resource constrained devices such as portable apparatus where encoding complexity typically carries with it a power consumption penalty.
  • this technique requires a significant bit overhead for transmitting the position of the SVT block within the macroblock as there are a large number of candidate positions.
  • This invention proceeds from the consideration that by using a reduced complexity spatially variable region or block within a macro block, the residual error coding process may produce a more optimally encoded image.
  • Embodiments of the present invention aim to address the above problem.
  • apparatus configured to: determine for each candidate pixel block of a plurality of candidate pixel blocks at least one criteria value, each candidate pixel block being a pixel block associated with a macroblock of pixels; and select at least one candidate pixel biock dependent on the at least one criteria value.
  • the apparatus may further comprise encoding at least one of the selected candidate pixel blocks.
  • the apparatus may be further configured to: determine at least one further criteria value for each of the selected at least one candidate block.
  • the apparatus may be further configured to select the at least one of the selected candidate pixel blocks dependent on the at least one further criteria value.
  • the further criteria value is preferably dependent on the correlation between the candidate pixel block and a corresponding set of pixels from the macroblock of pixels.
  • the macroblock is preferably associated with at least two motion compensation blocks and the criteria value for each candidate pixel block may comprise at least one of: whether the candidate pixel block straddles a boundary between motion compensation blocks; the number of motion compensation block boundaries straddled by the candidate pixel block; a reference frame index associated with the motion compensation blocks that the candidate pixel block straddles; at least one transform coefficient of the motion compensation blocks that the candidate pixel block straddles; a statistical distribution of the positions of current frame or previous frame candidate pixel blocks; a size of an overlapping region between the motion compensation block and the candidate pixel block; and a difference in the motion vector value associated with any motion compensation block boundaries straddled by the candidate pixel block.
  • the macroblock is preferably associated with one or more intra predicted blocks and the criteria value for each candidate pixel block may comprise at least one of: intra prediction direction of the macroblock; and intra prediction direction associated with the intra predicted blocks that the candidate pixel block straddles.
  • the apparatus may be further configured to select at least one candidate pixel block dependent on the motion vector value is less than or equal to a threshold value.
  • the threshold value is preferably 4 in units of quarter luma samples.
  • the apparatus may be further configured to encode an indicator of the at least one candidate pixel block selected.
  • the apparatus may be further configured to encode the indicator of the at least one candidate pixel block selected dependent on an order value associated with the candidate pixel block.
  • the order value associated with candidate block is preferably dependent on the at least one criteria.
  • the apparatus may be further configured to: determine for each candidate pixel block at least one additional criteria value; and generate for the selected at least one candidate pixel block the order value associated with the candidate block dependent on the at least one additional criteria value.
  • the apparatus may be further configured to encode the indicator of the at least one candidate pixel block selected using one of: entropy encoding; and variable length encoding.
  • an apparatus configured to: determine a first set of candidate pixel blocks, each candidate pixel block being a pixel block associated with a macroblock of pixels; determine a criteria value for each of the first set of candidate pixel blocks; and select at least one of the set of candidate pixel blocks dependent on the criteria value.
  • the macroblock is preferably associated with at least two motion compensation blocks and the criteria value for each candidate pixel block may comprise at least one of: whether the candidate pixel block straddles a boundary between motion compensation blocks; the number of motion compensation block boundaries straddled by the candidate pixel block; a reference frame index associated with the motion compensation blocks that the candidate pixel block straddles; at least one transform coefficient of the motion compensation blocks that the candidate pixel block straddles; a statistical distribution of the positions of current frame or previous frame candidate pixel blocks; a size of an overlapping region between the motion compensation block and the candidate pixel block; and a difference in the motion vector value associated with any motion compensation block boundaries straddled by the candidate pixel block.
  • the macroblock is preferably associated with one or more intra predicted blocks and the criteria value for each candidate pixel block may comprise at least one of: intra prediction direction of the macroblock; and intra prediction direction associated with the intra predicted blocks that the candidate pixel block straddles.
  • the apparatus may be further configured to select at least one candidate pixel block dependent on the motion vector value is less than or equal to a threshold value.
  • the threshold value is preferably 4 in units of quarter luma samples.
  • the apparatus may be further configured to generate for the selected at least one candidate pixel block an order value associated with the candidate block.
  • the order value associated with the candidate block is preferably dependent on the at least one criteria.
  • the apparatus may be further configured to: determine for each candidate pixel block at least one additional criteria value; and generate for the selected at least one candidate pixel block the order value associated with the candidate block dependent on the at least one additional criteria value.
  • the apparatus may be further configured to: determine a first indicator value; and select one of the at least one of the set of candidate pixel blocks selected as indicated by the first indicator value.
  • the apparatus may be further configured to select one of the at least one of the set of candidate pixel blocks selected as indicated by the first indicator value dependent on the an order value associated with the candidate block.
  • the apparatus may be further configured to: regenerate a first set of pixel values from the at least one of the set of candidate pixel blocks selected.
  • the apparatus may be further configured to regenerate the first set of pixel values by: dequantizing the at least one of the set of candidate pixel blocks selected; and inverse transforming a dequantized at least one of the set of candidate pixel blocks selected.
  • the apparatus may comprise an encoder.
  • the apparatus may comprise a decoder.
  • An electronic device may comprise apparatus as featured above.
  • a chipset may comprise apparatus as featured above.
  • a method comprising: determining for each candidate pixel block of a plurality of candidate pixel blocks at least one criteria value, each candidate pixel block being a pixel block associated with a macroblock of pixels; and selecting at least one candidate pixel block dependent on the at least one criteria value.
  • the method may further comprise encoding at least one of the selected candidate pixel blocks.
  • the method may further comprise determining at least one further criteria value for each of the selected at least one candidate block.
  • the method may further comprise selecting the at least one of the selected candidate pixel blocks dependent on the at least one further criteria value.
  • the further criteria value is preferably dependent on the correlation between the candidate pixel block and a corresponding set of pixels from the macroblock of pixels.
  • the macroblock is preferably associated with at least two motion compensation blocks and the criteria value for each candidate pixel block may comprise at least one of: whether the candidate pixel block straddles a boundary between motion compensation blocks; the number of motion compensation block boundaries straddled by the candidate pixel block; a reference frame index associated with the motion compensation blocks that the candidate pixel block straddles; at least one transform coefficient of the motion compensation blocks that the candidate pixel block straddles; a statistical distribution of the positions of current frame or previous frame candidate pixel blocks; a size of an overlapping region between the motion compensation biock and the candidate pixel block; and a difference in the motion vector value associated with any motion compensation block boundaries straddled by the candidate pixel block.
  • the macroblock is preferably associated with one or more intra predicted blocks and the criteria value for each candidate pixel block may comprise at least one of: intra prediction direction of the macroblock; and intra prediction direction associated with the intra predicted blocks that the candidate pixel block straddles.
  • the method may further comprise selecting at least one candidate pixel block dependent on the motion vector value is less than or equal to a threshold value.
  • the threshold value is preferably 4 in units of quarter luma samples.
  • the method may further comprise encoding an indicator of the at least one candidate pixel block selected.
  • the method may further comprise encoding the indicator of the at least one candidate pixel block selected dependent on an order value associated with the candidate pixel block.
  • the method may further comprise generating for the selected at least one candidate pixel block the order value associated with the candidate block.
  • the order value associated with the candidate block is preferably dependent on the at least one criteria.
  • the method may further comprise encoding the indicator of the at least one candidate pixel block selected using one of: entropy encoding; and variable length encoding.
  • a method comprising: determining a first set of candidate pixel blocks, each candidate pixel block being a pixel block associated with a macroblock of pixels; determining a criteria value for each of the first set of candidate pixel blocks; and selecting at least one of the set of candidate pixel blocks dependent on the criteria value.
  • the macroblock is preferably associated with at least two motion compensation blocks and the criteria value for each candidate pixel block may comprise at least one of: whether the candidate pixel block straddles a boundary between motion compensation blocks; the number of motion compensation block boundaries straddled by the candidate pixel block; a reference frame index associated with the motion compensation blocks that the candidate pixel block straddles; at least one transform coefficient of the motion compensation blocks that the candidate pixel block straddles; a statistical distribution of the positions of current frame or previous frame candidate pixel blocks; a size of an overlapping region between the motion compensation block and the candidate pixel block; and a difference in the motion vector value associated with any motion compensation block boundaries straddled by the candidate pixel block.
  • the macroblock is preferably associated with one or more intra predicted blocks and the criteria value for each candidate pixel block may comprise at least one of: intra prediction direction of the macroblock; and intra prediction direction associated with the intra predicted blocks that the candidate pixel block straddles.
  • the criteria is preferably a difference in the motion vector value associated with any motion compensation block boundaries straddled by the candidate pixel block
  • the method may be further configured to select at least one candidate pixel block dependent on the motion vector value is less than or equal to a threshold value.
  • the threshold value is preferably 4 in units of quarter luma samples.
  • the method may further comprise generating for the selected at least one candidate pixel block an order value associated with the candidate block.
  • the order value associated with the candidate block is preferably dependent on the at least one criteria.
  • the method may be further configured to: determining a first indicator value; selecting one of the at least one of the set of candidate pixel blocks selected as indicated by the first indicator value.
  • the method may further comprise selecting one of the at least one of the set of candidate pixel blocks selected as indicated by the first indicator value dependent on the an order value associated with the candidate block.
  • the method may further comprise regenerating a first set of pixel values from the at least one of the set of candidate pixel blocks selected.
  • the method may further comprise regenerating the first set of pixel values by: dequantizing the at least one of the set of candidate pixel blocks selected; and inverse transforming a dequantized at least one of the set of candidate pixel blocks selected.
  • a computer-readable medium encoded with instructions that, when executed by a computer, perform: determining for each candidate pixel block of a plurality of candidate pixel blocks at least one criteria value, each candidate pixel block being a pixel block associated with a macroblock of pixels; and selecting at least one candidate pixel block dependent on the at least one criteria value.
  • a computer-readable medium encoded with instructions that, when executed by a computer, perform: determining a first set of candidate pixel blocks, each candidate pixel block being a pixel block associated with a macroblock of pixels; determining a criteria value for each of the first set of candidate pixel blocks; and selecting at least one of the set of candidate pixel blocks dependent on the criteria value.
  • an apparatus comprising: means for determining for each candidate pixel block of a plurality of candidate pixel blocks at least one criteria value, each candidate pixel block being a pixel block associated with a macroblock of pixels; and means for selecting at least one candidate pixel block dependent on the at least one criteria value.
  • an apparatus comprising: means for determining a first set of candidate pixel blocks, each candidate pixel block being a pixel block associated with a macroblock of pixels; means for determining a criteria value for each of the first set of candidate pixel blocks; and means for selecting at least one of the set of candidate pixel blocks dependent on the criteria value.
  • Figure 1 shows schematically an electronic device employing embodiments of the invention
  • Figure 2 shows schematically a user equipment suitable for employing embodiments of the invention
  • FIG. 3 further shows schematically electronic devices employing embodiments of the invention connected using wireless and wired network connections
  • Figure 4 shows schematically an embodiment of the invention as incorporated within an encoder
  • Figure 5 shows a flow diagram showing the operation of an embodiment of the invention with respect to the residual encoder as shown in figure 4;
  • Figure 6 shows a schematic diagram of a decoder according to embodiments of the invention.
  • Figures 7a and 7b show a flow diagram of showing the operation of an embodiment of the invention with respect to the decoder shown in figure 6; and Figure 8 shows a simplified representation of a spatially varying transform block selection and offset from the macro block origin according to embodiments of the invention.
  • Figure 1 shows a schematic block diagram of an exemplary apparatus or electronic device 50, which may incorporate a codec according to an embodiment of the invention.
  • the electronic device 50 may for example be a mobile terminal or user equipment of a wireless communication system.
  • embodiments of the invention may be implemented within any electronic device or apparatus which may require encoding and decoding or encoding or decoding video images.
  • the apparatus 50 may comprise a housing 30 for incorporating and protecting the device.
  • the apparatus 50 further may comprise a display 32 in the form of a liquid crystal display, in other embodiments of the invention the display may be any suitable display technology suitable to display an image or video.
  • the apparatus 50 may further comprise a keypad 34.
  • any suitable data or user interface mechanism may be employed.
  • the user interface may be implemented as a virtual keyboard or data entry system as part of a touch-sensitive display.
  • the apparatus may comprise a microphone 36 or any suitable audio input which may be a digital or analogue signal input.
  • the apparatus 50 may further comprise an audio output device which in embodiments of the invention may be any one of: an earpiece 38, speaker, or an analogue audio or digital audio output connection.
  • the apparatus 50 may also comprise a battery 40 (or in other embodiments of the invention the device may be powered by any suitable mobile energy device such as solar cell, fuel cell or clockwork generator).
  • the apparatus may further comprise an infrared port 42 for short range line of sight communication to other devices.
  • the apparatus 50 may further comprise any suitable short range communication solution such as for example a Bluetooth wireless connection or a USB/firewire wired connection.
  • the apparatus 50 may comprise a controller 56 or processor for controlling the apparatus 50.
  • the controller 56 may be connected to memory 58 which in embodiments of the invention may store both data in the form of image and audio data and/or may also store instructions for implementation on the controller 56.
  • the controller 56 may further be connected to codec circuitry 54 suitable for carrying out coding and decoding of audio and/or video data or assisting in coding and decoding carried out by the controller 56.
  • the apparatus 50 may further comprise a card reader 48 and a smart card 46, for example a UICC and UICC reader for providing user information and being suitable for providing authentication information for authentication and authorization of the user at a network.
  • the apparatus 50 may comprise radio interface circuitry 52 connected to the controller and suitable for generating wireless communication signals for example for communication with a cellular communications network, a wireless communications system or a wireless local area network.
  • the apparatus 50 further may comprise an antenna 44 connected to the radio interface circuitry 52 for transmitting and receiving radio frequency signals generated at the radio interface circuitry 52.
  • the apparatus 50 comprises a camera capable of recording or detecting individual frames which are then passed to the codec 54 or controller for processing.
  • the apparatus may receive the video image data for processing from an adjacent device prior to transmission and/or storage.
  • the apparatus 50 may receive either wirelessly or by a wired connection the image for coding/decoding.
  • the system 10 comprises multiple communication devices which can communicate through one or more networks.
  • the system 10 may comprise any combination of wired or wireless networks including, but not limited to a wireless cellular telephone network (such as a GSM, UMTS, CDMA network etc), a wireless local area network (WL-AN) such as defined by any of the IEEE 802.x standards, a Bluetooth personal area network, an Ethernet local area network, a token ring local area network, a wide area network, and the Internet.
  • the system 10 may include both wired and wireless communication devices or apparatus 50 suitable for implementing embodiments of the invention.
  • the system shown in Figure 3 shows a mobile telephone network 11 and a representation of the internet 28.
  • Connectivity to the internet 28 may include, but is not limited to, long range wireless connections, short range wireless connections, and various wired connections including, but not limited to, telephone lines, cabie lines, power lines, and similar communication pathways.
  • the example communication devices shown in the system 10 may include, but are not limited to, an electronic device or apparatus 50, a combination personal digital assistant (PDA) and mobile telephone 14, a PDA 16, an integrated messaging device (IMD) 18, a desktop computer 20, a notebook computer 22.
  • the apparatus 50 may be stationary or mobile when carried by an individual who is moving.
  • the apparatus 50 may also be located in a mode of transport including, but not limited to, a car, a truck, a taxi, a bus, a train, a boat, an aeroplane, a bicycle, a motorcycle or any similar suitable mode of transport.
  • Some or further apparatus may send and receive calls and messages and communicate with service providers through a wireless connection 25 to a base station 24.
  • the base station 24 may be connected to a network server 26 that allows communication between the mobile telephone network 11 and the internet 28.
  • the system may include additional communication devices and communication devices of various types.
  • the communication devices may communicate using various transmission technologies including, but not limited to, code division multiple access (CDMA), global systems for mobile communications (GSM), universal mobile telecommunications system (UMTS), time divisional multiple access (TDMA), frequency division multiple access (FDMA), transmission control protocol-internet protocol (TCP-IP), short messaging service (SMS), multimedia messaging service (MMS), email, instant messaging service (IMS), Bluetooth, IEEE 802.11 and any similar wireless communication technology.
  • CDMA code division multiple access
  • GSM global systems for mobile communications
  • UMTS universal mobile telecommunications system
  • TDMA time divisional multiple access
  • FDMA frequency division multiple access
  • TCP-IP transmission control protocol-internet protocol
  • SMS short messaging service
  • MMS multimedia messaging service
  • email instant messaging service
  • IMS instant messaging service
  • Bluetooth IEEE 802.11 and any similar wireless communication technology.
  • a communications device involves in implementing various embodiments of the present invention may communicate using various media including, but not limited to, radio, infrared, laser, cable connections, and any suitable
  • FIG. 4 a block diagram of a video encoder suitable for carrying out embodiments of the invention is shown. Furthermore, with respect to Figure 5, the operation of the encoder exemplifying embodiments of the invention specifically with respect to the residual macro block encoding process is shown in detail.
  • Figure 4 shows the encoder as comprising a pixel predictor 302, prediction error encoder 303 and prediction error decoder 304.
  • the pixel predictor 302 receives the image 300 to be encoded at both the inter- predictor 306 (which determines the difference between the image and a reference frame 318) and the intra-predictor 308 (which determines the image based only on the current frame or picture).
  • the output of both the inter-predictor and the intra- predictor are passed to the mode selector 310.
  • the mode selector 310 also receives a copy of the image 300.
  • the output of the mode selector is the predicted representation of an image block 312 from either the intra-predictor 306 or intra- predictor 308 which is passed to a first summing device 321.
  • the first summing device may subtract the pixel predictor 302 output from the image 300 to produce a first prediction error signal 320 which is input to the prediction error encoder 303.
  • the pixel predictor 302 further receives from a preliminary reconstructor 339 the combination of the prediction representation of the image block 312 and the output 338 of the prediction error decoder 304.
  • the preliminary reconstructed image 314 may be passed to the intra-predictor 308 and to a filter 316.
  • the filter 316 receiving the preliminary representation may filter the preliminary representation and output a final reconstructed image 340 which may be saved in a reference frame memory 318.
  • the reference frame memory 318 may be connected to the inter-predictor 306 to be used as the reference image against which the image 300 is compared against in inter-prediction operations.
  • the operation of the pixel predictor 302 may be configured to carry out any known pixel prediction algorithm known in the art.
  • the encoder generates images in terms of 16x16 pixel macroblocks which go to form the full image or picture.
  • the pixel predictor 302 and the first summing device 321 output a series of 16x16 pixel residual data macroblocks which may represent the difference between a first macro-block in the image against a similar macro-block in the reference image or picture (in the inter- prediction mode) or an image macro-block itself (in the intra-prediction mode).
  • a selected 8x8 pixel block with respect to the selected block size it would be appreciated that different size selected blocks may be used in other embodiments of the invention.
  • the prediction error encoder 303 comprises a controller 355 which controls a block processor 351 , block tester 353 and block filterer 357.
  • the block processor 351 may receive the selected 16x16 pixel residual macroblock 320.
  • the output of the block processor 351 is connected to the block tester 353.
  • the block tester 353 is further connected to the block filterer 357.
  • the output of the block filterer 357 is passed to the entropy encoder 330 and also to the prediction error decoder 304.
  • the entropy encoder 330 receives the output of the prediction error encoder and may perform a suitable entropy encoding/variable length encoding on the signal to provide error detection and correction capability. Any suitable entropy encoding algorithm may be employed.
  • the prediction error decoder 304 receives the output from the prediction error encoder 303 and performs the opposite processes of the prediction error encoder 303 to produce a decoded prediction error signal 338 which when combined with the prediction representation of the image block 312 at the second summing device 339 produces the preliminary reconstructed image 314.
  • the prediction error decoder may be considered to comprise a block decoder which extracts the block values further described below, a block regenerator processor 361 which regenerates the block from the block decoder 359 values and a macrobjock filter 363 which may filter the regenerated macroblock according to further decoded information and filter parameters.
  • the block processor 351 receives the 16x16 pixel residual macroblock or in other words, a 16x16 pixel residual macroblock is selected as shown in Figure 5, step 501.
  • the controller 355 then initiates a loop control mechanism where the block processor 351 selects an 8x8 pixel residual block or Spatially Varying Transform (SVT) region from the 16x16 pixel residual macroblock.
  • SVT Spatially Varying Transform
  • Figure 8 also shows that the 16x16 pixel macroblock is formed from 4 8x8 pixel blocks which are designated the A block (top left) 803, the B block (top right) 809, the C block (bottom left) 805, the D block (bottom right) 807.
  • the SVT region 8x8 pixel block 811 may be defined with respect to the origin of the 16x16 pixel residual macroblock 801 (located at the top left corner), by a first offset value representing the horizontal offset ⁇ x 903 and a second offset value representing the vertical offset ⁇ y 903.
  • the vertical offset is zero in order to simplify the explanation of the embodiments of the invention.
  • the block processor 351 using the selected 8x8 pixel SVT candidate block 811 then generates a criteria value dependent on the selection.
  • the criteria value is the difference between the motion vector for the underlying 8x8 pixel motion compensation blocks 803, 805, 807, 809 that the spatially varying transform 8x8 candidate block 811 straddles.
  • the candidate SVT block in Figure 8 straddles the A (top left) 8x8 motion compensation block 803 and the B (top right) 8x8 motion compensation block 809.
  • the block processor calculates or determines the following criteria:
  • MVA ⁇ f ,MVB ref MVA ⁇ f ,MVB ref
  • MVA and MVB indicate the motion vector for the blocks A and B respectively and the subscript x, y and ref indicate the x, y component of the motion vector and the associated reference index respectively.
  • the candidate SVT block 811 may straddle more than two horizontally adjacent motion compensation blocks but may, where there is any vertical offset also straddle vertically adjacent motion compensation blocks. Therefore in the example shown in figure 8, the candidate may therefore straddle the boundaries between not only the A/B motion compensation blocks, but also the A/C boundary, the B/D boundary and the C/D boundary.
  • the motion partition of the macrobiock may be any practical shape and size.
  • the motion partition for a 16x16 pixel macroblock may be two 8x16 pixel motion partition blocks, or two 16x8 pixel motion partition blocks, or one 16x16 pixel motion partition block.
  • SVT block sizes 8x8 pixels
  • any practical size block may be used as the SVT block size.
  • the SVT may be 16x4, 4x16, or 4x4 pixels.
  • the difference is calculated for both the x and y motion vector differences, or a combination of the motion vector differences. In some embodiments the difference is calculated between the two motion compensation blocks within which the candidate SVT block 811 lies a majority within.
  • the difference between motion vectors is calculated by averaging or combining the differences in motion vector value for all of the differences.
  • the averaging is biased dependent on the proportion of the SVT block candidate which straddles the motion compensation block boundaries.
  • further criteria value may be generated using different motion compensation block motion vector difference combinations.
  • the criteria may be any one or more of the following: Whether the SVT candidate block straddles over a number of motion compensation block boundaries;
  • the criteria may be determined based on intra predicted blocks and the criteria value for each candidate pixel block comprises at least one of: intra prediction direction of the macroblock; and intra prediction direction associated with the intra predicted blocks that the candidate pixel block straddles.
  • the controller 355 determines whether or not all of the SVT candidate blocks have been tested. This checking operation is shown in Figure 5 by step 507.
  • step 508 the controller 355 shifts the candidate index so that the next SVT candidate block is selected in step 503. This candidate index shifting is shown in Figure 5 by step 508.
  • step 509 the block processor 351 then filters the tested SVT candidate blocks.
  • the filtering is carried out dependent on the criteria values of the tested SVT candidate blocks.
  • the block processor 351 filtering operation only selects SVT candidate blocks where the motion vector difference is less than or equal to a determined threshold and where the motion vector reference is the same. Using the example shown in figure 8 and a predefined threshold of 4, the block processor 351 may select SVT the candidate block 811 where the following conditions are true:
  • MVA and MVB indicate the motion vector for the motion compensation blocks A and B respectively and the subscript x, y and ref indicate the x, y component of the motion vector and the associated reference index respectively.
  • the block processor 351 may select the SVT candidate block if the motion vector difference between the underlying motion compensation blocks is small. As when the motion vector differences between blocks are large, the residua! values have a large motion block edge and transform coding across the boundary is less optimal.
  • the list generated dependent on the criteria further contains the reference index of the SVT candidate block.
  • the block processor 351 may then in some embodiments of the invention generate an ordered list dependent a criteria value.
  • the ordered list criteria may be the same criteria as the filtering criteria, for example the list may be generated as described above from candidate blocks with a small motion vector difference between the underlying motion compensation blocks, and the ordering of the generated list determined by the size of the motion vector difference.
  • the ordered list criteria may be different from the filtering criteria. For example, the list could be ordered based on the frequency of the candidate position used before. In another example, the list could be ordered based on the prediction mode or the prediction direction of the underlying macroblock.
  • step 510 The operation of ordering the list dependent on the same or different criteria values is shown in figure 5 by step 510.
  • the controller 355 may then initiate a further loop control mechanism where the block processor selects an SVT candidate block from the list.
  • step 511 The operation of the selection of one of the list SVT candidate blocks is shown in figure 5 by step 511.
  • the block processor 351 then performs a rate distortion optimisation (RDO) process on the selected list SVT candidate block.
  • RDO rate distortion optimisation
  • the rate distortion optimisation process may be determined using any known process. However, for completeness, an example of rate distortion optimisation process is described hereafter.
  • the block processor 351 then transforms the selected list SVT candidate block (8x8 pixel residual transform block) 811 using any suitable transformation.
  • the discrete cosine transform DCT
  • the block processor 351 performs a suitable quantisation on the selected list SVT candidate block 811. Any suitable quantisation scheme may be employed including but not exclusively vector quantisation. In other embodiments of the invention each coefficient may be quantised independently.
  • the block processor 351 furthermore generates a reconstruction value for the residual pixels in the remainder of the 16x16 pixel residual macroblock not selected as the selected list SVT candidate block.
  • the residual pixel values in the part of the 16x16 residual macroblock which are not selected for transform may either be represented individually or jointly.
  • each one of the pixels in the remaining area may be represented by a fixed value where each value may be selected from the following set of values -1 (11 , 2bit) 0 (0, 1 bit) , 1 (10, 2bit).
  • all the remaining pixel values may be represented as a single value selected from the above set of values.
  • the output of the block processor 351 in terms of the quantised selected list SVT candidate block 811 and the reconstruction value for the remainder of the 16x16 pixel residual macroblock are passed to the block tester 353.
  • the block processor 351 determines the mean square error (or some other error value) between a reconstructed value using the values provided by the block processor 351 and the residual error image input to the prediction error encoder 303.
  • the settings and the error value may be stored in a memory or within the controller 355.
  • the operation of testing performing rate distortion optimisation, i.e. the error between the transform and the quantised selected list SVT candidate transformed block 811 in combination with the reconstruction value for the remainder of 16x16 residual macroblock and the input 16x16 residual macroblock 801 is shown in Figure ⁇ by step 513.
  • the controller 355 determines whether or not all listed selected list SVT candidate blocks have been tested. This operation is shown in Figure 5 by step 515.
  • step 516 If all listed SVT candidate block options have not been tested, the operation selects the next listed SVT candidate block. This is shown in Figure 5 by step 516.
  • the operation then passes back to the step 513 and a further rate distortion optimisation value is generated and tested. If all listed SVT candidate blocks have been tested, the operation passes to the step of selecting the block with the lowest error.
  • the controller may encode the ⁇ x, ⁇ y values from the total list of values.
  • the controller encodes the index of the selected SVT position within the list.
  • This encoding of the index of the selected SVT position within the list may be carried out using for example a verbal length code or by an arithmetic coding dependent on embodiments of the invention. In embodiments of the invention which order the list by criteria this encoding may depend on the order of the list. Thus in some embodiments, for example, the candidate positions at the beginning of the list may have a shorter codeword, than the ones at the end of the list.
  • This information is furthermore passed to the multiplexer 523 to be signalled to the decoder. The operation of selecting or determining the coding options which minimize the cost function is shown in Figure 5 by step 517.
  • the controller 355 furthermore passes the SVT candidate 8x8 pixel transform block 811 information to the block filter 357.
  • the block filter 357 determines an internal filtering for the 16x16 pixei residual macroblock 801 with respect to the boundary of the block 811. With respect to Figure 8, the filtered boundary edges between the block 811 and the non-transformed areas of the residual macroblock 801 are shown.
  • the residual macroblock 801 and the 8x8 pixel transformation block 811 have a boundary 851 which is marked or designated for filtering.
  • the filtering may be a deblocking filtering and may in embodiments of the invention be a deblocking filter similar to the one used for the reconstructed frame.
  • the details of the filter may be further encoded and sent to the multiplexer.
  • the internal residual filtering determination operation is shown in Figure 5 by step 519.
  • the block filter 357 may determine an external 16x16 pixel residua! macroblock filtering process. This may be further described as being the coded block pattern (CBP) generation or derivation operation.
  • CBP coded block pattern
  • One such method for determining whether or not a deblocking filter is to be applied on a specific 8x8 pixel partition of the 16x16 pixel residual macroblock is shown in Figure 8.
  • the 16x16 pixel residual macroblock 801 is shown divided into four-parts or quarters each 8x8 pixels.
  • the first part 803 is the top-left quarter of the residual macroblock, the second part 805 the bottom-left quarter of the residual macroblock, the third part 809 the upper-right quarter of the residual macroblock and the fourth part 807 the bottom-right quarter of the residual macroblock.
  • the example shown in Figure 8 is that the CBP derivation indicates that the external deblocking filter needs to be applied to the external borders of the macroblock for the quarters where the 8x8 pixel transformation block 811 overlaps with the quarter.
  • the 8x8 pixel transformation block overlaps with the first and third parts - the upper left and upper right quarters only and therefore only the macro block boundary edges on the top-left 803 and top-right quarters 809 respectively are indicated as being suitable for filtering and the bottom-left quarter 805 and bottom-right quarter 807 are not indicated as being suitable for filtering.
  • the encoder may only determine one of the internal and external filtering processes. For example, in an embodiment of the invention in the CBP derivation process only the CBP for the four 8x8 blocks inside the macroblock are derived and it is not decided whether to filter the normal inner edges and macroblock boundary edges.
  • the filtering of the normal inner edges and macroblock boundary edges may in embodiments of the invention be decided according to other criteria besides CBP.
  • CBP coded block pattern
  • step 521 The determination of the external 16x16 pixel residual macro block filtering is shown in Figure 5 by step 521.
  • the operations of filter determination may be carried out during the testing of the cost function.
  • the cost of the filtering in terms of the processing and signalling information required to be transferred may also be used as a factor in the cost function determination and as such the configuration of the filtering of the macrobiocks may be determined dependent on the cost function optimisation process.
  • the encoded ⁇ x, ⁇ y, reconstruction values, and any internal or external filter information or coded block pattern values may be passed to a multiplexer which then multiplexes these values together with any reference information to form the output sequence of frame information.
  • the application of the entropy encoding process by the entropy encoder may be implemented following multiplexing of the information. The multiplexing of these values is shown in Figure 5 by step 523.
  • two or more separate areas are selected and encoded in order to further reduce the error.
  • the size of the pixel transform block is other then 8x8 pixels.
  • the 8x8 pixel block is encoded using spatial coding in other words is not transformed.
  • the reconstruction value of the remainder of the residual macroblock may be determined dependent on the quantisation step and signalled separately in the sequence or the picture header.
  • the encoding of the ⁇ x and ⁇ y values are jointly encoded to further exploit any correlation between the ⁇ x and ⁇ y values.
  • the ⁇ x and ⁇ y values are encoded separately.
  • the coding used for ⁇ x and ⁇ y is selected dependent on factors such as the motion vector used for the macroblock or from information derived from neighbouring macroblocks.
  • the index of the optimized SVT are coded using entropy coding methods such as variable length coding tables.
  • the invention as implemented in embodiments of the invention therefore has advantages in that the encoder determines a region of the residua! macroblock that is most optimally selected for transformation and attempts to more optimally exploit the correlation between the predicted image block and the image block. Furthermore, as will be described later, the decoder only requires coefficients for a single 8x8 pixel block transform to be decoded and thus the complexity of the decoder may be reduced while achieving a higher coding efficiency. Furthermore, by performing a filtering action upon the initial candidate set to produce a list of SVT candidate blocks with more optima! encoding indicated by the criteria enables a simplified encoder as not al! of the SVT candidate blocks need to be fully rate distortion optimised.
  • Figure 6 shows a block diagram of a video decoder suitable for employing embodiments of the invention.
  • the decoder shows an entropy decoder 600 which performs an entropy decoding on the received signal.
  • the entropy decoder thus performs the inverse operation to the entropy encoder 330 of the encoder described above.
  • the entropy decoder 600 outputs the results of the entropy decoding to a prediction error decoder 602 and pixel predictor 604.
  • the pixel predictor 604 receives the output of the entropy decoder 600 and a predictor selector 614 within the pixel predictor 604 determines that either an intra- prediction or an inter-prediction operation is to be carried out.
  • the predictor selector furthermore outputs a predicted representation of an image block 616 to a first combiner 613.
  • the predicted representation of the image block 616 is used in conjunction with the reconstructed prediction error signal 612 to generate a preliminary reconstructed image 618.
  • the preliminary reconstructed image 618 may be used in the predictor 614 or may be passed to a filter 620.
  • the filter 620 applies a filtering which outputs a final predicted signal 622.
  • the final predicted signal 622 may be stored in a reference frame memory 624, the reference frame memory 624 further being connected to the predictor 614 for prediction operations.
  • the operation of the prediction error decoder 602 is described in further detail with respect to the flow diagram of Figure 7a and 7b.
  • the prediction error decoder 602 receives the output of the entropy decoder 600.
  • the decoder selects the 16x16 pixel residual macroblock to regenerate.
  • the selection of the 16x16 pixel residual macroblock to be regenerated is shown in step 700.
  • the decoder decodes relevant 8x8 pixel motion compensation partition blocks. These values are passed to the criteria processor 651.
  • the decoder decodes any relevant 8x8 motion compensation partition motion vector values. These motion vector values MVA and MVB and MVC and MVD are passed to the criteria processor 651.
  • the criteria processor 651 having received the relevant 8x8 pixel block motion compensation partition values and the relevant 8x8 pixel block motion vector values from the decoder 601 then initiates a loop control mechanism where the candidate processor 651 selects a 8x8 pixel residual block SVT candidate from the 16x16 pixel residual macroblock.
  • the selection loop is the same as applied in the selection loop of the encoder for testing all possible SVT candidates for a criteria value.
  • a criteria is then generated for the SVT candidate block.
  • the criteria generated matches the criteria generated within the block processor of the encoder.
  • the criteria generator 651 similarly generates a criteria value or values of the difference between the motion vector values in both the x, y directions and the reference index. These values are assigned to the SVT candidate blocks.
  • the criteria processor 651 determines whether all of the candidate 8x8 pixel blocks suitable for SVT have been tested. The operation of checking is shown in Figure 7a by step 704. If not all of the candidate blocks have been tested, the operation passes to step 705 where the criteria processor 651 shifts the candidate index value and the operation passes back the step 703 where the criteria values of a different SVT candidate block are generated.
  • the filter processor 653 receives all of the SVT candidate criteria values and filters the candidate block values, selecting only those which have suitable criteria.
  • the suitability of the criteria is dependent on the embodiment of the invention and is the same as the filtering operation carried out by the block processor of the encoder.
  • the filter processor 653 filters the SVT candidate blocks to generate a list dependent on the criteria where the difference in the x-direction motion vector value is less or equal to 4 in the straddled motion blocks, the difference in the y direction is less than or equal to 4 in the straddled motion blocks and where the reference value of the motion vectors of the straddled motion blocks are the same.
  • the filter processor 553 may then in some embodiments of the invention generate an ordered list dependent a criteria value in a manner similar to the ordering of the list carried out within the encoder.
  • the ordered list criteria may be the same criteria as the filtering criteria, for example the list may be generated as described above from candidate blocks with a small motion vector difference between the underlying motion compensation blocks, and the ordering of the generated list determined by the size of the motion vector difference.
  • the ordered list criteria may be different from the filtering criteria. For example, the list could be ordered based on the frequency of the candidate position used before.
  • step 706A The operation of ordering the list dependent on the same or different criteria values is shown in figure 7 by step 706A.
  • the iist and the candidate index values for the list are passed to the dequantizer 708.
  • the decoder decodes the list value indicator generated by a decoding mechanism to reverse the encoding of the list value index generated within the encoder.
  • the operation of decoding the list value depends on the order of the list (for example the candidate positions at the beginning of the list has a shorter codeword, than the ones at the end of the list).
  • this value is passed to the dequantizer 708.
  • the dequantizer 708 receiving the list value indicator and the list with index references is thus able to select the SVT block or 8x8 pixel block from the list using the index value.
  • the dequantiser 608 dequantises the selected 8x8 pixel transformed block.
  • the dequantisation of the 8x8 pixel transformed block is shown in Figure 7b by step 708.
  • the inverse transformer 606 furthermore performs an inverse transformation on the selected dequantised 8x8 pixel transformed block.
  • the operation of performing the inverse transformation is shown in Figure 7b by step 709.
  • the inverse transformation carried out is dependent upon the transformation carried out within the encoder.
  • the reconstructor 603 furthermore decodes the reconstruction values and sets the remainder of the 16x16 pixel residua! macroblock dependent on the value of the reconstruction value.
  • the block filter 605 receives the combined data from the 8x8 pixel transformed block and the reconstructed remainder of the 16x16 pixel residual macroblock and performs any internal edge filtering in a manner similar to that identified by the encoder.
  • step 711 The operation of internal edge filtering is shown in Figure 7b by step 711. Furthermore, the block filter 605 performs external edge filtering on the reconstructed 16x16 pixel residua! macroblock dependent on the value of the coded block pattern information.
  • the block filter 605 and prediction error decoder 604 thus output the reconstructed 16x16 pixel residual macroblock to be combined with the current reference image output by the intra-prediction operation or inter-prediction operation to create a preliminary reconstructed image 618 as described above.
  • embodiments of the invention operating within a codec within an electronic device, it would be appreciated that the invention as described below may be implemented as part of any video codec. Thus, for example, embodiments of the invention may be implemented in a video codec which may implement video coding over fixed or wired communication paths.
  • user equipment may comprise a video codec such as those described in embodiments of the invention above.
  • user equipment is intended to cover any suitable type of wireless user equipment, such as mobile telephones, portable data processing devices or portable web browsers.
  • elements of a public land mobile network may also comprise video codecs as described above.
  • the various embodiments of the invention may be implemented in hardware or special purpose circuits, software, logic or any combination thereof.
  • some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto.
  • firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto.
  • While various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.
  • the embodiments of this invention may be implemented by computer software executable by a data processor of the mobile device, such as in the processor entity, or by hardware, or by a combination of software and hardware.
  • any blocks of the logic flow as in the Figures may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions.
  • the software may be stored on such physical media as memory chips, or memory blocks implemented within the processor, magnetic media such as hard disk or floppy disks, and optical media such as for example DVD and the data variants thereof, CD.
  • the memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory.
  • the data processors may be of any type suitable to the local technical environment, and may include one or more of genera! purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multi-core processor architecture, as non-limiting examples.
  • Embodiments of the inventions may be practiced in various components such as integrated circuit modules.
  • the design of integrated circuits is by and large a highly automated process.
  • Complex and powerful software tools are available for converting a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.
  • Programs such as those provided by Synopsys, Inc. of Mountain View, California and Cadence Design, of San Jose, California automatically route conductors and locate components on a semiconductor chip using well established rules of design as well as libraries of pre-stored design modules.
  • the resultant design in a standardized electronic format (e.g., Opus, GDSIl, or the like) may be transmitted to a semiconductor fabrication facility or "fab" for fabrication.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Selon l'invention, le codage de transformation n'est pas restreint à l'intérieur d'une limite de bloc normale mais est adapté aux caractéristiques de l'erreur de prévision. Ainsi il est possible d'obtenir une amélioration de l'efficacité de codage par la sélection et le codage de la meilleure partie de l'erreur de prévision en fonction d'une négociation de distorsion de débit. Il est proposé un procédé pour détecter efficacement le bloc le mieux adapté pour le codage de transformation dans un macrobloc à mouvement prévu. Il est également proposé un procédé pour transmettre efficacement le déplacement du bloc de transformation par rapport au macrobloc à mouvement compensé.
PCT/EP2008/065088 2008-11-06 2008-11-06 Algorithme de sélection de blocs rapide pour le codage vidéo utilisant une transformation variable dans l'espace WO2010051846A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2008/065088 WO2010051846A1 (fr) 2008-11-06 2008-11-06 Algorithme de sélection de blocs rapide pour le codage vidéo utilisant une transformation variable dans l'espace

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2008/065088 WO2010051846A1 (fr) 2008-11-06 2008-11-06 Algorithme de sélection de blocs rapide pour le codage vidéo utilisant une transformation variable dans l'espace

Publications (1)

Publication Number Publication Date
WO2010051846A1 true WO2010051846A1 (fr) 2010-05-14

Family

ID=40364374

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2008/065088 WO2010051846A1 (fr) 2008-11-06 2008-11-06 Algorithme de sélection de blocs rapide pour le codage vidéo utilisant une transformation variable dans l'espace

Country Status (1)

Country Link
WO (1) WO2010051846A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012532561A (ja) * 2009-07-06 2012-12-13 トムソン ライセンシング 空間変化残差符号化を行う方法および装置
US8964833B2 (en) 2011-07-19 2015-02-24 Qualcomm Incorporated Deblocking of non-square blocks for video coding
US9338476B2 (en) 2011-05-12 2016-05-10 Qualcomm Incorporated Filtering blockiness artifacts for video coding
CN110719477A (zh) * 2018-07-11 2020-01-21 腾讯美国有限责任公司 视频编解码的方法、装置、计算机设备及计算机可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1285538A1 (fr) * 2000-05-10 2003-02-26 Robert Bosch Gmbh Procede pour coder par transformation des sequences d'images animees
US20030156648A1 (en) * 2001-12-17 2003-08-21 Microsoft Corporation Sub-block transform coding of prediction residuals
US20050249291A1 (en) * 2004-05-07 2005-11-10 Stephen Gordon Method and system for generating a transform size syntax element for video decoding
US20070206679A1 (en) * 2004-09-08 2007-09-06 Lim Chong S Motion Image Encoding Method and Motion Image Decoding Method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1285538A1 (fr) * 2000-05-10 2003-02-26 Robert Bosch Gmbh Procede pour coder par transformation des sequences d'images animees
US20030156648A1 (en) * 2001-12-17 2003-08-21 Microsoft Corporation Sub-block transform coding of prediction residuals
US20050249291A1 (en) * 2004-05-07 2005-11-10 Stephen Gordon Method and system for generating a transform size syntax element for video decoding
US20070206679A1 (en) * 2004-09-08 2007-09-06 Lim Chong S Motion Image Encoding Method and Motion Image Decoding Method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
BJONTEGAARD G: "Larger transform for residual signal", VIDEO STANDARDS AND DRAFTS, XX, XX, no. VCEG-Y10, 12 January 2005 (2005-01-12), XP030003439 *
CIXUN ZHANG KEMAL UGUR JANI LAINEMA ET AL: "Video Coding Using Spatially Varying Transform", LECTURE NOTES IN COMPUTER SCIENCE; VOL. 5414; PROCEEDINGS OF THE 3RD PACIFIC RIM SYMPOSIUM ON ADVANCES IN IMAGE AND VIDEO TECHNOLOGY,, vol. 5414, 13 January 2009 (2009-01-13), pages 796 - 806, XP007907612, ISBN: 978-3-540-92956-7 *
WIEN M: "Variable block-size transforms for H.264/AVC", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 13, no. 7, 1 July 2003 (2003-07-01), pages 604 - 613, XP011099253, ISSN: 1051-8215 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012532561A (ja) * 2009-07-06 2012-12-13 トムソン ライセンシング 空間変化残差符号化を行う方法および装置
US9736500B2 (en) 2009-07-06 2017-08-15 Thomson Licensing Methods and apparatus for spatially varying residue coding
US9338476B2 (en) 2011-05-12 2016-05-10 Qualcomm Incorporated Filtering blockiness artifacts for video coding
US8964833B2 (en) 2011-07-19 2015-02-24 Qualcomm Incorporated Deblocking of non-square blocks for video coding
CN110719477A (zh) * 2018-07-11 2020-01-21 腾讯美国有限责任公司 视频编解码的方法、装置、计算机设备及计算机可读存储介质
CN110719477B (zh) * 2018-07-11 2022-03-25 腾讯美国有限责任公司 视频编解码的方法、装置、计算机设备及计算机可读存储介质
CN114900695A (zh) * 2018-07-11 2022-08-12 腾讯美国有限责任公司 视频编解码的方法、装置、计算机设备及计算机可读存储介质

Similar Documents

Publication Publication Date Title
US11368700B2 (en) Apparatus, a method and a computer program for video coding
KR101215682B1 (ko) 공간적으로 가변하는 변환을 이용하는 비디오 코딩 장치, 방법 및 컴퓨터 판독가능 저장 매체
EP2522144B1 (fr) Appareil et procédé pour codage vidéo
AU2020201908A1 (en) Method for video coding and an apparatus
EP4250732B1 (fr) Prédiction de mouvement dans un codage vidéo
US8848801B2 (en) Apparatus, a method and a computer program for video processing
US9280835B2 (en) Method for coding and an apparatus based on a DC prediction value
US9432699B2 (en) Methods, apparatuses and computer programs for video coding
US20120243606A1 (en) Methods, apparatuses and computer programs for video coding
WO2010116268A1 (fr) Procédé et appareil pour coder et décoder une image et une vidéo
WO2010051846A1 (fr) Algorithme de sélection de blocs rapide pour le codage vidéo utilisant une transformation variable dans l'espace

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08875288

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 08875288

Country of ref document: EP

Kind code of ref document: A1