WO2014120373A1 - Content adaptive entropy coding of coded/not-coded data for next generation video - Google Patents

Content adaptive entropy coding of coded/not-coded data for next generation video Download PDF

Info

Publication number
WO2014120373A1
WO2014120373A1 PCT/US2013/077967 US2013077967W WO2014120373A1 WO 2014120373 A1 WO2014120373 A1 WO 2014120373A1 US 2013077967 W US2013077967 W US 2013077967W WO 2014120373 A1 WO2014120373 A1 WO 2014120373A1
Authority
WO
WIPO (PCT)
Prior art keywords
coded
video data
dimensional
coding technique
scan
Prior art date
Application number
PCT/US2013/077967
Other languages
French (fr)
Inventor
Atul Puri
Daniel Socek
Original Assignee
Intel Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intel Corporation filed Critical Intel Corporation
Priority to US14/375,690 priority Critical patent/US9794568B2/en
Priority to EP13873217.7A priority patent/EP2952001A4/en
Priority to KR1020157016934A priority patent/KR20150090178A/en
Priority to CN201380068583.5A priority patent/CN104885471B/en
Publication of WO2014120373A1 publication Critical patent/WO2014120373A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103Selection of coding mode or of prediction mode
    • H04N19/105Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117Filters, e.g. for pre-processing or post-processing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124Quantisation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124Quantisation
    • H04N19/126Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/137Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/14Coding unit complexity, e.g. amount of activity or edge presence estimation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167Position within a video image, e.g. region of interest [ROI]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/1887Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a variable length codeword
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/192Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/31Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/513Processing of motion vectors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/513Processing of motion vectors
    • H04N19/517Processing of motion vectors by encoding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/523Motion estimation or motion compensation with sub-pixel accuracy
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/53Multi-resolution motion estimation; Hierarchical motion estimation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/573Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/577Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96Tree coding, e.g. quad-tree coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • H04N19/147Data rate or code amount at the encoder output according to rate distortion criteria
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/513Processing of motion vectors
    • H04N19/517Processing of motion vectors by encoding
    • H04N19/52Processing of motion vectors by encoding by predictive encoding

Definitions

  • a video encoder compresses video information so that more information can be sent over a given bandwidth.
  • the compressed signal may then be transmitted to a receiver having a decoder that decodes or decompresses the signal prior to display.
  • High Efficient Video Coding is the latest video compression standard, which is being developed by the Joint Collaborative Team on Video Coding (JCT-VC) formed by ISO/IEC Moving Picture Experts Group (MPEG) and ITU-T Video Coding Experts Group (VCEG).
  • JCT-VC Joint Collaborative Team on Video Coding
  • MPEG Moving Picture Experts Group
  • VCEG ITU-T Video Coding Experts Group
  • HEVC is being developed in response to the previous H.264/AVC (Advanced Video Coding) standard not providing enough compression for evolving higher resolution video applications. Similar to previous video coding standards, HEVC includes basic functional modules such as intra/inter prediction, transform, quantization, in-loop filtering, and entropy coding.
  • the ongoing HEVC standard may attempt to improve on limitations of the
  • H.264/AVC standard such as limited choices for allowed prediction partitions and coding partitions, limited allowed multiple references and prediction generation, limited transform block sizes and actual transforms, limited mechanisms for reducing coding artifacts, and inefficient entropy encoding techniques.
  • the ongoing HEVC standard may use iterative approaches to solving such problems.
  • NVM Next Generation Video
  • H.264/AVC was not perceived to be providing high enough compression for evolving higher resolution video applications.
  • FIG. 1 is an illustrative diagram of an example next generation video encoder
  • FIG. 2 is an illustrative diagram of an example next generation video decoder
  • FIG. 3(a) is an illustrative diagram of an example next generation video encoder and subsystems
  • FIG. 3(b) is an illustrative diagram of an example next generation video decoder and subsystems
  • FIG. 4 is an illustrative diagram of an example entropy encoder module
  • FIG. 5 is an illustrative diagram of an example entropy decoder module
  • FIG. 6 is an illustrative diagram of an example entropy encoder module
  • FIG. 7 is an illustrative diagram of an example entropy decoder module
  • FIG. 8 is a flow diagram illustrating an example process
  • FIG 9 illustrates an example bitstream
  • FIG. 10 is a flow diagram illustrating an example process
  • FIG. 11 is an illustrative diagram of an example video coding system and video coding process in operation
  • FIG. 11 is an illustrative diagram of an example video coding system
  • FIG. 12 is a flow diagram illustrating an example encoding process
  • FIG. 13 illustrates an example bitstream
  • FIG. 14 is a flow diagram illustrating an example decoding process
  • FIGS. 15(A), 15(B), and 15(C) provide an illustrative diagram of an example video coding system and video coding process in operation
  • FIG. 16 is an illustrative diagram of an example video coding system
  • FIG. 17 is an illustrative diagram of an example system
  • FIG. 18 illustrates an example device
  • FIG. 19 is an illustrative diagram of an example picture of coded/not-coded inter- blocks
  • FIG. 20 is an illustrative diagram of example coded/not-coded inter-blocks
  • FIG. 21 is an illustrative diagram of an example coded/not-coded data encoder module
  • FIG. 22 is a flow diagram illustrating an example process
  • FIG. 23 is a flow diagram illustrating an example process
  • FIG. 24 is a flow diagram illustrating an example process
  • FIG. 25 is a flow diagram illustrating an example process
  • FIG. 26 is a flow diagram illustrating an example process, all arranged in accordance with at least some implementations of the present disclosure.
  • SoC system-on-a-chip
  • implementation of the techniques and/or arrangements described herein are not restricted to particular architectures and/or computing systems and may be implemented by any architecture and/or computing system for similar purposes.
  • various architectures employing, for example, multiple integrated circuit (IC) chips and/or packages, and/or various computing devices and/or consumer electronic (CE) devices such as set top boxes, smart phones, etc. may implement the techniques and/or arrangements described herein.
  • IC integrated circuit
  • CE consumer electronic
  • claimed subject matter may be practiced without such specific details.
  • some material such as, for example, control structures and full software instruction sequences, may not be shown in detail in order not to obscure the material disclosed herein.
  • a machine-readable medium may include any medium and/or mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing device).
  • a machine-readable medium may include read only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.), and others.
  • references in the specification to "one implementation”, “an implementation”, “an example implementation”, etc., indicate that the implementation described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same implementation. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other implementations whether or not explicitly described herein.
  • NGV video coding may incorporate significant content based adaptivity in the video coding process to achieve higher compression.
  • the H.264/AVC standard may have a variety of limitations and ongoing attempts to improve on the standard, such as, for example, the HEVC standard may use iterative approaches to address such limitations.
  • an NGV system including an encoder and a decoder will be described.
  • the H.264 standard may support two modes of entropy coding.
  • Adaptive VLC Very Length Coding
  • transform coefficients may be coded using Context Adaptive VLC (CAVLC) and all other syntax elements (e.g., overhead data, modes, motion vectors, and so on) may be coded using Exponential Golomb coding.
  • CAVLC Context Adaptive VLC
  • all other syntax elements e.g., overhead data, modes, motion vectors, and so on
  • the CAVLC coded data and the Exponential Golomb coded data may be multiplexed to generate an encoded bitstream.
  • CABAC Context Adaptive Binary Arithmetic Coding
  • the corresponding decoder may also operate in two modes, disassembling and decoding the multiplexed bit stream in the first mode and decoding the CABAC encoded bitstream in the second mode.
  • Such techniques may have limitations.
  • CABAC coding may be efficient but may be complex such that throughput in higher resolution contexts may be limited. Further, by grouping data types together for coding, efficiency and complexity of the entropy coding may not be optimized.
  • entropy coding and decoding of various data such as overhead data, modes, motion vectors, and/or transform coefficients may be a significant contributor to the coding efficiency and complexity of the system.
  • the techniques discussed herein may balance coding efficiency and system complexity.
  • first video data and second video data may be received for entropy encoding at an entropy encoder module.
  • the first video data and the second video data may be different data types (e.g., header data, morphing parameters, synthesizing parameters, or global maps data or motion vectors or intra-prediction partition data or so on, as is discussed further herein).
  • a first entropy encoding technique may be determined for the first video data based on a parameter associated with the first video data such as, for example, a number of compressed bits of the first video data, a predetermined indicator or flag associated with the first video data, a predetermined threshold, or a heuristically determined threshold or the like.
  • the first entropy encoding technique may be chosen from one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique.
  • the first video data may be entropy encoded using the first entropy encoding technique and the second video data may be entropy encoded using the first entropy encoding technique.
  • the second entropy encoding technique may be determined for the second video data based on a parameter as discussed with respect to the first video data.
  • the second entropy encoding technique may be determined from options including an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, an adaptive vector variable length coding, an adaptive 1 -dimensional variable length coding technique, an adaptive 2-dimensional variable length coding technique, or other techniques as discussed herein.
  • the second entropy encoding technique may be predetermined for the data type of the second video data. Entropy encoding the first video data and the second video data may generate first compressed video data and second compressed video data. The first and second compressed video data may be assembled to generate an output bitstream.
  • the output bitstream may be transmitted from the encoder to a decoder, which may disassemble the bitstream to determine the first and second compressed video data.
  • the compressed video data may be entropy decoded to generate entropy decoded video data, which may be further decoded to generate a video frame.
  • the video frame may be transmitted to a display device for presentment to a user.
  • additional video data types may be received for entropy encoding at the entropy encoder module.
  • third, fourth, fifth, sixth, seventh, or more video data may be entropy encoded to generate associated compressed video data, which may be assembled in the output bitstream, transmitted, and subsequently entropy decoded via a decoder.
  • coder may refer to an encoder and/or a decoder.
  • the term “coding” may refer to performing video encoding via an encoder and/or performing video decoding via a decoder.
  • a video encoder and video decoder may both be examples of coders capable of coding video data.
  • codec may refer to any process, program or set of operations, such as, for example, any combination of software, firmware, and/or hardware, that may implement an encoder and/or a decoder.
  • video data may refer to any type of data associated with video coding such as, for example, video frames, image data, encoded bitstream data, or the like.
  • FIG. 1 is an illustrative diagram of an example next generation video encoder 100, arranged in accordance with at least some implementations of the present disclosure.
  • encoder 100 may receive input video 101.
  • Input video 101 may include any suitable input video for encoding such as, for example, input frames of a video sequence.
  • input video 101 may be received via a content pre-analyzer module 102.
  • Content pre-analyzer module 102 may be configured to perform analysis of the content of video frames of input video 101 to determine various types of parameters for improving video coding efficiency and speed performance. For example, content pre-analyzer module 102 may determine horizontal and vertical gradient information (e.g., Rs, Cs), variance, spatial complexity per picture, temporal complexity per picture, scene change detection, motion range estimation, gain detection, prediction distance estimation, number of objects estimation, region boundary detection, spatial complexity map computation, focus estimation, film grain estimation, or the like.
  • the parameters generated by content pre- analyzer module 102 may be used by encoder 100 (e.g., via encode controller 103) and/or quantized and communicated to a decoder.
  • video frames and/or other data may be transmitted from content pre-analyzer module 102 to adaptive picture organizer module 104, which may determine the picture type (e.g., I-, P-, or F/B-picture ) of each video frame and reorder the video frames as needed.
  • adaptive picture organizer module 104 may include a frame portion generator configured to generate frame portions.
  • content pre-analyzer module 102 and adaptive picture organizer module 104 may together be considered a pre-analyzer subsystem of encoder 100.
  • video frames and/or other data may be transmitted from adaptive picture organizer module 104 to prediction partitions generator module 105.
  • prediction partitions generator module 105 may divide a frame or picture into tiles or super-fragments or the like.
  • an additional module e.g., between modules 104 and 105 may be provided for dividing a frame or picture into tiles or super- fragments.
  • Prediction partitions generator module 105 may divide each tile or super- fragment into potential prediction partitionings or partitions.
  • the potential prediction partitionings may be determined using a partitioning technique such as, for example, a k-d tree partitioning technique, a bi-tree partitioning technique, or the like, which may be determined based on the picture type (e.g., I-, P-, or F/B-picture ) of individual video frames, a characteristic of the frame portion being partitioned, or the like.
  • the determined potential prediction partitionings may be partitions for prediction (e.g., inter- or intra-prediction) and may be described as prediction partitions or prediction blocks or the like.
  • a selected prediction partitioning (e.g., prediction partitions) may be determined from the potential prediction partitionings.
  • the selected prediction partitioning may be based on determining, for each potential prediction partitioning, predictions using characteristics and motion based multi-reference predictions or intra-predictions, and determining prediction parameters.
  • a potential prediction error may be determined by differencing original pixels with prediction pixels and the selected prediction partitioning may be the potential prediction partitioning with the minimum prediction error.
  • the selected prediction partitioning may be determined based on a rate distortion optimization including a weighted scoring based on number of bits for coding the partitioning and a prediction error associated with the prediction partitioning.
  • the original pixels of the selected prediction partitioning may be differenced with predicted partitions (e.g., a prediction of the prediction partition of the current frame based on a reference frame or frames and other predictive data such as inter- or intra-prediction data) at differencer 106.
  • predicted partitions e.g., a prediction of the prediction partition of the current frame based on a reference frame or frames and other predictive data such as inter- or intra-prediction data
  • the determination of the predicted partitions will be described further below and may include a decode loop as shown in Fig. 1.
  • Any residuals or residual data (e.g., partition prediction error data) from the differencing may be transmitted to coding partitions generator module 107.
  • coding partitions generator module 107 may be bypassed via switches 107a and 107b. In such examples, only a single level of partitioning may be performed. Such partitioning may be described as prediction partitioning (as discussed) or coding partitioning or both. In various examples, such partitioning may be performed via prediction partitions generator module 105 (as discussed) or, as is discussed further herein, such partitioning may be performed via a k-d tree intra-prediction/coding partitioner module or a bi-tree intra-prediction/coding partitioner module implemented via coding partitions generator module 107.
  • the partition prediction error data may not be significant enough to warrant encoding.
  • coding partitions generator module 107 may determine coding partitions of the prediction partitions. In some examples, coding partitions generator module 107 may not be needed as the partition may be encoded without coding partitioning (e.g., as shown via the bypass path available via switches 107a and 107b).
  • the partition prediction error data (which may subsequently be described as coding partitions in either event) may be transmitted to adaptive transform module 108 in the event the residuals or residual data require encoding.
  • prediction partitions generator module 105 and coding partitions generator module 107 may together be considered a partitioner subsystem of encoder 100.
  • coding partitions generator module 107 may operate on partition prediction error data, original pixel data, residual data, or wavelet data.
  • Coding partitions generator module 107 may generate potential coding partitionings (e.g., coding partitions) of, for example, partition prediction error data using bi-tree and/or k-d tree partitioning techniques or the like.
  • the potential coding partitions may be transformed using adaptive or fixed transforms with various block sizes via adaptive transform module 108 and a selected coding partitioning and selected transforms (e.g., adaptive or fixed) may be determined based on a rate distortion optimization or other basis.
  • the selected coding partitioning and/or the selected transform(s) may be determined based on a predetermined selection method based on coding partitions size or the like.
  • adaptive transform module 108 may include a first portion or component for performing a parametric transform to allow locally optimal transform coding of small to medium size blocks and a second portion or component for performing globally stable, low overhead transform coding using a fixed transform, such as a discrete cosine transform (DCT) or a picture based transform from a variety of transforms, including parametric transforms, or any other configuration as is discussed further herein.
  • a fixed transform such as a discrete cosine transform (DCT) or a picture based transform from a variety of transforms, including parametric transforms, or any other configuration as is discussed further herein.
  • DCT discrete cosine transform
  • PHT Parametric Haar Transform
  • transforms may be performed on 2D blocks of rectangular sizes between about 4x4 pixels and 64x64 pixels, with actual sizes depending on a number of factors such as whether the transformed data is luma or chroma, or inter or intra, or if the determined transform used is PHT or DCT or the like.
  • the resultant transform coefficients may be transmitted to adaptive quantize module 109.
  • Adaptive quantize module 109 may quantize the resultant transform coefficients.
  • any data associated with a parametric transform may be transmitted to either adaptive quantize module 109 (if quantization is desired) or adaptive entropy encoder module 1 10.
  • the quantized coefficients may be scanned and transmitted to adaptive entropy encoder module 110.
  • Adaptive entropy encoder module 1 10 may entropy encode the quantized coefficients and include them in output bitstream 1 11.
  • adaptive transform module 108 and adaptive quantize module 109 may together be considered a transform encoder subsystem of encoder 100.
  • encoder 100 includes a local decode loop.
  • the local decode loop may begin at adaptive inverse quantize module 1 12.
  • Adaptive inverse quantize module 112 may be configured to perform the opposite operation(s) of adaptive quantize module 109 such that an inverse scan may be performed and quantized coefficients may be de-scaled to determine transform coefficients. Such an adaptive quantize operation may be lossy, for example.
  • the transform coefficients may be transmitted to an adaptive inverse transform module 113.
  • Adaptive inverse transform module 1 13 may perform the inverse transform as that performed by adaptive transform module 108, for example, to generate residuals or residual values or partition prediction error data (or original data or wavelet data, as discussed) associated with coding partitions.
  • adaptive inverse quantize module 112 and adaptive inverse transform module 113 may together be considered a transform decoder subsystem of encoder 100.
  • the partition prediction error data may be transmitted to optional coding partitions assembler 1 14.
  • Coding partitions assembler 1 14 may assemble coding partitions into decoded prediction partitions as needed (as shown, in some examples, coding partitions assembler 1 14 may be skipped via switches 114a and 114b such that decoded prediction partitions may have been generated at adaptive inverse transform module 113) to generate prediction partitions of prediction error data or decoded residual prediction partitions or the like.
  • the decoded residual prediction partitions may be added to predicted partitions (e.g., prediction pixel data) at adder 115 to generate reconstructed prediction partitions.
  • the reconstructed prediction partitions may be transmitted to prediction partitions assembler 1 16.
  • Prediction partitions assembler 1 16 may assemble the reconstructed prediction partitions to generate reconstructed tiles or super-fragments.
  • coding partitions assembler module 114 and prediction partitions assembler module 1 16 may together be considered an un-partitioner subsystem of encoder 100.
  • the reconstructed tiles or super-fragments may be transmitted to blockiness analyzer and deblock filtering module 1 17.
  • Blockiness analyzer and deblock filtering module 117 may deblock and dither the reconstructed tiles or super-fragments (or prediction partitions of tiles or super-fragments).
  • the generated deblock and dither filter parameters may be used for the current filter operation and/or coded in output bitstream 11 1 for use by a decoder, for example.
  • the output of blockiness analyzer and deblock filtering module 1 17 may be transmitted to a quality analyzer and quality restoration filtering module 1 18.
  • Quality analyzer and quality restoration filtering module 1 18 may determine QR filtering parameters (e.g., for a QR decomposition) and use the determined parameters for filtering.
  • the QR filtering parameters may also be coded in output bitstream 1 11 for use by a decoder.
  • the output of quality analyzer and quality restoration filtering module 1 18 may be transmitted to decoded picture buffer 1 19.
  • the output of quality analyzer and quality restoration filtering module 118 may be a final reconstructed frame that may be used for prediction for coding other frames (e.g., the final reconstructed frame may be a reference frame or the like).
  • blockiness analyzer and deblock filtering module 117 and quality analyzer and quality restoration filtering module 1 18 may together be considered a filtering subsystem of encoder 100.
  • prediction operations may include inter- and/or intra-prediction.
  • inter-prediction may be performed by one or more modules including morphing analyzer and morphed picture generation module 120, synthesizing analyzer and generation module 121, and characteristics and motion filtering predictor module 123.
  • Morphing analyzer and morphed picture generation module 120 may analyze a current picture to determine parameters for changes in gain, changes in dominant motion, changes in registration, and changes in blur with respect to a reference frame or frames with which itmay beto be coded.
  • the determined morphing parameters may be quantized/de-quantized and used (e.g., by morphing analyzer and morphed picture generation module 120) to generate morphed reference frames that that may be used by motion estimator module 122 for computing motion vectors for efficient motion (and characteristics) compensated prediction of a current frame.
  • Synthesizing analyzer and generation module 121 may generate super resolution (SR) pictures and projected interpolation (PI) pictures or the like for motion for determining motion vectors for efficient motion compensated prediction in these frames.
  • Motion estimator module 122 may generate motion vector data based on morphed reference frame(s) and/or super resolution (SR) pictures and projected interpolation (PI) pictures along with the current frame.
  • motion estimator module 122 may be considered an inter-prediction module.
  • the motion vector data may be used for inter-prediction. If inter-prediction is applied, characteristics and motion compensated filtering predictor module 123 may apply motion compensation as part of the local decode loop as discussed.
  • Intra-prediction may be performed by intra-directional prediction analyzer and prediction generation module 124.
  • Intra-directional prediction analyzer and prediction generation module 124 may be configured to perform spatial directional prediction and may use decoded neighboring partitions. In some examples, both the determination of direction and generation of prediction may be performed by intra-directional prediction analyzer and prediction generation module 124. In some examples, intra-directional prediction analyzer and prediction generation module 124 may be considered an intra- prediction module.
  • prediction modes and reference types analyzer module 125 may allow for selection of prediction modes from among, “skip”, “auto”, “inter”, “split”, “multi”, and “intra”, for each prediction partition of a tile (or super- fragment), all of which may apply to P- and F/B-pictures. In addition to prediction modes, it also allows for selection of reference types that can be different depending on "inter” or “multi” mode, as well as for P- and F/B-pictures.
  • the prediction signal at the output of prediction modes and reference types analyzer module 125 may be filtered by prediction analyzer and prediction fusion filtering module 126.
  • Prediction analyzer and prediction fusion filtering module 126 may determine parameters (e.g., filtering coefficients, frequency, overhead) to use for filtering and may perform the filtering.
  • filtering the prediction signal may fuse different types of signals representing different modes (e.g., intra, inter, multi, split, skip, and auto).
  • intra-prediction signals may be different than all other types of inter-prediction signal(s) such that proper filtering may greatly enhance coding efficiency.
  • the filtering parameters may be encoded in output bitstream 11 1 for use by a decoder.
  • the filtered prediction signal may provide the second input (e.g., prediction partition(s)) to differencer 106, as discussed above, that may determine the prediction difference signal (e.g., partition prediction error) for coding discussed earlier. Further, the same filtered prediction signal may provide the second input to adder 115, also as discussed above. As discussed, output bitstream 1 1 1 may provide an efficiently encoded bitstream for use by a decoder for the presentment of video.
  • FIG. 1 illustrates example control signals associated with operation of video encoder 100, where the following abbreviations may represent the associated information: scnchg Scene change information
  • qs quantizer information set (includes Quantizer parameter (Qp), Quantizer matrix (QM) choice)
  • the various signals and data items that may need to be sent to the decoder ie, pgst, ptyp, prp, pptn, cptn, modes, reftype, ethp, xmtyp, xmdir, xmmod, idir, mv, qs, mop, syp, ddi, qri, api, fii, quant coefficients and others may then be entropy encoded by adaptive entropy encoder 1 10 that may include different entropy coders collectively referred to as an entropy encoder subsystem. While these control signals are illustrated as being associated with specific example functional modules of encoder 100 in Fig.
  • control signals may include a different distribution of control signals among the functional modules of encoder 300.
  • the present disclosure is not limited in this regard and, in various examples, implementation of the control signals herein may include the undertaking of only a subset of the specific example control signals shown, additional control signals, and/or in a different arrangement than illustrated.
  • FIG. 2 is an illustrative diagram of an example next generation video decoder 200, arranged in accordance with at least some implementations of the present disclosure.
  • decoder 200 may receive an input bitstream 201.
  • input bitstream 201 may be encoded via encoder 100 and/or via the encoding techniques discussed herein.
  • input bitstream 201 may be received by an adaptive entropy decoder module 202.
  • Adaptive entropy decoder module 202 may decode the various types of encoded data (e.g., overhead, motion vectors, transform coefficients, etc.).
  • adaptive entropy decoder 202 may use a variable length decoding technique.
  • adaptive entropy decoder 202 may perform the inverse operation(s) of adaptive entropy encoder module 1 10 discussed above.
  • the decoded data may be transmitted to adaptive inverse quantize module 203.
  • Adaptive inverse quantize module 203 may be configured to inverse scan and de-scale quantized coefficients to determine transform coefficients. Such an adaptive quantize operation may be lossy, for example.
  • adaptive inverse quantize module 203 may be configured to perform the opposite operation of adaptive quantize module 109 (e.g., substantially the same operations as adaptive inverse quantize module 112).
  • the transform coefficients (and, in some examples, transform data for use in a parametric transform) may be transmitted to an adaptive inverse transform module 204.
  • Adaptive inverse transform module 204 may perform an inverse transform on the transform coefficients to generate residuals or residual values or partition prediction error data (or original data or wavelet data) associated with coding partitions.
  • adaptive inverse transform module 204 may be configured to perform the opposite operation of adaptive transform module 108 (e.g., substantially the same operations as adaptive inverse transform module 113).
  • adaptive inverse transform module 204 may perform an inverse transform based on other previously decoded data, such as, for example, decoded neighboring partitions.
  • adaptive inverse quantize module 203 and adaptive inverse transform module 204 may together be considered a transform decoder subsystem of decoder 200.
  • the residuals or residual values or partition prediction error data may be transmitted to coding partitions assembler 205.
  • Coding partitions assembler 205 may assemble coding partitions into decoded prediction partitions as needed (as shown, in some examples, coding partitions assembler 205 may be skipped via switches 205a and 205b such that decoded prediction partitions may have been generated at adaptive inverse transform module 204).
  • the decoded prediction partitions of prediction error data (e.g., prediction partition residuals) may be added to predicted partitions (e.g., prediction pixel data) at adder 206 to generate reconstructed prediction partitions.
  • the reconstructed prediction partitions may be transmitted to prediction partitions assembler 207.
  • Prediction partitions assembler 207 may assemble the reconstructed prediction partitions to generate reconstructed tiles or super-fragments.
  • coding partitions assembler module 205 and prediction partitions assembler module 207 may together be considered an un-partitioner subsystem of decoder 200.
  • the reconstructed tiles or super-fragments may be transmitted to deblock filtering module 208.
  • Deblock filtering module 208 may deblock and dither the reconstructed tiles or super- fragments (or prediction partitions of tiles or super- fragments).
  • the generated deblock and dither filter parameters may be determined from input bitstream 201, for example.
  • the output of deblock filtering module 208 may be transmitted to a quality restoration filtering module 209.
  • Quality restoration filtering module 209 may apply quality filtering based on QR parameters, which may be determined from input bitstream 201, for example. As shown in FIG. 2, the output of quality restoration filtering module 209 may be transmitted to decoded picture buffer 210.
  • the output of quality restoration filtering module 209 may be a final reconstructed frame that may be used for prediction for coding other frames (e.g., the final reconstructed frame may be a reference frame or the like).
  • deblock filtering module 208 and quality restoration filtering module 209 may together be considered a filtering subsystem of decoder 200.
  • compensation due to prediction operations may include inter- and/or intra-prediction compensation.
  • inter-prediction compensation may be performed by one or more modules including morphing generation module 211, synthesizing generation module 212, and characteristics and motion compensated filtering predictor module 213.
  • Morphing generation module 211 may use de-quantized morphing parameters (e.g., determined from input bitstream 201) to generate morphed reference frames.
  • Synthesizing generation module 212 may generate super resolution (SR) pictures and projected interpolation (PI) pictures or the like based on parameters determined from input bitstream 201. If inter-prediction is applied, characteristics and motion compensated filtering predictor module 213 may apply motion compensation based on the received frames and motion vector data or the like in input bitstream 201.
  • SR super resolution
  • PI projected interpolation
  • Intra-prediction compensation may be performed by intra-directional prediction generation module 214.
  • Intra-directional prediction generation module 214 may be configured to perform spatial directional prediction and may use decoded neighboring partitions according to intra-prediction data in input bitstream 201.
  • prediction modes selector module 215 may determine a prediction mode selection from among, “skip”, “auto”, “inter”, “multi”, and “intra”, for each prediction partition of a tile, all of which may apply to P- and F/B-pictures, based on mode selection data in input bitstream 201. In addition to prediction modes, it also allows for selection of reference types that can be different depending on "inter” or “multi” mode, as well as for P- and F/B-pictures.
  • the prediction signal at the output of prediction modes selector module 215 may be filtered by prediction fusion filtering module 216. Prediction fusion filtering module 216 may perform filtering based on parameters (e.g., filtering coefficients, frequency, overhead) determined via input bitstream 201.
  • filtering the prediction signal may fuse different types of signals representing different modes (e.g., intra, inter, multi, skip, and auto).
  • intra-prediction signals may be different than all other types of inter-prediction signal(s) such that proper filtering may greatly enhance coding efficiency.
  • the filtered prediction signal may provide the second input (e.g., prediction partition(s)) to differencer 206, as discussed above.
  • the output of quality restoration filtering module 209 may be a final reconstructed frame.
  • Final reconstructed frames may be transmitted to an adaptive picture re-organizer 217, which may re-order or re-organize frames as needed based on ordering parameters in input bitstream 201.
  • Re-ordered frames may be transmitted to content post- restorer module 218.
  • Content post-restorer module 218 may be an optional module configured to perform further improvement of perceptual quality of the decoded video. The improvement processing may be performed in response to quality improvement parameters in input bitstream 201 or it may be performed as standalone operation.
  • content post-restorer module 218 may apply parameters to improve quality such as, for example, an estimation of film grain noise or residual blockiness reduction (e.g., even after the deblocking operations discussed with respect to deblock filtering module 208).
  • decoder 200 may provide display video 219, which may be configured for display via a display device (not shown).
  • FIG. 2 illustrates example control signals associated with operation of video decoder 200, where the indicated abbreviations may represent similar information as discussed with respect to FIG. 1 above. While these control signals are illustrated as being associated with specific example functional modules of decoder 200 in Fig. 4, other implementations may include a different distribution of control signals among the functional modules of encoder 100. The present disclosure is not limited in this regard and, in various examples, implementation of the control signals herein may include the undertaking of only a subset of the specific example control signals shown, additional control signals, and/or in a different arrangement than illustrated.
  • FIGS. 1 through 2 illustrate particular encoding and decoding modules
  • various other coding modules or components not depicted may also be utilized in accordance with the present disclosure.
  • the present disclosure is not limited to the particular components illustrated in FIGS. 1 and 2 and/or to the manner in which the various components are arranged.
  • Various components of the systems described herein may be implemented in software, firmware, and/or hardware and/or any combination thereof.
  • various components of encoder 100 and/or decoder 200 may be provided, at least in part, by hardware of a computing System-on-a-Chip (SoC) such as may be found in a computing system such as, for example, a mobile phone.
  • SoC computing System-on-a-Chip
  • encoder 100 may be associated with and/or provided by a content provider system including, for example, a video content server system, and that output bitstream 1 11 may be transmitted or conveyed to decoders such as, for example, decoder 200 by various communications components and/or systems such as transceivers, antennae, network systems, and the like not depicted in FIGS. 1 and 2.
  • a content provider system including, for example, a video content server system
  • output bitstream 1 11 may be transmitted or conveyed to decoders such as, for example, decoder 200 by various communications components and/or systems such as transceivers, antennae, network systems, and the like not depicted in FIGS. 1 and 2.
  • decoder 200 may be associated with a client system such as a computing device (e.g., a desktop computer, laptop computer, tablet computer, convertible laptop, mobile phone, or the like) that is remote to encoder 100 and that receives input bitstream 201 via various communications components and/or systems such as transceivers, antennae, network systems, and the like not depicted in FIGS. 1 and 2.
  • a computing device e.g., a desktop computer, laptop computer, tablet computer, convertible laptop, mobile phone, or the like
  • various communications components and/or systems such as transceivers, antennae, network systems, and the like not depicted in FIGS. 1 and 2.
  • FIG. 3(a) is an illustrative diagram of an example next generation video encoder 300a, arranged in accordance with at least some implementations of the present disclosure.
  • FIG. 3(a) presents a similar encoder to that shown in FIGS. 1(a) and 1(b), and similar elements will not be repeated for the sake of brevity. As shown in FIG.
  • encoder 300a may include preanalyzer subsystem 310a, partitioner subsystem 320a, prediction encoding subsystem 330a, transform encoder subsystem 340a, filtering encoding subsystem 350a, entropy encoder system 360a, transform decoder subsystem 370a, and/or unpartioner subsystem 380a.
  • Preanalyzer subsystem 310a may include content pre-analyzer module 102 and/or adaptive picture organizer module 104.
  • Partitioner subsystem 320a may include prediction partitions generator module 105, and/or coding partitions generator 107.
  • Prediction encoding subsystem 330a may include motion estimator module 122, characteristics and motion compensated filtering predictor module 123, and/or intra- directional prediction analyzer and prediction generation module 124.
  • Transform encoder subsystem 340a may include adaptive transform module 108 and/or adaptive quantize module 109.
  • Filtering encoding subsystem 350a may include blockiness analyzer and deblock filtering module 117, quality analyzer and quality restoration filtering module 118, motion estimator module 122, characteristics and motion compensated filtering predictor module 123, and/or prediction analyzer and prediction fusion filtering module 126.
  • Entropy coding subsystem 360a may include adaptive entropy encoder module 1 10.
  • Transform decoder subsystem 370a may include adaptive inverse quantize module 112 and/or adaptive inverse transform module 1 13.
  • Unpartioner subsystem 380a may include coding partitions assembler 1 14 and/or prediction partitions assembler 1 16.
  • Partitioner subsystem 320a of encoder 300a may include two partitioning subsystems: prediction partitions generator module 105 that may perform analysis and partitioning for prediction, and coding partitions generator module 107 that may perform analysis and partitioning for coding.
  • Another partitioning method may include adaptive picture organizer 104 which may segment pictures into regions or slices may also be optionally considered as being part of this partitioner.
  • Prediction encoder subsystem 330a of encoder 300a may include motion estimator 122 and characteristics and motion compensated filtering predictor 123 that may perform analysis and prediction of "inter" signal, and intra-directional prediction analyzer and prediction generation module 124 that may perform analysis and prediction of "intra" signal.
  • Motion estimator 122 and characteristics and motion compensated filtering predictor 123 may allow for increasing predictability by first compensating for other sources of differences (such as gain, global motion, registration), followed by actual motion compensation. They may also allow for use of data modeling to create synthesized frames (super resolution, and projection) that may allow better predictions, followed by use of actual motion compensation in such frames.
  • Transform encoder subsystem 340a of encoder 300a may perform analysis to select the type and size of transform and may include two major types of components.
  • the first type of component may allow for using parametric transform to allow locally optimal transform coding of small to medium size blocks; such coding however may require some overhead.
  • the second type of component may allow globally stable, low overhead coding using a generic/fixed transform such as the DCT, or a picture based transform from a choice of small number of transforms including parametric transforms.
  • a generic/fixed transform such as the DCT
  • a picture based transform from a choice of small number of transforms including parametric transforms.
  • PHT Parametric Haar Transform
  • Transforms may be performed on 2D blocks of rectangular sizes between 4x4 and 64x64, with actual sizes that may depend on a number of factors such as if the transformed data is luma or chroma, inter or intra, and if the transform used is PHT or DCT.
  • the resulting transform coefficients may be quantized, scanned and entropy coded.
  • Entropy encoder subsystem 360a of encoder 300a may include a number of efficient but low complexity components each with the goal of efficiently coding a specific type of data (various types of overhead, motion vectors, or transform coefficients).
  • Components of this subsystem may belong to a generic class of low complexity variable length coding techniques, however, for efficient coding, each component may be custom optimized for highest efficiency. For instance, a custom solution may be designed for coding of
  • Filtering encoder subsystem 350a of encoder 300a may perform analysis of parameters as well as multiple filtering of the reconstructed pictures based on these parameters, and may include several subsystems. For example, a first subsystem, blockiness analyzer and deblock filtering module 117 may deblock and dither to reduce or mask any potential block coding artifacts. A second example subsystem, quality analyzer and quality restoration filtering module 1 18, may perform general quality restoration to reduce the artifacts due to quantization operation in any video coding. A third example subsystem, which may include motion estimator 122 and characteristics and motion compensated filtering predictor module 123, may improve results from motion
  • a fourth example subsystem, prediction fusion analyzer and filter generation module 126 may allow adaptive filtering of the prediction signal (which may reduce spurious artifacts in prediction, often from intra prediction) thereby reducing the prediction error which needs to be coded.
  • Encode controller module 103 of encoder 300a may be responsible for overall video quality under the constraints of given resources and desired encoding speed. For instance, in full RDO (Rate Distortion Optimization) based coding without using any shortcuts, the encoding speed for software encoding may be simply a consequence of computing resources (speed of processor, number of processors, hyperthreading, DDR3 memory etc.) availability. In such case, encode controller module 103 may be input every single combination of prediction partitions and coding partitions and by actual encoding, and the bitrate may be calculated along with reconstructed error for each case and, based on lagrangian optimization equations, the best set of prediction and coding partitions may be sent for each tile of each frame being coded.
  • RDO Rate Distortion Optimization
  • the full RDO based mode may result in best compression efficiency and may also be the slowest encoding mode.
  • content analysis parameters from content preanalyzer module 102 and using them to make RDO simplification (not test all possible cases) or only pass a certain percentage of the blocks through full RDO, quality versus speed tradeoffs may be made allowing speedier encoding.
  • VBR variable bitrate
  • Encode controller module 103 may also include a rate controller that can be invoked in case of constant bitrate (CBR) controlled coding.
  • preanalyzer subsystem 310a of encoder 300a may perform analysis of content to compute various types of parameters useful for improving video coding efficiency and speed performance. For instance, it may compute horizontal and vertical gradient information (Rs, Cs), variance, spatial complexity per picture, temporal complexity per picture, scene change detection, motion range estimation, gain detection, prediction distance estimation, number of objects estimation, region boundary detection, spatial complexity map computation, focus estimation, film grain estimation etc.
  • the parameters generated by preanalyzer subsystem 310a may either be consumed by the encoder or be quantized and communicated to decoder 200. While subsystems 310a through 380a are illustrated as being associated with specific example functional modules of encoder 300a in Fig.
  • implementations of encoder 300a herein may include a different distribution of the functional modules of encoder 300a among subsystems 310a through 380a.
  • the present disclosure is not limited in this regard and, in various examples, implementation of the example subsystems 310a through 380a herein may include the undertaking of only a subset of the specific example functional modules of encoder 300a shown, additional functional modules, and/or in a different arrangement than illustrated.
  • FIG. 3(b) is an illustrative diagram of an example next generation video decoder 300b, arranged in accordance with at least some implementations of the present disclosure.
  • decoder 300b may include prediction decoder subsystem 330b, filtering decoder subsystem 350b, entropy decoder subsystem 360b, transform decoder subsystem 370b, unpartitioner_2 subsystem 380b, unpartitioner_l subsystem 351b, filtering decoder subsystem 350b, and/or postrestorer subsystem 390b.
  • Prediction decoder subsystem 330b may include characteristics and motion compensated filtering predictor module 213 and/or intra-directional prediction generation module 214.
  • Filtering decoder subsystem 350b may include deblock filtering module 208, quality restoration filtering module 209, characteristics and motion compensated filtering predictor module 213, and/or prediction fusion filtering module 216.
  • Entropy decoder subsystem 360b may include adaptive entropy decoder module 202.
  • Transform decoder subsystem 370b may include adaptive inverse quantize module 203 and/or adaptive inverse transform module 204.
  • Unpartitioner_2 subsystem 380b may include coding partitions assembler 205.
  • Unpartitioner_l subsystem 351b may include prediction partitions assembler 207.
  • Postrestorer subsystem 790 may include content post restorer module 218 and/or adaptive picture re-organizer 217.
  • Entropy decoding subsystem 360b of decoder 300b may perform the inverse operation of the entropy encoder subsystem 360a of encoder 300a, i.e., it may decode various data (types of overhead, motion vectors, transform coefficients) encoded by entropy encoder subsystem 360a using a class of techniques loosely referred to as variable length decoding.
  • various types of data to be decoded may include "Coded/Not Coded" data, "Modes and Ref Types" data, "Motion Vector” data, "Prediction and Coding Partitions" data, and "Transform Coefficient" data.
  • Transform decoder subsystem 370b of decoder 300b may perform inverse operation to that of transform encoder subsystem 340a of encoder 300a.
  • Transform decoder subsystem 370b may include two types of components.
  • the first type of example component may support use of the parametric inverse PHT transform of small to medium block sizes, while the other type of example component may support inverse DCT transform for all block sizes.
  • the PHT transform used for a block may depend on analysis of decoded data of the neighboring blocks.
  • Output bitstream 1 11 and/or input bitstream 201 may carry information about partition/block sizes for PHT transform as well as in which direction of the 2D block to be inverse transformed the PHT may be used (the other direction uses DCT).
  • the partition/ block sizes information may be also retrieved from output bitstream 1 11 and/or input bitstream 20 land used to apply inverse DCT of appropriate size.
  • Unpartitioner subsystem 380b of decoder 300b may perform inverse operation to that of partitioner subsystem 320a of encoder 300a and may include two unpartitioning subsystems, coding partitions assembler module 205 that may perform unpartitioning of coded data and prediction partitions assembler module 207 that may perform
  • adaptive picture organizer module 104 is used at encoder 300a for region segmentation or slices, adaptive picture re-organizer module 217 may be needed at the decoder.
  • Prediction decoder subsystem 330b of decoder 300b may include characteristics and motion compensated filtering predictor module 213 that may perform prediction of "inter" signal and intra-directional prediction generation module 214 that may perform prediction of "intra” signal. Characteristics and motion compensated filtering predictor module 213 may allow for increasing predictability by first compensating for other sources of differences (such as gain, global motion, registration) or creation of synthesized frames (super resolution, and projection), followed by actual motion compensation.
  • characteristics and motion compensated filtering predictor module 213 may allow for increasing predictability by first compensating for other sources of differences (such as gain, global motion, registration) or creation of synthesized frames (super resolution, and projection), followed by actual motion compensation.
  • Filtering decoder subsystem 350b of decoder 300b may perform multiple filtering of the reconstructed pictures based on parameters sent by encoder 300a and may include several subsystems.
  • the first example subsystem, deblock filtering modue 208 may deblock and dither to reduce or mask any potential block coding artifacts.
  • the second example subsystem, quality restoration filtering module 209 may perform general quality restoration to reduce the artifacts due to quantization operation in any video coding.
  • the third example subsystem, characteristics and motion compensated filtering predictor module 213, may improve results from motion compensation by using a filter that may adapt to the motion characteristics (motion speed/degree of blurriness) of the content.
  • the fourth example subsystem, prediction fusion filtering module 216 may allow adaptive filtering of the prediction signal (which may reduce spurious artifacts in prediction, often from intra prediction) thereby reducing the prediction error which may need to be coded.
  • Postrestorer subsystem 390b of decoder 300b is an optional block that may perform further improvement of perceptual quality of decoded video. This processing can be done either in response to quality improvement parameters sent by encoder 100, or it can be standalone decision made at the postrestorer subsystem 390b.
  • specific parameters computed at encoder 100 that can be used to improve quality at postrestorer subsystem 390b may be estimation of film grain noise and residual blockiness at encoder 100 (even after deblocking).
  • the film grain noise if parameters can be computed and sent via output bitstream 11 1 and/or input bitstream 201 to decoder 200, then these parameters may be used to synthesize the film grain noise.
  • postrestorer subsystem 390b may decode these parameters and may use them to optionally perform additional deblocking prior to display.
  • encoder 100 also may have access to scene change, spatial complexity, temporal complexity, motion range, and prediction distance information that may help in quality restoration in postrestorer subsystem 390b.
  • subsystems 330b through 390b are illustrated as being associated with specific example functional modules of decoder 300b in Fig. 3(b), other implementations of decoder 300b herein may include a different distribution of the functional modules of decoder 300b among subsystems 330b through 390b.
  • the present disclosure is not limited in this regard and, in various examples, implementation of the example subsystems 330b through 390b herein may include the undertaking of only a subset of the specific example functional modules of decoder 300b shown, additional functional modules, and/or in a different arrangement than illustrated.
  • FIG. 4 is an illustrative diagram of an example entropy encoder module 1 10, arranged in accordance with at least some implementations of the present disclosure.
  • entropy encoder module 1 10 may include bitstream headers, parameters and maps data encoder module 401, picture partitions, prediction partitions, and coding partitions encoder module 402, coding modes and reference types encoder module 403, coded/not- coded data encoder module 404, transform coefficients encoder module 405, motion vector prediction and differential motion vector encoder module 406, intra-prediction type and direction data encoder module 407, and/or bitstream assembler module 408.
  • each of modules 401 ⁇ 107 may be shortened to encoder module 401, encoder module 404, or the like for the sake of brevity.
  • encoder modules 401-407 may receive video data 411— 417, respectively, via adaptive entropy encoder 110.
  • received video data 41 1-417 may be received from various modules of encoder 100 as discussed herein.
  • encoder modules 401-407 may compress the received video data 41 1-417 to generated compressed video data 421-427.
  • Compressed video data 421-427 may be provided to bitstream assembler 408, which may assemble compressed video data 421 ⁇ 127 to generate output bitstream 11 1.
  • encoder modules 401 ⁇ 107 may each include one or more specialized component(s) for efficiently encoding the type of data associated with received video data 41 1-417.
  • one or more of encoder modules 401-407 may preprocess the received video data 41 1-417 and/or select an entropy coding technique based on a parameter, parameters, or characteristics of the received video data 411— 417 or other system information.
  • encoder module 401 may receive overhead data 41 1, which may include bitstream header data (e.g., sequence and/or picture level bitstream headers), morphing parameters, synthesizing parameters, or global maps data (e.g., quantizer maps of pictures indicating quantizers to be used on a partition basis).
  • bitstream header data e.g., sequence and/or picture level bitstream headers
  • morphing parameters e.g., sequence and/or picture level bitstream headers
  • synthesizing parameters e.g., global maps data (e.g., quantizer maps of pictures indicating quantizers to be used on a partition basis).
  • global maps data e.g., quantizer maps of pictures indicating quantizers to be used on a partition basis.
  • encoder module 401 may implement an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression of video data 411.
  • encoder module 41 1 may determine which technique provides the greatest compression efficiency (e.g., the fewest bits for compressed video data 421) such that the parameter(s) associated with video data 411 may be the number of bits needed for each coding technique or the like. Encoder module 411 may entropy encode video data 411 to generate compressed video data 421 (e.g., compressed overhead data), which may be transmitted to bitstream assembler 408 as shown.
  • the parameter associated with the video data may be a fewest number of attainable bits, most efficient encoding technique, or the like.
  • the parameter associated with the video data may be a predefined or predetermine parameter such that encoding technique is predetermined.
  • the parameter associated with the video data may be based on a characteristic of the video data such that the determined encoding technique may be adaptive to the received video data as is discussed further herein.
  • encoder module 402 may receive partition data 412, which may include picture slices or regions data, intra-prediction partition data, and/or inter-prediction partition and coding partition data.
  • partition data 412 may include picture slices or regions data, intra-prediction partition data, and/or inter-prediction partition and coding partition data.
  • encoder module 412 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for the compression of the intra-prediction partition data and/or inter-prediction partition data portions of video data 412 based on a parameter associated with the intra- prediction partition data and/or inter-prediction partition data (e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter,
  • encoder module 412 may implement adaptive codebook variable length coding for the slices or regions data portions of video data 412 to generate compressed video data 422 (e.g., compressed partition data), which may be transmitted to bitstream assembler 408 as shown.
  • the intra- prediction partition data and/or inter-prediction partition data may include data indicating the partitioning of tiles into partitions, partitions into sub-partitions, or the like.
  • the partitions and/or sub-partitions may include prediction partitions and/or subpartitions.
  • partitions and/or sub-partitions may include coding partitions and/or sub-partitions.
  • encoder module 403 may receive modes and reference types data 413, which may include modes (e.g., intra, split, skip, auto, inter, or multi) data and/or references data for each prediction partition.
  • modes e.g., intra, split, skip, auto, inter, or multi
  • the mode split information may indicate whether a partition is further divided or not. If a partition is further divided, the mode data may further include direction information indicating whether the split is a horizontal split (e.g., hor) or a vertical split (e.g., vert).
  • modes and reference types data 413 may include modes (e.g., intra, split, skip, auto, inter, or multi) data and/or references data for each prediction partition.
  • the mode split information may indicate whether a partition is further divided or not. If a partition is further divided, the mode data may further include direction information indicating whether the split is a horizontal split (e.g., hor) or a vertical split (e.g., vert).
  • encoder module 403 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for separate coding of split/non-split partition information data, separate coding of split/non-split split information data, or prediction reference information data based on a parameter associated with the data (e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter, characteristics of the pertinent portions of video data 413, or the like), and encoder module 403 may implement adaptive variable length coding for joint coding of modes and split information to generate compressed video data 423 (e.g., compressed modes and reference types data), which may be transmitted to bitstream assembler 408 as shown.
  • a parameter associated with the data e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter, characteristics of the pertinent portions of video data 413, or the like
  • encoder module 403 may implement adaptive variable length coding for joint coding of modes and split information to generate compressed video
  • encoder module 404 may receive coded / not-coded data 414, which may include coded/not-coded data as discussed herein.
  • a partition or sub-partition
  • a partition or sub-partition
  • coded/not-coded data may not be needed for partitions having an intra or skip mode.
  • coded/not-coded data may be needed for partitions having an auto, inter, or multi mode. As is discussed further below with respect to FIG.
  • encoder module 404 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for coded/not- coded data based on a parameter associated with the coded/not-coded data (e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter, characteristics of video data 414, or the like) to generate compressed video data 424 (e.g., compressed coded / not-coded data), which may be transmitted to bitstream assembler 408 as shown.
  • a parameter associated with the coded/not-coded data e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter, characteristics of video data 414, or the like
  • compressed video data 424 e.g., compressed coded / not-coded data
  • encoder module 405 may receive transform data 415, which may include transform coefficient data. For example, for blocks or partitions or sub-partitions that are coded (e.g., have one or more nonzero transform coefficients), transform coefficient data may be received for entropy encoding. As is discussed further with respect to FIG. 6, encoder module 405 may implement adaptive vector variable length coding for blocks or partitions or sub-partitions having a block or partition or sub-partition size of 2 in one dimension (e.g., 2xK partitions or Kx2 sized partitions).
  • encoder module 405 may implement adaptive 1 -dimensional variable length coding for blocks or partitions or sub-partitions of size 4x4 and adaptive 2-dimensional variable length coding for all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on).
  • the generated compressed video data 425 (e.g., compressed transform data) may be transmitted to bitstream assembler 408 as shown.
  • the transform coefficient data may result from a forward transform of rectangular (or square or the like) partitions of pixel data or rectangular (or square or the like ) of pixel difference values implemented via adaptive transform module 108 followed by a quantization of the resulting coefficients via adaptive quantize module 109.
  • the transform coefficient data may be scanned to convert it to a 1 -dimensional frequency ordered partition via encoder module 405.
  • Such conversion may be highly adaptive any partition size (e.g., 24 or more or 32 or more partition sizes), different data types (e.g., discrete cosine transform coefficients or hybrid parametric Haar transform coefficients or the like of either intra or inter partitions), and/or different quantizer set characteristics (e.g., various combinations of quantizer parameters and/or matrices).
  • a block or partition or sub-partition may belong to different picture types: I-picture (e.g., intra compensation only), P-picture (e.g., predictive) or F-picture (e.g., functional) and/or may represent different types of signal or data (e.g., luma or chroma or the like), which may be quantized with different quantizer setting.
  • I-picture e.g., intra compensation only
  • P-picture e.g., predictive
  • F-picture e.g., functional
  • signal or data e.g., luma or chroma or the like
  • encoder module 406 may receive motion data 416, which may include motion vector data. As is discussed further with respect to FIG. 6, motion vector prediction may be performed based on video data 416 to generate one or more predicted motion vectors. A predicted motion vector may be differenced with an original motion data of video data 416 to generate a difference motion vector. In some examples, encoder module 416 may implement an adaptive classified variable length coding for the difference motion vector(s) to generate compressed video data 426 (e.g., compressed motion data), which may be transmitted to bitstream assembler 408 as shown.
  • compressed video data 426 e.g., compressed motion data
  • encoder module 407 may receive intra-prediction data 417, which may include intra-prediction type or intra-prediction direction data.
  • intra coding may use prediction, which may use neighboring past decoded partition(s) within the same frame to generate spatial prediction.
  • the predictors may include dc, slope, directional, BTPC, feature matching, or the like.
  • the directional predictor may be adaptive for different partition sizes.
  • specifying a directional predictor may include providing an technique for determining angular prediction pixel partition(s) for coding using causal neighboring decoded partitions and/or specifying a technique for entropy coding spatial prediction directions.
  • such techniques may be performed via encoder module 407. As is discussed further below with respect to FIG.
  • encoder module 407 may implement an adaptive variable length coding technique or an arithmetic coding technique for intra-prediction type or intra-prediction direction data based on a parameter associated with the intra-prediction type or intra-prediction direction data (e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter, characteristics of video data 417, or the like) to generate compressed video data 427 (e.g., compressed intra-prediction data), which may be transmitted to bitstream assembler 408 as shown.
  • a parameter associated with the intra-prediction type or intra-prediction direction data e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter, characteristics of video data 417, or the like
  • compressed video data 427 e.g., compressed intra-prediction data
  • adaptive entropy encoder 1 10 may include bitstream assembler 408.
  • encoder modules 401 ⁇ 107 may provide entropy coded compressed video data 421 ⁇ 127 at different instances in time.
  • one or some of compressed video data 421-427 may be picture based, region or slice based, tile based, prediction partition based, coding partition based, or any combination thereof.
  • bitstream assembler may multiplex (the potentially different) compressed video data 421 ⁇ 127 to create a valid bitstream such as, for example, output bitstream 1 11.
  • the valid bitstream may be a valid next generation video (NGV) coded bitstream, which may following a NGV bitstream syntax specification.
  • NGV next generation video
  • output bitstream 1 11 may be a video only bitstream.
  • output bitstream 1 11 may be multiplexed (e.g., Transport or a Media File Format) with uncoded or coded audio to create a multiplexed audio-visual stream.
  • the bitstream may be used local decode, storage, or transmission to a decoder as discussed herein.
  • FIG. 5 is an illustrative diagram of an example entropy decoder module 202, arranged in accordance with at least some implementations of the present disclosure.
  • entropy decoder module 202 may include bitstream headers, parameters and maps data decoder module 501, picture partitions, prediction partitions, and coding partitions decoder module 502, coding modes and reference types decoder module 503, coded/not- coded data decoder module 504, transform coefficients decoder module 505, motion vector and differential motion vector decoder module 506, intra-prediction and direction data decoder module 507, and/or bitstream disassembler module 508.
  • each of modules 501-507 may be shortened to decoder module 501, decoder module 505, or the like for the sake of brevity.
  • bitstream disassembler module 508 may receive input bitstream 201.
  • input bitstream 201 may be a valid bitstream such as, for example, a valid next generation video (NGV) coded bitstream, which may follow a NGV bitstream syntax specification.
  • NGV next generation video
  • input bitstream 201 may be a video only bitstream.
  • input bitstream 201 may be a multiplexed audio-visual stream as discussed herein.
  • Bitstream disassembler module 508 may disassemble input bitstream 201 to determine compressed video data 51 1-517 as shown. For example, bitstream disassembler module 508 may use a predefined syntax or specification to divide input bitstream 201 into component compressed video data 51 1-517 by data type for decompression via decoder modules 501-507. In some examples, bitstream disassembler module 508 may perform an inverse operation with respect to bitstream assembler module 508.
  • decoder modules 501-507 may receive compressed video data 511-517, respectively, and generate video data 521-527.
  • Video data 521-527 may be transmitted to various components of decoder 200 for further decoding as discussed herein.
  • Decoder 200 may thereby generate video frame(s) for presentment to a user via a display device (not shown).
  • decoder modules 501-507 may each perform an inverse operation with respect to encoder modules 401-407.
  • decoder modules 501-507 may each include one or more specialized component(s) for efficiently entropy decoding the type of data associated with compressed video data 511-517.
  • decoder module 501 may receive compressed overhead data 51 1, which may include compressed bitstream header data (e.g., sequence and/or picture level bitstream headers), morphing parameters, synthesizing parameters, or global maps data (e.g., quantizer maps of pictures indicating quantizers to be used on a partition basis).
  • decoder module 511 may implement an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables for decompression of compressed overhead data 511 to generate overhead data 521.
  • decoder module 501 may determine which coding technique to implement based on a parameter or indicator provided via bitstream 201.
  • decoder module 502 may receive compressed partition data 512, which may include compressed picture slices or regions data, intra-prediction partition data, and/or inter-prediction partition data.
  • decoder module 512 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for the decompression of the intra- prediction partition data and/or inter-prediction partition data portions of compressed partition data 512, and decoder module 512 may implement an adaptive codebook variable length coding for the decompression of the slices or regions data portions of compressed partition data 512 to generate partition data 522.
  • the intra-prediction partition data and/or inter-prediction partition data may include data indicating the partitioning of tiles into partitions, partitions into sub-partitions, or the like.
  • the partitions and/or sub-partitions may include prediction partitions and/or subpartitions.
  • partitions and/or sub-partitions may include coding partitions and/or sub-partitions.
  • decoder module 502 may determine which coding technique to implement for the decompression of the intra-prediction partition data and/or inter-prediction partition data portions of compressed video data 512 based on a parameter or indicator provided via bitstream 201.
  • decoder module 503 may receive compressed modes and reference types data 513, which may include compressed modes (e.g., intra, split, skip, auto, inter, or multi) data and/or references data for each prediction partition.
  • the mode split information may indicate whether a partition is further divided or not. If a partition is further divided, the mode data may further include direction information indicating whether the split is a horizontal split (e.g., hor) or a vertical split (e.g., vert).
  • decoder module 503 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for decompression of separate coding of split/non-split partition information data, separate coding of split/non-split split information data, or prediction reference information data, and decoder module 503 may implement an adaptive variable length coding for decompression of joint coding of modes and split information to generate modes and reference types data 523. In some examples, decoder module 503 may determine which coding technique to implement for decompression of separate coding of split/non-split partition information data, separate coding of split/non-split split information data, or prediction reference information data based on a parameter or indicator provided via bitstream 201.
  • decoder module 504 may receive compressed coded / not-coded data 514, which may include coded/not-coded data as discussed herein.
  • a partition or sub-partition
  • a partition or sub-partition
  • coded/not-coded data may not be needed for partitions having an intra or skip mode.
  • coded/not-coded data may be needed for partitions having an auto, inter, or multi mode.
  • decoder module 504 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for decompression of coded/not-coded data to generate coded/not-coded data 524. In some examples, decoder module 504 may determine which coding technique to implement for decompression based on a parameter or indicator provided via bitstream 201.
  • decoder module 505 may receive compressed transform data 515, which may include transform coefficient data.
  • compressed transform data 515 may include transform coefficient data.
  • decoder module 505 may implement an adaptive vector variable length coding for decompression of blocks or partitions or sub-partitions having a block or partition or sub-partition size of 2 in one dimension (e.g., 2xK partitions or Kx2 sized partitions.
  • decoder module 505 may implement an adaptive 1 -dimensional variable length coding for decompression of blocks or partitions or sub-partitions of size 4x4 and adaptive 2-dimensional variable length coding for decompression of all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on).
  • the generated transform data 525 may be transmitted to other module(s) of decoder 200 as shown.
  • decoder module 506 may receive compressed motion data 516, which may include motion vector data.
  • compressed motion data 516 may be decompressed using an adaptive classified variable length coding technique to generate predicted difference motion vectors.
  • the predicted difference motion vectors may be added to motion vector prediction to generate reconstructed motion vectors.
  • the motion vector prediction may be generated based on previously decoded motion vectors of neighboring blocks or partitions using the inverse of the technique implemented via encoder module 506, for example, and/or motion vectors.
  • the reconstructed motion vectors may be transmitted to other module(s) of decoder 200 via motion data 526 as shown.
  • decoder module 507 may receive compressed intra- prediction data 517, which may include intra-prediction type or intra-prediction direction data.
  • intra coding may use prediction, which may use neighboring past decoded partition(s) within the same frame to generate spatial prediction.
  • the predictors may include dc, slope, directional, BTPC, feature matching, or the like.
  • the directional predictor may be adaptive for different partition sizes.
  • specifying a directional predictor may include providing an technique for determining angular prediction pixel partition(s) for coding using causal neighboring decoded partitions and/or specifying a technique for entropy coding spatial prediction directions.
  • decoder module 517 may implement an adaptive variable length coding technique or an arithmetic coding technique for decompression of intra-prediction type or intra-prediction direction data to generate intra-prediction data 527.
  • decoder module 507 may determine which coding technique to implement for decompression based on a parameter or indicator provided via bitstream 201.
  • entropy coding techniques may be implemented on various data types for lossless compression of video data to generate compressed video data at an entropy encoder and for decompression of the compressed video data to generate duplicate video data at an entropy decoder.
  • an adaptive symbol-run variable length coding technique may be implemented.
  • encoder and decoder modules 401, 501, 402, 502, 403, 503, and/or 404, 504 may implement an adaptive symbol-run variable length coding technique on some or all of the video data or compressed video data received.
  • an adaptive symbol-run variable length coding technique may include coding of relative difference in addresses between non-skipped blocks within a frame in video coding that allows one to determine the number of consecutive skipped blocks.
  • encoder module 504 instead of sending one bit (e.g., bit-map) for each block to signal coded/not-coded (e.g., skipped) blocks, encoder module 504, for example, may encode a run of skipped blocks. In such implementations, the longer the run of skipped blocks, the more efficiently the data may be compressed.
  • adaptive symbol-run variable length coding technique may include converting the first video data from bit map data to at least one of an inverted bitmap, a differential bit map, or a gradient predictive bit map before applying adaptive symbol-run variable length coding.
  • adaptive symbol-run variable length coding technique may be used to entropy encode substantially any type of event (e.g., symbol/run combination).
  • symbol/run events may be used to code multi-level (e.g., 0, 1, 2, 3, etc.) or binary (e.g., 0, 1) events.
  • adaptive symbol-run variable length coding technique may be applied a number of consecutive times, breaking a multi-level map into a number of binary sub-maps, with each previous sub-map, excluded from next level's sub-map, or the like.
  • an adaptive proxy variable length coding technique may be implemented.
  • encoder and decoder modules 401, 501, 402, 502, 403, 503, and/or 404, 504 may implement an adaptive proxy variable length coding technique on some or all of the video data or compressed video data received.
  • an adaptive proxy variable length coding technique may include substitution of original fixed length ID blocks (e.g., groups) of bits with variable length codes (e.g., patterns of sequence of bits) such that after the replacement the resulting bitstream may be smaller in size.
  • the process may be repeated (or inversed) resulting in original intended bitstream.
  • the original blocks of bits replaced may be of fixed small sizes (e.g., groups of 2 bits, groups of 3 bits, or groups of 4 bits, or the like).
  • the replacement codes may be of small size and variable length in nature.
  • the adaptive proxy variable length coding discussed herein may be characterized as Short VLCs (e.g., variable length codes).
  • the adaptive proxy variable length coding technique described herein may be adaptive to content by providing multiple replacement options.
  • 1- dimensional blocks/groups of 2 bits may be replaced with 1-3 bit long codes.
  • 1-dimensional blocks/groups (or collections of blocks/groups) of 3 bits with codes may be replaced with 1-5 bit long codes.
  • the adaptive proxy variable length coding technique may exploit statistical redundancy within a bitstream.
  • the adaptive proxy variable length coding technique may provide a compression gain of about 1-1.3.
  • adaptive proxy variable length coding technique may offer the advantage of being amenable to application to short sequence of bits.
  • an adaptive block-of-symbols variable length coding technique may be implemented.
  • encoder and decoder modules 405, 505 may implement an adaptive block-of-symbols variable length coding technique on some or all of the video data or compressed video data received.
  • an adaptive block-of-symbols variable length coding technique may include two sub-coding techniques, as will be discussed further with respect to FIG. 7.
  • the adaptive block-of-symbols variable length coding technique may include an adaptive vector variable length coding technique and an adaptive 1D/2D (1- dimensional/2-dimensional) variable length coding technique.
  • the adaptive block-of-symbols variable length coding technique may be used to encode blocks of closely related symbols such as blocks of transform coefficients as discussed herein.
  • the adaptive vector variable length coding technique of the adaptive block-of-symbols variable length coding technique may encode relatively small 2D blocks or partitions of symbols by use of a joint single codeword such that coding a block of symbols may result in a VLC (variable length coding) codebook.
  • VLC variable length coding
  • the larger the size of the block or partition the larger the size of the codebook.
  • the adaptive vector variable length coding technique may be applied to block or partition sizes having a size 2 in one dimension (e.g., 2xK or Kx2 blocks or partitions). By applying the adaptive vector variable length coding technique to blocks or partitions of such sizes, the size of the VLC codebook may be advantageously limited.
  • the adaptive ID variable length coding technique of the adaptive block-of-symbols variable length coding technique may be used for coding 4x4 transform coefficient block or partition sizes, is essentially same as the CAVLC coder. This coder is primarily used for coding 4x4.
  • the adaptive ID variable length coding technique may be implemented via a content adaptive variable length coding technique with a number of different VLC Tables used based on the context of the coefficient(s) being coded. For example, based on the context of the coefficient(s) being coded encoder and/or decoder modules 505, 505 may switch VLC Tables.
  • the adaptive 2D variable length coding technique of the adaptive block-of-symbols variable length coding technique may utilize two dimensional properties of a block of symbols to switch based on context between a number of different VCL Tables.
  • the adaptive 2D variable length coding technique may be characterized as a CA2DVLC (Content Adaptive 2D Variable Length) coder.
  • the adaptive 2D variable length coding technique may be used to encode all remaining transform coefficient block or petition sizes besides 2xK, Kx2 blocks and 4x4 blocks (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on).
  • FIG. 6 is an illustrative diagram of an example entropy encoder module 1 10, arranged in accordance with at least some implementations of the present disclosure.
  • entropy encoder module 110 may include bitstream headers, parameters and maps data encoder module 401, picture partitions, prediction partitions, and coding partitions encoder module 402, coding modes and reference types encoder module 403, coded/not-coded data encoder module 404, transform coefficients encoder module 405, motion vector and differential motion vector encoder module 406, intra-prediction and direction data encoder module 407, and/or bitstream assembler module 408.
  • encoder module 401 may include adaptive VLC, symbol-run VLC, and/or proxy VLC encoder for bitstream headers, parameters, and maps data module 611 and may receive video data 411.
  • Video data 411 may have a data type such that video data 411 may include bitstream header data (e.g., sequence and/or picture level bitstream headers), morphing parameters, synthesizing parameters, and/or global maps data (e.g., quantizer maps of pictures indicating quantizers to be used on a partition basis).
  • bitstream header data e.g., sequence and/or picture level bitstream headers
  • morphing parameters e.g., sequence and/or picture level bitstream headers
  • synthesizing parameters e.g., synthesizing parameters
  • global maps data e.g., quantizer maps of pictures indicating quantizers to be used on a partition basis.
  • an entropy encoding technique may be determined for video data 411 based on a parameter, parameters or characteristics of video data 41
  • the entropy encoding technique for video data 41 1 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above, or a variable length coding table or tables compression technique.
  • the determined entropy encoding technique may be applied to video data 41 1 to generate compressed video data 421.
  • a variable length coding table or tables compression technique may include a content adaptive variable length coding technique with one or more tables available for coding based on video data 411 , for example.
  • encoder module 404 may determine which technique provides the greatest compression efficiency (e.g., the fewest bits for compressed video data 421) such that the parameter(s) associated with video data 41 1 may be the number of bits needed for each coding technique or the like.
  • encoder module 402 may include adaptive codebook VLC encoder for picture partitions module 621, symbol-run VLC and/or proxy VLC encoder for intra-prediction partitions module 622, and/or symbol-run VLC and/or proxy VLC encoder for inter-prediction partitions and coding partitions module 623. Also as shown, encoder module 402 may receive video data 412. In some examples, video data 412 may include picture slices or regions data 624, intra-prediction partition data 625, and/or inter- prediction and coding partition data 626.
  • picture slices or regions data 624 may be received via adaptive codebook VLC encoder for picture partitions module 621, which may apply adaptive codebook variable length coding to picture slices or regions data 624 to generate compressed picture slices or regions data 627.
  • picture slices or regions data 624 may include region boundaries for pictures, slices, regions, or the like.
  • adaptive codebook variable length coding may include content adaptive variable length coding using an codebook adaptive to the content of picture slices or regions data 624 or other system parameters or the like.
  • intra-prediction partition data 625 may be received via symbol-run VLC and/or proxy VLC encoder for intra-prediction partitions module 622.
  • an entropy encoding technique may be determined for intra-prediction partition data 625 based on a parameter, parameters or characteristics of intra -prediction partition data 625 or other system parameters (e.g., compression efficiency, a characteristic of the data, and so on), as discussed herein.
  • the determined entropy encoding technique may be applied to intra-prediction partition data 625 to generate compressed intra-prediction partition data 628.
  • the entropy encoding technique for intra-prediction partition data 625 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above.
  • intra-prediction partition data 625 may include partitions based on bi-tree partitioning or k-d tree partitioning, or the like.
  • inter-prediction and coding partition data 626 may be received via symbol-run VLC and/or proxy VLC encoder for inter-prediction partitions and coding partitions module 623.
  • an entropy encoding technique may be determined for or inter-prediction and coding partition data 626 based on a parameter, parameters or characteristics of or inter-prediction and coding partition data 626 or other system parameters (e.g., compression efficiency, a characteristic of the data, and so on), as discussed herein.
  • the determined entropy encoding technique may be applied to or inter- prediction and coding partition data 626 to generate compressed or inter-prediction and coding partition data 629.
  • the entropy encoding technique for or inter-prediction and coding partition data 626 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above.
  • or inter-prediction and coding partition data 426 may include inter-partitions and coding tree partitions, or the like.
  • encoder module 403 may include adaptive VLC encoder for joint modes and splits module 631, symbol-run VLC and/or proxy VLC encoder for modes module 632, symbol-run VLC and/or proxy VLC encoder for splits module 633, and/or symbol-run VLC and/or proxy VLC encoder for reference types module 634. Also as shown, encoder module 403 may receive video data 412. In some examples, video data 412 may joint coding of modes and splits data 635, modes information data 636, split/not- split information data 637, and/or prediction reference information data 638.
  • joint coding of modes and splits data 635 may be received via adaptive VLC encoder for joint modes and splits module 631, which may apply adaptive variable length coding to joint coding of modes and splits data 635 to generate compressed joint coding of modes and splits data 639.
  • adaptive variable length coding may include content adaptive variable length coding adaptive to the content of joint coding of modes and splits data 635 or other system parameters or the like.
  • modes and splits data may be coded jointly via adaptive VLC encoder for joint modes and splits module 631.
  • modes and splits data may be coded separately via symbol-run VLC and/or proxy VLC encoder for modes module 632 and symbol-run VLC and/or proxy VLC encoder for splits module 633, as is discussed below.
  • encoder 100 (via, e.g., adaptive entropy encoder 1 10 and/or encode controller 103) to code jointly or separately based on results of a comparison of the coding techniques to determine which technique compresses the data most efficiently.
  • modes information data 636 may be received via symbol-run VLC and/or proxy VLC encoder for modes module 632.
  • an entropy encoding technique may be determined for modes information data 636 based on a parameter, parameters or characteristics of modes information data 636 or other system parameters (e.g., compression efficiency, a characteristic of the data, and so on), as discussed herein.
  • the determined entropy encoding technique may be applied to modes information data 636 to generate compressed modes information data 642.
  • the entropy encoding technique for modes information data 636 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above.
  • split/not-split information data 637 may be received via symbol-run VLC and/or proxy VLC encoder for splits module 633.
  • an entropy encoding technique may be determined for split/not-split information data 637 based on a parameter, parameters or characteristics of split/not-split information data 637 or other system parameters (e.g., compression efficiency, a characteristic of the data, and so on), as discussed herein.
  • the determined entropy encoding technique may be applied to split/not- split information data 637 to generate compressed split/not-split information data 643.
  • the entropy encoding technique for split/not-split information data 637 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above.
  • prediction reference information data 638 may be received via symbol- run VLC and/or proxy VLC encoder for reference types module 634.
  • an entropy encoding technique may be determined for prediction reference information data 638 based on a parameter, parameters or characteristics of prediction reference information data 638 or other system parameters (e.g., compression efficiency, a characteristic of the data, and so on), as discussed herein.
  • the determined entropy encoding technique may be applied to prediction reference information data 638 to generate compressed prediction reference information data 644.
  • the entropy encoding technique for prediction reference information data 638 may be one of an adaptive symbol- run variable length coding technique or an adaptive proxy variable length coding technique, as described above.
  • encoder module 404 may include symbol-run VLC and/or proxy VLC encoder for coded/not-coded data 641 and may receive video data 414.
  • Video data 414 may have a data type such that video data 41 1 may coded/not-coded data.
  • a partition or sub-partition
  • a partition or sub-partition
  • a partition or sub-partition
  • coded/not-coded data may not be needed for partitions having an intra or skip mode.
  • coded/not-coded data may be needed for partitions having an auto, inter, or multi mode.
  • an entropy encoding technique may be determined for video data 414 based on a parameter, parameters or characteristics of video data 414 or other system parameters.
  • the entropy encoding technique for video data 414 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above.
  • the determined entropy encoding technique may be applied to video data 414 to generate compressed video data 424.
  • encoder module 404 may determine which technique provides the greatest compression efficiency as discussed such that the parameter(s) associated with video data 411 may be the number of bits needed for each coding technique or the like.
  • the parameter associated with video data 411 may be based on a characteristic of the video data such that the determined encoding technique may be adaptive to the received video data.
  • encoder module 405 may include an adaptive vector VLC encoder for transform coefficients module 651 and/or an adaptive ID and 2D VLC encoder for transform coefficients module 652.
  • adaptive vector VLC encoder for transform coefficients module 651 may receive transform coefficients data 653, which may include transform coefficient data for blocks or partitions or sub-partitions having a block or partition or sub-partition size of 2 in one dimension (e.g., 2xK partitions or Kx2 sized partitions).
  • An adaptive vector variable length coding technique may be applied to transform coefficients data 653 to generate compressed transform coefficients data 655.
  • the adaptive vector variable length coding technique may include a quad-tree division of the block or partition or the, representing each quadrant generated via the quad-tree division with a single vector codeword that represents all coefficients with a single index value, and entropy coding the codeword using a variable length coding technique or the like.
  • adaptive ID and 2D VLC encoder for transform coefficients module 652 may receive transform coefficient data 654 and may implement an adaptive 1- dimensional variable length coding for blocks or partitions or sub-partitions of size 4x4 and an adaptive 2-dimensional variable length coding for all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on) to generate compressed transform coefficient data 656.
  • transform coefficient data 653, 654 may result from a forward transform of rectangular (or square or the like) partitions of pixel data or rectangular (or square or the like ) of pixel difference values implemented via adaptive transform module 108 followed by a quantization of the resulting coefficients via adaptive quantize module 109.
  • the transform coefficient data may be scanned to convert it to a 1 -dimensional frequency ordered partition via encoder module 405.
  • Such conversion may be highly adaptive any partition size (e.g., 24 or more or 32 or more partition sizes), different data types (e.g., discrete cosine transform coefficients or hybrid parametric Haar transform coefficients or the like of either intra or inter partitions), and/or different quantizer set characteristics (e.g., various combinations of quantizer parameters and/or matrices).
  • a block or partition or sub-partition may belong to different picture types: I-picture (e.g., intra compensation only), P-picture (e.g., predictive) or F-picture (e.g., functional) and/or may represent different types of signal or data (e.g., luma or chroma or the like), which may be quantized with different quantizer setting.
  • I-picture e.g., intra compensation only
  • P-picture e.g., predictive
  • F-picture e.g., functional
  • signal or data e.g., luma or chroma or the like
  • encoder module 406 may include a motion vector predictor module 661, an adaptive VLC encoder for motion vector differences module 662, and/or a differencer 663.
  • encoder module 406 may receive video data 416, which may include motion vector data, via motion vector predictor module 661.
  • Motion vector predictor module 661 may perform motion vector prediction based on video data 416 (e.g., the motion vector data of video data 516) using original motion vector(s) to generate associated predicted motion vector(s).
  • the motion vector prediction may be based on immediate neighbors to left, right, above, or below the motion vector being predicted. In some examples, other spatial neighbors that may share the same or similar characteristics may be used.
  • Differencer 663 may difference the predicted motion vector(s) and the original motion vector(s) to generate difference motion vector(s) for entropy coding.
  • adaptive VLC encoder for motion vector differences module 662 may apply an adaptive variable length coding technique to the difference motion vector(s) to generate compressed video data 526.
  • differential (e.g., difference) motion vectors may have twice the range of original motion vectors.
  • Further l/8 th pel precision motion compensation may expand the range of the difference motion vector by a factor of 8.
  • classification of large space(s) into smaller subintervals and indexing of vectors inside the subinterval may be used.
  • encoder module 407 may include adaptive VLC and/or arithmetic encoder for intra-prediction and directions data module 671 and may receive video data 417.
  • Video data 517 may have a data type such that video data 417 may include intra-prediction type or intra-prediction direction data.
  • an entropy encoding technique may be determined for video data 417 based on a parameter, parameters or characteristics of video data 417 or other system parameters (e.g., compression efficiency or the like) as discussed herein.
  • the entropy encoding technique for video data 417 may be one of an adaptive variable length coding technique or an arithmetic coding technique, as described above.
  • the determined entropy encoding technique may be applied to video data 417 to generate compressed video data 427.
  • the adaptive variable length coding technique may include a content adaptive variable length coding based on the content of video data 417.
  • the arithmetic coding technique may include a content adaptive binary arithmetic coding based on the content of video data 417.
  • video data 417 may support 9 or more prediction directions and a variety of prediction types including planar, Binary Tree Predictive Coding (BTPC), or the like.
  • the output encoder modules 401-407 may be input to bitstream assembler 408, which may output a multiplexed bitstream formatted per the bitstream syntax, as discussed above.
  • FIG. 7 is an illustrative diagram of an example entropy decoder module 202, arranged in accordance with at least some implementations of the present disclosure.
  • entropy decoder module 202 may include headers, parameters and maps data decoder module 501, picture partitions, prediction partitions, and coding partitions decoder module 502, coding modes and reference types decoder module 503, coded/not-coded data decoder module 504, transform coefficients decoder module 505, motion vector and differential motion vector decoder module 506, intra-prediction and direction data decoder module 507, and/or bitstream disassembler module 508.
  • entropy decoder module 202 may perform an inverse technique with respect to entropy encoder module 110 (and the pertinent sub-modules) such that there may be a one-to-one correspondence between encoder modules (and sub-modules) and decoder modules (and sub-modules)
  • bitstream disassembler module 508 may receive input bitstream 201.
  • input bitstream 201 may be a valid bitstream such as, for example, a valid next generation video (NGV) coded bitstream, which may follow a NGV bitstream syntax specification or any valid bitstream as discussed herein.
  • NGV next generation video
  • bitstream disassembler module 508 may disassemble input bitstream 201 to determine compressed video data 51 1-517, which may each have one or more component parts as discussed further below.
  • bitstream disassembler module 508 may use a predefined syntax or specification to divide input bitstream 201 into component compressed video data 51 1-517 by data type for decompression via decoder modules 501— 507.
  • bitstream disassembler module 508 may perform an inverse operation with respect to bitstream assembler module 308.
  • the disassembling of input bitstream 201 may be characterized as a de-multiplexing.
  • decoder module 501 may include adaptive VLC, symbol-run VLC, and/or proxy VLC decoder for headers, parameters, and maps data module 711 and may receive compressed video data 51 1.
  • compressed video data 51 1 may include header data (e.g., sequence and/or picture level bitstream headers), morphing parameters, synthesizing parameters, and/or global maps data entropy encoded using one of an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression technique.
  • adaptive VLC, symbol-run VLC, and/or proxy VLC decoder for headers, parameters, and maps data module 71 1 may determine an entropy decoding technique applicable to compressed video data 51 1 and decode compressed video data 511 using the applicable technique to generate video data 521.
  • the applicable technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
  • decoder module 502 may include adaptive codebook VLC decoder for picture partitions module 721, symbol-run VLC and/or proxy VLC decoder for intra- prediction partitions data module 722, and/or symbol-run VLC and/or proxy VLC decoder for inter-prediction partitions and coding partitions data module 723 and may receive compressed video data 512.
  • compressed picture slices or regions data 724 may be received via adaptive codebook VLC decoder for picture partitions module 721.
  • adaptive codebook VLC decoder for picture partitions module 721 may apply adaptive codebook variable length coding to compressed picture slices or regions data 724 to generate picture slices or regions data 727.
  • adaptive codebook variable length coding may include content adaptive variable length coding using an codebook adaptive to the content of compressed picture slices or regions data 724 or other system parameters or the like.
  • the codebook may be implemented via adaptive codebook VLC decoder for picture partitions module 721.
  • symbol-run VLC and/or proxy VLC decoder for intra-prediction partitions data 722 may receive compressed intra-prediction partition data 725.
  • compressed intra-prediction partition data 725 may include compressed intra- prediction partition data entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described herein.
  • symbol-run VLC and/or proxy VLC decoder for intra prediction partitions data 722 may determine an entropy decoding technique applicable to compressed intra-prediction partition data 725 and entropy decode compressed intra- prediction partition data 725 using the applicable technique to generate intra-prediction partition data 728.
  • the applicable coding technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201
  • symbol-run VLC and/or proxy VLC decoder for inter-prediction partitions and coding partitions data 723 may receive compressed inter-prediction and coding partition data 726.
  • compressed inter-prediction and coding partition data 726 may include compressed inter-prediction and coding partition data entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described herein.
  • symbol-run VLC and/or proxy VLC decoder for inter-prediction partitions and coding partitions data 723 may determine an entropy decoding technique applicable to compressed inter-prediction and coding partition data 726 and entropy decode compressed inter-prediction and coding partition data 726 using the applicable technique to generate inter-prediction and coding partition data 729.
  • the applicable coding technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
  • decoder module 503 may include adaptive VLC decoder for joint modes and splits module 731, symbol-run VLC and/or proxy VLC decoder for modes module 732, symbol-run VLC and/or proxy VLC decoder for splits module 733, and/or symbol- run VLC and/or proxy VLC decoder for reference types module 734 and may receive compressed video data 513.
  • modes and splits data may be coded jointly and, in some examples, modes and splits data may be coded separately.
  • adaptive VLC decoder for joint modes and splits module 731 may decode jointly coded data
  • symbol-run VLC and/or proxy VLC decoder for modes module 732 and symbol-run VLC and/or proxy VLC decoder for splits module 733 may decode separately coded data.
  • whether data is jointly or separately coded may be indicated via input bitstream 201.
  • compressed joint coding of modes and splits data 735 may be received via adaptive VLC decoder for joint modes and splits module 731.
  • adaptive VLC decoder for joint modes and splits module 731 may apply adaptive variable length coding to compressed joint coding of modes and splits data 735 to generate joint coding of modes and splits data 739.
  • adaptive variable length coding may be content adaptive variable length coding adaptive to the content of compressed joint coding of modes and splits data 735 or other system parameters or the like.
  • symbol-run VLC and/or proxy VLC decoder for modes module 732 may receive compressed modes information data 736.
  • compressed modes information data 736 may include compressed modes information data entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described herein.
  • symbol-run VLC and/or proxy VLC decoder for modes module 732 may determine an entropy decoding technique applicable to compressed modes information data 736 and entropy decode compressed modes information data 736 using the applicable technique to generate modes information data 742.
  • the applicable coding technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
  • symbol-run VLC and/or proxy VLC decoder for splits module 733 may receive compressed splits information data 737.
  • compressed splits information data 737 may include compressed splits information data entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described herein.
  • symbol-run VLC and/or proxy VLC decoder for splits module 733 may determine an entropy decoding technique applicable to compressed splits information data 737 and entropy decode compressed splits information data 737 using the applicable technique to generate splits information data 743.
  • the applicable coding technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
  • symbol-run VLC and/or proxy VLC decoder for reference types module 734 may receive compressed reference types information data 738.
  • compressed reference types information data 738 may include compressed reference types information data entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described herein.
  • symbol-run VLC and/or proxy VLC decoder for reference types module 734 may determine an entropy decoding technique applicable to compressed reference types information data 738 and entropy decode compressed reference types information data 738 using the applicable technique to generate reference types information data 744.
  • the applicable coding technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
  • decoder module 504 may include symbol-run VLC and/or proxy VLC decoder for coded/not-coded data module 741 and may receive compressed video data 514.
  • compressed video data 514 may include coded/not-coded, as discussed herein, entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique.
  • symbol-run VLC and/or proxy VLC decoder for coded/not-coded data module 741 may determine an entropy decoding technique applicable to compressed video data 514 and decode compressed video data 514 using the applicable technique to generate video data 524.
  • the applicable technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
  • the applicable technique may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique.
  • decoder module 505 may include adaptive vector VLC decoder for transform coefficients module 751 and/or adaptive ID and 2D VLC decoder for transform coefficients module 752 and may receive compressed video data 515.
  • adaptive vector VLC decoder for transform coefficients module 751 may receive compressed transform coefficients data 753, which may include compressed transform coefficients data for blocks or partitions or sub-partitions having a size of 2 in one dimension (e.g., 2xK partitions or Kx2 sized partitions), as discussed herein.
  • adaptive vector VLC decoder for transform coefficients module 751 may apply an adaptive vector variable length coding technique to entropy decode compressed transform coefficients data 753 to generate transform coefficients data 755.
  • the adaptive vector variable length coding technique may include using a variable length codeword to generate all coefficients of a quad-tree division of the block, which may be generated via merging the quad-tree division.
  • adaptive ID and 2D VLC decoder for transform coefficients module 752 may receive compressed transform coefficients data 754, which may include compressed transform coefficients data for blocks or partitions or sub-partitions of size 4x4 and all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on).
  • compressed transform coefficients data for blocks or partitions or sub-partitions of size 4x4 and all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on).
  • adaptive ID and 2D VLC decoder for transform coefficients module 752 may apply an adaptive 1 -dimensional variable length coding for blocks or partitions or sub-partitions of size 4x4 and an adaptive 2-dimensional variable length coding for all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on) to entropy decode compressed transform coefficients data 754 to generate transform coefficients data 756.
  • an adaptive 1 -dimensional variable length coding for blocks or partitions or sub-partitions of size 4x4
  • an adaptive 2-dimensional variable length coding for all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on)
  • decoder module 506 may include an adaptive VLC decoder for motion vector differences module 762, a motion vector predictor 761 and an adder 763, and may receive compressed video data 516.
  • adaptive VLC decoder for motion vector differences module 762 may decode compressed video data 516 to generate motion vector differences.
  • motion vector predictor 761 may generate prediction motion vectors using previously decoded neighboring motion vectors in analogy to the techniques discussed with respect to motion vector predictor module 661.
  • decoded difference motion vector(s) may be added via adder 763 to prediction motion vector(s) to generate reconstructed motion vector(s), which may be output as a part of video data 526 and further used to perform motion vector prediction for other motion vectors via motion vector predictor module 761.
  • decoder module 507 may include adaptive VLC and/or arithmetic decoder for intra-prediction type and direction module 771 and may receive compressed video data 517.
  • compressed video data 517 may include intra-prediction type and direction data entropy encoded using one of an adaptive VLC technique or an arithmetic coding technique.
  • adaptive VLC and/or arithmetic decoder for intra- prediction type and direction module 771 may determine an entropy decoding technique applicable to compressed video data 517 and decode compressed video data 517 using the applicable technique to generate video data 527.
  • the applicable technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
  • video data 521-527 may be transmitted to various components of decoder 200. Further, as discussed, decoder 200 may use entropy decoded video data 521-527 to generate video frames, which may be output, via display video 219, for presentment or display via a display device for a user.
  • FIG. 8 is a flow diagram illustrating an example process 800, arranged in accordance with at least some implementations of the present disclosure.
  • Process 800 may include one or more operations, functions or actions as illustrated by one or more of operations 802, 804, 806, and/or 808.
  • Process 800 may form at least part of a next generation video coding process.
  • process 800 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or entropy encoder module 1 10 of FIGS. 5 or 7.
  • Process 800 may begin at operation 802, "Obtain First and Second Video Data of Different Types for Entropy Encoding", where first and second (or additional) video data of different types may be obtained or received for entropy coding.
  • first and second (or additional) video data of different types may be obtained or received for entropy coding.
  • two or more of video data 41 1-417 (and/or any sub-components of video data 511-517) may be received.
  • video data 411— 417 may be received via adaptive entropy encoder 110.
  • the first and second (or additional) video data may be of different types such as any of the types or sub-component types as discussed with respect to video data 41 1-417 or elsewhere herein.
  • Process 800 may continue at operation 804, "Determine an Entropy Encoding Technique for the First Video Data", where a first entropy encoding technique may be determined for the first video data.
  • a first entropy encoding technique may be determined for the first video data.
  • one or more of encoder modules 401 ⁇ 107 may determine a coding technique for video data 411— 417 from various coding technique options.
  • encoder module 401 may determine a coding technique for video data 411 from one of an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression technique.
  • encoder module 402 may determine a coding technique for intra-prediction partition data 625 of video data 402 from one of an adaptive symbol-run variable length coding technique and an adaptive proxy variable length coding technique, and so on.
  • a coding technique for intra-prediction partition data 625 of video data 402 from one of an adaptive symbol-run variable length coding technique and an adaptive proxy variable length coding technique, and so on.
  • two ore more types of video data may be received.
  • an entropy encoding technique may be determined for two, three, or more types of video data as described herein.
  • an entropy encoding technique may be determined for video data.
  • a selected coding table associated with the first encoding technique may be determined from two or more available tables. For example, the table selection may be made based on a bit count comparison, a coding efficiency, or the like between the available tables. For example, any of the described entropy encoding techniques discussed herein may have multiple available tables.
  • a table indicator may be generated and provided in bitstream 900 indicating the selected table for the video data.
  • bitstream 900 may include indicators indicating the selected encoding technique and the selected coding table. Further an indicator or indicators associated with the length of the video data may be included into bitstream 900.
  • the length or number of tuples in the video data may be provided and any remainder portion (e.g., if the length is not evenly divided into tuples) may be coded using a bit map technique and provided into the output bitstream.
  • Process 800 may continue at operation 806, "Entropy Encode the First Video Data using the First Entropy Encoding Technique and Entropy Encode the Second Video Data to Generate First and Second Compressed Video Data", where the first video data may be entropy encoded using the first entropy encoding technique to generate first compressed video data and the second video data may be compressed using a second entropy encoding technique to generate first and second compressed video data.
  • Process 800 may continue at operation 808, "Assemble the First and Second Compressed Video Data to Generate an Output Bitstream", where the first compressed video data and the second compressed video data may be assembled to generate an output bitstream.
  • bitstream assembler 508 may assemble or multiplex the first compressed video data and the second compressed video data to generate output bitstream 1 11 as discussed herein.
  • output bitstream 1 11 may be multiplexed with an audio stream (coded or uncoded) to generate a multiplexed audio-visual stream.
  • one or more of the video data may be preprocessed or otherwise manipulated prior to entropy coding. For example, for motion vector data, motion vector prediction may be performed to generate predicted motion vector(s), the predicted motion vector(s) may be differenced with the original motion vector(s) to generate difference motion vector(s), and the difference motion vector(s) may be entropy coded as discussed herein.
  • each of the seven encoder modules 401 ⁇ 107 may be implemented simultaneously to operate on seven types of video data 411— 417.
  • Process 800 may be implemented via adaptive entropy encoder module 110 as discussed herein. Further, process 800 may be repeated either in serial or in parallel on any number of instantiations of video data.
  • video data of different types may be entropy coded using a variety of determined (or predetermined) adaptive entropy coding techniques to generate compressed video data.
  • the compressed video data may be assembled to generate an output bitstream.
  • FIG. 9 illustrates an example bitstream 900, arranged in accordance with at least some implementations of the present disclosure.
  • bitstream 900 may correspond to output bitstream 11 1 as shown in FIGS. 1, 3a, 4, and 6 and/or input bitstream 201 as shown in FIGS. 2, 3b, 5, and 7.
  • bitstream 900 may include a header portion and a data portion.
  • bitstream 900 may include data, indicators, index values, mode selection data, or the like associated with encoding compressed video as discussed herein.
  • bitstream 900 may include indicator data 901, compressed video data 421, compressed video data 422, compressed video data 423, compressed video data 424, compressed video data 425, compressed video data 426, and/or compressed video data 427.
  • the illustrated data may be in any order in bitstream 900 and may be adjacent or separated by any other of a variety of additional data for coding video.
  • bitstream 900 may also include indicators indicating the selected encoding technique and the selected coding table (e.g., in indicator data 901). Further an indicator or indicators associated with the length of the video data may be included into bitstream 900. For example, the length or number of tuples in the video data (e.g.
  • compressed video data 421-427 may include any compressed video data encoded via any technique as discussed herein.
  • indicator data 901 may include header data, mode indicator data, and/or data indicating entropy encoding techniques associated with compressed video data 421 ⁇ 127.
  • indicator data 901 may indicate an entropy coding technique used for compressing video data 421, an entropy coding technique used for compressing portions of video data 422, an entropy coding technique used for compressing portions of video data 423, an entropy coding technique used for compressing video data 424, and/or an entropy coding technique used for compressing video data 427, as discussed herein with respect to FIGS. 4 and 6.
  • bitstream 900 may be generated by an encoder such as, for example, encoder 100 and/or received by a decoder 200 for decoding such that video frames may be presented via a display device.
  • FIG. 10 is a flow diagram illustrating an example process 1000, arranged in accordance with at least some implementations of the present disclosure.
  • Process 1000 may include one or more operations, functions or actions as illustrated by one or more of operations 1002, 1004, 1006, and/or 1008.
  • Process 1000 may form at least part of a next generation video coding process.
  • process 1000 may form at least part of a next generation video decoding process as undertaken by decoder system 200 of FIG. 2.
  • Process 1000 may begin at operation 1002, "Receive Entropy Encoded Bitstream", where an entropy encoded bitstream may be received.
  • a bitstream encoded as discussed herein may be received at a video decoder.
  • bitstream 100 or 900 or the like may be received via decoder 200.
  • Process 1000 may continue at operation 1004, "Disassemble the Entropy Encoded Bitstream to First Compressed Video Data and Second Compressed Video Data", where the received bitstream may be disassembled to determine different types of compressed video data.
  • bitstream 201 may be disassembled via bitstream disassembler 508 to generate compressed video data 51 1-517 (and any sub-component video data) as illustrated in FIG. 7.
  • the disassembled compressed video data may include first compressed video and second compressed video data.
  • Process 1000 may continue at operation 1006, "Entropy Decode the First and Second Compressed Video Data to Generate First Video Data and Second Video Data", where first and second compressed video data may be entropy decoded to generate (decompressed) first and second video data.
  • decoder modules 501-507 may determine a coding technique for compressed video data 511-517 from various coding technique options.
  • decoder module 501 may determine a coding technique for compressed video data 511 from one of an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression technique based on an indicator or indicators provided via the input bitstream.
  • decoder module 502 may determine a coding technique for compressed intra-prediction partition data 725 of video data 512 from one of an adaptive symbol-run variable length coding technique and an adaptive proxy variable length coding technique, and so on.
  • a coding technique for compressed intra-prediction partition data 725 of video data 512 from one of an adaptive symbol-run variable length coding technique and an adaptive proxy variable length coding technique, and so on.
  • two ore more types of video data may be received.
  • an entropy decoding technique may be determined for two, three, or more types of video data as described herein.
  • Process 1000 may continue at operation 1008, "Decode the First and Second Video Data to Generate a Video Frame", where the first and second video data (and any other entropy decoded video data) may be decoded to generate video frame(s).
  • the video frames may be suitable for presentment to a user via a display device, for example.
  • one or more of the entropy decoded video data may be post-processed or otherwise manipulated prior to further decoding. For example, entropy decoded difference motion vector(s) may be added to prediction motion vector(s) to generate reconstructed motion vector(s), which may be output for use in motion vector prediction (e.g., inter-prediction) via decoder 200.
  • entropy decoded difference motion vector(s) may be added to prediction motion vector(s) to generate reconstructed motion vector(s), which may be output for use in motion vector prediction (e.g., inter-prediction) via decoder 200.
  • various components of the systems described herein may be implemented in software, firmware, and/or hardware and/or any combination thereof.
  • various components of encoder 100 or encoder 200 may be provided, at least in part, by hardware of a computing System-on-a-Chip (SoC) such as may be found in a computing system such as, for example, a smart phone.
  • SoC System-on-a-Chip
  • systems described herein may include additional components that have not been depicted in the corresponding figures.
  • the systems discussed herein may include additional components such as bit stream multiplexer or de-multiplexer modules and the like that have not been depicted in the interest of clarity.
  • FIG. 11 is an illustrative diagram of an example video coding system 1610 and video coding process 1 100 in operation, arranged in accordance with at least some
  • process 1 100 may include one or more operations, functions or actions as illustrated by one or more of actions 1 100-11 12.
  • process 1100 will be described herein with reference to example video coding system 1610 including encoder 100 of FIG. 1 and decoder 200 of FIG. 2, as is discussed further herein below with respect to FIG. 16.
  • process 1 100 may be undertaken by a system including both an encoder and decoder or by separate systems with one system employing an encoder (and optionally a decoder) and another system employing a decoder (and optionally an encoder).
  • an encoder may include a local decode loop employing a local decoder as a part of the encoder system.
  • video coding system 1610 may include logic circuitry 1150, the like, and/or combinations thereof.
  • logic circuitry 1 150 may include encoder 100 and may include any modules as discussed with respect to FIG. 1 and/or FIGS. 3 and 5 and decoder 200 and may include any modules as discussed with respect to FIG. 2 and/or FIGS. 4 and 6.
  • video coding system 1610 as shown in FIG. 16, may include one particular set of blocks or actions associated with particular modules, these blocks or actions may be associated with different modules than the particular module illustrated here.
  • process 1100 is directed to encoding and decoding, the concepts and/or operations described may be applied to encoding and/or decoding separately, and, more generally, to video coding.
  • Process 1100 may begin at operation 1 101, "Obtain Video Data of Different Types", where video data of different types may be received for entropy encoding.
  • video data of different types may be received for entropy encoding.
  • two or more types of video data may be received for entropy encoding.
  • two or more of video data 41 1 ⁇ 417 (and/or any sub-components of video data 411— 4 IV) may be received via adaptive entropy encoder 1 10.
  • the first and second (or additional) video data may be of different types such as any of the types or sub-component types as discussed with respect to video data 41 1-417 or elsewhere herein.
  • Process 1100 may continue from operation 1101 to operation 1102, "Determine Entropy Encoding Technique(s) for one or more Types of Video Data", where entropy encoding technique(s) may be determined for one or more of the video data types.
  • one or more of encoder modules 401 ⁇ 107 may determine a coding technique for video data 41 1— 417 from various coding technique options.
  • encoder module 401 may determine a coding technique for video data 41 1 from one of an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression technique.
  • encoder module 402 may determine a coding technique for intra- prediction partition data 625 of video data 402 from one of an adaptive symbol-run variable length coding technique and an adaptive proxy variable length coding technique, and so on as discussed herein.
  • Process 1100 may continue at operation 1103, "Entropy Encode the Video Data using the Determined Technique(s)", the video data may be entropy encoded using the determined technique(s).
  • first video data may be entropy encoded using a first entropy encoding technique to generate first compressed video data.
  • second, third, or more additional video data may be entropy encoded using second, third or more additional entropy encoding techniques as discussed herein to generate first, second, third, and so on respective compressed video data.
  • Process 1100 may continue from operation 1 103 to operation 1 104, "Assemble the Compressed Video Data", where the compressed video data of different types may be assembled to generate an output bitstream.
  • bitstream assembler 408 may assemble or multiplex the compressed video data to generate output bitstream 11 1 as discussed herein.
  • Process 1100 may continue from operation 1104 to operation 1105, "Optionally Multiplex Video Bitstream with Audio Stream", where the video bitstream may be optionally multiplexed with a coded or uncoded audio stream to generate an audio-visual bitstream.
  • Process 1100 may continue from operation 1105 or operation 1 104 to operation 1 106, "Transmit Bitstream", where the bitstream may be transmitted.
  • video coding system 1610 may transmit output bitstream 11 1 or bitstream 800 or the like via an antenna 1102 (please refer to FIG. 16).
  • Operations 1101-1106 may provide for video encoding and bitstream transmission techniques, which may be employed by an encoder system as discussed herein.
  • operations 1 107-1 112 may provide for video decoding and video display techniques, which may be employed by a decoder system as discussed herein.
  • Process 1100 may continue at operation 1107, "Receive Bitstream", where the encoded bitstream may be received.
  • the encoded bitstream may be received via decoder 200.
  • the bitstream may include different types of entropy encoded data as discussed herein.
  • Process 1100 may continue from operation 1107 to operation 1108, "Disassamble Bitstream to Generate Compressed Video Data of Different Types", where the received bitstream may be disassembled to determine different types of compressed video data.
  • bitstream 201 may be disassembled via bitstream disassembler 508 to generate compressed video data 51 1-517 (and any sub-component video data) as illustrated in FIG. 7.
  • the disassembled compressed video data may include first, second, third, or more compressed video data.
  • Process 1100 may continue from operation 1 108 to operation 1109, "Determine Entropy Decoding Technique(s) for one or more Types of Compressed Video Data", entropy decoding technique(s) may be determined for one or more of the compressed video data types. In some examples, the entropy decoding technique(s) may be determined based on a flag or indicator or the like conveyed via the received bitstream. As discussed, in some examples, one or more of decoder modules 501-507 may determine a coding technique for compressed video data 511-517 from various coding technique options.
  • decoder module 401 may determine a coding technique for compressed video data 511 from one of an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression technique based on an indicator or indicators provided via the input bitstream. Further, decoder module 502 may determine a coding technique for compressed intra- prediction partition data 725 of video data 512 from one of an adaptive symbol-run variable length coding technique and an adaptive proxy variable length coding technique, and so on. A wide range of examples have been provided herein and will not be repeated here for the sake of brevity. As discussed, in some examples, two ore more types of video data may be received.
  • an entropy decoding technique may be determined for two, three, or more types of video data as described herein.
  • Process 1100 may continue from operation 1 109 to operation 1610, "Entropy Decode the Compressed Video Data", where the compressed video data may be entropy decoded based on the determined entropy decoding techniques.
  • compressed video data 511-517 may be entropy decoded via decode modules 501-507.
  • Process 1100 may continue from operation 1610 to operation 1 11 1, "Decode the Entropy Decoded Video Data to Generate Video Frame(s)", where the first and second video data (and any other entropy decoded video data) may be decoded to generate video frame(s).
  • the video frames may be suitable for presentment to a user via a display device, for example.
  • the video frame may be determined based on the
  • Process 1100 may continue from operation 1 11 1 to operation 1 1 12, "Transmit Video Frames for Presentment via a Display Device", where generated video frame(s) may be transmitted for presentment via a display device.
  • the video frame(s) may be transmitted to a display device 1 105 (as shown in FIG. 16) for presentment.
  • display device 1 105 may display the video frames to a user, for example.
  • implementation of the example processes herein may include the undertaking of all operations shown in the order illustrated, the present disclosure is not limited in this regard and, in various examples, implementation of the example processes herein may include the undertaking of only a subset of the operations shown and/or in a different order than illustrated.
  • any one or more of the operations discussed herein may be undertaken in response to instructions provided by one or more computer program products.
  • Such program products may include signal bearing media providing instructions that, when executed by, for example, a processor, may provide the functionality described herein.
  • the computer program products may be provided in any form of one or more machine-readable media.
  • a processor including one or more processor core(s) may undertake one or more of the operations of the example processes herein in response to program code and/or instructions or instruction sets conveyed to the processor by one or more machine-readable media.
  • a machine-readable medium may convey software in the form of program code and/or instructions or instruction sets that may cause any of the devices and/or systems described herein to implement at least portions of the video systems as discussed herein.
  • module refers to any combination of software logic, firmware logic and/or hardware logic configured to provide the functionality described herein.
  • the software may be embodied as a software package, code and/or instruction set or instructions, and "hardware", as used in any implementation described herein, may include, for example, singly or in any combination, hardwired circuitry, programmable circuitry, state machine circuitry, and/or firmware that stores instructions executed by programmable circuitry.
  • the modules may, collectively or individually, be embodied as circuitry that forms part of a larger system, for example, an integrated circuit (IC), system on-chip (SoC), and so forth.
  • a module may be embodied in logic circuitry for the implementation via software, firmware, or hardware of the coding systems discussed herein.
  • FIG. 12 is a flow diagram illustrating an example process 1200, arranged in accordance with at least some implementations of the present disclosure.
  • Process 1200 may include one or more operations, functions or actions as illustrated by one or more operations.
  • Process 1200 may form at least part of a next generation video coding process.
  • process 1200 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or any other encoder system or subsystems described herein.
  • Process 1200 may begin at operation 1202, "Receive Input Video Frames of a Video Sequence", where input video frames of a video sequence may be received via encoder 100 for example.
  • Process 1200 may continue at operation 1204, "Associate a Picture Type with each Video Frame", where a picture type may be associated with each video frame in a group of pictures via content pre-analyzer module 102 for example.
  • the picture type may be F/B-picture , P-picture, or I-picture, or the like.
  • a video sequence may include groups of pictures and the processing described herein (e.g., operations 1203 through 1211) may be performed on a frame or picture of a group of pictures and the processing may be repeated for all frames or pictures of a group and then repeated for all groups of pictures in a video sequence.
  • Process 1200 may continue at operation 1206, "Divide a Picture into Tiles and/or Super-fragments and Potential Prediction Partitionings", where a picture may be divided into tiles or super-fragments and potential prediction partitions via prediction partitions generator 105 for example.
  • Process 1200 may continue at operation 1210, "For Potential Prediction Partitioning, Determine Potential Prediction Error", where, for each potential prediction partitioning, a potential prediction error may be determined.
  • a prediction error may be determined for each prediction partitioning (and associated prediction partitions, prediction(s), and prediction parameters).
  • determining the potential prediction error may include differencing original pixels (e.g., original pixel data of a prediction partition) with prediction pixels.
  • the associated prediction parameters may be stored.
  • the prediction error data partition may include prediction error data generated based at least in part on a previously decoded frame generated using at least one of a morphing technique or a synthesizing technique.
  • Process 1200 may continue at operation 1212, "Select Prediction Partitioning and Prediction Type and Save Parameters", where a prediction partitioning and prediction type may be selected and the associated parameters may be saved.
  • a prediction partitioning and prediction type may be selected and the associated parameters may be saved.
  • the potential prediction partitioning with a minimum prediction error may be selected.
  • the potential prediction partitioning may be selected based on a rate distortion optimization (RDO).
  • RDO rate distortion optimization
  • Process 1200 may continue at operation 1214, "Perform Transforms on Potential Coding Partitionings", where fixed or content adaptive transforms with various block sizes may be performed on various potential coding partitionings of partition prediction error data.
  • partition prediction error data may be partitioned to generate a plurality of coding partitions.
  • the partition prediction error data may be partitioned by a bi-tree coding partitioner module or a k-d tree coding partitioner module of coding partitions generator module 107 as discussed herein.
  • partition prediction error data associated with an F/B- or P-picture may be partitioned by a bi-tree coding partitioner module.
  • video data associated with an I-picture may be partitioned by a k-d tree coding partitioner module.
  • a coding partitioner module may be chosen or selected via a switch or switches.
  • the partitions may be generated by coding partitions generator module 107.
  • Process 1200 may continue at operation 1216, "Determine the Best Coding
  • Partitioning, Transform Block Sizes, and Actual Transform where the best coding partitioning, transform block sizes, and actual transforms may be determined.
  • various coding partitionings e.g., having various coding partitions
  • a selected coding partitioning which may also include further division of coding partitions into transform blocks when coding partitions to not match a transform block size as discussed.
  • Process 1200 may continue at operation 1218, "Quantize and Scan Transform Coefficients", where transform coefficients associated with coding partitions (and/or transform blocks) may be quantized and scanned in preparation for entropy coding.
  • Process 1200 may continue at operation 1222, "Entropy Encode Data associated with Each Tile or Super-fragment Decode, Such As Coding Partition Indicator(s), Block Size Data, Transform Type Data, Quantizer (Qp), and Quantized Transform Coefficients, Motion Vectors and Reference Type Data, Characteristic Parameters (e.g., mop, syp)", where data may be entropy encoded.
  • the entropy encoded data may include the coding partition indicators, block size data, transform type data, quantizer (Qp), quantized transform coefficients, motion vectors and reference type data, characteristic parameters (e.g., mop, syp), the like, and/or combinations thereof.
  • the entropy encoded data may include prediction partitioning, prediction parameters, the selected coding partitioning, the selected characteristics data, motion vector data, quantized transform coefficients, filter parameters, selection data (such as mode selection data), and indictors.
  • Process 1200 may continue at operation 1223 "Apply DD/DB Filter, Reconstruct Pixel Data, Assemble into a Picture", where deblock filtering (e.g., DD or DB filters) may be applied, pixel data may be reconstructed, and assembled into a picture.
  • deblock filtering e.g., DD or DB filters
  • pixel data may be reconstructed, and assembled into a picture.
  • a local decode loop e.g., including inverse scan, inverse transform, and assembling coding partitions
  • prediction error data partitions may be generated.
  • the prediction error data partitions may be added with a prediction partition to generate reconstructed prediction partitions, which may be assembled into tiles or super-fragments.
  • the assembled tiles or super-fragments may be optionally processed via deblock filtering and/or quality restoration filtering and assembled to generate a picture.
  • Process 1200 may continue at operation 1224 "Apply QR/LF Filter Save in
  • Reference Picture Buffers where quality restoration filtering (e.g., QR or LF filtering) may be applied, and the assembled picture may be saved in reference picture buffers.
  • quality restoration filtering e.g., QR or LF filtering
  • the assembled tiles or super-fragments may be optionally processed via quality restoration filtering and assembled to generate a picture.
  • the picture may be saved in decoded picture buffer 1 19 as a reference picture for prediction of other (e.g., following) pictures.
  • Process 1200 may continue at operation 1225, "Apply AP/AM Filter, Determine Modifying (e.g., Morphing or Synthesizing) Characteristic Parameters for Generating Morphed or Synthesized Prediction Reference(s) and Perform Prediction(s)", where, modifying (e.g., morphing or synthesizing) characteristic parameters and prediction(s) may be performed and adaptive motion filtering or adaptive precision filtering (e.g., AP/AM Filter) may be applied. For example, modifying (e.g., morphing or synthesizing) characteristic parameters for generating morphed or synthesized prediction reference(s) may be generated and prediction(s) may be performed. Additionally, adaptive motion filtering or adaptive precision filtering may be applied at this point in the process.
  • modifying e.g., morphing or synthesizing characteristic parameters for generating morphed or synthesized prediction reference(s) may be generated and prediction(s) may be performed.
  • inter-prediction may be performed.
  • up to 4 decoded past and/or future pictures and several morphing/synthesis predictions may be used to generate a large number of reference types (e.g., reference pictures).
  • reference types e.g., reference pictures
  • 'inter' mode up to nine reference types may be supported in P- pictures, and up to ten reference types may be supported for F/B-pictures.
  • 'multi' mode may provide a type of inter prediction mode in which instead of 1 reference picture, 2 reference pictures may be used and P- and F/B-pictures respectively may allow 3, and up to 8 reference types.
  • prediction may be based on a previously decoded frame generated using at least one of a morphing technique or a synthesizing technique.
  • the bitstream discussed below with respect to operation 1212
  • the bitstream may include a frame reference, morphing parameters, or synthesizing parameters associated with the prediction partition.
  • Process 1200 may continue at operation 1229 "Optionally Apply EP Filter and/or Optionally apply FI FP Filter", where enhanced predicted partition (e.g., EP Filtering) or FI/FP Filtering (e.g., fusion filtering or fusion improvement filtering) may be optionally applied.
  • enhanced predicted partition e.g., EP Filtering
  • FI/FP Filtering e.g., fusion filtering or fusion improvement filtering
  • a decision may be made regarding whether to utilize some form or FI/FP Filter (fusion improvement filtering/fusion filtering) or not to use FI/FP Filtering.
  • some form or FI FP Filter e.g., fusion filtering or fusion improvement filtering
  • the selected predicted partition and a second selected predicted partition may be assembled to generate at least a portion of an assembled picture.
  • FI/FP Filtering may be applied to filter the portion of the assembled picture.
  • FI/FP Filtering parameters e.g., filtering parameters or fusion improvement filtering parameters
  • FI/FP Filtering parameters associated with the FI/FP Filtering may be generated and sent to the entropy coder subsystem.
  • an indicator may be generated that indicates to the decoder system whether to use the enhanced predicted partition (e.g., EP Filtering) or the predicted partition data as the selected predicted partition for the prediction partition.
  • Operations 1202 through 1229 may provide for video encoding and bitstream transmission techniques, which may be employed by an encoder system as discussed herein.
  • FIG. 13 illustrates an example bitstream 1300, arranged in accordance with at least some implementations of the present disclosure.
  • bitstream 1300 may correspond to output bitstream 11 1 as shown in FIG. 1 and/or input bitstream 201 as shown in FIG. 2.
  • bitstream 1300 may include a header portion and a data portion.
  • bitstream 1300 may include data, indicators, index values, mode selection data, or the like associated with encoding a video frame as discussed herein.
  • bitstream 1300 may be generated by an encoder such as, for example, encoder 100 and/or received by a decoder 200 for decoding such that decoded video frames may be presented via a display device.
  • FIG. 14 is a flow diagram illustrating an example process 1400, arranged in accordance with at least some implementations of the present disclosure.
  • Process 1400 may include one or more operations, functions or actions as illustrated by one or more operations.
  • Process 1400 may form at least part of a next generation video coding process.
  • process 1400 may form at least part of a next generation video decoding process as undertaken by decoder system 200 and/or any other decoder system or subsystems described herein.
  • Process 1400 may begin at operation 1402, "Receive Encoded Bitstream", where a bitstream may be received.
  • a bitstream encoded as discussed herein may be received at a video decoder.
  • bitstream 900 or 1300 may be received via decoder 200.
  • Process 1400 may continue at operation 1404, "Decode the Entropy Encoded Bitstream to Determine Coding Partition Indicator(s), Block Size Data, Transform Type Data, Quantizer (Qp), Quantized Transform Coefficients, Motion Vectors and Reference Type Data, Characteristic Parameters (e.g., mop, syp)", where the bitstream may be decoded to determine coding partition indicators, block size data, transform type data, quantizer (Qp), quantized transform coefficients, motion vectors and reference type data, characteristic parameters (e.g., mop, syp), the like, and/or combinations thereof.
  • the bitstream may be decoded to determine coding partition indicators, block size data, transform type data, quantizer (Qp), quantized transform coefficients, motion vectors and reference type data, characteristic parameters (e.g., mop, syp), the like, and/or combinations thereof.
  • the entropy encoded data may include prediction partitioning, prediction parameters, the selected coding partitioning, the selected characteristics data, motion vector data, quantized transform coefficients, filter parameters, selection data (such as mode selection data), and indictors.
  • Process 1400 may continue at operation 1406, "Apply Quantizer (Qp) on Quantized Coefficients to Generate Inverse Quantized Transform Coefficients", where quantizer (Qp) may be applied to quantized transform coefficients to generate inverse quantized transform coefficients.
  • Qp may be applied via adaptive inverse quantize module 203.
  • Process 1400 may continue at operation 1408, "On each Decoded Block of
  • Coefficients in a Coding (or Intra Predicted) Partition Perform Inverse Transform based on Transform Type and Block Size Data to Generate Decoded Prediction Error Partitions", where, on each decode block of transform coefficients in a coding (or intra predicted) partition, an inverse transform based on the transform type and block size data may be performed to generate decoded prediction error partitions.
  • the inverse transform may include an inverse fixed transform.
  • the inverse transform may include an inverse content adaptive transform.
  • performing the inverse content adaptive transform may include determining basis functions associated with the inverse content adaptive transform based on a neighboring block of decoded video data, as discussed herein.
  • Any forward transform used for encoding as discussed herein may be used for decoding using an associated inverse transform.
  • the inverse transform may be performed by adaptive inverse transform module 204.
  • generating the decoded prediction error partitions may also include assembling coding partitions via coding partitions assembler 205.
  • Process 1400 may continue at operation 1423 "Apply DD/DB Filter, Reconstruct Pixel Data, Assemble into a Picture", where deblock filtering (e.g., DD or DB filters) may be applied, pixel data may be reconstructed, and assembled into a picture.
  • deblock filtering e.g., DD or DB filters
  • the prediction error data partitions may be added with a prediction partition to generate reconstructed prediction partitions, which may be assembled into tiles or super-fragments.
  • the assembled tiles or super-fragments may be optionally processed via deblock filtering.
  • Process 1400 may continue at operation 1424 "Apply QR/LF Filter Save in
  • Reference Picture Buffers where quality restoration filtering (e.g., QR or LF filtering) may be applied, and the assembled picture may be saved in reference picture buffers.
  • quality restoration filtering e.g., QR or LF filtering
  • the assembled tiles or super-fragments may be optionally processed via quality restoration filtering and assembled to generate a picture.
  • the picture may be saved in decoded picture buffer 1 19 as a reference picture for prediction of other (e.g., following) pictures.
  • Process 1400 may continue at operation 1425, "Apply AP/AM Filter, Use Decoded Modifying Characteristics (e.g., mop, syp) to Generate Modified References for Prediction and Use Motion Vectors and Reference Info, Predicted Partition Info, and Modified References to Generate Predicted Partition", where modified references for prediction may be generated and predicted partitions may be generated as well, and where adaptive motion filtering or adaptive precision filtering (e.g., AP/AM Filter ) may be applied.
  • AP/AM Filter Use Decoded Modifying Characteristics
  • modified references for prediction may be generated based at least in part on decoded modifying characteristics (e.g., mop, syp) and predicted partitions may be generated based at least in part on motion vectors and reference information, predicted partition information, and modified references.
  • decoded modifying characteristics e.g., mop, syp
  • predicted partitions may be generated based at least in part on motion vectors and reference information, predicted partition information, and modified references.
  • adaptive motion filtering or adaptive precision filtering may be applied at this point in the process.
  • Process 1400 may continue at operation 1429 "Optionally Apply EP Filter and/or Optionally apply FI FP Filter", where enhanced predicted partition (e.g., EP Filtering) or FI/FP Filtering (e.g., fusion filtering or fusion improvement filtering) may be optionally applied.
  • enhanced predicted partition e.g., EP Filtering
  • FI/FP Filtering e.g., fusion filtering or fusion improvement filtering
  • a decision may be made regarding whether to utilize some form or FI FP Filter (fusion improvement filtering/fusion filtering) or not to use FI/FP Filtering.
  • some form or FI/FP Filter e.g., fusion filtering or fusion improvement filtering
  • the selected predicted partition and a second selected predicted partition may be assembled to generate at least a portion of an assembled picture.
  • FI/FP Filtering may be applied to filter the portion of the assembled picture.
  • FI/FP Filtering parameters e.g., filtering parameters or fusion improvement filtering parameters
  • FI/FP Filtering parameters associated with the FI/FP Filtering may be generated and sent to the entropy coder subsystem.
  • an indicator may be received from the encoder system that indicates to the decoder system whether to use the enhanced predicted partition (e.g., EP Filtering) or the predicted partition data as the selected predicted partition for the prediction partition.
  • the enhanced predicted partition e.g., EP Filtering
  • Process 1400 may continue at operation 1430, "Add Prediction Partition to the Decoded Prediction Error Data Partition to Generate a Reconstructed Partition", where a prediction partition my be added to the decoded prediction error data partition to generate a reconstructed prediction partition.
  • the decoded prediction error data partition may be added to the associated prediction partition via adder 206.
  • Process 1400 may continue at operation 1432, "Assemble Reconstructed Partitions to Generate a Tile or Super-Fragment", where the reconstructed prediction partitions may be assembled to generate tiles or super-fragments.
  • the reconstructed prediction partitions may be assembled to generate tiles or super- fragments via prediction partitions assembler module 207.
  • Process 1400 may continue at operation 1434, "Assemble Tiles or Super-Fragments of a Picture to Generate a Full Decoded Picture", where the tiles or super- fragments of a picture may be assembled to generate a full decoded picture.
  • tiles or super- fragments may be assembled to generate a full decoded picture, which may be stored via decoded picture buffer 210 and/or transmitted for presentment via a display device after processing via adaptive picture re-organizer module 217 and content post-restorer module 218.
  • FIGS. 15(A), 15(B), and 15(C) provide an illustrative diagram of an example video coding system 1600 and video coding process 1500 in operation, arranged in accordance with at least some implementations of the present disclosure.
  • FIGS. 15(A), 15(B), and 15(C) provide an illustrative diagram of an example video coding system 1600 and video coding process 1500 in operation, arranged in accordance with at least some implementations of the present disclosure. In the illustrated
  • process 1500 may include one or more operations, functions or actions as illustrated by one or more of actions 1501 through 1580.
  • process 1500 will be described herein with reference to example video coding system 1600 including encoder 100 of FIG. 1 and decoder 200 of FIG. 2, as is discussed further herein below with respect to FIG. 16.
  • process 1500 may be undertaken by a system including both an encoder and decoder or by separate systems with one system employing an encoder (and optionally a decoder) and another system employing a decoder (and optionally an encoder).
  • an encoder may include a local decode loop employing a local decoder as a part of the encoder system.
  • video coding system 1600 may include logic circuitry 1650, the like, and/or combinations thereof.
  • logic circuitry 1650 may include encoder system 100 of FIG. 1 and/or decoder system 200 of FIG. 2 and may include any modules as discussed with respect to any of the encoder systems or subsystems described herein and/or decoder systems or subsystems described herein.
  • video coding system 1600 as shown in FIGS. 15(A)-(C) may include one particular set of blocks or actions associated with particular modules, these blocks or actions may be associated with different modules than the particular modules illustrated here.
  • process 1500 is directed to encoding and decoding, the concepts and/or operations described may be applied to encoding and/or decoding separately, and, more generally, to video coding.
  • Process 1500 may begin at operation 1501, "Receive Input Video Frames of a Video Sequence", where input video frames of a video sequence may be received via encoder 100 for example.
  • Process 1500 may continue at operation 1502, "Associate a Picture Type with each Video Frame in a Group of Pictures", where a picture type may be associated with each video frame in a group of pictures via content pre-analyzer module 102 for example.
  • the picture type may be F/B-picture , P-picture, or I-picture, or the like.
  • a video sequence may include groups of pictures and the processing described herein (e.g., operations 1503 through 151 1) may be performed on a frame or picture of a group of pictures and the processing may be repeated for all frames or pictures of a group and then repeated for all groups of pictures in a video sequence.
  • Process 1500 may continue at operation 1503, "Divide a Picture into Tiles and/or Super-fragments and Potential Prediction Partitionings", where a picture may be divided into tiles or super- fragments and potential prediction partitions via prediction partitions generator 105 for example.
  • Process 1500 may continue at operation 1504, "For Each Potential Prediction Partitioning, Perform Prediction(s) and Determine Prediction Parameters", where, for each potential prediction partitionings, prediction(s) may be performed and prediction parameters may be determined. For example, a range of potential prediction partitionings (each having various prediction partitions) may be generated and the associated prediction(s) and prediction parameters may be determined.
  • the prediction(s) may include prediction(s) using characteristics and motion based multi-reference predictions or intra-predictions.
  • inter-prediction may be performed.
  • up to 4 decoded past and/or future pictures and several morphing/synthesis predictions may be used to generate a large number of reference types (e.g., reference pictures).
  • reference types e.g., reference pictures
  • up to 9 reference types may be supported in P- pictures, and up to 10 reference types may be supported for F/B-pictures.
  • 'multi' mode may provide a type of inter prediction mode in which instead of 1 reference picture, 2 reference pictures may be used and P- and F/B-pictures respectively may allow 3, and up to 8 reference types.
  • prediction may be based on a previously decoded frame generated using at least one of a morphing technique or a synthesizing technique.
  • the bitstream discussed below with respect to operation 1512
  • the bitstream may include a frame reference, morphing parameters, or synthesizing parameters associated with the prediction partition.
  • Process 1500 may continue at operation 1505, "For Each Potential Prediction Partitioning, Determine Potential Prediction Error", where, for each potential prediction partitioning, a potential prediction error may be determined. For example, for each prediction partitioning (and associated prediction partitions, prediction(s), and prediction parameters), a prediction error may be determined. For example, determining the potential prediction error may include differencing original pixels (e.g., original pixel data of a prediction partition) with prediction pixels. In some examples, the associated prediction parameters may be stored. As discussed, in some examples, the prediction error data partition may include prediction error data generated based at least in part on a previously decoded frame generated using at least one of a morphing technique or a synthesizing technique.
  • Process 1500 may continue at operation 1506, "Select Prediction Partitioning and Prediction Type and Save Parameters", where a prediction partitioning and prediction type may be selected and the associated parameters may be saved.
  • the potential prediction partitioning with a minimum prediction error may be selected.
  • the potential prediction partitioning may be selected based on a rate distortion optimization (RDO).
  • RDO rate distortion optimization
  • Process 1500 may continue at operation 1507, "Perform Fixed or Content Adaptive Transforms with Various Block Sizes on Various Potential Coding Partitionings of Partition Prediction Error Data", where fixed or content adaptive transforms with various block sizes may be performed on various potential coding partitionings of partition prediction error data.
  • partition prediction error data may be partitioned to generate a plurality of coding partitions.
  • the partition prediction error data may be partitioned by a bi-tree coding partitioner module or a k-d tree coding partitioner module of coding partitions generator module 107 as discussed herein.
  • partition prediction error data associated with an F/B- or P-picture may be partitioned by a bi-tree coding partitioner module.
  • video data associated with an I- picture may be partitioned by a k-d tree coding partitioner module.
  • a coding partitioner module may be chosen or selected via a switch or switches.
  • the partitions may be generated by coding partitions generator module 107.
  • Process 1500 may continue at operation 1508, "Determine the Best Coding
  • Partitioning, Transform Block Sizes, and Actual Transform where the best coding partitioning, transform block sizes, and actual transforms may be determined.
  • various coding partitionings e.g., having various coding partitions
  • a selected coding partitioning which may also include further division of coding partitions into transform blocks when coding partitions to not match a transform block size as discussed.
  • Process 1500 may continue at operation 1509, "Quantize and Scan Transform Coefficients", where transform coefficients associated with coding partitions (and/or transform blocks) may be quantized and scanned in preparation for entropy coding.
  • Process 1500 may continue at operation 151 1, "Entropy Encode Data associated with Each Tile or Super- fragment", where data associated with each tile or super- fragment may be entropy encoded.
  • data associated with each tile or super-fragment of each picture of each group of pictures of each video sequence may be entropy encoded.
  • the entropy encoded data may include the prediction partitioning, prediction parameters, the selected coding partitioning, the selected characteristics data, motion vector data, quantized transform coefficients, filter parameters, selection data (such as mode selection data), and indictors.
  • Process 1500 may continue at operation 1512, "Generate Bitstream" where a bitstream may be generated based on the entropy encoded data.
  • the bitstream may include a frame or picture reference, morphing parameters, or synthesizing parameters associated with a prediction partition.
  • Process 1500 may continue at operation 1513, "Transmit Bitstream", where the bitstream may be transmitted.
  • video coding system 2400 may transmit output bitstream 11 1, bitstream 2100, or the like via an antenna 2402 (please refer to FIG. 34).
  • Process 1500 may continue at operation 1520, "Reconstruct Pixel Data, Assemble into a Picture, and Save in Reference Picture Buffers", where pixel data may be reconstructed, assembled into a picture, and saved in reference picture buffers.
  • a local decode loop e.g., including inverse scan, inverse transform, and assembling coding partitions
  • prediction error data partitions may be generated.
  • the prediction error data partitions may be added with a prediction partition to generate reconstructed prediction partitions, which may be assembled into tiles or super-fragments.
  • the assembled tiles or super-fragments may be optionally processed via deblock filtering and/or quality restoration filtering and assembled to generate a picture.
  • the picture may be saved in decoded picture buffer 1 19 as a reference picture for prediction of other (e.g., following) pictures.
  • Process 1500 may continue at operation 1523 "Apply DD/DB Filter, Reconstruct Pixel Data, Assemble into a Picture", where deblock filtering (e.g., DD or DB filters) may be applied, pixel data may be reconstructed, and assembled into a picture.
  • deblock filtering e.g., DD or DB filters
  • pixel data may be reconstructed, and assembled into a picture.
  • a local decode loop e.g., including inverse scan, inverse transform, and assembling coding partitions
  • prediction error data partitions may be generated.
  • the prediction error data partitions may be added with a prediction partition to generate reconstructed prediction partitions, which may be assembled into tiles or super-fragments.
  • the assembled tiles or super-fragments may be optionally processed via deblock filtering and/or quality restoration filtering and assembled to generate a picture.
  • Process 1500 may continue at operation 1524 "Apply QR/LF Filter Save in
  • Reference Picture Buffers where quality restoration filtering (e.g., QR or LF filtering) may be applied, and the assembled picture may be saved in reference picture buffers.
  • quality restoration filtering e.g., QR or LF filtering
  • the assembled tiles or super-fragments may be optionally processed via quality restoration filtering and assembled to generate a picture.
  • the picture may be saved in decoded picture buffer 1 19 as a reference picture for prediction of other (e.g., following) pictures.
  • Process 1500 may continue at operation 1525, "Generate Modifying Characteristic Parameters", where, modified characteristic parameters may be generated.
  • modified characteristic parameters may be generated.
  • a second modified prediction reference picture and second modifying characteristic parameters associated with the second modified prediction reference picture may be generated based at least in part on the second decoded prediction reference picture, where the second modified reference picture may be of a different type than the first modified reference picture.
  • Process 1500 may continue at operation 1526, "Generate Modified Prediction Reference Pictures", where modified prediction reference pictures may be generated, for example, a first modified prediction reference picture and first modifying characteristic parameters associated with the first modified prediction reference picture may be generated based at least in part on the first decoded prediction reference picture.
  • modified prediction reference pictures may be generated, for example, a first modified prediction reference picture and first modifying characteristic parameters associated with the first modified prediction reference picture may be generated based at least in part on the first decoded prediction reference picture.
  • Process 1500 may continue at operation 1527, "Generate Motion Data", where, motion estimation data may be generated.
  • motion data associated with a prediction partition of a current picture may be generated based at least in part on one of the first modified prediction reference picture or the second modified prediction reference picture.
  • Process 1500 may continue at operation 1528, "Apply AP/AM Filter Perform Motion Compensation", where, motion compensation may be performed. For example, motion compensation may be performed based at least in part on the motion data and at least one of the first modified prediction reference picture or the second modified prediction reference picture to generate prediction partition data for the prediction partition and adaptive motion filtering or adaptive precision filtering (e.g., AP/AM Filter) may be applied.
  • Process 1500 may feed this information back to operation 1504 where each decoded prediction error partition (e.g., including zero prediction error partitions) may be added to the corresponding prediction partition to generate a reconstructed prediction partition. Additionally, adaptive motion filtering or adaptive precision filtering may be applied at this point in the process.
  • Process 1500 may continue at operation 1529 "Optionally Apply EP", where enhanced predicted partition (e.g., EP Filtering) may be optionally applied.
  • enhanced predicted partition e.g., EP Filtering
  • an indicator may be generated that indicates to the decoder system whether to use the enhanced predicted partition (e.g., EP Filtering) or the predicted partition data as the selected predicted partition for the prediction partition.
  • Process 1500 may continue at operation 1530 "Optionally apply FI/FP Filter", where FI/FP Filtering (e.g., fusion filtering or fusion improvement filtering) may be optionally applied.
  • FI/FP Filtering e.g., fusion filtering or fusion improvement filtering
  • a decision may be made regarding whether to utilize some form or FI/FP Filter (fusion improvement filtering/fusion filtering) or not to use FI/FP Filtering.
  • some form or FI/FP Filter e.g., fusion filtering or fusion improvement filtering
  • FI/FP Filtering may be applied to filter the portion of the assembled picture.
  • FI FP Filtering parameters e.g., filtering parameters or fusion improvement filtering parameters
  • associated with the FI FP Filtering may be generated and sent to the entropy coder subsystem.
  • Operations 1501 through 1530 may provide for video encoding and bitstream transmission techniques, which may be employed by an encoder system as discussed herein.
  • the following operations, operations 1554 through 1568 may provide for video decoding and video display techniques, which may be employed by a decoder system as discussed herein.
  • Process 1500 may continue at operation 1554, "Receive Bitstream", where the bitstream may be received.
  • bitstream may include data associated with a coding partition, one or more indicators, and/or data defining coding partition(s) as discussed above.
  • the bitstream may include the prediction partitioning, prediction parameters, the selected coding partitioning, the selected characteristics data, motion vector data, quantized transform coefficients, filter parameters, selection data (such as mode selection data), and indictors.
  • Process 1500 may continue at operation 1555, "Decode Bitstream", where the received bitstream may be decoded via adaptive entropy decoder module 202 for example.
  • received bitstream may be entropy decoded to determine the prediction partitioning, prediction parameters, the selected coding partitioning, the selected characteristics data, motion vector data, quantized transform coefficients, filter parameters, selection data (such as mode selection data), and indictors.
  • Process 1500 may continue at operation 1556, "Perform Inverse Scan and Inverse Quantization on Each Block of Each Coding Partition", where an inverse scan and inverse quantization may be performed on each block of each coding partition for the prediction partition being processed.
  • the inverse scan and inverse quantization may be performed via adaptive inverse quantize module 203.
  • Process 1500 may continue at operation 1557, "Perform Fixed or Content Adaptive Inverse Transform to Decode Transform Coefficients to Determine Decoded Prediction Error Data Partitions", where a fixed or content adaptive inverse transform may be performed to decode transform coefficients to determine decoded prediction error data partitions.
  • the inverse transform may include an inverse content adaptive transform such as a hybrid parametric Haar inverse transform such that the hybrid parametric Haar inverse transform may include a parametric Haar inverse transform in a direction of the parametric transform direction and a discrete cosine inverse transform in a direction orthogonal to the parametric transform direction.
  • the fixed inverse transform may include a discrete cosine inverse transform or a discrete cosine inverse transform approximator.
  • the fixed or content adaptive transform may be performed via adaptive inverse transform module 204.
  • the content adaptive inverse transform may be based on other previously decoded data, such as, for example, decoded neighboring partitions or blocks.
  • generating the decoded prediction error data partitions may include assembling decoded coding partitions via coding partitions assembler module 205.
  • Process 1500 may continue at operation 1558, "Generate Prediction Pixel Data for Each Prediction Partition", where prediction pixel data may be generated for each prediction partition.
  • prediction pixel data may be generated using the selected prediction type (e.g., based on characteristics and motion, or intra-, or other types) and associated prediction parameters.
  • Process 1500 may continue at operation 1559, "Add to Each Decoded Prediction Error Partition the Corresponding Prediction Partition to Generate Reconstructed
  • each decoded prediction error partition (e.g., including zero prediction error partitions) may be added to the corresponding prediction partition to generated a reconstructed prediction partition.
  • prediction partitions may be generated via the decode loop illustrated in FIG. 2 and added via adder 206 to decoded prediction error partitions.
  • Process 1500 may continue at operation 1560, "Assemble Reconstructed Prediction Partitions to Generate Decoded Tiles or Super- fragments", where reconstructed prediction partitions may be assembled to generate decoded tiles or super-fragments.
  • prediction partitions may be assembled to generate decoded tiles or super-fragments via prediction partitions assembler module 207.
  • Process 1500 may continue at operation 1561, "Apply Deblock Filtering and/or QR Filtering to Generate Final Decoded Tiles or Super- fragments", where optional deblock filtering and/or quality restoration filtering may be applied to the decoded tiles or super- fragments to generate final decoded tiles or super-fragments.
  • optional deblock filtering may be applied via deblock filtering module 208 and/or optional quality restoration filtering may be applied via quality restoration filtering module 209.
  • Process 1500 may continue at operation 1562, "Assemble Decoded Tiles or Super- fragments to Generate a Decoded Video Picture, and Save in Reference Picture Buffers", where decoded (or final decoded) tiles or super- fragments may be assembled to generate a decoded video picture, and the decoded video picture may be saved in reference picture buffers (e.g., decoded picture buffer 210) for use in future prediction.
  • reference picture buffers e.g., decoded picture buffer 2
  • Process 1500 may continue at operation 1563, "Transmit Decoded Video Frames for Presentment via a Display Device", where decoded video frames may be transmitted for presentment via a display device.
  • decoded video pictures may be further processed via adaptive picture re-organizer 217 and content post restorer module 218 and transmitted to a display device as video frames of display video 219 for presentment to a user.
  • the video frame(s) may be transmitted to a display device 2405 (as shown in FIG. 34) for presentment.
  • Process 1500 may continue at operation 1573 "Apply DD/DB Filter, Reconstruct Pixel Data, Assemble into a Picture", where deblock filtering (e.g., DD or DB filters) may be applied, pixel data may be reconstructed, and assembled into a picture.
  • deblock filtering e.g., DD or DB filters
  • the prediction error data partitions may be added with a prediction partition to generate reconstructed prediction partitions, which may be assembled into tiles or super-fragments.
  • the assembled tiles or super-fragments may be optionally processed via deblock filtering.
  • Process 1500 may continue at operation 1574 "Apply QR/LF Filter Save in
  • Reference Picture Buffers where quality restoration filtering (e.g., QR or LF filtering) may be applied, and the assembled picture may be saved in reference picture buffers.
  • quality restoration filtering e.g., QR or LF filtering
  • the assembled tiles or super-fragments may be optionally processed via quality restoration filtering and assembled to generate a picture.
  • the picture may be saved in a picture buffer as a reference picture for prediction of other (e.g., following) pictures.
  • Process 1500 may continue at operation 1576, "Generate Modified Prediction Reference Pictures", where modified prediction reference pictures may be generated, for example, at least a portion of a third modified prediction reference picture may be generated based at least in part on the third modifying characteristic parameters. Similarly, at least a portion a fourth modified prediction reference picture may be generated based at least in part on the second modifying characteristic parameters associated.
  • Process 1500 may continue at operation 1577, "Generate Motion Data", where, motion estimation data may be generated.
  • motion data associated with a prediction partition of a current picture may be generated based at least in part on one of the third modified prediction reference picture or the third modified prediction reference picture.
  • Process 1500 may continue at operation 1578, "Apply AP/AM Filter and Perform Motion Compensation", where, motion compensation may be performed and where adaptive motion filtering or adaptive precision filtering (e.g., AP/AM Filter ) may be applied. For example, motion compensation may be performed based at least in part on the motion data and at least one of the third modified prediction reference picture or the fourth modified prediction reference picture to generate prediction partition data for the prediction partition.
  • Process 1300 may feed this information back to operation 1559 where each decoded prediction error partition (e.g., including zero prediction error partitions) may be added to the corresponding prediction partition to generate a reconstructed prediction partition. Additionally, adaptive motion filtering or adaptive precision filtering may be applied at this point in the process.
  • Process 1500 may continue at operation 1579 "Optionally Apply EP Filter", where enhanced predicted partition (e.g., EP Filtering) may be optionally applied.
  • enhanced predicted partition e.g., EP Filtering
  • an indicator may be received from the encoder system that indicates to the decoder system whether to use the enhanced predicted partition (e.g., EP Filtering) or the predicted partition data as the selected predicted partition for the prediction partition.
  • Process 1500 may continue at operation 1580 "Optionally apply FI/FP Filter", where FI/FP Filtering (e.g., fusion filtering or fusion improvement filtering) may be optionally applied.
  • FI/FP Filtering e.g., fusion filtering or fusion improvement filtering
  • a decision may be made regarding whether to utilize some form or FI/FP Filter (fusion improvement filtering/fusion filtering) or not to use FI/FP Filtering.
  • some form or FI/FP Filter e.g., fusion filtering or fusion improvement filtering
  • FI/FP Filtering may be applied to filter the portion of the assembled picture.
  • FI FP Filtering parameters e.g., filtering parameters or fusion improvement filtering parameters
  • associated with the FI FP Filtering may be generated and sent to the entropy coder subsystem.
  • Process 1500 may be implemented via any of the coder systems as discussed herein. Further, process 1500 may be repeated either in serial or in parallel on any number of instantiations of video data such as prediction error data partitions, original data partitions, or wavelet data or the like.
  • implementation of the example processes herein may include the undertaking of all operations shown in the order illustrated, the present disclosure is not limited in this regard and, in various examples, implementation of the example processes herein may include the undertaking of only a subset of the operations shown and/or in a different order than illustrated.
  • FIG. 16 is an illustrative diagram of example video coding system 1600, arranged in accordance with at least some implementations of the present disclosure.
  • video coding system 1600 may include imaging device(s) 1601, video encoder 100 and/or a video encoder implemented via logic circuitry 1650 of processing unit(s) 1620, video decoder 200 and/or a video decoder implemented via logic circuitry 1650 of processing unit(s) 1620, an antenna 1602, one or more processor(s) 1603, one or more memory store(s) 2004, and/or a display device 1605.
  • imaging device(s) 1601, antenna 1602, processing unit(s) 1620, logic circuitry 1650, video encoder 100, video decoder 200, processor(s) 1603, memory store(s) 1604, and/or display device 1605 may be capable of communication with one another.
  • video coding system 1600 may include only video encoder 100 or only video decoder 200 in various examples.
  • video coding system 1600 may include antenna 1602. Antenna 1602 may be configured to transmit or receive an encoded bitstream of video data, for example. Further, in some examples, video coding system 1600 may include display device 1605. Display device 1605 may be configured to present video data. As shown, in some example, logic circuitry 1650 may be implemented via processing unit(s) 1620. Processing unit(s) 1620 may include application-specific integrated circuit (ASIC) logic, graphics processor(s), general purpose processor(s), or the like. Video coding system 1600 also may include optional processor(s) 1603, which may similarly include application-specific integrated circuit (ASIC) logic, graphics processor(s), general purpose processor(s), or the like.
  • ASIC application-specific integrated circuit
  • logic circuitry 1650 may be implemented via hardware or video coding dedicated hardware or the like, and processor(s) 1603 may implemented general purpose software or operating systems or the like.
  • memory stores 1604 may be any type of memory such as volatile memory (e.g., Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.), and so forth.
  • memory stores 1604 may be implemented by cache memory.
  • logic circuitry 1650 may access memory stores 1604 (for implementation of an image buffer for example).
  • logic circuitry 1650 and/or processing unit(s) 1620 may include memory stores (e.g., cache or the like) for the implementation of an image buffer or the like.
  • video encoder 100 implemented via logic circuitry may include an image buffer (e.g., via either processing unit(s) 1620 or memory store(s) 1604)) and a graphics processing unit (e.g., via processing unit(s) 1620).
  • the graphics processing unit may be communicatively coupled to the image buffer.
  • the graphics processing unit may include video encoder 100 as implemented via logic circuitry 1650 to embody the various modules as discussed with respect to FIG. 1 and FIGS. 3 and 5.
  • the graphics processing unit may include entropy encoder logic circuitry, and so on.
  • the logic circuitry may be configured to perform the various operations as discussed herein.
  • the entropy encoder logic circuitry may be configured to receive first video data and second video data of different types for entropy encoding, determine a first entropy encoding technique for the first video data based at least in part on a parameter associated with the first video data such that the first entropy encoding technique comprises at least one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, entropy encode the first video data using the first encoding technique to generate first compressed video data and the second video data using a second encoding technique to generate second compressed video data, and assemble the first compressed video data and the second compressed video data to generate an output bitstream.
  • Video decoder 200 may be implemented in a similar manner.
  • antenna 1602 of video coding system 1600 may be configured to receive an entropy encoded bitstream of video data.
  • the bitstream may include two or more compressed video data types.
  • Video coding system 1600 may also include video decoder 200 coupled to antenna 1602 and configured to decode the encoded bitstream.
  • video decoder 200 may be configured to disassemble the entropy encoded bitstream to determine first compressed video data and second compressed video data, determine a first entropy decoding technique for the first compressed video data, such that the first entropy decoding technique comprises at least one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, entropy decode the first compressed video data based on the first entropy decoding technique to generate first video data and the second compressed video data based on a second entropy decoding technique to generate second video data, and decode the first video data and the second video data to generate a video frame.
  • features described herein may be undertaken in response to instructions provided by one or more computer program products.
  • Such program products may include signal bearing media providing instructions that, when executed by, for example, a processor, may provide the functionality described herein.
  • the computer program products may be provided in any form of one or more machine-readable media.
  • a processor including one or more processor core(s) may undertake one or more features described herein in response to program code and/or instructions or instruction sets conveyed to the processor by one or more machine-readable media.
  • a machine-readable medium may convey software in the form of program code and/or instructions or instruction sets that may cause any of the devices and/or systems described herein to implement at least portions of the features described herein.
  • system 1700 may be a media system although system 1700 is not limited to this context.
  • system 1700 may be incorporated into a personal computer (PC), laptop computer, ultra-laptop computer, tablet, touch pad, portable computer, handheld computer, palmtop computer, personal digital assistant (PDA), cellular telephone, combination cellular telephone/PDA, television, smart device (e.g., smart phone, smart tablet or smart television), mobile internet device (MID), messaging device, data communication device, cameras (e.g. point-and-shoot cameras, super-zoom cameras, digital single-lens reflex (DSLR) cameras), and so forth.
  • system 1700 includes a platform 1702 coupled to a display 1720.
  • Platform 1702 may receive content from a content device such as content services device(s) 1730 or content delivery device(s) 1740 or other similar content sources.
  • a navigation controller 1750 including one or more navigation features may be used to interact with, for example, platform 1702 and/or display 1720. Each of these components is described in greater detail below.
  • platform 1702 may include any combination of a chipset 1705, processor 1710, memory 1712, antenna 1713, storage 1714, graphics subsystem 1715, applications 1716 and/or radio 1718.
  • Chipset 1705 may provide intercommunication among processor 1710, memory 1712, storage 1714, graphics subsystem 1715, applications 1716 and/or radio 1718.
  • chipset 1705 may include a storage adapter (not depicted) capable of providing intercommunication with storage 1714.
  • Processor 1710 may be implemented as a Complex Instruction Set Computer (CISC) or Reduced Instruction Set Computer (RISC) processors, x86 instruction set compatible processors, multi-core, or any other microprocessor or central processing unit (CPU). In various implementations, processor 1710 may be dual-core processor(s), dual-core mobile processor(s), and so forth.
  • CISC Complex Instruction Set Computer
  • RISC Reduced Instruction Set Computer
  • processor 1710 may be dual-core processor(s), dual-core mobile processor(s), and so forth.
  • Memory 1712 may be implemented as a volatile memory device such as, but not limited to, a Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), or Static RAM (SRAM).
  • RAM Random Access Memory
  • DRAM Dynamic Random Access Memory
  • SRAM Static RAM
  • Storage 1714 may be implemented as a non-volatile storage device such as, but not limited to, a magnetic disk drive, optical disk drive, tape drive, an internal storage device, an attached storage device, flash memory, battery backed-up SDRAM (synchronous DRAM), and/or a network accessible storage device.
  • storage 1714 may include technology to increase the storage performance enhanced protection for valuable digital media when multiple hard drives are included, for example.
  • Graphics subsystem 1715 may perform processing of images such as still or video for display.
  • Graphics subsystem 1715 may be a graphics processing unit (GPU) or a visual processing unit (VPU), for example.
  • An analog or digital interface may be used to communicatively couple graphics subsystem 1715 and display 1720.
  • the interface may be any of a High-Definition Multimedia Interface, DisplayPort, wireless HDMI, and/or wireless HD compliant techniques.
  • Graphics subsystem 1715 may be integrated into processor 1710 or chipset 1705.
  • graphics subsystem 1715 may be a stand-alone device communicatively coupled to chipset 1705.
  • graphics and/or video processing techniques described herein may be implemented in various hardware architectures.
  • graphics and/or video functionality may be integrated within a chipset.
  • a discrete graphics and/or video processor may be used.
  • the graphics and/or video functions may be provided by a general purpose processor, including a multi-core processor.
  • the functions may be implemented in a consumer electronics device.
  • Radio 1718 may include one or more radios capable of transmitting and receiving signals using various suitable wireless communications techniques. Such techniques may involve communications across one or more wireless networks.
  • Example wireless networks include (but are not limited to) wireless local area networks (WLANs), wireless personal area networks (WPANs), wireless metropolitan area network (WMANs), cellular networks, and satellite networks. In communicating across such networks, radio 1718 may operate in accordance with one or more applicable standards in any version.
  • display 1720 may include any television type monitor or display.
  • Display 1720 may include, for example, a computer display screen, touch screen display, video monitor, television-like device, and/or a television.
  • Display 1720 may be digital and/or analog.
  • display 1720 may be a holographic display.
  • display 1720 may be a transparent surface that may receive a visual projection.
  • projections may convey various forms of information, images, and/or objects.
  • such projections may be a visual overlay for a mobile augmented reality (MAR) application.
  • MAR mobile augmented reality
  • platform 1702 Under the control of one or more software applications 1716, platform 1702 may display user interface 1722 on display 1720.
  • MAR mobile augmented reality
  • content services device(s) 1730 may be hosted by any national, international and/or independent service and thus accessible to platform 1702 via the Internet, for example.
  • Content services device(s) 1730 may be coupled to platform 1702 and/or to display 1720.
  • Platform 1702 and/or content services device(s) 1730 may be coupled to a network 1760 to communicate (e.g., send and/or receive) media information to and from network 1760.
  • Content delivery device(s) 1740 also may be coupled to platform 1702 and/or to display 1720.
  • content services device(s) 1730 may include a cable television box, personal computer, network, telephone, Internet enabled devices or appliance capable of delivering digital information and/or content, and any other similar device capable of unidirectionally or bidirectionally communicating content between content providers and platform 1702 and/display 1720, via network 1760 or directly. It will be appreciated that the content may be communicated unidirectionally and/or
  • Examples of content may include any media information including, for example, video, music, medical and gaming information, and so forth.
  • Content services device(s) 1730 may receive content such as cable television programming including media information, digital information, and/or other content.
  • content providers may include any cable or satellite television or radio or Internet content providers. The provided examples are not meant to limit implementations in accordance with the present disclosure in any way.
  • platform 1702 may receive control signals from navigation controller 1750 having one or more navigation features.
  • the navigation features of controller 1750 may be used to interact with user interface 1722, for example.
  • navigation controller 1750 may be a pointing device that may be a computer hardware component (specifically, a human interface device) that allows a user to input spatial (e.g., continuous and multi-dimensional) data into a computer.
  • GUI graphical user interfaces
  • televisions and monitors allow the user to control and provide data to the computer or television using physical gestures.
  • Movements of the navigation features of controller 1750 may be replicated on a display (e.g., display 1720) by movements of a pointer, cursor, focus ring, or other visual indicators displayed on the display.
  • a display e.g., display 1720
  • the navigation features located on navigation controller 1750 may be mapped to virtual navigation features displayed on user interface 1722, for example.
  • controller 1750 may not be a separate component but may be integrated into platform 1702 and/or display 1720. The present disclosure, however, is not limited to the elements or in the context shown or described herein.
  • drivers may include technology to enable users to instantly turn on and off platform 1702 like a television with the touch of a button after initial boot-up, when enabled, for example.
  • Program logic may allow platform 1702 to stream content to media adaptors or other content services device(s) 1730 or content delivery device(s) 1740 even when the platform is turned "off.”
  • chipset 1705 may include hardware and/or software support for 5.1 surround sound audio and/or high definition 7.1 surround sound audio, for example.
  • Drivers may include a graphics driver for integrated graphics platforms.
  • the graphics driver may comprise a peripheral component interconnect (PCI) Express graphics card.
  • PCI peripheral component interconnect
  • any one or more of the components shown in system 1700 may be integrated.
  • platform 1702 and content services device(s) 1730 may be integrated, or platform 1702 and content delivery device(s) 1740 may be integrated, or platform 1702, content services device(s) 1730, and content delivery device(s) 1740 may be integrated, for example.
  • platform 1702 and display 1720 may be an integrated unit. Display 1720 and content service device(s) 1730 may be integrated, or display 1720 and content delivery device(s) 1740 may be integrated, for example. These examples are not meant to limit the present disclosure.
  • system 1700 may be implemented as a wireless system, a wired system, or a combination of both.
  • system 1700 may include components and interfaces suitable for communicating over a wireless shared media, such as one or more antennas, transmitters, receivers, transceivers, amplifiers, filters, control logic, and so forth.
  • a wireless shared media may include portions of a wireless spectrum, such as the RF spectrum and so forth.
  • system 1700 may include components and interfaces suitable for communicating over wired communications media, such as input/output (I/O) adapters, physical connectors to connect the I/O adapter with a corresponding wired communications medium, a network interface card (NIC), disc controller, video controller, audio controller, and the like.
  • wired communications media may include a wire, cable, metal leads, printed circuit board (PCB), backplane, switch fabric,
  • Platform 1702 may establish one or more logical or physical channels to
  • the information may include media information and control information.
  • Media information may refer to any data representing content meant for a user. Examples of content may include, for example, data from a voice conversation, videoconference, streaming video, electronic mail ("email") message, voice mail message, alphanumeric symbols, graphics, image, video, text and so forth. Data from a voice conversation may be, for example, speech information, silence periods, background noise, comfort noise, tones and so forth.
  • Control information may refer to any data representing commands, instructions or control words meant for an automated system. For example, control information may be used to route media information through a system, or instruct a node to process the media information in a predetermined manner. The embodiments, however, are not limited to the elements or in the context shown or described in FIG. 17.
  • system 1700 may be embodied in varying physical styles or form factors.
  • FIG. 18 illustrates implementations of a small form factor device 1800 in which system 1800 may be embodied.
  • device 1800 may be implemented as a mobile computing device a having wireless capabilities.
  • a mobile computing device may refer to any device having a processing system and a mobile power source or supply, such as one or more batteries, for example.
  • examples of a mobile computing device may include a personal computer (PC), laptop computer, ultra-laptop computer, tablet, touch pad, portable computer, handheld computer, palmtop computer, personal digital assistant (PDA), cellular telephone, combination cellular telephone/PDA, television, smart device (e.g., smart phone, smart tablet or smart television), mobile internet device (MID), messaging device, data communication device, cameras (e.g. point-and-shoot cameras, super-zoom cameras, digital single-lens reflex (DSLR) cameras), and so forth.
  • PC personal computer
  • laptop computer ultra-laptop computer
  • tablet touch pad
  • portable computer handheld computer
  • palmtop computer personal digital assistant
  • MID mobile internet device
  • Examples of a mobile computing device also may include computers that are arranged to be worn by a person, such as a wrist computer, finger computer, ring computer, eyeglass computer, belt-clip computer, arm-band computer, shoe computers, clothing computers, and other wearable computers.
  • a mobile computing device may be implemented as a smart phone capable of executing computer applications, as well as voice communications and/or data communications.
  • voice communications and/or data communications may be described with a mobile computing device implemented as a smart phone by way of example, it may be appreciated that other embodiments may be implemented using other wireless mobile computing devices as well. The embodiments are not limited in this context.
  • device 1800 may include a housing 1802, a display 1804, an input/output (I/O) device 1806, and an antenna 1808.
  • Device 1800 also may include navigation features 1812.
  • Display 1804 may include any suitable display unit for displaying information appropriate for a mobile computing device.
  • I/O device 1806 may include any suitable I/O device for entering information into a mobile computing device. Examples for I/O device 1806 may include an alphanumeric keyboard, a numeric keypad, a touch pad, input keys, buttons, switches, rocker switches, microphones, speakers, voice recognition device and software, and so forth. Information also may be entered into device 1800 by way of microphone (not shown). Such information may be digitized by a voice recognition device (not shown). The embodiments are not limited in this context.
  • Various embodiments may be implemented using hardware elements, software elements, or a combination of both.
  • hardware elements may include processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so forth), integrated circuits, application specific integrated circuits (ASIC), programmable logic devices (PLD), digital signal processors (DSP), field programmable gate array (FPGA), logic gates, registers, semiconductor device, chips, microchips, chip sets, and so forth.
  • Examples of software may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application program interfaces (API), instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. Determining whether an embodiment is implemented using hardware elements and/or software elements may vary in accordance with any number of factors, such as desired computational rate, power levels, heat tolerances, processing cycle budget, input data rates, output data rates, memory resources, data bus speeds and other design or performance constraints. One or more aspects of at least one embodiment may be implemented by
  • IP cores may be stored on a tangible, machine readable medium and supplied to various customers or manufacturing facilities to load into the fabrication machines that actually make the logic or processor.
  • systems, apparatus, articles, and methods are described herein related to content adaptive entropy coding for video systems.
  • systems, apparatus, articles, and methods are described herein related to content adaptive entropy coding for coded / not coded video data (e.g., data indicating whether blocks or partitions or the like include non-zero transform coefficients) for video systems.
  • a selected entropy coding technique may be determined for coded/not-coded video data from a plurality of entropy coding techniques.
  • the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique
  • the plurality of entropy coding techniques may include two or more proxy variable length coding techniques implementing different variable length codes for different sequences (e.g., via proxy variable length coding tables)
  • the plurality of entropy coding techniques may include two or more symbol-run coding techniques implementing different codes (e.g., via symbol-run coding tables), or the like.
  • the selected entropy coding technique may be based on evaluations of various processed bitstreams associated with the coded/not-coded video data.
  • the processed bitstream(s) may include a pass-through bitstream (e.g., a bitstream that is the coded/not-coded video data), a reversed bitstream (e.g., a reversal of the coded/not-coded video data), an inverted bitstream (e.g., a bit inversion of the coded/not-coded video data), or a differential bitstream (e.g., a bit difference of the coded/not-coded video data).
  • a pass-through bitstream e.g., a bitstream that is the coded/not-coded video data
  • a reversed bitstream e.g., a reversal of the coded/not-coded video data
  • an inverted bitstream e.g., a bit inversion of the coded/not-coded video data
  • the various processed bitstreams may be evaluated based on various entropy coding techniques (e.g., proxy variable length coding technique(s) and symbol-run coding technique(s) as discussed) to determine a processed bitstream and entropy coding technique having a minimum bit cost (e.g., the fewest bits of the evaluated processed bitstreams and entropy coding techniques).
  • the processed bitstream may be entropy encoded to generate an encoded coded/not-coded video data bitstream and the encoded coded/not-coded video data bitstream may be assembled to generate an output bitstream.
  • the output bitstream may also include a coded/not-coded video data header indicating the selected entropy coding technique and the type of the processed bitstream (e.g., pass-through, reversed, inverted, differential, or the like).
  • a coded/not-coded video data header indicating the selected entropy coding technique and the type of the processed bitstream (e.g., pass-through, reversed, inverted, differential, or the like).
  • encoder module 404 may include symbol-run VLC and/or proxy VLC encoder for coded/not-coded data 641 (please refer to FIG. 6).
  • Encoder module 404 may receive video data 414, which may include coded/not-coded symbol bits as is discussed further herein, via symbol-run VLC and/or proxy VLC encoder for coded/not- coded data 641.
  • Symbol-run VLC and/or proxy VLC encoder for coded/not-coded data 641 may determine an entropy encoding technique for video data 414 based on a parameter, parameters or characteristics of video data 414 or other system parameters.
  • the entropy encoding technique for video data 414 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, or multiple adaptive symbol-run variable length coding techniques or multiple adaptive proxy variable length coding technique, or combinations thereof as described herein.
  • the determined entropy encoding technique may be applied to video data 414 to generate compressed video data 424.
  • encoder module 404 may determine which technique provides the greatest compression efficiency that the parameter(s) associated with video data 414 may be the number of bits needed for each coding technique or the like.
  • the parameter associated with video data 314 may be based on a characteristic of the video data such that the determined encoding technique may be adaptive to the received video data.
  • next generation video coding may use 3 picture types, I-, P-, and F- pictures.
  • I- and P-pictures while they may have differences in details of prediction and coding, may be functionally similar.
  • inter-partitions e.g., as referred herein to inter-blocks
  • inter-partitions of P- or F- pictures after transform and quantization (please refer to FIG. 1), may result in either any non-zero quantized transform coefficients that may need to be sent to decoder or all zero quantized transform coefficients (e.g., no non-zero quantized transform coefficients). If a block (e.g., partition) has any non-zero transform coefficients, the block may be classified as coded.
  • a block e.g., partition
  • This coded/not-coded (e.g., CNC) data may be communicated to a decoder via output bitstream 1 11 or the like.
  • the coded / not-coded data may be substantial due to the use of variable block size partitioning that may result in lots of small blocks, particularly at higher bitrates.
  • FIG. 19 is an illustrative diagram of an example picture 1900 of coded/not-coded inter-blocks, arranged in accordance with at least some implementations of the present disclosure.
  • picture 1900 may include tiles 1910, which may include one or more blocks, partitions, sub-partitions, or the like, as discussed herein.
  • the term block may be used for the sake of clarity; however, it is to be understood the term block may be representative of a prediction partition or sub-partition or the like, which may be substantially any size or shape.
  • picture 1900 may be a P-picture having a tile size of 32x32 for coding using a quantizer value (e.g., quantization parameter, Qp) of 8.
  • Qp quantization parameter
  • a quantizer value may determine a step size in the quantization of transform coefficients (e.g., by adaptive quantize module 109, please see FIG. 1).
  • coded inter-blocks 1920 are illustrated in light gray and not-coded inter-blocks 1930 are illustrated in dark gray.
  • Blocks which are illustrated in white may be other types of blocks (e.g., intra, skip, or auto blocks) such that inter-prediction does not apply to such blocks. The prediction of such blocks may not be relevant to the present discussion.
  • the partitioning for coding may result in a significant number of coded inter-blocks 1920 and not-coded inter- blocks 1930.
  • coded/not-coded inter- blocks 1920, 1930 may be assigned one bit.
  • the coded / not coded (e.g., CNC) data may result in a substantial bitstream of data.
  • FIG. 20 is an illustrative diagram of an example picture 2000 of coded/not-coded inter-blocks, arranged in accordance with at least some implementations of the present disclosure.
  • picture 2000 may include tiles 2010, which may include one or more blocks, partitions, prediction partitions, sub-partitions, or the like, as discussed herein.
  • picture 2000 may be an F-picture having a tile size of 64x64 for coding using a quantizer value of 32.
  • coded inter-blocks 2020 are illustrated in light gray and not-coded inter-blocks 2030 are illustrated in dark gray.
  • FIG. 20 is an illustrative diagram of an example picture 2000 of coded/not-coded inter-blocks, arranged in accordance with at least some implementations of the present disclosure.
  • picture 2000 may include tiles 2010, which may include one or more blocks, partitions, prediction partitions, sub-partitions, or the like, as discussed herein.
  • picture 2000 may be an F-picture having a tile size of 64x64 for coding using
  • blocks illustrated in white may be other types of blocks (e.g., intra, skip, or auto blocks) such that inter-prediction does not apply to such blocks and they may not be relevant to the present discussion.
  • the partitioning for coding may result in a smaller number of coded inter-blocks 2020 and not-coded inter- blocks 2030.
  • F-pictures may have a lesser density of coded/not-coded inter-blocks as compared with P-pictures.
  • each block (e.g. of coded/not-coded blocks 2020, 2030) may be assigned one bit.
  • entropy coding techniques discussed herein may be used to encode the coded/not-coded data.
  • an adaptive symbol-run variable coding technique may be used.
  • an adaptive proxy variable length coding technique may be used. Such techniques have been described herein and will not be repeated for the sake of brevity.
  • various techniques discussed with respect to adaptive symbol-run variable coding and/or adaptive proxy variable length coding as discussed elsewhere herein may be used with respect to encoder module 404, sub-modules, and the various methods and modes discussed herein.
  • the described entropy coding techniques may each be modified based on the codes, symbols, or the like used in the entropy coding implementation.
  • Such implementations will be described further herein with respect to symbol run variable length coding tables and proxy variable length coding tables such as Tables 3-12 below.
  • a selected entropy coding technique may operate on a processed bitstream associated with the coded/not-coded video data.
  • a frame of video data may have a block (or partition) structure.
  • picture 1910 may have a block structure as shown via coded/not-coded blocks 1920, 1930.
  • the block structure may include the partitioning of picture 1910 into blocks (or partitions).
  • the block structure may be substantially complicated and may be adaptive to a variety of characteristics and/or parameters of picture 1910, adjacent pictures (or frames), coding settings, or the like.
  • a block structure associated with a video frame may be read (e.g., by adaptive entropy encoder 110 or a module therein).
  • the block structure may be associated with the coded/not-coded video data such that the bits of the coded/not-coded video data indicate a 1 for blocks of the block structure that are coded (e.g., have non-zero transform coefficients) and a 0 for blocks of the block structure that are not-coded (e.g., have no non-zero transform coefficients).
  • the coded/not-coded video data may subsequently be processed.
  • a 1 -dimensional raster scan may be generated based on the coded/not-coded video data and a 1 -dimensional block-based scan may be generated based on the coded/not- coded video data.
  • the 1 -dimensional raster scan and the 1 -dimensional block-based scan may be considered processed bitstreams associated with the coded/not- coded video data.
  • the 1 -dimensional raster scan or the 1 -dimensional block-based scan or both may be further processed by performing bit inversions, reversals, bit differentials, or the like. Such further processed bitstreams may also be considered processed bitstreams as described herein.
  • a processed bitstream associated with the coded/not-coded video data may be entropy encoded using a selected entropy coding technique (e.g., including selected codes, symbols, or the like as described herein using tables).
  • the selected entropy coding technique may be chosen based on evaluating multiple entropy coding techniques with respect to multiple processed bitstreams to determine the selected entropy coding technique and the relevant (e.g., selected) processed bitstream.
  • FIG. 21 is an illustrative diagram of an example coded/not-coded data encoder module 404, arranged in accordance with at least some implementations of the present disclosure.
  • coded/not-coded data encoder module 404 which may be described as encoder module 404 herein for the sake of brevity, may include symbol-run VLC and/or proxy VLC encoder for coded/not-coded data 641, as discussed with respect to FIG. 6.
  • coded/not-coded data encoder module 404 may not include symbol-run VLC and/or proxy VLC encoder for coded/not-coded data 641.
  • coded/not-coded data encoder module 404 may provide efficiency by incorporating a number of adaptive techniques. Further, coded/not-coded data encoder module 404 may allow coding efficiency while providing substantially low complexity.
  • coded/not-coded data encoder module 404 may include a coded/not-coded data processing module 2101, a symbol-run generation and variable length coding (VLC) lookup table (LUT) module 211 1, a proxy variable length coding (VLC) lookup table (LUT) module 2120, a coded/not-coded variable length coding (VLC) encoder module 2130, and a minimum bits selector 2140.
  • symbol-run generation and VLC LUT module 21 11 may include multiple symbol-run generation and VLC LUT modules 21 12, 21 13, 21 14.
  • Symbol-run generation and VLC LUT module 21 11 may include any number of modules such as modules corresponding to the symbol-run VLC tables discussed herein or the like.
  • proxy VLC LUT module 2121 may include multiple proxy VLC LUT modules 2122, 2123, 2124.
  • Proxy VLC LUT module 211 1 may include any number of modules such as modules corresponding to the proxy VLC tables discussed herein or the like.
  • coded/not-coded data processing module 2101 may include a bits pass-through module 2102, a bits rescanner/reverser module 2103, a bits inverter module 2104, and a bits differencer module 2105.
  • coded/not-coded data processing module 2101 and coded/not-coded VLC encoder module 2130 may receive video data 414.
  • Video data 414 may include any suitable coded/not-coded video data (e.g., including coded/not symb bits as shown).
  • video data 414 may include coded/not- coded video data associated with a block structure of a video frame or picture.
  • video data 414 may include a representation of a 2-dimensional description of various size blocks (or partitions) that may be coded/not-coded (e.g., 1 may indicate coded and 0 may indicate not-coded).
  • Video data 414 may be received and processed by coded/not-coded data processing module 2101.
  • coded/not-coded data processing module 2101 may process 414 via one or more of modules 2102, 2103, 2104, 2105.
  • coded/not- coded data processing module 2101 may process video data 414 via all of modules 2102, 2103, 2104, 2105 in parallel.
  • coded/not-coded data processing module 2101 may include bits pass-through module 2102, bits rescanner/reverser module 2103, bits inverter module 2104, and/or a bits differencer module 2105.
  • bits pass- through module 2102 may pass-through one or more bitstreams associated with coded/not- coded video data of video data 414.
  • the coded/not-coded data may be processed or scanned to produce one or more scans associated with the coded/not-coded video data.
  • the coded/not-coded video data may be processed to generate a 1- dimensional raster scan, a 1 -dimensional block-based scan, a 1 -dimensional tile-based scan, or the like.
  • generating a 1 -dimensional raster scan may include a frame or picture based scan that processes from the top-left of the frame to the bottom-right of the frame and generates a 1 bit for each coded block and a 0 bit for each not-coded block (or vice versa).
  • the scan may follow a pattern from top-left to bottom-right by moving horizontally from left to right at a top row and then dropping down one row and repeating the left to right scans vertically down the picture.
  • the size and/or shape of the block may be ignored such that each size and shape of block may be given a 1 or 0 as described.
  • blocks may be processed when the block's top-left corner is scanned and, if the block's edge is scanned in subsequent horizontal scans, the block may be skipped as previously scanned.
  • Other portions of the described codec may indicate the size, shape, and location of each block (e.g., via partitioning and the like) such that that information may not be needed in the coded/not-coded data scan.
  • generating a 1 -dimensional tile-based scan may include performing a scan (e.g. a scan from top-left to bottom-right as discussed) within a top-left tile and then moving to the next tile and repeating the scan within each tile.
  • the ordering of the tiles may also follow a pattern from top-left to bottom-right of the frame or picture by moving horizontally from left to right and then dropping down one row and repeating the left to right scans progressing vertically down the picture until the picture is complete.
  • generating a 1 -dimensional block-based scan may include performing a scan (again from top-left to bottom-right as discussed) within a block (not corresponding to the blocks being coded), such that the blocks are portions of a tile. For example, a tile may be divided in half vertically and then each half may be quartered horizontally to define 8 8x16 blocks that may each be scanned.
  • the scan may be performed from top-left to bottom-right within a block, then within a tile, and then within a picture.
  • a tile prior to the dividing steps described, a tile may be flipped vertically about a vertical centerline prior to dividing vertically into halves and quartering each half horizontally.
  • other block sizes and/or divisions of tiles may be used.
  • Bits pass-through module 2102 may pass-through the generated or processed bitstream, for example, without further processing.
  • the generated or processed bitstream may be generated by coded/not-coded data processing module 2101 or another module of coded/not-coded data encoder module 404.
  • bits rescanner/reverser module 2103 may reverse the order of the scan or bitstream to generate a reversed scan such as, for example, 1 -dimensional reversed raster scan, a 1 -dimensional reversed block-based scan, a 1 -dimensional reversed tile- based scan, or the like.
  • bits inverter module 2104 may invert the bits (e.g., bit invert) of the scan or bitstream (e.g., change Os to Is and vice versa) to generate an inverted bits scan such as, for example, a 1 -dimensional inverted bits raster scan, a 1 -dimensional inverted bits block-based scan, a 1 -dimensional inverted bits tile- based scan, or the like.
  • bits inverter module 2104 may invert the bits (e.g., bit invert) of the scan or bitstream (e.g., change Os to Is and vice versa) to generate an inverted bits scan such as, for example, a 1 -dimensional inverted bits raster scan, a 1 -dimensional inverted bits block-based scan, a 1 -dimensional inverted bits tile- based scan, or the like.
  • bits differencer module 2105 may difference the bits (e.g., bit difference) of the scan or bitstream (e.g., take a difference along sequential bits) to generate a differential bits scan such as, for example, a 1- dimensional differential bits raster scan, a 1 -dimensional differential bits block-based scan, a 1 -dimensional differential bits tile-based scan, or the like. Further, in some examples, bitstreams or scans may be passed between modules 2102, 2103, 2104, 2105.
  • an inverted scan generated by bits inverter module 2104 may be operated on by bits differencer module 2105 to generate an inverted differential scan such as, for example, a 1- dimensional inverted differential bits raster scan, a 1 -dimensional inverted differential bits block-based scan, a 1 -dimensional inverted differential bits tile-based scan, or the like.
  • one or more or all of the generated processed bitstreams as discussed may be transmitted to one or both of symbol-run generation and VLC LUT module 21 1 1 and/or proxy VLC LUT module 2121.
  • the same processed bitstreams may be transmitted to both-run generation and VLC LUT module 211 1 and/or proxy VLC LUT module 2121.
  • different processed bitstreams may be transmitted to symbol-run generation and VLC LUT module 211 1 and/or proxy VLC LUT module 2121.
  • the processed bitstream(s) received by symbol-run generation and VLC LUT module 211 1 may be processed to determine bit cost(s) associated with coding the processed bitstream(s) using symbol-run coding with one or more symbol / code combinations. For example, a bit cost may be determined for a processed bitstream using a first and second or a first, second, and third (or the like) symbol-run coding LUT. For example, symbol-run generation and VLC LUT module 211 1 may allow for the counting of the bit cost(s) of events (e.g., symbol-run combination(s)).
  • the processed bitstream(s) received by proxy VLC LUT module 2121 may be processed to determine bit cost(s) associated with coding the processed bitstream(s) using symbol-run coding with one or more symbol / code combinations. For example, a bit cost may be determined for a processed bitstream using one, a first and second, or a first, second, and third (or the like) proxy VLC LUT.
  • proxy VLC LUT module 2121 may allow for the counting of bits by trying any number of different proxy VLC codes that may replace groups of bits/patterns with shorter codes.
  • the processing by symbol-run generation and VLC LUT module 211 1 and/or proxy VLC LUT module 2121 may be performed via a bit counting by look up or the like (e.g., actual encoding and bit counting may not be required). Such processing may substantially reduce the complexity of encoder module 404, for example.
  • bit costs may thereby be determined based on combinations of processed bitstream(s), encoding technique (e.g., symbol-run or proxy VLC), and encoding technique implementation (e.g., look up table for the pertinent encoding technique).
  • encoding technique e.g., symbol-run or proxy VLC
  • encoding technique implementation e.g., look up table for the pertinent encoding technique.
  • example bit costs are provided for P-pictures and F-pictures, respectively, herein.
  • P-pictures and F-pictures respectively, herein.
  • the determined bit costs may be transmitted to minimum bits selector 2140 and/or coded/not-coded VLC encoder module 2130.
  • Minimum bits selector 2140 and/or coded/not-coded VLC encoder module 2130.
  • Minimum bits selector 2140 may determine the minimum bit cost based on the received bit cost to determine the selected entropy coding technique and the selected processed bitstream. Minimum bits selector 2140 may transmit the selected entropy coding technique and the selected processed bitstream to coded/not-coded VLC encoder module 2130. For example, minimum bits selector 2140 may determine a best solution (e.g., fewest number of bits) for representing the coded/not-coded data for a current frame. For example, the output of minimum bits selector 2140 may be an index to the entropy coding technique and table that produces the highest compression.
  • Coded/not-coded VLC encoder module 2130 may generate the selected processed bitstream associated with the coded/not-coded video data, as discussed, and entropy encode the processed bitstream using the selected entropy coding technique to generate an encoded coded/not-coded video data bitstream, which may be a part of or implemented as compressed video data 324.
  • coded/not-coded VLC encoder module 2130 may use the index provided via minimum bits selector 2140 to access the selected entropy coding technique and table to generate a compressed representation of the coded/not-coded video data as, for example compressed video data 424.
  • compressed video data 424 may be transmitted to bitstream assembler 408, which may assemble compressed video data and other compressed video data (e.g., compressed video data 401- 403 and 405-407) to generate output bitstream 11 1.
  • an original stream e.g., an original scan such as a 1 -dimensional raster scan, a 1 -dimensional block-based scan, or a 1- dimensional tile-based scan
  • further processed stream(s) to determine a solution that is most efficient in compression (e.g., a minimum bit cost).
  • FIG. 22 is a flow diagram illustrating an example process 2200, arranged in accordance with at least some implementations of the present disclosure.
  • Process 2200 may include one or more operations, functions or actions as illustrated by one or more of operations 2202, 2204, and/or 2206.
  • Process 2200 may form at least part of a next generation video coding process.
  • process 2200 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or entropy encoder module 1 10 as discussed herein.
  • process 2200 may include reading a block structure associated with a frame of video data as discussed with respect to FIGS. 19 and 20.
  • the block structure may be associated with coded/not-coded video data as discussed such that the coded/not-coded video data has a bit associated with each block of the block structure indicting whether the block is coded (e.g., is associated with a bit of 1) or not-coded (e.g., is associated with a bit of 0).
  • Process 2200 may begin (or continue from reading the block structure and associating the block structure with coded/not-coded data) at operation 2202, "Determine a Selected Entropy Coding Technique for Coded/Not-Coded Video Data", where a selected entropy coding technique may be determined for coded/not-coded video data from multiple entropy coding techniques as discussed herein.
  • the entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, a single proxy variable length coding technique and multiple symbol-run coding techniques (e.g., implemented via different tables), multiple proxy variable length coding techniques (e.g., implemented via different tables) and a single symbol-run coding technique, multiple proxy variable length coding techniques, or multiple symbol-run coding techniques, or the like.
  • the selected entropy coding technique may be determined based on estimating bit costs for the multiple entropy coding techniques and with various processed bitstreams (e.g., a pass-through of the coded/not-coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, 1 -dimensional scans, bit inversions, bit differences, bits reversals, or the like), as discussed herein, and selecting the entropy coding technique and processed bitstream having the lowest bit cost.
  • the selected entropy coding technique may be determined via coded/not-coded data encoder module 304.
  • Process 2200 may continue at operation 2204, "Entropy Encode a Processed Bitstream Associated with the Coded/Not-Coded Video Data Using the Selected Entropy Coding Technique", where a processed bitstream associated with the coded/not-coded video data may be entropy encoded using the selected entropy coding technique to generate an encoded coded/not-coded video data bitstream.
  • the processed bitstream may be a selected bitstream such that the selected bitstream and selected entropy coding technique provided the lowest bit cost.
  • the processed bitstream may be entropy encoded via coded/not-coded VLC encoder module 2130.
  • Process 2200 may continue at operation 2206, "Assemble the Encoded Coded/Not- Coded Video into an Output Bitstream", where the encoded coded/not-coded video data may be assembled into an output bitstream.
  • bitstream assembler 308 may assemble the encoded coded/not-coded video data (e.g., compressed video data 424) and other compressed video data (e.g., compressed video data 421-423 and 425-427) into output bitstream 11 1.
  • a coded/not-coded video data header associated with the selected entropy coding technique and/or the selected processed bitstream may be assembled into the output bitstream.
  • compressed video data 424 may include the coded/not-coded video data header, which may be assembled as discussed into output bitstream 11 1.
  • output bitstream 1 11 may be transmitted via encoder system 100 as implemented via video coding system 1100 or the like to one or more video decoders.
  • FIG. 23 is a flow diagram illustrating an example process 2300, arranged in accordance with at least some implementations of the present disclosure.
  • Process 2300 may include one or more operations, functions or actions as illustrated by one or more of operations 2302, 2304, 2306, and/or 230.
  • Process 2300 may form at least part of a next generation video coding process.
  • process 2300 may form at least part of a next generation video encoding process as undertaken by decoder system 200 of FIG. 2 and/or entropy decoder module 202 as discussed herein.
  • Process 2300 may begin at operation 2302, "Receive an Entropy Encoded
  • Bitstream where an entropy encoded bitstream may be received.
  • output bitstream 1 1 1, input bitstream 201, or bitstream 800 or the like may be received via video decoder 200 as implemented via video coding system 1 100 or the like.
  • Process 2300 may continue at operation 2304, "Disassemble the Entropy Encoded Bitstream to Determine a Coded/Not-Coded Video Data Header and an Encoded
  • Coded/Not-Coded Video Data Bitstream where the entropy encoded bitstream may be disassembled to determine a coded/not-coded video data header and encoded coded/not- coded video data.
  • the entropy encoded bitstream may be disassembled via bitstream disassembler 408.
  • Process 2300 may continue at operation 2306, "Determine a Selected Entropy Coding Technique Based on the Coded/Not-Coded Video Data Header", where a selected entropy coding technique may be determined for the encoded coded/not-coded video data from multiple entropy coding techniques based at least in part on the coded/not-coded video data header.
  • the coded/not-coded video data header may include a code indicating a selected entropy coding technique (and/or selected table for the entropy coding technique) for the encoded coded/not-coded video data and a format or type of the bitstream of the encoded coded/not-coded video data.
  • the selected entropy coding technique may include any technique as discussed herein (e.g., symbol-run coding technique(s) as implemented via various tables and/or proxy variable length coding technique(s) as implemented via various tables) and the bitstream may include any bitstream type or processing as discussed herein (e.g., 1 -dimensional scans, bit inversions, bit differences, bits reversals, or the like) encoded or compressed via the selected entropy coding technique.
  • any technique as discussed herein e.g., symbol-run coding technique(s) as implemented via various tables and/or proxy variable length coding technique(s) as implemented via various tables
  • the bitstream may include any bitstream type or processing as discussed herein (e.g., 1 -dimensional scans, bit inversions, bit differences, bits reversals, or the like) encoded or compressed via the selected entropy coding technique.
  • Process 2300 may continue at operation 2308, "Entropy Decode the Encoded Coded/Not-Coded Video Data Bitstream Using the Selected Entropy Coding Technique to Determine Coded/Not-Coded Video Data", where the encoded coded/not-coded video data may be entropy decoded using the selected entropy coding technique to determine coded/not-coded video data.
  • the encoded coded/not-coded video data may be decompressed to using the selected entropy coding technique to generate a processed bitstream.
  • the processed bitstream may include any processed bitstream (e.g., 1 -dimensional scans, bit inversions, bit differences, bits reversals, or the like) as discussed herein.
  • the processed bitstream may be decoded via the inverse of the operation used to generate the processed bitstream (e.g., by pass-through, bit inversion, bit reversal, or bit differential as discussed) to generate the coded/not-coded video data.
  • Process 2300 may continue at operation 2310, "Generate a Video Frame Based on the Coded/Not-Coded Video Data", where the coded/not-coded video data may be used to generate a video frame as discussed herein with respect to decoder system 200. Further, the generated video frame may be transmitted to a display device for presentment to a viewer or user.
  • multiple combinations of entropy coding techniques and processed bitstreams may be evaluated to determine a minimum bit cost combination.
  • a subset of such combinations may be chosen for evaluation. For example, those combinations likely to produce minimum bit costs or efficient coding may be selected for evaluation.
  • the chosen combinations may be based on the frame type (e.g., F- or P-picture) of the video data.
  • FIG. 24 provides a flow diagram illustrating an example process 2400, arranged in accordance with at least some implementations of the present disclosure.
  • Process A1900 may include one or more operations, functions or actions as illustrated by one or more of operations 2400-2415.
  • Process 2400 may form at least part of a next generation video coding process.
  • process 2400 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or entropy encoder module 1 10 as discussed herein.
  • process 2400 may be applied to P-pictures.
  • Process 2400 may begin at operation 2401 "Input Frame Level CNC Data", where frame-level tile/block structure and coded/not-coded video data (e.g., binary information; CNC data) for inter mode blocks may be read. For example, a block structure associated with a frame of video data may be read.
  • Process 2400 may continue at operation 2402 "Number of Inter Blocks ⁇ Thr (25)?", where, if the number of inter blocks in the frame is less than a threshold value (e.g., 25 in the example), then the CNC data may be coded with a bitmap method (e.g., bit by bit) as shown at operation 2414 "Code CNC Data with a Bitmap Method" and process 2400 may terminate as shown at operation 2415 "Finish Processing Current Frame".
  • a threshold value e.g. 25 in the example
  • operation 2402 may check for a simple case such that determining a compression technique may not be advantageous and the frame may be processed with bitmap coding. If the number of inter blocks in the frame is greater than (or equal to) the threshold, process 2400 may continue at operations 2403 and 2404.
  • a 1 -dimensional raster scan of the coded/not-coded data may be generated.
  • a 1-dimensional block based scan of the coded/not- coded data may be generated.
  • the 1 -dimensional raster scan may include a normal raster scan of blocks (e.g., frame-wise) and the 1-dimensional block based scan may include a 8x16 block-based scan (e.g., a scan of blocks in raster order within the 8x16 areas while processing 8x16 areas in raster order) or an equivalent 2x4 scale scan.
  • Process 2400 may continue at operations 2405 "Invert Bits" and 2406 "Invert Bits", where the respective scans may be bit inverted.
  • the 1- dimensional raster scan may be bit inverted to generate an inverted bits 1-dimensional raster scan.
  • the 1-dimensional block-based scan may be bit inverted to generate an inverted bits 1-dimensional block-based scan.
  • operations 2403-2406 may generated processed bitstreams.
  • Process 2400 may continue at operations 2407 "Estimate Cost of Symbol-Run Coding (Table SO)", 2408 "Estimate Cost of Proxy VLC Coding (Tables V2,V3,V6,V1)", 2409 “Estimate Cost of Proxy VLC Coding (Tables V3 & V6)", and 2410 "Estimate Cost of Symbol-Run Coding (Table S2)", where the bit costs of the illustrated processed bitstream and entropy coding technique combinations may be estimated. As discussed, in some examples, the bit costs estimations may be determined based on a table look up or the like (e.g., and not based on an actual encoding and bit counting of the processed bitstreams).
  • a first bit cost associated with a symbol run coding technique (e.g., using Table SO) of the inverted bits 1 -dimensional raster scan may be estimated.
  • a second, a third, a fourth, and a fifth bit cost associated with multiple proxy variable length coding techniques (e.g., one each using Tables V2, V3, V6, and VI) of the inverted bits 1 -dimensional raster scan may be estimated.
  • a sixth and a seventh bit cost associated with a proxy variable length coding techniques (e.g., one each using Tables V3 and V6) of the 1- dimensional raster scan may be estimated.
  • an eighth bit cost associated with a proxy variable length coding technique (e.g., using Table S2) of the 1 -dimensional raster scan may be estimated.
  • Operations 2407-1910 may provide advantageous coding technique (including coding table) and processed bitstream combinations for P-pictures. In some examples, some of the provided combinations may be eliminated. In some examples, other available combinations may be added. For example, a difference bits operation may be added to process 2400 to provide additional processed bitstreams for bit cost estimation(s).
  • Process 2400 may continue at operation 241 1 "Select Method with the Smallest Cost (get_min_ind)", where a minimum (e.g., smallest) bit cost based on the first through eighth bit costs may be determined (e.g., as illustrated via action “get_min_ind”). For example, the estimated bits for all methods may be compared to determine the one with the smallest cost.
  • Process 2400 may continue at operation 2412 "Code the Selected Method Index (See Table 1)", where a header code associated with the selected entropy coding technique (and table if applicable) may be encoded. For example, Table 1 as follows illustrated example header codes:
  • Process 2400 may continue at operation 2413 "Code CNC Data with the Selected Method", where the processed bitstream associated with the coded/not-coded video data may be entropy encoded using the selected entropy coding technique to generate encoded coded/not-coded video data. Further, as discussed, the encoded coded/not-coded video data bitstream may be assembled into an output bitstream. For example, the selected method may be applied to the CNC data and the result may be written to bitstream. Process 2400 may terminate at operation 2415 "Finish Processing Current Frame". Process 2400 may be repeated as needed to encode any number of frames such as, for example, P-pictures or P- pictures.
  • FIG. 25 provides a flow diagram illustrating an example process 2500, arranged in accordance with at least some implementations of the present disclosure.
  • Process A2000 may include one or more operations, functions or actions as illustrated by one or more of operations 2500-2514.
  • Process 2500 may form at least part of a next generation video coding process.
  • process 2500 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or entropy encoder module 1 10 as discussed herein.
  • process 2500 may be applied to F-pictures.
  • Process 2500 may begin at operation 2501 "Input Frame Level CNC Data", where frame-level tile/block structure and coded/not-coded video data (e.g., binary information; CNC data) for inter mode blocks may be read. For example, a block structure associated with a frame of video data may be read.
  • Process 2500 may continue at operation 2502 "Number of Inter Blocks ⁇ Thr (25)?", where, if the number of inter blocks in the frame is less than a threshold value (e.g., 25 in the example), then the CNC data may be coded with a bitmap method (e.g., bit by bit) as shown at operation 2513 "Code CNC Data with a Bitmap Method” and process 2500 may terminate as shown at operation 2514 "Finish Processing Current Frame".
  • a threshold value e.g. 25 in the example
  • operation 2502 may check for a simple case such that determining a compression technique may not be advantageous and the frame may be processed with bitmap coding. If the number of inter blocks in the frame is greater than (or equal to) the threshold, process 2500 may continue at operations 2503 and 2504.
  • a 1-dimensional tile-based scan of the coded/not-coded data may be generated.
  • a 1-dimensional raster scan of the coded/not-coded data may be generated.
  • the 1-dimensional raster scan may include a normal raster scan of blocks (e.g., frame-wise) and the 1-dimensional tile-based scan may include a scan of blocks in raster order within the tile areas while processing tiles in raster order).
  • Process 2500 may continue at operations 2505 "Invert Bits” and 2506 "Invert Bits", where the respective scans may be bit inverted.
  • the 1- dimensional tile-based scan may be bit inverted to generate an inverted bits 1-dimensional tile-based scan.
  • the 1-dimensional raster scan may be bit inverted to generate an inverted bits 1-dimensional raster scan.
  • Process 2500 may continue at operations 2507 "Difference Bits” and 2508 "Difference Bits”, where the respective scans may be bit differenced.
  • the inverted bits 1-dimensional tile-based scan may be bit differenced to generate an inverted differential bits 1- dimensional tile-based scan.
  • the inverted bits 1- dimensional raster scan may be bit differenced to generate an inverted differential bits 1- dimensional raster scan.
  • operations 2503-2508 may generated processed bitstreams.
  • Process 2500 may continue at operations set 2509, which may include operations "Estimate Cost of Proxy VLC Coding (Tables V3,V8,V5,V10)” (first instance), “Estimate Cost of Symbol-Run Coding (Tables SO & S2)", “Estimate Cost of Proxy VLC Coding (Tables V3 & V5)”, “Estimate Cost of Symbol-Run Coding (Table S2)”, “Estimate Cost of Proxy VLC Coding (Table V8)” (first instance), “Estimate Cost of Proxy VLC Coding (Tables V3,V8,V5,V10)” (second instance), “Estimate Cost of Proxy VLC Coding (Tables V0 & V3)”, “Estimate Cost of Proxy VLC Coding (Table V8)” (second instance), where the bit costs of the illustrated processed bitstream and entropy coding technique combinations may be estimated. As discussed, in some examples, the bit costs estimations may be determined based on a table look up or the like (e.g., and not
  • a first, a second, a third, and a fourth bit cost associated with multiple proxy variable length codings (e.g., one each using Tables V3, V8, V5, and V10) of the inverted bits 1 -dimensional tile-based scan may be estimated.
  • a fifth and a sixth bit cost associated with multiple symbol-run codings e.g., one each using Tables SO and VS2 of the 1 -dimensional tile-based scan may be estimated.
  • a seventh and an eighth bit cost associated with multiple proxy variable length codings (e.g., one each using Tables V3 and V5) of the 1 -dimensional tile-based scan may be estimated.
  • a ninth bit cost associated with a symbol-run coding (e.g., using Table S2) of the inverted differential bits 1 -dimensional tile-based scan may be estimated.
  • a tenth bit cost associated with a proxy variable length coding (e.g., using Table V8) of the inverted differential bits 1 -dimensional tile-based scan may be estimated.
  • an eleventh, a twelfth, a thirteenth, and a fourteenth bit cost associated with multiple proxy variable length codings (e.g., one each using Tables V3, V8, V5, and V10) of the inverted bits 1 -dimensional raster scan may be estimated.
  • a fifteenth and a sixteenth bit cost associated with multiple proxy variable length codings (e.g., one each using Tables V0 and V3) of the 1 -dimensional raster scan may be estimated.
  • a seventeenth bit cost associated with a proxy variable length coding (e.g., using Table V8) of the inverted differential bits 1 -dimensional tile-based scan may be estimated.
  • Operations set 2509 may provide advantageous coding technique (including coding table) and processed bitstream combinations for F-pictures. In some examples, some of the provided combinations may be eliminated. In some examples, other available
  • Process 2500 may continue at operation 2510 "Select Method with the Smallest Cost", where a minimum (e.g., smallest) bit cost based on the first through seventeenth bit costs may be determined. For example, the estimated bits for all methods may be compared to determine the one with the smallest cost.
  • Process 2500 may continue at operation 2511 "Code the Selected Method Index (See Table 2)", where a header code associated with the selected entropy coding technique (and table if applicable) may be encoded. For example, Table 2 as follows illustrated example header codes:
  • Process 2500 may continue at operation 2512 "Code CNC Data with the Selected Method", where the processed bitstream associated with the coded/not-coded video data may be entropy encoded using the selected entropy coding technique to generate encoded coded/not-coded video data. Further, as discussed, the encoded coded/not-coded video data bitstream may be assembled into an output bitstream. For example, the selected method may be applied to the CNC data and the result may be written to bitstream. Process 2500 may terminate at operation 2514 "Finish Processing Current Frame". Process 2500 may be repeated as needed to encode any number of frames such as, for example, P-pictures or P- pictures.
  • the entropy encoding technique(s) may use various tables.
  • the tables as referenced herein are presented as follows and labeled in accordance with the previous discussion and, in particular the discussion presented with respect to FIGS. 24 and 25 and Tables 1 and 2 above.
  • techniques discussed herein may include estimating bit cost(s) of proxy variable length coding and symbol-run coding in various scenarios.
  • estimating bit cost(s) associated with proxy VLC may include a look up operation.
  • estimating bit cost(s) associated with symbol-run coding may implement the process as illustrated in FIG. 26.
  • FIG. 26 is a flow diagram illustrating an example process 2600, arranged in accordance with at least some implementations of the present disclosure.
  • Process 2600 may include one or more operations, functions or actions as illustrated by one or more of operations 2302, 2304, 2306, and/or 230.
  • Process 2600 may form at least part of a next generation video coding process.
  • process 2600 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or entropy decoder module 1 10 as discussed herein.
  • Process 2600 may begin at operation 2601 "Input Binary Mask A and Symbol-Run Table T", where a binary mask (e.g., any processed bitstream as discussed herein) and a symbol-run table (e.g., a symbol-run table as illustrated herein or the like) may be input.
  • process 2600 may continue at operation 261 1 "Return Cost as Estimated Cost of Symbol-Run Coding of A with Table T", where the cost as estimated with respect to symbol-run coding of binary mask A with Table may be returned as running cost c. If not, process 2600 may continue at operation 2603, as discussed above.
  • Process 2600 may continue at operation 2611, where the cost as estimated with respect to symbol-run coding of binary mask A with Table may be returned as running cost c as discussed.
  • process 2600 may provide bit cost(s) estimates associated with symbol-run coding techniques as discussed herein.
  • Both in P-pictures and F-pictures for coding of coded/not-coded data for each of the combinations of operations or bitstream processing (e.g., none, pass-through, scanning, bit inversion, bit differencing, or the like), coding techniques (e.g., symbol-run or proxy VLC), and options within each technique (e.g., multiple tables as described herein), the best combination (e.g., minimum bit cost) that yields the most compression may be selected as the coding method (e.g., entropy coding technique) for the given video data (e.g., frame, picture, or slice, or the like).
  • bitstream processing e.g., none, pass-through, scanning, bit inversion, bit differencing, or the like
  • coding techniques e.g., symbol-run or proxy VLC
  • options within each technique e.g., multiple tables as described herein
  • the best combination e.g., minimum bit cost
  • the coding method e.g., en
  • example video coding system 1600 may include imaging device(s) 1601, video encoder 100 and/or a video encoder implemented via logic circuitry 1650 of processing unit(s) 1620, video decoder 200 and/or a video decoder implemented via logic circuitry 1650 of processing unit(s) 1620, an antenna 1602, one or more processor(s) 1603, one or more memory store(s) 2004, and/or a display device 2005.
  • video encoder 100 implemented via logic circuitry 1650 may include an image buffer (e.g., via either processing unit(s) 1620 or memory store(s) 1604)) and a graphics processing unit (e.g., via processing unit(s) 1620).
  • the graphics processing unit may be communicatively coupled to the image buffer.
  • the graphics processing unit may include video encoder 100 as implemented via logic circuitry 1650 to embody the various modules as discussed with respect to FIG. 1 and FIGS. 4, 6 and A 16.
  • the graphics processing unit may include entropy encoder logic circuitry, and so on.
  • the logic circuitry may be configured to perform the various operations as discussed herein.
  • the entropy encoder logic circuitry may be configured to determine a selected entropy coding technique for coded/not-coded video data from a plurality of entropy coding techniques such that the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, entropy encode a processed bitstream associated with the coded/not-coded video data using the selected entropy coding technique to generate encoded coded/not-coded video data, and assemble the encoded coded/not-coded video data into an output bitstream.
  • Video decoder 200 may be implemented in a similar manner.
  • antenna 1602 of video coding system 1600 may be configured to receive an entropy encoded bitstream of video data.
  • the bitstream may include compressed video data of various types.
  • Video coding system 1600 may also include video decoder 200 coupled to antenna 1602 and configured to decode the encoded bitstream.
  • video decoder 200 may be configured to disassemble the entropy encoded bitstream to determine a coded/not-coded video data header and encoded coded/not-coded video data, determine a selected entropy coding technique for the encoded coded/not-coded video data from a plurality of entropy coding techniques based at least in part on the coded/not-coded video data header such that the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, entropy decode the encoded coded/not-coded video data using the selected entropy coding technique to determine coded/not-coded video data, and generate a video frame based at least in part on the coded/not-coded video data.
  • the video frame may be configured to be presented to a user via a display device, for example.
  • video data of different types may be entropy coded to generate compressed (e.g.,. coded) video data.
  • the compressed video data may be assembled to generate an output bitstream.
  • example bitstream 800 may include indicator data 801, compressed video data 421, compressed video data 422, compressed video data 423, compressed video data 424, compressed video data 425, compressed video data 426, and/or compressed video data 427.
  • compressed video data 424 may include coded coded/not-coded video data, or the like.
  • indicator data 801 and/or compressed video data 424 may include a coded/not-coded video data header associated with the selected entropy coding technique or the processed bitstream (e.g.
  • the coded/not- coded video data header may indicate a selected entropy coding technique or a processed bitstream type— a pass-through of the coded/not-coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, a 1 -dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block-based scan based on the coded/not- coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data, or the like).
  • a computer-implemented method for video coding may include determining a selected entropy coding technique for coded/not-coded video data from a plurality of entropy coding techniques such that the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, entropy encoding a processed bitstream associated with the coded/not-coded video data - I l l - using the selected entropy coding technique to generate encoded coded/not-coded video data, assembling the encoded coded/not-coded video data into an output bitstream.
  • a computer-implemented method for video coding may further include reading a block structure associated with a frame of video data, associating the block structure and the coded/not-coded video data, encoding a coded/not-coded video data header associated with at least one of the selected entropy coding technique or the processed bitstream, transmitting the output bitstream, receiving the output bitstream, disassembling the output bitstream to determine the coded/not-coded video data header and the encoded coded/not-coded video data, determining the selected entropy coding technique based at least in part on the coded/not-coded video data header, entropy decoding the encoded coded/not-coded video data bitstream using the selected entropy coding technique to determine the coded/not-coded video data, and generating a video frame based at least in part on the coded/not-coded video data.
  • the coded/not-coded video data may include a plurality of bits such that each bit may indicate whether an associated block of the block structure associated with the frame of video data is coded or not-coded, such that a bit of 1 may indicate the associated block is coded and a bit of 0 indicates the associated block is not-coded, such that the coded/not-coded video data ma include symbol bits, and such that a mode associated with the frame of video data may be an inter mode.
  • the processed bitstream associated with the coded/not-coded video data may include at least one of a pass-through of the coded/not-coded video data, a reverse of the coded/not- coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, a 1 -dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block-based scan based on the coded/not-coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data.
  • Assembling the encoded coded/not-coded video into the output bitstream may include assembling the coded/not-coded video data header into the output bitstream.
  • the frame of video data may be a P-picture and determining the selected entropy coding technique may include determining a number of inter-blocks in the frame of video data is greater than a threshold such that the threshold may be 25, generating a 1 -dimensional raster scan based on the coded/not-coded video data, generating a 1 -dimensional block-based scan based on the coded/not-coded video data such that the 1 -dimensional block-based scan may include a 2x4 scale scan, bit inverting the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan, bit inverting the 1 -dimensional block-based scan to generate an inverted bits 1 -dimensional block-based scan, estimating a first bit cost associated with the symbol run coding technique based on the inverted bits 1 -dimensional rast
  • the frame of video data may be an F-picture and determining the selected entropy coding technique may include determining a number of inter-blocks in the frame of video data is greater than a threshold such that the threshold may be 25, generating a 1 -dimensional tile-based scan based on the coded/not-coded video data, generating a 1 -dimensional raster scan based on the coded/not-coded video data, bit inverting the 1 -dimensional tile-based scan to generate an inverted bits 1 -dimensional tile-based scan, bit inverting the 1 -dimensional raster scan to generate an inverted bits 1 -dimensional raster scan, bit differencing the inverted bits 1- dimensional tile-based scan to generate an inverted differential bits 1 -dimensional tile- based scan, bit differencing the inverted bits 1 -dimensional raster scan to generate an inverted differential bits 1 -dimensional raster scan, estimating a first bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional
  • a video encoder may include an image buffer and a graphics processing unit having entropy encoder logic circuitry.
  • the graphics processing unit may be communicatively coupled to the image buffer and the entropy encoder logic circuitry may be configured to determine a selected entropy coding technique for coded/not-coded video data from a plurality of entropy coding techniques such that the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, entropy encode a processed bitstream associated with the coded/not- coded video data using the selected entropy coding technique to generate encoded coded/not-coded video data, and assemble the encoded coded/not-coded video data into an output bitstream.
  • the entropy encoder logic circuitry may be further configured to read a block structure associated with a frame of video data, associate the block structure and the coded/not-coded video data, and encode a coded/not-coded video data header associated with at least one of the selected entropy coding technique or the processed bitstream.
  • the coded/not-coded video data may include a plurality of bits such that each bit may indicate whether an associated block of the block structure associated with the frame of video data is coded or not-coded, such that a bit of 1 may indicate the associated block is coded and a bit of 0 indicates the associated block is not- coded, such that the coded/not-coded video data ma include symbol bits, and such that a mode associated with the frame of video data may be an inter mode.
  • the processed bitstream associated with the coded/not-coded video data may include at least one of a pass-through of the coded/not-coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not- coded video data, a 1 -dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block-based scan based on the coded/not-coded video data, or a 1- dimensional tile-based scan based on the coded/not-coded video data.
  • To assemble the encoded coded/not-coded video into the output bitstream may include the entropy encoder logic circuitry being configured to assemble the coded/not-coded video data header into the output bitstream.
  • the frame of video data may be a P-picture and to determine the selected entropy coding technique may include the entropy encoder logic circuitry being configured to determine a number of inter-blocks in the frame of video data is greater than a threshold such that the threshold may be 25, generate a 1 -dimensional raster scan based on the coded/not-coded video data, generate a 1 -dimensional block-based scan based on the coded/not-coded video data such that the 1 -dimensional block-based scan may include a 2x4 scale scan, bit invert the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan, bit invert the 1 -dimensional block-based scan to generate an inverted bits 1 -dimensional block-based scan, estimate a first bit cost associated with
  • the frame of video data may be an F-picture and to determine the selected entropy coding technique may include the entropy encoder logic circuitry being configured to determine a number of inter-blocks in the frame of video data is greater than a threshold such that the threshold may be 25, generate a 1 -dimensional tile- based scan based on the coded/not-coded video data, generate a 1 -dimensional raster scan based on the coded/not-coded video data, bit invert the 1 -dimensional tile-based scan to generate an inverted bits 1 -dimensional tile-based scan, bit invert the 1 -dimensional raster scan to generate an inverted bits 1 -dimensional raster scan, bit difference the inverted bits 1 -dimensional tile-based scan to generate an inverted differential bits 1 -dimensional tile- based scan, bit difference the inverted bits 1 -dimensional raster scan to generate an inverted differential bits 1 -dimensional raster scan, estimate a first bit cost associated with the proxy variable length coding technique based on the inverted
  • a system may include an antenna configured to receive an entropy encoded bitstream of video data and a video decoder communicatively coupled to the antenna and configured to decode the encoded bitstream.
  • the video decoder may be configured to disassemble the entropy encoded bitstream to determine a coded/not-coded video data header and encoded coded/not-coded video data, determine a selected entropy coding technique for the encoded coded/not-coded video data from a plurality of entropy coding techniques based at least in part on the coded/not-coded video data header such that the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, entropy decode the encoded coded/not- coded video data using the selected entropy coding technique to determine coded/not- coded video data, and generate a video frame based at least in part on the coded/not-coded video data.
  • the system may also include a display device configured to present video frames.
  • the coded/not-coded video data may include a plurality of bits such that each bit may indicate whether an associated block of the block structure associated with a frame of video data is coded or not-coded and such that a bit of 1 may indicate the associated block is coded and a bit of 0 may indicate the associated block is not-coded.
  • To entropy decode the encoded coded/not-coded video data using the selected entropy coding technique to determine coded/not-coded video data may include the video decoder being configured to decompress the encoded coded/not-coded video data to generate a processed bitstream and decode the processed bitstream to generate the coded/not-coded video data such that the processed bitstream associated with the coded/not-coded video data may include at least one of a pass-through of the coded/not- coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, a 1- dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block- based scan based on the coded/not-coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data.
  • an apparatus may include means for performing the methods according to any one of the above examples.
  • the above examples may include specific combination of features. However, such the above examples are not limited in this regard and, in various implementations, the above examples may include the undertaking only a subset of such features, undertaking a different order of such features, undertaking a different combination of such features, and/or undertaking additional features than those features explicitly listed. For example, all features described with respect to the example methods may be implemented with respect to the example apparatus, the example systems, and/or the example articles, and vice versa.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Image Analysis (AREA)
  • Television Systems (AREA)

Abstract

Techniques related to content adaptive entropy coding of coded/not-coded data are described. A computer-implemented method for video coding comprises the steps of determining a selected entropy coding technique for coded/not- coded video data from a plurality of entropy coding techniques, wherein the plurality of entropy coding techniques comprises a proxy variable length coding technique and a symbol- run coding technique, entropy encoding a processed bitstream associated with the coded/not- coded video data using the selected entropy coding technique to generate encoded coded/not-coded video data, and assembling the encoded coded/not-coded video data into an output bitstream.

Description

CONTENT ADAPTIVE ENTROPY CODING OF CODED/NOT-CODED DATA FOR
NEXT GENERATION VIDEO
RELATED APPLICATIONS
This application claims the benefit of U.S. Provisional Application No. 61/758,314 filed 30 JAN 2013 and titled "NEXT GENERATION VIDEO CODING", the contents of which are hereby incorporated in their entirety.
BACKGROUND
A video encoder compresses video information so that more information can be sent over a given bandwidth. The compressed signal may then be transmitted to a receiver having a decoder that decodes or decompresses the signal prior to display.
High Efficient Video Coding (HEVC) is the latest video compression standard, which is being developed by the Joint Collaborative Team on Video Coding (JCT-VC) formed by ISO/IEC Moving Picture Experts Group (MPEG) and ITU-T Video Coding Experts Group (VCEG). HEVC is being developed in response to the previous H.264/AVC (Advanced Video Coding) standard not providing enough compression for evolving higher resolution video applications. Similar to previous video coding standards, HEVC includes basic functional modules such as intra/inter prediction, transform, quantization, in-loop filtering, and entropy coding.
The ongoing HEVC standard may attempt to improve on limitations of the
H.264/AVC standard such as limited choices for allowed prediction partitions and coding partitions, limited allowed multiple references and prediction generation, limited transform block sizes and actual transforms, limited mechanisms for reducing coding artifacts, and inefficient entropy encoding techniques. However, the ongoing HEVC standard may use iterative approaches to solving such problems.
For instance, with ever increasing resolution of video to be compressed and expectation of high video quality, the corresponding bitrate/bandwidth required for coding using existing video coding standards such as H.264 or even evolving standards such as H.265/HEVC, is relatively high. The aforementioned standards use expanded forms of traditional approaches to implicitly address the insufficient compression/quality problem, but often the results are limited.
The present description, developed within the context of a Next Generation Video (NGV) codec project, addresses the general problem of designing an advanced video codec that maximizes the achievable compression efficiency while remaining sufficiently practical for implementation on devices. For instance, with ever increasing resolution of video and expectation of high video quality due to availability of good displays, the corresponding bitrate/bandwidth required using existing video coding standards such as earlier MPEG standards and even the more recent H.264/AVC standard, is relatively high.
H.264/AVC was not perceived to be providing high enough compression for evolving higher resolution video applications.
BRIEF DESCRIPTION OF THE DRAWINGS
The material described herein is illustrated by way of example and not by way of limitation in the accompanying figures. For simplicity and clarity of illustration, elements illustrated in the figures are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference labels have been repeated among the figures to indicate corresponding or analogous elements. In the figures: FIG. 1 is an illustrative diagram of an example next generation video encoder;
FIG. 2 is an illustrative diagram of an example next generation video decoder;
FIG. 3(a) is an illustrative diagram of an example next generation video encoder and subsystems;
FIG. 3(b) is an illustrative diagram of an example next generation video decoder and subsystems;
FIG. 4 is an illustrative diagram of an example entropy encoder module;
FIG. 5 is an illustrative diagram of an example entropy decoder module;
FIG. 6 is an illustrative diagram of an example entropy encoder module;
FIG. 7 is an illustrative diagram of an example entropy decoder module; FIG. 8 is a flow diagram illustrating an example process;
FIG 9 illustrates an example bitstream;
FIG. 10 is a flow diagram illustrating an example process; FIG. 11 is an illustrative diagram of an example video coding system and video coding process in operation;
FIG. 11 is an illustrative diagram of an example video coding system;
FIG. 12 is a flow diagram illustrating an example encoding process;
FIG. 13 illustrates an example bitstream;
FIG. 14 is a flow diagram illustrating an example decoding process;
FIGS. 15(A), 15(B), and 15(C) provide an illustrative diagram of an example video coding system and video coding process in operation;
FIG. 16 is an illustrative diagram of an example video coding system;
FIG. 17 is an illustrative diagram of an example system;
FIG. 18 illustrates an example device;
FIG. 19 is an illustrative diagram of an example picture of coded/not-coded inter- blocks;
FIG. 20 is an illustrative diagram of example coded/not-coded inter-blocks;
FIG. 21 is an illustrative diagram of an example coded/not-coded data encoder module;
FIG. 22 is a flow diagram illustrating an example process;
FIG. 23 is a flow diagram illustrating an example process;
FIG. 24 is a flow diagram illustrating an example process;
FIG. 25 is a flow diagram illustrating an example process; and
FIG. 26 is a flow diagram illustrating an example process, all arranged in accordance with at least some implementations of the present disclosure.
DETAILED DESCRIPTION One or more embodiments or implementations are now described with reference to the enclosed figures. While specific configurations and arrangements are discussed, it should be understood that this is done for illustrative purposes only. Persons skilled in the relevant art will recognize that other configurations and arrangements may be employed without departing from the spirit and scope of the description. It will be apparent to those skilled in the relevant art that techniques and/or arrangements described herein may also be employed in a variety of other systems and applications other than what is described herein.
While the following description sets forth various implementations that may be manifested in architectures such as system-on-a-chip (SoC) architectures for example, implementation of the techniques and/or arrangements described herein are not restricted to particular architectures and/or computing systems and may be implemented by any architecture and/or computing system for similar purposes. For instance, various architectures employing, for example, multiple integrated circuit (IC) chips and/or packages, and/or various computing devices and/or consumer electronic (CE) devices such as set top boxes, smart phones, etc., may implement the techniques and/or arrangements described herein. Further, while the following description may set forth numerous specific details such as logic implementations, types and interrelationships of system components, logic partitioning/integration choices, etc., claimed subject matter may be practiced without such specific details. In other instances, some material such as, for example, control structures and full software instruction sequences, may not be shown in detail in order not to obscure the material disclosed herein.
The material disclosed herein may be implemented in hardware, firmware, software, or any combination thereof. The material disclosed herein may also be implemented as instructions stored on a machine-readable medium, which may be read and executed by one or more processors. A machine-readable medium may include any medium and/or mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing device). For example, a machine-readable medium may include read only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.), and others.
References in the specification to "one implementation", "an implementation", "an example implementation", etc., indicate that the implementation described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same implementation. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other implementations whether or not explicitly described herein.
Systems, apparatus, articles, and methods are described below related to content adaptive entropy coding for video systems.
Next generation video (NGV) systems, apparatus, articles, and methods are described below. NGV video coding may incorporate significant content based adaptivity in the video coding process to achieve higher compression. As discussed above, the H.264/AVC standard may have a variety of limitations and ongoing attempts to improve on the standard, such as, for example, the HEVC standard may use iterative approaches to address such limitations. Herein, an NGV system including an encoder and a decoder will be described.
For example, the H.264 standard may support two modes of entropy coding. In the first mode, Adaptive VLC (Variable Length Coding), transform coefficients may be coded using Context Adaptive VLC (CAVLC) and all other syntax elements (e.g., overhead data, modes, motion vectors, and so on) may be coded using Exponential Golomb coding. The CAVLC coded data and the Exponential Golomb coded data may be multiplexed to generate an encoded bitstream. In the second mode, all data may be encoded using Context Adaptive Binary Arithmetic Coding (CABAC). The corresponding decoder may also operate in two modes, disassembling and decoding the multiplexed bit stream in the first mode and decoding the CABAC encoded bitstream in the second mode. Such techniques may have limitations. For example, CABAC coding may be efficient but may be complex such that throughput in higher resolution contexts may be limited. Further, by grouping data types together for coding, efficiency and complexity of the entropy coding may not be optimized.
In some video codec implementations, entropy coding and decoding of various data such as overhead data, modes, motion vectors, and/or transform coefficients may be a significant contributor to the coding efficiency and complexity of the system. In some examples, the techniques discussed herein may balance coding efficiency and system complexity.
In some examples, first video data and second video data may be received for entropy encoding at an entropy encoder module. The first video data and the second video data may be different data types (e.g., header data, morphing parameters, synthesizing parameters, or global maps data or motion vectors or intra-prediction partition data or so on, as is discussed further herein). A first entropy encoding technique may be determined for the first video data based on a parameter associated with the first video data such as, for example, a number of compressed bits of the first video data, a predetermined indicator or flag associated with the first video data, a predetermined threshold, or a heuristically determined threshold or the like. In some examples, the first entropy encoding technique may be chosen from one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique. The first video data may be entropy encoded using the first entropy encoding technique and the second video data may be entropy encoded using the first entropy encoding technique. In some examples, the second entropy encoding technique may be determined for the second video data based on a parameter as discussed with respect to the first video data. In some examples, the second entropy encoding technique may be determined from options including an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, an adaptive vector variable length coding, an adaptive 1 -dimensional variable length coding technique, an adaptive 2-dimensional variable length coding technique, or other techniques as discussed herein. In some examples, the second entropy encoding technique may be predetermined for the data type of the second video data. Entropy encoding the first video data and the second video data may generate first compressed video data and second compressed video data. The first and second compressed video data may be assembled to generate an output bitstream.
The output bitstream may be transmitted from the encoder to a decoder, which may disassemble the bitstream to determine the first and second compressed video data. The compressed video data may be entropy decoded to generate entropy decoded video data, which may be further decoded to generate a video frame. The video frame may be transmitted to a display device for presentment to a user.
In some examples, additional video data types may be received for entropy encoding at the entropy encoder module. For example, third, fourth, fifth, sixth, seventh, or more video data may be entropy encoded to generate associated compressed video data, which may be assembled in the output bitstream, transmitted, and subsequently entropy decoded via a decoder. The various data types and encoding / decoding technique options and implementations are discussed further herein.
As used herein, the term "coder" may refer to an encoder and/or a decoder.
Similarly, as used herein, the term "coding" may refer to performing video encoding via an encoder and/or performing video decoding via a decoder. For example a video encoder and video decoder may both be examples of coders capable of coding video data. In addition, as used herein, the term "codec" may refer to any process, program or set of operations, such as, for example, any combination of software, firmware, and/or hardware, that may implement an encoder and/or a decoder. Further, as used herein, the phrase "video data" may refer to any type of data associated with video coding such as, for example, video frames, image data, encoded bitstream data, or the like.
FIG. 1 is an illustrative diagram of an example next generation video encoder 100, arranged in accordance with at least some implementations of the present disclosure. As shown, encoder 100 may receive input video 101. Input video 101 may include any suitable input video for encoding such as, for example, input frames of a video sequence. As shown, input video 101 may be received via a content pre-analyzer module 102.
Content pre-analyzer module 102 may be configured to perform analysis of the content of video frames of input video 101 to determine various types of parameters for improving video coding efficiency and speed performance. For example, content pre-analyzer module 102 may determine horizontal and vertical gradient information (e.g., Rs, Cs), variance, spatial complexity per picture, temporal complexity per picture, scene change detection, motion range estimation, gain detection, prediction distance estimation, number of objects estimation, region boundary detection, spatial complexity map computation, focus estimation, film grain estimation, or the like. The parameters generated by content pre- analyzer module 102 may be used by encoder 100 (e.g., via encode controller 103) and/or quantized and communicated to a decoder. As shown, video frames and/or other data may be transmitted from content pre-analyzer module 102 to adaptive picture organizer module 104, which may determine the picture type (e.g., I-, P-, or F/B-picture ) of each video frame and reorder the video frames as needed. In some examples, adaptive picture organizer module 104 may include a frame portion generator configured to generate frame portions. In some examples, content pre-analyzer module 102 and adaptive picture organizer module 104 may together be considered a pre-analyzer subsystem of encoder 100.
As shown, video frames and/or other data may be transmitted from adaptive picture organizer module 104 to prediction partitions generator module 105. In some examples, prediction partitions generator module 105 may divide a frame or picture into tiles or super-fragments or the like. In some examples, an additional module (e.g., between modules 104 and 105) may be provided for dividing a frame or picture into tiles or super- fragments. Prediction partitions generator module 105 may divide each tile or super- fragment into potential prediction partitionings or partitions. In some examples, the potential prediction partitionings may be determined using a partitioning technique such as, for example, a k-d tree partitioning technique, a bi-tree partitioning technique, or the like, which may be determined based on the picture type (e.g., I-, P-, or F/B-picture ) of individual video frames, a characteristic of the frame portion being partitioned, or the like. In some examples, the determined potential prediction partitionings may be partitions for prediction (e.g., inter- or intra-prediction) and may be described as prediction partitions or prediction blocks or the like.
In some examples, a selected prediction partitioning (e.g., prediction partitions) may be determined from the potential prediction partitionings. For example, the selected prediction partitioning may be based on determining, for each potential prediction partitioning, predictions using characteristics and motion based multi-reference predictions or intra-predictions, and determining prediction parameters. For each potential prediction partitioning, a potential prediction error may be determined by differencing original pixels with prediction pixels and the selected prediction partitioning may be the potential prediction partitioning with the minimum prediction error. In other examples, the selected prediction partitioning may be determined based on a rate distortion optimization including a weighted scoring based on number of bits for coding the partitioning and a prediction error associated with the prediction partitioning.
As shown, the original pixels of the selected prediction partitioning (e.g., prediction partitions of a current frame) may be differenced with predicted partitions (e.g., a prediction of the prediction partition of the current frame based on a reference frame or frames and other predictive data such as inter- or intra-prediction data) at differencer 106. The determination of the predicted partitions will be described further below and may include a decode loop as shown in Fig. 1. Any residuals or residual data (e.g., partition prediction error data) from the differencing may be transmitted to coding partitions generator module 107. In some examples, such as for intra-prediction of prediction partitions in any picture type (I-, F/B- or P-pictures), coding partitions generator module 107 may be bypassed via switches 107a and 107b. In such examples, only a single level of partitioning may be performed. Such partitioning may be described as prediction partitioning (as discussed) or coding partitioning or both. In various examples, such partitioning may be performed via prediction partitions generator module 105 (as discussed) or, as is discussed further herein, such partitioning may be performed via a k-d tree intra-prediction/coding partitioner module or a bi-tree intra-prediction/coding partitioner module implemented via coding partitions generator module 107.
In some examples, the partition prediction error data, if any, may not be significant enough to warrant encoding. In other examples, where it may be desirable to encode the partition prediction error data and the partition prediction error data is associated with inter-prediction or the like, coding partitions generator module 107 may determine coding partitions of the prediction partitions. In some examples, coding partitions generator module 107 may not be needed as the partition may be encoded without coding partitioning (e.g., as shown via the bypass path available via switches 107a and 107b).
With or without coding partitioning, the partition prediction error data (which may subsequently be described as coding partitions in either event) may be transmitted to adaptive transform module 108 in the event the residuals or residual data require encoding. In some examples, prediction partitions generator module 105 and coding partitions generator module 107 may together be considered a partitioner subsystem of encoder 100.
In various examples, coding partitions generator module 107 may operate on partition prediction error data, original pixel data, residual data, or wavelet data.
Coding partitions generator module 107 may generate potential coding partitionings (e.g., coding partitions) of, for example, partition prediction error data using bi-tree and/or k-d tree partitioning techniques or the like. In some examples, the potential coding partitions may be transformed using adaptive or fixed transforms with various block sizes via adaptive transform module 108 and a selected coding partitioning and selected transforms (e.g., adaptive or fixed) may be determined based on a rate distortion optimization or other basis. In some examples, the selected coding partitioning and/or the selected transform(s) may be determined based on a predetermined selection method based on coding partitions size or the like.
For example, adaptive transform module 108 may include a first portion or component for performing a parametric transform to allow locally optimal transform coding of small to medium size blocks and a second portion or component for performing globally stable, low overhead transform coding using a fixed transform, such as a discrete cosine transform (DCT) or a picture based transform from a variety of transforms, including parametric transforms, or any other configuration as is discussed further herein. In some examples, for locally optimal transform coding a Parametric Haar Transform (PHT) may be performed, as is discussed further herein. In some examples, transforms may be performed on 2D blocks of rectangular sizes between about 4x4 pixels and 64x64 pixels, with actual sizes depending on a number of factors such as whether the transformed data is luma or chroma, or inter or intra, or if the determined transform used is PHT or DCT or the like.
As shown, the resultant transform coefficients may be transmitted to adaptive quantize module 109. Adaptive quantize module 109 may quantize the resultant transform coefficients. Further, any data associated with a parametric transform, as needed, may be transmitted to either adaptive quantize module 109 (if quantization is desired) or adaptive entropy encoder module 1 10. Also as shown in FIG. 1, the quantized coefficients may be scanned and transmitted to adaptive entropy encoder module 110. Adaptive entropy encoder module 1 10 may entropy encode the quantized coefficients and include them in output bitstream 1 11. In some examples, adaptive transform module 108 and adaptive quantize module 109 may together be considered a transform encoder subsystem of encoder 100.
As also shown in FIG. 1, encoder 100 includes a local decode loop. The local decode loop may begin at adaptive inverse quantize module 1 12. Adaptive inverse quantize module 112 may be configured to perform the opposite operation(s) of adaptive quantize module 109 such that an inverse scan may be performed and quantized coefficients may be de-scaled to determine transform coefficients. Such an adaptive quantize operation may be lossy, for example. As shown, the transform coefficients may be transmitted to an adaptive inverse transform module 113. Adaptive inverse transform module 1 13 may perform the inverse transform as that performed by adaptive transform module 108, for example, to generate residuals or residual values or partition prediction error data (or original data or wavelet data, as discussed) associated with coding partitions. In some examples, adaptive inverse quantize module 112 and adaptive inverse transform module 113 may together be considered a transform decoder subsystem of encoder 100.
As shown, the partition prediction error data (or the like) may be transmitted to optional coding partitions assembler 1 14. Coding partitions assembler 1 14 may assemble coding partitions into decoded prediction partitions as needed (as shown, in some examples, coding partitions assembler 1 14 may be skipped via switches 114a and 114b such that decoded prediction partitions may have been generated at adaptive inverse transform module 113) to generate prediction partitions of prediction error data or decoded residual prediction partitions or the like.
As shown, the decoded residual prediction partitions may be added to predicted partitions (e.g., prediction pixel data) at adder 115 to generate reconstructed prediction partitions. The reconstructed prediction partitions may be transmitted to prediction partitions assembler 1 16. Prediction partitions assembler 1 16 may assemble the reconstructed prediction partitions to generate reconstructed tiles or super-fragments. In some examples, coding partitions assembler module 114 and prediction partitions assembler module 1 16 may together be considered an un-partitioner subsystem of encoder 100.
The reconstructed tiles or super-fragments may be transmitted to blockiness analyzer and deblock filtering module 1 17. Blockiness analyzer and deblock filtering module 117 may deblock and dither the reconstructed tiles or super-fragments (or prediction partitions of tiles or super-fragments). The generated deblock and dither filter parameters may be used for the current filter operation and/or coded in output bitstream 11 1 for use by a decoder, for example. The output of blockiness analyzer and deblock filtering module 1 17 may be transmitted to a quality analyzer and quality restoration filtering module 1 18. Quality analyzer and quality restoration filtering module 1 18 may determine QR filtering parameters (e.g., for a QR decomposition) and use the determined parameters for filtering. The QR filtering parameters may also be coded in output bitstream 1 11 for use by a decoder. As shown, the output of quality analyzer and quality restoration filtering module 1 18 may be transmitted to decoded picture buffer 1 19. In some examples, the output of quality analyzer and quality restoration filtering module 118 may be a final reconstructed frame that may be used for prediction for coding other frames (e.g., the final reconstructed frame may be a reference frame or the like). In some examples, blockiness analyzer and deblock filtering module 117 and quality analyzer and quality restoration filtering module 1 18 may together be considered a filtering subsystem of encoder 100.
In encoder 100, prediction operations may include inter- and/or intra-prediction. As shown in FIG. 1(a), inter-prediction may be performed by one or more modules including morphing analyzer and morphed picture generation module 120, synthesizing analyzer and generation module 121, and characteristics and motion filtering predictor module 123. Morphing analyzer and morphed picture generation module 120 may analyze a current picture to determine parameters for changes in gain, changes in dominant motion, changes in registration, and changes in blur with respect to a reference frame or frames with which itmay beto be coded. The determined morphing parameters may be quantized/de-quantized and used (e.g., by morphing analyzer and morphed picture generation module 120) to generate morphed reference frames that that may be used by motion estimator module 122 for computing motion vectors for efficient motion (and characteristics) compensated prediction of a current frame. Synthesizing analyzer and generation module 121 may generate super resolution (SR) pictures and projected interpolation (PI) pictures or the like for motion for determining motion vectors for efficient motion compensated prediction in these frames.
Motion estimator module 122 may generate motion vector data based on morphed reference frame(s) and/or super resolution (SR) pictures and projected interpolation (PI) pictures along with the current frame. In some examples, motion estimator module 122 may be considered an inter-prediction module. For example, the motion vector data may be used for inter-prediction. If inter-prediction is applied, characteristics and motion compensated filtering predictor module 123 may apply motion compensation as part of the local decode loop as discussed.
Intra-prediction may be performed by intra-directional prediction analyzer and prediction generation module 124. Intra-directional prediction analyzer and prediction generation module 124 may be configured to perform spatial directional prediction and may use decoded neighboring partitions. In some examples, both the determination of direction and generation of prediction may be performed by intra-directional prediction analyzer and prediction generation module 124. In some examples, intra-directional prediction analyzer and prediction generation module 124 may be considered an intra- prediction module.
As shown in FIG. 1, prediction modes and reference types analyzer module 125 may allow for selection of prediction modes from among, "skip", "auto", "inter", "split", "multi", and "intra", for each prediction partition of a tile (or super- fragment), all of which may apply to P- and F/B-pictures. In addition to prediction modes, it also allows for selection of reference types that can be different depending on "inter" or "multi" mode, as well as for P- and F/B-pictures. The prediction signal at the output of prediction modes and reference types analyzer module 125 may be filtered by prediction analyzer and prediction fusion filtering module 126. Prediction analyzer and prediction fusion filtering module 126 may determine parameters (e.g., filtering coefficients, frequency, overhead) to use for filtering and may perform the filtering. In some examples, filtering the prediction signal may fuse different types of signals representing different modes (e.g., intra, inter, multi, split, skip, and auto). In some examples, intra-prediction signals may be different than all other types of inter-prediction signal(s) such that proper filtering may greatly enhance coding efficiency. In some examples, the filtering parameters may be encoded in output bitstream 11 1 for use by a decoder. The filtered prediction signal may provide the second input (e.g., prediction partition(s)) to differencer 106, as discussed above, that may determine the prediction difference signal (e.g., partition prediction error) for coding discussed earlier. Further, the same filtered prediction signal may provide the second input to adder 115, also as discussed above. As discussed, output bitstream 1 1 1 may provide an efficiently encoded bitstream for use by a decoder for the presentment of video.
FIG. 1 illustrates example control signals associated with operation of video encoder 100, where the following abbreviations may represent the associated information: scnchg Scene change information
spcpx Spatial complexity information
tpcpx Temporal complexity information
pdist Temporal prediction distance information
pap Pre Analysis parameters (placeholder for all other pre analysis parameters
except scnchg, spcpx, tpcpx, pdist)
ptyp Picture types information
pgst Picture group structure information
pptn cand. Prediction partitioning candidates
cptn cand. Coding Partitioning Candidates
prp Preprocessing
xmtyp Transform type information
xmdir Transform direction information
xmmod Transform mode
ethp One eighth (l/8th) pel motion prediction
pptn Prediction Partitioning
cptn Coding Partitioning
mot&cod cost Motion and Coding Cost
qs quantizer information set (includes Quantizer parameter (Qp), Quantizer matrix (QM) choice)
mv Motion vectors
mop Morphing Paramters
syp Synthesizing Parameters
ddi Deblock and dither information
qri Quality Restoration filtering index/information
api Adaptive Precision filtering index/information
fii Fusion Filtering index/information
mod Mode information
reftyp Reference type information
idir Intra Prediction Direction
The various signals and data items that may need to be sent to the decoder, ie, pgst, ptyp, prp, pptn, cptn, modes, reftype, ethp, xmtyp, xmdir, xmmod, idir, mv, qs, mop, syp, ddi, qri, api, fii, quant coefficients and others may then be entropy encoded by adaptive entropy encoder 1 10 that may include different entropy coders collectively referred to as an entropy encoder subsystem. While these control signals are illustrated as being associated with specific example functional modules of encoder 100 in Fig. 1, other implementations may include a different distribution of control signals among the functional modules of encoder 300. The present disclosure is not limited in this regard and, in various examples, implementation of the control signals herein may include the undertaking of only a subset of the specific example control signals shown, additional control signals, and/or in a different arrangement than illustrated.
FIG. 2 is an illustrative diagram of an example next generation video decoder 200, arranged in accordance with at least some implementations of the present disclosure. As shown, decoder 200 may receive an input bitstream 201. In some examples, input bitstream 201 may be encoded via encoder 100 and/or via the encoding techniques discussed herein. As shown, input bitstream 201 may be received by an adaptive entropy decoder module 202. Adaptive entropy decoder module 202 may decode the various types of encoded data (e.g., overhead, motion vectors, transform coefficients, etc.). In some examples, adaptive entropy decoder 202 may use a variable length decoding technique. In some examples, adaptive entropy decoder 202 may perform the inverse operation(s) of adaptive entropy encoder module 1 10 discussed above.
The decoded data may be transmitted to adaptive inverse quantize module 203. Adaptive inverse quantize module 203 may be configured to inverse scan and de-scale quantized coefficients to determine transform coefficients. Such an adaptive quantize operation may be lossy, for example. In some examples, adaptive inverse quantize module 203 may be configured to perform the opposite operation of adaptive quantize module 109 (e.g., substantially the same operations as adaptive inverse quantize module 112). As shown, the transform coefficients (and, in some examples, transform data for use in a parametric transform) may be transmitted to an adaptive inverse transform module 204. Adaptive inverse transform module 204 may perform an inverse transform on the transform coefficients to generate residuals or residual values or partition prediction error data (or original data or wavelet data) associated with coding partitions. In some examples, adaptive inverse transform module 204 may be configured to perform the opposite operation of adaptive transform module 108 (e.g., substantially the same operations as adaptive inverse transform module 113). In some examples, adaptive inverse transform module 204 may perform an inverse transform based on other previously decoded data, such as, for example, decoded neighboring partitions. In some examples, adaptive inverse quantize module 203 and adaptive inverse transform module 204 may together be considered a transform decoder subsystem of decoder 200.
As shown, the residuals or residual values or partition prediction error data may be transmitted to coding partitions assembler 205. Coding partitions assembler 205 may assemble coding partitions into decoded prediction partitions as needed (as shown, in some examples, coding partitions assembler 205 may be skipped via switches 205a and 205b such that decoded prediction partitions may have been generated at adaptive inverse transform module 204). The decoded prediction partitions of prediction error data (e.g., prediction partition residuals) may be added to predicted partitions (e.g., prediction pixel data) at adder 206 to generate reconstructed prediction partitions. The reconstructed prediction partitions may be transmitted to prediction partitions assembler 207. Prediction partitions assembler 207 may assemble the reconstructed prediction partitions to generate reconstructed tiles or super-fragments. In some examples, coding partitions assembler module 205 and prediction partitions assembler module 207 may together be considered an un-partitioner subsystem of decoder 200.
The reconstructed tiles or super-fragments may be transmitted to deblock filtering module 208. Deblock filtering module 208 may deblock and dither the reconstructed tiles or super- fragments (or prediction partitions of tiles or super- fragments). The generated deblock and dither filter parameters may be determined from input bitstream 201, for example. The output of deblock filtering module 208 may be transmitted to a quality restoration filtering module 209. Quality restoration filtering module 209 may apply quality filtering based on QR parameters, which may be determined from input bitstream 201, for example. As shown in FIG. 2, the output of quality restoration filtering module 209 may be transmitted to decoded picture buffer 210. In some examples, the output of quality restoration filtering module 209 may be a final reconstructed frame that may be used for prediction for coding other frames (e.g., the final reconstructed frame may be a reference frame or the like). In some examples, deblock filtering module 208 and quality restoration filtering module 209 may together be considered a filtering subsystem of decoder 200.
As discussed, compensation due to prediction operations may include inter- and/or intra-prediction compensation. As shown, inter-prediction compensation may be performed by one or more modules including morphing generation module 211, synthesizing generation module 212, and characteristics and motion compensated filtering predictor module 213. Morphing generation module 211 may use de-quantized morphing parameters (e.g., determined from input bitstream 201) to generate morphed reference frames. Synthesizing generation module 212 may generate super resolution (SR) pictures and projected interpolation (PI) pictures or the like based on parameters determined from input bitstream 201. If inter-prediction is applied, characteristics and motion compensated filtering predictor module 213 may apply motion compensation based on the received frames and motion vector data or the like in input bitstream 201.
Intra-prediction compensation may be performed by intra-directional prediction generation module 214. Intra-directional prediction generation module 214 may be configured to perform spatial directional prediction and may use decoded neighboring partitions according to intra-prediction data in input bitstream 201.
As shown in FIG. 2, prediction modes selector module 215 may determine a prediction mode selection from among, "skip", "auto", "inter", "multi", and "intra", for each prediction partition of a tile, all of which may apply to P- and F/B-pictures, based on mode selection data in input bitstream 201. In addition to prediction modes, it also allows for selection of reference types that can be different depending on "inter" or "multi" mode, as well as for P- and F/B-pictures. The prediction signal at the output of prediction modes selector module 215 may be filtered by prediction fusion filtering module 216. Prediction fusion filtering module 216 may perform filtering based on parameters (e.g., filtering coefficients, frequency, overhead) determined via input bitstream 201. In some examples, filtering the prediction signal may fuse different types of signals representing different modes (e.g., intra, inter, multi, skip, and auto). In some examples, intra-prediction signals may be different than all other types of inter-prediction signal(s) such that proper filtering may greatly enhance coding efficiency. The filtered prediction signal may provide the second input (e.g., prediction partition(s)) to differencer 206, as discussed above.
As discussed, the output of quality restoration filtering module 209 may be a final reconstructed frame. Final reconstructed frames may be transmitted to an adaptive picture re-organizer 217, which may re-order or re-organize frames as needed based on ordering parameters in input bitstream 201. Re-ordered frames may be transmitted to content post- restorer module 218. Content post-restorer module 218 may be an optional module configured to perform further improvement of perceptual quality of the decoded video. The improvement processing may be performed in response to quality improvement parameters in input bitstream 201 or it may be performed as standalone operation. In some examples, content post-restorer module 218 may apply parameters to improve quality such as, for example, an estimation of film grain noise or residual blockiness reduction (e.g., even after the deblocking operations discussed with respect to deblock filtering module 208). As shown, decoder 200 may provide display video 219, which may be configured for display via a display device (not shown).
FIG. 2 illustrates example control signals associated with operation of video decoder 200, where the indicated abbreviations may represent similar information as discussed with respect to FIG. 1 above. While these control signals are illustrated as being associated with specific example functional modules of decoder 200 in Fig. 4, other implementations may include a different distribution of control signals among the functional modules of encoder 100. The present disclosure is not limited in this regard and, in various examples, implementation of the control signals herein may include the undertaking of only a subset of the specific example control signals shown, additional control signals, and/or in a different arrangement than illustrated.
While FIGS. 1 through 2 illustrate particular encoding and decoding modules, various other coding modules or components not depicted may also be utilized in accordance with the present disclosure. Further, the present disclosure is not limited to the particular components illustrated in FIGS. 1 and 2 and/or to the manner in which the various components are arranged. Various components of the systems described herein may be implemented in software, firmware, and/or hardware and/or any combination thereof. For example, various components of encoder 100 and/or decoder 200 may be provided, at least in part, by hardware of a computing System-on-a-Chip (SoC) such as may be found in a computing system such as, for example, a mobile phone.
Further, it may be recognized that encoder 100 may be associated with and/or provided by a content provider system including, for example, a video content server system, and that output bitstream 1 11 may be transmitted or conveyed to decoders such as, for example, decoder 200 by various communications components and/or systems such as transceivers, antennae, network systems, and the like not depicted in FIGS. 1 and 2. It may also be recognized that decoder 200 may be associated with a client system such as a computing device (e.g., a desktop computer, laptop computer, tablet computer, convertible laptop, mobile phone, or the like) that is remote to encoder 100 and that receives input bitstream 201 via various communications components and/or systems such as transceivers, antennae, network systems, and the like not depicted in FIGS. 1 and 2.
Therefore, in various implementations, encoder 100 and decoder subsystem 200 may be implemented either together or independent of one another. FIG. 3(a) is an illustrative diagram of an example next generation video encoder 300a, arranged in accordance with at least some implementations of the present disclosure. FIG. 3(a) presents a similar encoder to that shown in FIGS. 1(a) and 1(b), and similar elements will not be repeated for the sake of brevity. As shown in FIG. 3(a), encoder 300a may include preanalyzer subsystem 310a, partitioner subsystem 320a, prediction encoding subsystem 330a, transform encoder subsystem 340a, filtering encoding subsystem 350a, entropy encoder system 360a, transform decoder subsystem 370a, and/or unpartioner subsystem 380a. Preanalyzer subsystem 310a may include content pre-analyzer module 102 and/or adaptive picture organizer module 104. Partitioner subsystem 320a may include prediction partitions generator module 105, and/or coding partitions generator 107.
Prediction encoding subsystem 330a may include motion estimator module 122, characteristics and motion compensated filtering predictor module 123, and/or intra- directional prediction analyzer and prediction generation module 124. Transform encoder subsystem 340a may include adaptive transform module 108 and/or adaptive quantize module 109. Filtering encoding subsystem 350a may include blockiness analyzer and deblock filtering module 117, quality analyzer and quality restoration filtering module 118, motion estimator module 122, characteristics and motion compensated filtering predictor module 123, and/or prediction analyzer and prediction fusion filtering module 126.
Entropy coding subsystem 360a may include adaptive entropy encoder module 1 10.
Transform decoder subsystem 370a may include adaptive inverse quantize module 112 and/or adaptive inverse transform module 1 13. Unpartioner subsystem 380a may include coding partitions assembler 1 14 and/or prediction partitions assembler 1 16.
Partitioner subsystem 320a of encoder 300a may include two partitioning subsystems: prediction partitions generator module 105 that may perform analysis and partitioning for prediction, and coding partitions generator module 107 that may perform analysis and partitioning for coding. Another partitioning method may include adaptive picture organizer 104 which may segment pictures into regions or slices may also be optionally considered as being part of this partitioner.
Prediction encoder subsystem 330a of encoder 300a may include motion estimator 122 and characteristics and motion compensated filtering predictor 123 that may perform analysis and prediction of "inter" signal, and intra-directional prediction analyzer and prediction generation module 124 that may perform analysis and prediction of "intra" signal. Motion estimator 122 and characteristics and motion compensated filtering predictor 123 may allow for increasing predictability by first compensating for other sources of differences (such as gain, global motion, registration), followed by actual motion compensation. They may also allow for use of data modeling to create synthesized frames (super resolution, and projection) that may allow better predictions, followed by use of actual motion compensation in such frames. Transform encoder subsystem 340a of encoder 300a may perform analysis to select the type and size of transform and may include two major types of components. The first type of component may allow for using parametric transform to allow locally optimal transform coding of small to medium size blocks; such coding however may require some overhead. The second type of component may allow globally stable, low overhead coding using a generic/fixed transform such as the DCT, or a picture based transform from a choice of small number of transforms including parametric transforms. For locally adaptive transform coding, PHT (Parametric Haar Transform) may be used. Transforms may be performed on 2D blocks of rectangular sizes between 4x4 and 64x64, with actual sizes that may depend on a number of factors such as if the transformed data is luma or chroma, inter or intra, and if the transform used is PHT or DCT. The resulting transform coefficients may be quantized, scanned and entropy coded.
Entropy encoder subsystem 360a of encoder 300a may include a number of efficient but low complexity components each with the goal of efficiently coding a specific type of data (various types of overhead, motion vectors, or transform coefficients). Components of this subsystem may belong to a generic class of low complexity variable length coding techniques, however, for efficient coding, each component may be custom optimized for highest efficiency. For instance, a custom solution may be designed for coding of
"Coded/Not Coded" data, another for "Modes and Ref Types" data, yet another for "Motion Vector" data, and yet another one for "Prediction and Coding Partitions" data. Finally, because a very large portion of data to be entropy coded is "transform coefficient" data, multiple approaches for efficient handling of specific block sizes, as well as an algorithm that may adapt between multiple tables may be used.
Filtering encoder subsystem 350a of encoder 300a may perform analysis of parameters as well as multiple filtering of the reconstructed pictures based on these parameters, and may include several subsystems. For example, a first subsystem, blockiness analyzer and deblock filtering module 117 may deblock and dither to reduce or mask any potential block coding artifacts. A second example subsystem, quality analyzer and quality restoration filtering module 1 18, may perform general quality restoration to reduce the artifacts due to quantization operation in any video coding. A third example subsystem, which may include motion estimator 122 and characteristics and motion compensated filtering predictor module 123, may improve results from motion
compensation by using a filter that adapts to the motion characteristics (motion speed/degree of blurriness) of the content. A fourth example subsystem, prediction fusion analyzer and filter generation module 126, may allow adaptive filtering of the prediction signal (which may reduce spurious artifacts in prediction, often from intra prediction) thereby reducing the prediction error which needs to be coded.
Encode controller module 103 of encoder 300a may be responsible for overall video quality under the constraints of given resources and desired encoding speed. For instance, in full RDO (Rate Distortion Optimization) based coding without using any shortcuts, the encoding speed for software encoding may be simply a consequence of computing resources (speed of processor, number of processors, hyperthreading, DDR3 memory etc.) availability. In such case, encode controller module 103 may be input every single combination of prediction partitions and coding partitions and by actual encoding, and the bitrate may be calculated along with reconstructed error for each case and, based on lagrangian optimization equations, the best set of prediction and coding partitions may be sent for each tile of each frame being coded. The full RDO based mode may result in best compression efficiency and may also be the slowest encoding mode. By using content analysis parameters from content preanalyzer module 102 and using them to make RDO simplification (not test all possible cases) or only pass a certain percentage of the blocks through full RDO, quality versus speed tradeoffs may be made allowing speedier encoding. Up to now we have described a variable bitrate (VBR) based encoder operation. Encode controller module 103 may also include a rate controller that can be invoked in case of constant bitrate (CBR) controlled coding.
Lastly, preanalyzer subsystem 310a of encoder 300a may perform analysis of content to compute various types of parameters useful for improving video coding efficiency and speed performance. For instance, it may compute horizontal and vertical gradient information (Rs, Cs), variance, spatial complexity per picture, temporal complexity per picture, scene change detection, motion range estimation, gain detection, prediction distance estimation, number of objects estimation, region boundary detection, spatial complexity map computation, focus estimation, film grain estimation etc. The parameters generated by preanalyzer subsystem 310a may either be consumed by the encoder or be quantized and communicated to decoder 200. While subsystems 310a through 380a are illustrated as being associated with specific example functional modules of encoder 300a in Fig. 3(a), other implementations of encoder 300a herein may include a different distribution of the functional modules of encoder 300a among subsystems 310a through 380a. The present disclosure is not limited in this regard and, in various examples, implementation of the example subsystems 310a through 380a herein may include the undertaking of only a subset of the specific example functional modules of encoder 300a shown, additional functional modules, and/or in a different arrangement than illustrated.
FIG. 3(b) is an illustrative diagram of an example next generation video decoder 300b, arranged in accordance with at least some implementations of the present disclosure.
FIG. 3(b) presents a similar decoder to that shown in FIG. 2, and similar elements will not be repeated for the sake of brevity. As shown in FIG. 3(b), decoder 300b may include prediction decoder subsystem 330b, filtering decoder subsystem 350b, entropy decoder subsystem 360b, transform decoder subsystem 370b, unpartitioner_2 subsystem 380b, unpartitioner_l subsystem 351b, filtering decoder subsystem 350b, and/or postrestorer subsystem 390b. Prediction decoder subsystem 330b may include characteristics and motion compensated filtering predictor module 213 and/or intra-directional prediction generation module 214. Filtering decoder subsystem 350b may include deblock filtering module 208, quality restoration filtering module 209, characteristics and motion compensated filtering predictor module 213, and/or prediction fusion filtering module 216.
Entropy decoder subsystem 360b may include adaptive entropy decoder module 202. Transform decoder subsystem 370b may include adaptive inverse quantize module 203 and/or adaptive inverse transform module 204. Unpartitioner_2 subsystem 380b may include coding partitions assembler 205. Unpartitioner_l subsystem 351b may include prediction partitions assembler 207. Postrestorer subsystem 790 may include content post restorer module 218 and/or adaptive picture re-organizer 217.
Entropy decoding subsystem 360b of decoder 300b may perform the inverse operation of the entropy encoder subsystem 360a of encoder 300a, i.e., it may decode various data (types of overhead, motion vectors, transform coefficients) encoded by entropy encoder subsystem 360a using a class of techniques loosely referred to as variable length decoding. Specifically, various types of data to be decoded may include "Coded/Not Coded" data, "Modes and Ref Types" data, "Motion Vector" data, "Prediction and Coding Partitions" data, and "Transform Coefficient" data. Transform decoder subsystem 370b of decoder 300b may perform inverse operation to that of transform encoder subsystem 340a of encoder 300a. Transform decoder subsystem 370b may include two types of components. The first type of example component may support use of the parametric inverse PHT transform of small to medium block sizes, while the other type of example component may support inverse DCT transform for all block sizes. The PHT transform used for a block may depend on analysis of decoded data of the neighboring blocks. Output bitstream 1 11 and/or input bitstream 201 may carry information about partition/block sizes for PHT transform as well as in which direction of the 2D block to be inverse transformed the PHT may be used (the other direction uses DCT). For blocks coded purely by DCT, the partition/ block sizes information may be also retrieved from output bitstream 1 11 and/or input bitstream 20 land used to apply inverse DCT of appropriate size.
Unpartitioner subsystem 380b of decoder 300b may perform inverse operation to that of partitioner subsystem 320a of encoder 300a and may include two unpartitioning subsystems, coding partitions assembler module 205 that may perform unpartitioning of coded data and prediction partitions assembler module 207 that may perform
unpartitioning for prediction. Further if optional adaptive picture organizer module 104 is used at encoder 300a for region segmentation or slices, adaptive picture re-organizer module 217 may be needed at the decoder.
Prediction decoder subsystem 330b of decoder 300b may include characteristics and motion compensated filtering predictor module 213 that may perform prediction of "inter" signal and intra-directional prediction generation module 214 that may perform prediction of "intra" signal. Characteristics and motion compensated filtering predictor module 213 may allow for increasing predictability by first compensating for other sources of differences (such as gain, global motion, registration) or creation of synthesized frames (super resolution, and projection), followed by actual motion compensation.
Filtering decoder subsystem 350b of decoder 300b may perform multiple filtering of the reconstructed pictures based on parameters sent by encoder 300a and may include several subsystems. The first example subsystem, deblock filtering modue 208, may deblock and dither to reduce or mask any potential block coding artifacts. The second example subsystem, quality restoration filtering module 209, may perform general quality restoration to reduce the artifacts due to quantization operation in any video coding. The third example subsystem, characteristics and motion compensated filtering predictor module 213, may improve results from motion compensation by using a filter that may adapt to the motion characteristics (motion speed/degree of blurriness) of the content. The fourth example subsystem, prediction fusion filtering module 216, may allow adaptive filtering of the prediction signal (which may reduce spurious artifacts in prediction, often from intra prediction) thereby reducing the prediction error which may need to be coded.
Postrestorer subsystem 390b of decoder 300b is an optional block that may perform further improvement of perceptual quality of decoded video. This processing can be done either in response to quality improvement parameters sent by encoder 100, or it can be standalone decision made at the postrestorer subsystem 390b. In terms of specific parameters computed at encoder 100 that can be used to improve quality at postrestorer subsystem 390b may be estimation of film grain noise and residual blockiness at encoder 100 (even after deblocking). As regards the film grain noise, if parameters can be computed and sent via output bitstream 11 1 and/or input bitstream 201 to decoder 200, then these parameters may be used to synthesize the film grain noise. Likewise, for any residual blocking artifacts at encoder 100, if they can be measured and parameters sent via output bitstream 11 1 and/or bitstream 201, postrestorer subsystem 390b may decode these parameters and may use them to optionally perform additional deblocking prior to display. In addition, encoder 100 also may have access to scene change, spatial complexity, temporal complexity, motion range, and prediction distance information that may help in quality restoration in postrestorer subsystem 390b.
While subsystems 330b through 390b are illustrated as being associated with specific example functional modules of decoder 300b in Fig. 3(b), other implementations of decoder 300b herein may include a different distribution of the functional modules of decoder 300b among subsystems 330b through 390b. The present disclosure is not limited in this regard and, in various examples, implementation of the example subsystems 330b through 390b herein may include the undertaking of only a subset of the specific example functional modules of decoder 300b shown, additional functional modules, and/or in a different arrangement than illustrated.
FIG. 4 is an illustrative diagram of an example entropy encoder module 1 10, arranged in accordance with at least some implementations of the present disclosure. As shown, entropy encoder module 1 10 may include bitstream headers, parameters and maps data encoder module 401, picture partitions, prediction partitions, and coding partitions encoder module 402, coding modes and reference types encoder module 403, coded/not- coded data encoder module 404, transform coefficients encoder module 405, motion vector prediction and differential motion vector encoder module 406, intra-prediction type and direction data encoder module 407, and/or bitstream assembler module 408. In the discussion herein, each of modules 401^107 may be shortened to encoder module 401, encoder module 404, or the like for the sake of brevity.
As shown, encoder modules 401-407 may receive video data 411— 417, respectively, via adaptive entropy encoder 110. In some examples, received video data 41 1-417 may be received from various modules of encoder 100 as discussed herein. As shown, encoder modules 401-407 may compress the received video data 41 1-417 to generated compressed video data 421-427. Compressed video data 421-427 may be provided to bitstream assembler 408, which may assemble compressed video data 421^127 to generate output bitstream 11 1.
In some examples, encoder modules 401^107 may each include one or more specialized component(s) for efficiently encoding the type of data associated with received video data 41 1-417. In some examples, one or more of encoder modules 401-407 may preprocess the received video data 41 1-417 and/or select an entropy coding technique based on a parameter, parameters, or characteristics of the received video data 411— 417 or other system information.
For example, encoder module 401 may receive overhead data 41 1, which may include bitstream header data (e.g., sequence and/or picture level bitstream headers), morphing parameters, synthesizing parameters, or global maps data (e.g., quantizer maps of pictures indicating quantizers to be used on a partition basis). As is discussed further below with respect to FIG. 6, in some examples, encoder module 401 may implement an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression of video data 411. In some examples, encoder module 41 1 may determine which technique provides the greatest compression efficiency (e.g., the fewest bits for compressed video data 421) such that the parameter(s) associated with video data 411 may be the number of bits needed for each coding technique or the like. Encoder module 411 may entropy encode video data 411 to generate compressed video data 421 (e.g., compressed overhead data), which may be transmitted to bitstream assembler 408 as shown.
As discussed, in some examples, the parameter associated with the video data (e.g., any of video data 41 1-417) may be a fewest number of attainable bits, most efficient encoding technique, or the like. In other examples, the parameter associated with the video data may be a predefined or predetermine parameter such that encoding technique is predetermined. In some examples, the parameter associated with the video data may be based on a characteristic of the video data such that the determined encoding technique may be adaptive to the received video data as is discussed further herein.
As shown, in some examples, encoder module 402 may receive partition data 412, which may include picture slices or regions data, intra-prediction partition data, and/or inter-prediction partition and coding partition data. As is discussed further below with respect to FIG. 6, in some examples, encoder module 412 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for the compression of the intra-prediction partition data and/or inter-prediction partition data portions of video data 412 based on a parameter associated with the intra- prediction partition data and/or inter-prediction partition data (e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter,
characteristics of video data 412, or the like), and encoder module 412 may implement adaptive codebook variable length coding for the slices or regions data portions of video data 412 to generate compressed video data 422 (e.g., compressed partition data), which may be transmitted to bitstream assembler 408 as shown. In some examples, the intra- prediction partition data and/or inter-prediction partition data may include data indicating the partitioning of tiles into partitions, partitions into sub-partitions, or the like. In some examples, the partitions and/or sub-partitions may include prediction partitions and/or subpartitions. In some examples, partitions and/or sub-partitions may include coding partitions and/or sub-partitions.
Further as shown, in some examples, encoder module 403 may receive modes and reference types data 413, which may include modes (e.g., intra, split, skip, auto, inter, or multi) data and/or references data for each prediction partition. For example, the mode split information may indicate whether a partition is further divided or not. If a partition is further divided, the mode data may further include direction information indicating whether the split is a horizontal split (e.g., hor) or a vertical split (e.g., vert). As is discussed further below with respect to FIG. 6, in some examples, encoder module 403 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for separate coding of split/non-split partition information data, separate coding of split/non-split split information data, or prediction reference information data based on a parameter associated with the data (e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter, characteristics of the pertinent portions of video data 413, or the like), and encoder module 403 may implement adaptive variable length coding for joint coding of modes and split information to generate compressed video data 423 (e.g., compressed modes and reference types data), which may be transmitted to bitstream assembler 408 as shown.
Further, in some examples, encoder module 404 may receive coded / not-coded data 414, which may include coded/not-coded data as discussed herein. For example, a partition (or sub-partition) may be coded if it has any nonzero transform coefficients and a partition (or sub-partition) may be not-coded if it has all zero transform coefficients. In some examples, coded/not-coded data may not be needed for partitions having an intra or skip mode. In some examples, coded/not-coded data may be needed for partitions having an auto, inter, or multi mode. As is discussed further below with respect to FIG. 6, in some examples, encoder module 404 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for coded/not- coded data based on a parameter associated with the coded/not-coded data (e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter, characteristics of video data 414, or the like) to generate compressed video data 424 (e.g., compressed coded / not-coded data), which may be transmitted to bitstream assembler 408 as shown.
In some examples, encoder module 405 may receive transform data 415, which may include transform coefficient data. For example, for blocks or partitions or sub-partitions that are coded (e.g., have one or more nonzero transform coefficients), transform coefficient data may be received for entropy encoding. As is discussed further with respect to FIG. 6, encoder module 405 may implement adaptive vector variable length coding for blocks or partitions or sub-partitions having a block or partition or sub-partition size of 2 in one dimension (e.g., 2xK partitions or Kx2 sized partitions). Further, encoder module 405 may implement adaptive 1 -dimensional variable length coding for blocks or partitions or sub-partitions of size 4x4 and adaptive 2-dimensional variable length coding for all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on). The generated compressed video data 425 (e.g., compressed transform data) may be transmitted to bitstream assembler 408 as shown.
For example, the transform coefficient data may result from a forward transform of rectangular (or square or the like) partitions of pixel data or rectangular (or square or the like ) of pixel difference values implemented via adaptive transform module 108 followed by a quantization of the resulting coefficients via adaptive quantize module 109. In some examples, the transform coefficient data may be scanned to convert it to a 1 -dimensional frequency ordered partition via encoder module 405. Such conversion may be highly adaptive any partition size (e.g., 24 or more or 32 or more partition sizes), different data types (e.g., discrete cosine transform coefficients or hybrid parametric Haar transform coefficients or the like of either intra or inter partitions), and/or different quantizer set characteristics (e.g., various combinations of quantizer parameters and/or matrices).
Further, a block or partition or sub-partition may belong to different picture types: I-picture (e.g., intra compensation only), P-picture (e.g., predictive) or F-picture (e.g., functional) and/or may represent different types of signal or data (e.g., luma or chroma or the like), which may be quantized with different quantizer setting.
Further, in some examples, encoder module 406 may receive motion data 416, which may include motion vector data. As is discussed further with respect to FIG. 6, motion vector prediction may be performed based on video data 416 to generate one or more predicted motion vectors. A predicted motion vector may be differenced with an original motion data of video data 416 to generate a difference motion vector. In some examples, encoder module 416 may implement an adaptive classified variable length coding for the difference motion vector(s) to generate compressed video data 426 (e.g., compressed motion data), which may be transmitted to bitstream assembler 408 as shown.
Further, in some examples, encoder module 407 may receive intra-prediction data 417, which may include intra-prediction type or intra-prediction direction data. For example, as discussed, intra coding may use prediction, which may use neighboring past decoded partition(s) within the same frame to generate spatial prediction. In such examples, there may be predictors for indicating a past decoded partition or partitions. For example, the predictors may include dc, slope, directional, BTPC, feature matching, or the like. Further, in some examples, the directional predictor may be adaptive for different partition sizes. For example, specifying a directional predictor may include providing an technique for determining angular prediction pixel partition(s) for coding using causal neighboring decoded partitions and/or specifying a technique for entropy coding spatial prediction directions. In some examples, such techniques may be performed via encoder module 407. As is discussed further below with respect to FIG. 6, in some examples, encoder module 407 may implement an adaptive variable length coding technique or an arithmetic coding technique for intra-prediction type or intra-prediction direction data based on a parameter associated with the intra-prediction type or intra-prediction direction data (e.g., a fewest number of attainable bits, most efficient encoding technique, a predetermined parameter, characteristics of video data 417, or the like) to generate compressed video data 427 (e.g., compressed intra-prediction data), which may be transmitted to bitstream assembler 408 as shown.
As shown in FIG. 4, adaptive entropy encoder 1 10 may include bitstream assembler 408. In some examples, some or all of encoder modules 401^107 may provide entropy coded compressed video data 421^127 at different instances in time. Further, in some examples, one or some of compressed video data 421-427 may be picture based, region or slice based, tile based, prediction partition based, coding partition based, or any combination thereof. In some examples, bitstream assembler may multiplex (the potentially different) compressed video data 421^127 to create a valid bitstream such as, for example, output bitstream 1 11. For example, the valid bitstream may be a valid next generation video (NGV) coded bitstream, which may following a NGV bitstream syntax specification. In some examples, output bitstream 1 11 may be a video only bitstream. In some examples, output bitstream 1 11 may be multiplexed (e.g., Transport or a Media File Format) with uncoded or coded audio to create a multiplexed audio-visual stream. In any event, the bitstream may be used local decode, storage, or transmission to a decoder as discussed herein.
FIG. 5 is an illustrative diagram of an example entropy decoder module 202, arranged in accordance with at least some implementations of the present disclosure. As shown, entropy decoder module 202 may include bitstream headers, parameters and maps data decoder module 501, picture partitions, prediction partitions, and coding partitions decoder module 502, coding modes and reference types decoder module 503, coded/not- coded data decoder module 504, transform coefficients decoder module 505, motion vector and differential motion vector decoder module 506, intra-prediction and direction data decoder module 507, and/or bitstream disassembler module 508. In the discussion herein, each of modules 501-507 may be shortened to decoder module 501, decoder module 505, or the like for the sake of brevity.
As shown, bitstream disassembler module 508 may receive input bitstream 201. In some examples, input bitstream 201 may be a valid bitstream such as, for example, a valid next generation video (NGV) coded bitstream, which may follow a NGV bitstream syntax specification. In some examples, input bitstream 201 may be a video only bitstream. In some examples, input bitstream 201 may be a multiplexed audio-visual stream as discussed herein. Bitstream disassembler module 508 may disassemble input bitstream 201 to determine compressed video data 51 1-517 as shown. For example, bitstream disassembler module 508 may use a predefined syntax or specification to divide input bitstream 201 into component compressed video data 51 1-517 by data type for decompression via decoder modules 501-507. In some examples, bitstream disassembler module 508 may perform an inverse operation with respect to bitstream assembler module 508.
As shown in FIG. 5, decoder modules 501-507 may receive compressed video data 511-517, respectively, and generate video data 521-527. Video data 521-527 may be transmitted to various components of decoder 200 for further decoding as discussed herein.
Decoder 200 may thereby generate video frame(s) for presentment to a user via a display device (not shown). In some examples, decoder modules 501-507 may each perform an inverse operation with respect to encoder modules 401-407. In some examples, decoder modules 501-507 may each include one or more specialized component(s) for efficiently entropy decoding the type of data associated with compressed video data 511-517.
For example, decoder module 501 may receive compressed overhead data 51 1, which may include compressed bitstream header data (e.g., sequence and/or picture level bitstream headers), morphing parameters, synthesizing parameters, or global maps data (e.g., quantizer maps of pictures indicating quantizers to be used on a partition basis). In some examples, decoder module 511 may implement an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables for decompression of compressed overhead data 511 to generate overhead data 521. In some examples, decoder module 501 may determine which coding technique to implement based on a parameter or indicator provided via bitstream 201.
As shown, in some examples, decoder module 502 may receive compressed partition data 512, which may include compressed picture slices or regions data, intra-prediction partition data, and/or inter-prediction partition data. In some examples, decoder module 512 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for the decompression of the intra- prediction partition data and/or inter-prediction partition data portions of compressed partition data 512, and decoder module 512 may implement an adaptive codebook variable length coding for the decompression of the slices or regions data portions of compressed partition data 512 to generate partition data 522. In some examples, the intra-prediction partition data and/or inter-prediction partition data may include data indicating the partitioning of tiles into partitions, partitions into sub-partitions, or the like. In some examples, the partitions and/or sub-partitions may include prediction partitions and/or subpartitions. In some examples, partitions and/or sub-partitions may include coding partitions and/or sub-partitions. In some examples, decoder module 502 may determine which coding technique to implement for the decompression of the intra-prediction partition data and/or inter-prediction partition data portions of compressed video data 512 based on a parameter or indicator provided via bitstream 201.
Further, in some examples, decoder module 503 may receive compressed modes and reference types data 513, which may include compressed modes (e.g., intra, split, skip, auto, inter, or multi) data and/or references data for each prediction partition. For example, the mode split information may indicate whether a partition is further divided or not. If a partition is further divided, the mode data may further include direction information indicating whether the split is a horizontal split (e.g., hor) or a vertical split (e.g., vert). In some examples, decoder module 503 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for decompression of separate coding of split/non-split partition information data, separate coding of split/non-split split information data, or prediction reference information data, and decoder module 503 may implement an adaptive variable length coding for decompression of joint coding of modes and split information to generate modes and reference types data 523. In some examples, decoder module 503 may determine which coding technique to implement for decompression of separate coding of split/non-split partition information data, separate coding of split/non-split split information data, or prediction reference information data based on a parameter or indicator provided via bitstream 201.
Further, in some examples, decoder module 504 may receive compressed coded / not-coded data 514, which may include coded/not-coded data as discussed herein. For example, a partition (or sub-partition) may be coded if it has any nonzero transform coefficients and a partition (or sub-partition) may be not-coded if it has all zero transform coefficients. In some examples, coded/not-coded data may not be needed for partitions having an intra or skip mode. In some examples, coded/not-coded data may be needed for partitions having an auto, inter, or multi mode. In some examples, decoder module 504 may implement an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique for decompression of coded/not-coded data to generate coded/not-coded data 524. In some examples, decoder module 504 may determine which coding technique to implement for decompression based on a parameter or indicator provided via bitstream 201.
As shown, in some examples, decoder module 505 may receive compressed transform data 515, which may include transform coefficient data. For example, for blocks or partitions or sub-partitions that are coded (e.g., have one or more nonzero transform coefficients) compressed transform data 515 may include transform coefficient data. In some examples, decoder module 505 may implement an adaptive vector variable length coding for decompression of blocks or partitions or sub-partitions having a block or partition or sub-partition size of 2 in one dimension (e.g., 2xK partitions or Kx2 sized partitions. Further, decoder module 505 may implement an adaptive 1 -dimensional variable length coding for decompression of blocks or partitions or sub-partitions of size 4x4 and adaptive 2-dimensional variable length coding for decompression of all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on). The generated transform data 525 may be transmitted to other module(s) of decoder 200 as shown.
Further, in some examples, decoder module 506 may receive compressed motion data 516, which may include motion vector data. As is discussed further below with respect to FIG. 6, in some examples, compressed motion data 516 may be decompressed using an adaptive classified variable length coding technique to generate predicted difference motion vectors. The predicted difference motion vectors may be added to motion vector prediction to generate reconstructed motion vectors. The motion vector prediction may be generated based on previously decoded motion vectors of neighboring blocks or partitions using the inverse of the technique implemented via encoder module 506, for example, and/or motion vectors. The reconstructed motion vectors may be transmitted to other module(s) of decoder 200 via motion data 526 as shown.
Further, in some examples, decoder module 507 may receive compressed intra- prediction data 517, which may include intra-prediction type or intra-prediction direction data. For example, as discussed, intra coding may use prediction, which may use neighboring past decoded partition(s) within the same frame to generate spatial prediction. In such examples, there may be predictors for indicating a past decoded partition or partitions. For example, the predictors may include dc, slope, directional, BTPC, feature matching, or the like. Further, in some examples, the directional predictor may be adaptive for different partition sizes. For example, specifying a directional predictor may include providing an technique for determining angular prediction pixel partition(s) for coding using causal neighboring decoded partitions and/or specifying a technique for entropy coding spatial prediction directions. In some examples, decoder module 517 may implement an adaptive variable length coding technique or an arithmetic coding technique for decompression of intra-prediction type or intra-prediction direction data to generate intra-prediction data 527. In some examples, decoder module 507 may determine which coding technique to implement for decompression based on a parameter or indicator provided via bitstream 201.
As discussed above, a variety of entropy coding techniques may be implemented on various data types for lossless compression of video data to generate compressed video data at an entropy encoder and for decompression of the compressed video data to generate duplicate video data at an entropy decoder.
In some examples, an adaptive symbol-run variable length coding technique may be implemented. For example, encoder and decoder modules 401, 501, 402, 502, 403, 503, and/or 404, 504 may implement an adaptive symbol-run variable length coding technique on some or all of the video data or compressed video data received.
In some examples, an adaptive symbol-run variable length coding technique may include coding of relative difference in addresses between non-skipped blocks within a frame in video coding that allows one to determine the number of consecutive skipped blocks. For example, in the context of coded/not-coded data as encoded and decoded via encoder and decoder modules 504, 504, instead of sending one bit (e.g., bit-map) for each block to signal coded/not-coded (e.g., skipped) blocks, encoder module 504, for example, may encode a run of skipped blocks. In such implementations, the longer the run of skipped blocks, the more efficiently the data may be compressed.
Further, several types of adaptivity may be added to adaptive symbol-run variable length coding technique as described herein: adaptivity that may allow for use of multiple tables, adaptivity that may allow for use of either performing this type of coding on original bit map data, inverted bitmap, differential bitmap, or gradient predictive bitmap, or the like. For example, the adaptive symbol-run variable length coding technique may include converting the first video data from bit map data to at least one of an inverted bitmap, a differential bit map, or a gradient predictive bit map before applying adaptive symbol-run variable length coding. For example, adaptive symbol-run variable length coding technique may be used to entropy encode substantially any type of event (e.g., symbol/run combination). Further, symbol/run events may be used to code multi-level (e.g., 0, 1, 2, 3, etc.) or binary (e.g., 0, 1) events. In examples where multi-level events are encoded, adaptive symbol-run variable length coding technique may be applied a number of consecutive times, breaking a multi-level map into a number of binary sub-maps, with each previous sub-map, excluded from next level's sub-map, or the like.
In some examples, an adaptive proxy variable length coding technique may be implemented. For example, encoder and decoder modules 401, 501, 402, 502, 403, 503, and/or 404, 504 may implement an adaptive proxy variable length coding technique on some or all of the video data or compressed video data received.
In some examples, an adaptive proxy variable length coding technique may include substitution of original fixed length ID blocks (e.g., groups) of bits with variable length codes (e.g., patterns of sequence of bits) such that after the replacement the resulting bitstream may be smaller in size. In some examples, at the decoder, the process may be repeated (or inversed) resulting in original intended bitstream. In some examples, the original blocks of bits replaced may be of fixed small sizes (e.g., groups of 2 bits, groups of 3 bits, or groups of 4 bits, or the like). In some examples, the replacement codes may be of small size and variable length in nature. In some examples, the adaptive proxy variable length coding discussed herein may be characterized as Short VLCs (e.g., variable length codes). Further, the adaptive proxy variable length coding technique described herein may be adaptive to content by providing multiple replacement options. In some examples, 1- dimensional blocks/groups of 2 bits may be replaced with 1-3 bit long codes. In some examples, 1-dimensional blocks/groups (or collections of blocks/groups) of 3 bits with codes may be replaced with 1-5 bit long codes. In some examples, the adaptive proxy variable length coding technique may exploit statistical redundancy within a bitstream. In some examples, the adaptive proxy variable length coding technique may provide a compression gain of about 1-1.3. In some examples, adaptive proxy variable length coding technique may offer the advantage of being amenable to application to short sequence of bits.
In some examples, an adaptive block-of-symbols variable length coding technique may be implemented. For example, encoder and decoder modules 405, 505 may implement an adaptive block-of-symbols variable length coding technique on some or all of the video data or compressed video data received.
In some examples, an adaptive block-of-symbols variable length coding technique may include two sub-coding techniques, as will be discussed further with respect to FIG. 7. For example, the adaptive block-of-symbols variable length coding technique may include an adaptive vector variable length coding technique and an adaptive 1D/2D (1- dimensional/2-dimensional) variable length coding technique. In some examples, the adaptive block-of-symbols variable length coding technique may be used to encode blocks of closely related symbols such as blocks of transform coefficients as discussed herein.
In some examples, the adaptive vector variable length coding technique of the adaptive block-of-symbols variable length coding technique may encode relatively small 2D blocks or partitions of symbols by use of a joint single codeword such that coding a block of symbols may result in a VLC (variable length coding) codebook. In some examples, the larger the size of the block or partition, the larger the size of the codebook. In some examples, the adaptive vector variable length coding technique may be applied to block or partition sizes having a size 2 in one dimension (e.g., 2xK or Kx2 blocks or partitions). By applying the adaptive vector variable length coding technique to blocks or partitions of such sizes, the size of the VLC codebook may be advantageously limited.
In some examples, the adaptive ID variable length coding technique of the adaptive block-of-symbols variable length coding technique may be used for coding 4x4 transform coefficient block or partition sizes, is essentially same as the CAVLC coder. This coder is primarily used for coding 4x4. In some examples, the adaptive ID variable length coding technique may be implemented via a content adaptive variable length coding technique with a number of different VLC Tables used based on the context of the coefficient(s) being coded. For example, based on the context of the coefficient(s) being coded encoder and/or decoder modules 505, 505 may switch VLC Tables.
In some examples, the adaptive 2D variable length coding technique of the adaptive block-of-symbols variable length coding technique may utilize two dimensional properties of a block of symbols to switch based on context between a number of different VCL Tables. In some examples, the adaptive 2D variable length coding technique may be characterized as a CA2DVLC (Content Adaptive 2D Variable Length) coder. In some examples, In some examples, the adaptive 2D variable length coding technique may be used to encode all remaining transform coefficient block or petition sizes besides 2xK, Kx2 blocks and 4x4 blocks (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on).
FIG. 6 is an illustrative diagram of an example entropy encoder module 1 10, arranged in accordance with at least some implementations of the present disclosure. As shown and as discussed above with respect to FIG. 4, entropy encoder module 110 may include bitstream headers, parameters and maps data encoder module 401, picture partitions, prediction partitions, and coding partitions encoder module 402, coding modes and reference types encoder module 403, coded/not-coded data encoder module 404, transform coefficients encoder module 405, motion vector and differential motion vector encoder module 406, intra-prediction and direction data encoder module 407, and/or bitstream assembler module 408.
As shown, encoder module 401 may include adaptive VLC, symbol-run VLC, and/or proxy VLC encoder for bitstream headers, parameters, and maps data module 611 and may receive video data 411. Video data 411 may have a data type such that video data 411 may include bitstream header data (e.g., sequence and/or picture level bitstream headers), morphing parameters, synthesizing parameters, and/or global maps data (e.g., quantizer maps of pictures indicating quantizers to be used on a partition basis). In some examples, an entropy encoding technique may be determined for video data 411 based on a parameter, parameters or characteristics of video data 41 1 or other system parameters. In some examples, the entropy encoding technique for video data 41 1 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above, or a variable length coding table or tables compression technique. The determined entropy encoding technique may be applied to video data 41 1 to generate compressed video data 421. A variable length coding table or tables compression technique may include a content adaptive variable length coding technique with one or more tables available for coding based on video data 411 , for example. In some examples, encoder module 404 may determine which technique provides the greatest compression efficiency (e.g., the fewest bits for compressed video data 421) such that the parameter(s) associated with video data 41 1 may be the number of bits needed for each coding technique or the like. In some examples, the parameter associated with video data 41 1 may be based on a characteristic of the video data such that the determined encoding technique may be adaptive to the received video data. As shown in FIG. 6, encoder module 402 may include adaptive codebook VLC encoder for picture partitions module 621, symbol-run VLC and/or proxy VLC encoder for intra-prediction partitions module 622, and/or symbol-run VLC and/or proxy VLC encoder for inter-prediction partitions and coding partitions module 623. Also as shown, encoder module 402 may receive video data 412. In some examples, video data 412 may include picture slices or regions data 624, intra-prediction partition data 625, and/or inter- prediction and coding partition data 626.
As shown, picture slices or regions data 624 may be received via adaptive codebook VLC encoder for picture partitions module 621, which may apply adaptive codebook variable length coding to picture slices or regions data 624 to generate compressed picture slices or regions data 627. In some examples, picture slices or regions data 624 may include region boundaries for pictures, slices, regions, or the like. In some examples, adaptive codebook variable length coding may include content adaptive variable length coding using an codebook adaptive to the content of picture slices or regions data 624 or other system parameters or the like.
As shown, intra-prediction partition data 625 may be received via symbol-run VLC and/or proxy VLC encoder for intra-prediction partitions module 622. In some examples, an entropy encoding technique may be determined for intra-prediction partition data 625 based on a parameter, parameters or characteristics of intra -prediction partition data 625 or other system parameters (e.g., compression efficiency, a characteristic of the data, and so on), as discussed herein. The determined entropy encoding technique may be applied to intra-prediction partition data 625 to generate compressed intra-prediction partition data 628. As shown, in some examples, the entropy encoding technique for intra-prediction partition data 625 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above. In some examples, intra-prediction partition data 625 may include partitions based on bi-tree partitioning or k-d tree partitioning, or the like.
As shown, inter-prediction and coding partition data 626 may be received via symbol-run VLC and/or proxy VLC encoder for inter-prediction partitions and coding partitions module 623. In some examples, an entropy encoding technique may be determined for or inter-prediction and coding partition data 626 based on a parameter, parameters or characteristics of or inter-prediction and coding partition data 626 or other system parameters (e.g., compression efficiency, a characteristic of the data, and so on), as discussed herein. The determined entropy encoding technique may be applied to or inter- prediction and coding partition data 626 to generate compressed or inter-prediction and coding partition data 629. As shown, in some examples, the entropy encoding technique for or inter-prediction and coding partition data 626 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above. In some examples, or inter-prediction and coding partition data 426 may include inter-partitions and coding tree partitions, or the like.
As shown in FIG. 6, encoder module 403 may include adaptive VLC encoder for joint modes and splits module 631, symbol-run VLC and/or proxy VLC encoder for modes module 632, symbol-run VLC and/or proxy VLC encoder for splits module 633, and/or symbol-run VLC and/or proxy VLC encoder for reference types module 634. Also as shown, encoder module 403 may receive video data 412. In some examples, video data 412 may joint coding of modes and splits data 635, modes information data 636, split/not- split information data 637, and/or prediction reference information data 638.
As shown, joint coding of modes and splits data 635 may be received via adaptive VLC encoder for joint modes and splits module 631, which may apply adaptive variable length coding to joint coding of modes and splits data 635 to generate compressed joint coding of modes and splits data 639. In some examples, adaptive variable length coding may include content adaptive variable length coding adaptive to the content of joint coding of modes and splits data 635 or other system parameters or the like.
As discussed, in some examples, modes and splits data may be coded jointly via adaptive VLC encoder for joint modes and splits module 631. In some examples, modes and splits data may be coded separately via symbol-run VLC and/or proxy VLC encoder for modes module 632 and symbol-run VLC and/or proxy VLC encoder for splits module 633, as is discussed below. In some examples, encoder 100 (via, e.g., adaptive entropy encoder 1 10 and/or encode controller 103) to code jointly or separately based on results of a comparison of the coding techniques to determine which technique compresses the data most efficiently.
As shown, modes information data 636 may be received via symbol-run VLC and/or proxy VLC encoder for modes module 632. In some examples, an entropy encoding technique may be determined for modes information data 636 based on a parameter, parameters or characteristics of modes information data 636 or other system parameters (e.g., compression efficiency, a characteristic of the data, and so on), as discussed herein. The determined entropy encoding technique may be applied to modes information data 636 to generate compressed modes information data 642. As shown, in some examples, the entropy encoding technique for modes information data 636 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above.
As shown, split/not-split information data 637 may be received via symbol-run VLC and/or proxy VLC encoder for splits module 633. In some examples, an entropy encoding technique may be determined for split/not-split information data 637 based on a parameter, parameters or characteristics of split/not-split information data 637 or other system parameters (e.g., compression efficiency, a characteristic of the data, and so on), as discussed herein. The determined entropy encoding technique may be applied to split/not- split information data 637 to generate compressed split/not-split information data 643. As shown, in some examples, the entropy encoding technique for split/not-split information data 637 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above.
As shown, prediction reference information data 638 may be received via symbol- run VLC and/or proxy VLC encoder for reference types module 634. In some examples, an entropy encoding technique may be determined for prediction reference information data 638 based on a parameter, parameters or characteristics of prediction reference information data 638 or other system parameters (e.g., compression efficiency, a characteristic of the data, and so on), as discussed herein. The determined entropy encoding technique may be applied to prediction reference information data 638 to generate compressed prediction reference information data 644. As shown, in some examples, the entropy encoding technique for prediction reference information data 638 may be one of an adaptive symbol- run variable length coding technique or an adaptive proxy variable length coding technique, as described above.
As shown, encoder module 404 may include symbol-run VLC and/or proxy VLC encoder for coded/not-coded data 641 and may receive video data 414. Video data 414 may have a data type such that video data 41 1 may coded/not-coded data. For example, a partition (or sub-partition) may be coded if it has any nonzero transform coefficients and a partition (or sub-partition) may be not-coded if it has all zero transform coefficients. In some examples, coded/not-coded data may not be needed for partitions having an intra or skip mode. In some examples, coded/not-coded data may be needed for partitions having an auto, inter, or multi mode. In some examples, an entropy encoding technique may be determined for video data 414 based on a parameter, parameters or characteristics of video data 414 or other system parameters. In some examples, the entropy encoding technique for video data 414 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described above. The determined entropy encoding technique may be applied to video data 414 to generate compressed video data 424. In some examples, encoder module 404 may determine which technique provides the greatest compression efficiency as discussed such that the parameter(s) associated with video data 411 may be the number of bits needed for each coding technique or the like. In some examples, the parameter associated with video data 411 may be based on a characteristic of the video data such that the determined encoding technique may be adaptive to the received video data.
As shown, in some examples, encoder module 405 may include an adaptive vector VLC encoder for transform coefficients module 651 and/or an adaptive ID and 2D VLC encoder for transform coefficients module 652.
As shown, adaptive vector VLC encoder for transform coefficients module 651 may receive transform coefficients data 653, which may include transform coefficient data for blocks or partitions or sub-partitions having a block or partition or sub-partition size of 2 in one dimension (e.g., 2xK partitions or Kx2 sized partitions). An adaptive vector variable length coding technique may be applied to transform coefficients data 653 to generate compressed transform coefficients data 655. In some examples, the adaptive vector variable length coding technique may include a quad-tree division of the block or partition or the, representing each quadrant generated via the quad-tree division with a single vector codeword that represents all coefficients with a single index value, and entropy coding the codeword using a variable length coding technique or the like.
Also as shown, adaptive ID and 2D VLC encoder for transform coefficients module 652 may receive transform coefficient data 654 and may implement an adaptive 1- dimensional variable length coding for blocks or partitions or sub-partitions of size 4x4 and an adaptive 2-dimensional variable length coding for all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on) to generate compressed transform coefficient data 656. As discussed, transform coefficient data 653, 654 may result from a forward transform of rectangular (or square or the like) partitions of pixel data or rectangular (or square or the like ) of pixel difference values implemented via adaptive transform module 108 followed by a quantization of the resulting coefficients via adaptive quantize module 109. In some examples, the transform coefficient data may be scanned to convert it to a 1 -dimensional frequency ordered partition via encoder module 405. Such conversion may be highly adaptive any partition size (e.g., 24 or more or 32 or more partition sizes), different data types (e.g., discrete cosine transform coefficients or hybrid parametric Haar transform coefficients or the like of either intra or inter partitions), and/or different quantizer set characteristics (e.g., various combinations of quantizer parameters and/or matrices). Further, a block or partition or sub-partition may belong to different picture types: I-picture (e.g., intra compensation only), P-picture (e.g., predictive) or F-picture (e.g., functional) and/or may represent different types of signal or data (e.g., luma or chroma or the like), which may be quantized with different quantizer setting.
As shown, encoder module 406 may include a motion vector predictor module 661, an adaptive VLC encoder for motion vector differences module 662, and/or a differencer 663. As shown, encoder module 406 may receive video data 416, which may include motion vector data, via motion vector predictor module 661. Motion vector predictor module 661 may perform motion vector prediction based on video data 416 (e.g., the motion vector data of video data 516) using original motion vector(s) to generate associated predicted motion vector(s). In some examples, the motion vector prediction may be based on immediate neighbors to left, right, above, or below the motion vector being predicted. In some examples, other spatial neighbors that may share the same or similar characteristics may be used. For example, a number of different types of prediction may be adaptively selected and the selection information may provided to decoder 200 via bitstream 1 1 1. Differencer 663 may difference the predicted motion vector(s) and the original motion vector(s) to generate difference motion vector(s) for entropy coding. As shown adaptive VLC encoder for motion vector differences module 662 may apply an adaptive variable length coding technique to the difference motion vector(s) to generate compressed video data 526. In some examples, differential (e.g., difference) motion vectors may have twice the range of original motion vectors. Further l/8th pel precision motion compensation may expand the range of the difference motion vector by a factor of 8. In some examples, to address such expansion, classification of large space(s) into smaller subintervals and indexing of vectors inside the subinterval may be used.
As shown, encoder module 407 may include adaptive VLC and/or arithmetic encoder for intra-prediction and directions data module 671 and may receive video data 417. Video data 517 may have a data type such that video data 417 may include intra-prediction type or intra-prediction direction data. In some examples, an entropy encoding technique may be determined for video data 417 based on a parameter, parameters or characteristics of video data 417 or other system parameters (e.g., compression efficiency or the like) as discussed herein. In some examples, the entropy encoding technique for video data 417 may be one of an adaptive variable length coding technique or an arithmetic coding technique, as described above. The determined entropy encoding technique may be applied to video data 417 to generate compressed video data 427. In some examples, the adaptive variable length coding technique may include a content adaptive variable length coding based on the content of video data 417. In some examples, the arithmetic coding technique may include a content adaptive binary arithmetic coding based on the content of video data 417. In some examples, video data 417 may support 9 or more prediction directions and a variety of prediction types including planar, Binary Tree Predictive Coding (BTPC), or the like.
As shown in FIG. 6 and discussed above with respect to FIG. 4, the output encoder modules 401-407 (via the associated sub-modules) may be input to bitstream assembler 408, which may output a multiplexed bitstream formatted per the bitstream syntax, as discussed above.
FIG. 7 is an illustrative diagram of an example entropy decoder module 202, arranged in accordance with at least some implementations of the present disclosure. As shown and as discussed above with respect to FIG. 5, entropy decoder module 202 may include headers, parameters and maps data decoder module 501, picture partitions, prediction partitions, and coding partitions decoder module 502, coding modes and reference types decoder module 503, coded/not-coded data decoder module 504, transform coefficients decoder module 505, motion vector and differential motion vector decoder module 506, intra-prediction and direction data decoder module 507, and/or bitstream disassembler module 508. In some examples, entropy decoder module 202 (and the pertinent sub-modules) may perform an inverse technique with respect to entropy encoder module 110 (and the pertinent sub-modules) such that there may be a one-to-one correspondence between encoder modules (and sub-modules) and decoder modules (and sub-modules)
As shown, bitstream disassembler module 508 may receive input bitstream 201. In some examples, input bitstream 201 may be a valid bitstream such as, for example, a valid next generation video (NGV) coded bitstream, which may follow a NGV bitstream syntax specification or any valid bitstream as discussed herein. As discussed with respect to FIG. 5, bitstream disassembler module 508 may disassemble input bitstream 201 to determine compressed video data 51 1-517, which may each have one or more component parts as discussed further below. For example, bitstream disassembler module 508 may use a predefined syntax or specification to divide input bitstream 201 into component compressed video data 51 1-517 by data type for decompression via decoder modules 501— 507. In some examples, bitstream disassembler module 508 may perform an inverse operation with respect to bitstream assembler module 308. In some examples, the disassembling of input bitstream 201 may be characterized as a de-multiplexing.
As shown, decoder module 501 may include adaptive VLC, symbol-run VLC, and/or proxy VLC decoder for headers, parameters, and maps data module 711 and may receive compressed video data 51 1. In some examples, compressed video data 51 1 may include header data (e.g., sequence and/or picture level bitstream headers), morphing parameters, synthesizing parameters, and/or global maps data entropy encoded using one of an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression technique. In some examples, adaptive VLC, symbol-run VLC, and/or proxy VLC decoder for headers, parameters, and maps data module 71 1 may determine an entropy decoding technique applicable to compressed video data 51 1 and decode compressed video data 511 using the applicable technique to generate video data 521. In some examples, the applicable technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
As shown, decoder module 502 may include adaptive codebook VLC decoder for picture partitions module 721, symbol-run VLC and/or proxy VLC decoder for intra- prediction partitions data module 722, and/or symbol-run VLC and/or proxy VLC decoder for inter-prediction partitions and coding partitions data module 723 and may receive compressed video data 512.
As shown, compressed picture slices or regions data 724 may be received via adaptive codebook VLC decoder for picture partitions module 721. In some examples, adaptive codebook VLC decoder for picture partitions module 721 may apply adaptive codebook variable length coding to compressed picture slices or regions data 724 to generate picture slices or regions data 727. As discussed, adaptive codebook variable length coding may include content adaptive variable length coding using an codebook adaptive to the content of compressed picture slices or regions data 724 or other system parameters or the like. In some examples, the codebook may be implemented via adaptive codebook VLC decoder for picture partitions module 721.
As shown, symbol-run VLC and/or proxy VLC decoder for intra-prediction partitions data 722 may receive compressed intra-prediction partition data 725. In some examples, compressed intra-prediction partition data 725 may include compressed intra- prediction partition data entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described herein. In some examples, symbol-run VLC and/or proxy VLC decoder for intra prediction partitions data 722 may determine an entropy decoding technique applicable to compressed intra-prediction partition data 725 and entropy decode compressed intra- prediction partition data 725 using the applicable technique to generate intra-prediction partition data 728. In some examples, the applicable coding technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201
As shown, symbol-run VLC and/or proxy VLC decoder for inter-prediction partitions and coding partitions data 723 may receive compressed inter-prediction and coding partition data 726. In some examples, compressed inter-prediction and coding partition data 726 may include compressed inter-prediction and coding partition data entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described herein. In some examples, symbol-run VLC and/or proxy VLC decoder for inter-prediction partitions and coding partitions data 723 may determine an entropy decoding technique applicable to compressed inter-prediction and coding partition data 726 and entropy decode compressed inter-prediction and coding partition data 726 using the applicable technique to generate inter-prediction and coding partition data 729. In some examples, the applicable coding technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
As shown, decoder module 503 may include adaptive VLC decoder for joint modes and splits module 731, symbol-run VLC and/or proxy VLC decoder for modes module 732, symbol-run VLC and/or proxy VLC decoder for splits module 733, and/or symbol- run VLC and/or proxy VLC decoder for reference types module 734 and may receive compressed video data 513. As discussed above with respect to encoder module 403, in some examples, modes and splits data may be coded jointly and, in some examples, modes and splits data may be coded separately. In some examples, adaptive VLC decoder for joint modes and splits module 731 may decode jointly coded data, and symbol-run VLC and/or proxy VLC decoder for modes module 732 and symbol-run VLC and/or proxy VLC decoder for splits module 733 may decode separately coded data. In some examples, whether data is jointly or separately coded may be indicated via input bitstream 201.
As shown, compressed joint coding of modes and splits data 735 may be received via adaptive VLC decoder for joint modes and splits module 731. In some examples, adaptive VLC decoder for joint modes and splits module 731 may apply adaptive variable length coding to compressed joint coding of modes and splits data 735 to generate joint coding of modes and splits data 739. As discussed, adaptive variable length coding may be content adaptive variable length coding adaptive to the content of compressed joint coding of modes and splits data 735 or other system parameters or the like.
As shown, symbol-run VLC and/or proxy VLC decoder for modes module 732 may receive compressed modes information data 736. In some examples, compressed modes information data 736 may include compressed modes information data entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described herein. In some examples, symbol-run VLC and/or proxy VLC decoder for modes module 732 may determine an entropy decoding technique applicable to compressed modes information data 736 and entropy decode compressed modes information data 736 using the applicable technique to generate modes information data 742. In some examples, the applicable coding technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
As shown, symbol-run VLC and/or proxy VLC decoder for splits module 733 may receive compressed splits information data 737. In some examples, compressed splits information data 737 may include compressed splits information data entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described herein. In some examples, symbol-run VLC and/or proxy VLC decoder for splits module 733 may determine an entropy decoding technique applicable to compressed splits information data 737 and entropy decode compressed splits information data 737 using the applicable technique to generate splits information data 743. In some examples, the applicable coding technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
As shown, symbol-run VLC and/or proxy VLC decoder for reference types module 734 may receive compressed reference types information data 738. In some examples, compressed reference types information data 738 may include compressed reference types information data entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, as described herein. In some examples, symbol-run VLC and/or proxy VLC decoder for reference types module 734 may determine an entropy decoding technique applicable to compressed reference types information data 738 and entropy decode compressed reference types information data 738 using the applicable technique to generate reference types information data 744. In some examples, the applicable coding technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
As shown, decoder module 504 may include symbol-run VLC and/or proxy VLC decoder for coded/not-coded data module 741 and may receive compressed video data 514. In some examples, compressed video data 514 may include coded/not-coded, as discussed herein, entropy encoded using one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique. In some examples, symbol-run VLC and/or proxy VLC decoder for coded/not-coded data module 741 may determine an entropy decoding technique applicable to compressed video data 514 and decode compressed video data 514 using the applicable technique to generate video data 524. In some examples, the applicable technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201. In some examples, the applicable technique may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique.
As shown, decoder module 505 may include adaptive vector VLC decoder for transform coefficients module 751 and/or adaptive ID and 2D VLC decoder for transform coefficients module 752 and may receive compressed video data 515.
As shown, adaptive vector VLC decoder for transform coefficients module 751 may receive compressed transform coefficients data 753, which may include compressed transform coefficients data for blocks or partitions or sub-partitions having a size of 2 in one dimension (e.g., 2xK partitions or Kx2 sized partitions), as discussed herein. In some examples, adaptive vector VLC decoder for transform coefficients module 751 may apply an adaptive vector variable length coding technique to entropy decode compressed transform coefficients data 753 to generate transform coefficients data 755. As discussed, In some examples, the adaptive vector variable length coding technique may include using a variable length codeword to generate all coefficients of a quad-tree division of the block, which may be generated via merging the quad-tree division.
As shown, adaptive ID and 2D VLC decoder for transform coefficients module 752 may receive compressed transform coefficients data 754, which may include compressed transform coefficients data for blocks or partitions or sub-partitions of size 4x4 and all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on). In some examples, adaptive ID and 2D VLC decoder for transform coefficients module 752 may apply an adaptive 1 -dimensional variable length coding for blocks or partitions or sub-partitions of size 4x4 and an adaptive 2-dimensional variable length coding for all other block or partition or sub-partition sizes (e.g., 4x8, 8x4, 8x8, 16x16, 32x32, 64x64, and so on) to entropy decode compressed transform coefficients data 754 to generate transform coefficients data 756.
As shown, decoder module 506 may include an adaptive VLC decoder for motion vector differences module 762, a motion vector predictor 761 and an adder 763, and may receive compressed video data 516. In some examples, adaptive VLC decoder for motion vector differences module 762 may decode compressed video data 516 to generate motion vector differences. Furthermore, motion vector predictor 761 may generate prediction motion vectors using previously decoded neighboring motion vectors in analogy to the techniques discussed with respect to motion vector predictor module 661. As shown, decoded difference motion vector(s) may be added via adder 763 to prediction motion vector(s) to generate reconstructed motion vector(s), which may be output as a part of video data 526 and further used to perform motion vector prediction for other motion vectors via motion vector predictor module 761.
As shown, decoder module 507 may include adaptive VLC and/or arithmetic decoder for intra-prediction type and direction module 771 and may receive compressed video data 517. In some examples, compressed video data 517 may include intra-prediction type and direction data entropy encoded using one of an adaptive VLC technique or an arithmetic coding technique. In some examples, adaptive VLC and/or arithmetic decoder for intra- prediction type and direction module 771 may determine an entropy decoding technique applicable to compressed video data 517 and decode compressed video data 517 using the applicable technique to generate video data 527. In some examples, the applicable technique may be determined based on an indicator, parameter, header data, or the like conveyed via input bitstream 201.
As discussed, video data 521-527 (including various sub-components as discussed) may be transmitted to various components of decoder 200. Further, as discussed, decoder 200 may use entropy decoded video data 521-527 to generate video frames, which may be output, via display video 219, for presentment or display via a display device for a user.
FIG. 8 is a flow diagram illustrating an example process 800, arranged in accordance with at least some implementations of the present disclosure. Process 800 may include one or more operations, functions or actions as illustrated by one or more of operations 802, 804, 806, and/or 808. Process 800 may form at least part of a next generation video coding process. By way of non-limiting example, process 800 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or entropy encoder module 1 10 of FIGS. 5 or 7.
Process 800 may begin at operation 802, "Obtain First and Second Video Data of Different Types for Entropy Encoding", where first and second (or additional) video data of different types may be obtained or received for entropy coding. For example, two or more of video data 41 1-417 (and/or any sub-components of video data 511-517) may be received. In some examples, video data 411— 417 may be received via adaptive entropy encoder 110. As discussed, the first and second (or additional) video data may be of different types such as any of the types or sub-component types as discussed with respect to video data 41 1-417 or elsewhere herein.
Process 800 may continue at operation 804, "Determine an Entropy Encoding Technique for the First Video Data", where a first entropy encoding technique may be determined for the first video data. As discussed, in some examples, one or more of encoder modules 401^107 may determine a coding technique for video data 411— 417 from various coding technique options. For example, encoder module 401 may determine a coding technique for video data 411 from one of an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression technique. Further, encoder module 402 may determine a coding technique for intra-prediction partition data 625 of video data 402 from one of an adaptive symbol-run variable length coding technique and an adaptive proxy variable length coding technique, and so on. A wide range of examples have been provided herein with respect to FIGS. 4 and 6 and will not be repeated here for the sake of brevity. As discussed, in some examples, two ore more types of video data may be received. In some examples, an entropy encoding technique may be determined for two, three, or more types of video data as described herein.
As discussed, an entropy encoding technique may be determined for video data. In some examples, a selected coding table associated with the first encoding technique may be determined from two or more available tables. For example, the table selection may be made based on a bit count comparison, a coding efficiency, or the like between the available tables. For example, any of the described entropy encoding techniques discussed herein may have multiple available tables. In some examples, a table indicator may be generated and provided in bitstream 900 indicating the selected table for the video data. For example, bitstream 900 may include indicators indicating the selected encoding technique and the selected coding table. Further an indicator or indicators associated with the length of the video data may be included into bitstream 900. For example, the length or number of tuples in the video data (e.g. the length of the string of video) may be provided and any remainder portion (e.g., if the length is not evenly divided into tuples) may be coded using a bit map technique and provided into the output bitstream.
Process 800 may continue at operation 806, "Entropy Encode the First Video Data using the First Entropy Encoding Technique and Entropy Encode the Second Video Data to Generate First and Second Compressed Video Data", where the first video data may be entropy encoded using the first entropy encoding technique to generate first compressed video data and the second video data may be compressed using a second entropy encoding technique to generate first and second compressed video data.
Process 800 may continue at operation 808, "Assemble the First and Second Compressed Video Data to Generate an Output Bitstream", where the first compressed video data and the second compressed video data may be assembled to generate an output bitstream. For example, bitstream assembler 508 may assemble or multiplex the first compressed video data and the second compressed video data to generate output bitstream 1 11 as discussed herein.
As discussed, in some examples, output bitstream 1 11 may be multiplexed with an audio stream (coded or uncoded) to generate a multiplexed audio-visual stream. Further, as discussed, in some examples, one or more of the video data may be preprocessed or otherwise manipulated prior to entropy coding. For example, for motion vector data, motion vector prediction may be performed to generate predicted motion vector(s), the predicted motion vector(s) may be differenced with the original motion vector(s) to generate difference motion vector(s), and the difference motion vector(s) may be entropy coded as discussed herein. Further, in some examples, each of the seven encoder modules 401^107 may be implemented simultaneously to operate on seven types of video data 411— 417. Process 800 may be implemented via adaptive entropy encoder module 110 as discussed herein. Further, process 800 may be repeated either in serial or in parallel on any number of instantiations of video data.
As discussed, video data of different types may be entropy coded using a variety of determined (or predetermined) adaptive entropy coding techniques to generate compressed video data. The compressed video data may be assembled to generate an output bitstream.
FIG. 9 illustrates an example bitstream 900, arranged in accordance with at least some implementations of the present disclosure. In some examples, bitstream 900 may correspond to output bitstream 11 1 as shown in FIGS. 1, 3a, 4, and 6 and/or input bitstream 201 as shown in FIGS. 2, 3b, 5, and 7. Although not shown in FIG. 9 for the sake of clarity of presentation, in some examples bitstream 900 may include a header portion and a data portion. In various examples, bitstream 900 may include data, indicators, index values, mode selection data, or the like associated with encoding compressed video as discussed herein. As shown, in some examples, bitstream 900 may include indicator data 901, compressed video data 421, compressed video data 422, compressed video data 423, compressed video data 424, compressed video data 425, compressed video data 426, and/or compressed video data 427. The illustrated data may be in any order in bitstream 900 and may be adjacent or separated by any other of a variety of additional data for coding video. As discussed, bitstream 900 may also include indicators indicating the selected encoding technique and the selected coding table (e.g., in indicator data 901). Further an indicator or indicators associated with the length of the video data may be included into bitstream 900. For example, the length or number of tuples in the video data (e.g. the length of the string of video) may be provided and any remainder portion (e.g., if the length is not evenly divided into tuples) may be coded using a bit map technique and provided into the output bitstream. In some examples, compressed video data 421-427 may include any compressed video data encoded via any technique as discussed herein. In some examples, indicator data 901 may include header data, mode indicator data, and/or data indicating entropy encoding techniques associated with compressed video data 421^127. For example, indicator data 901 may indicate an entropy coding technique used for compressing video data 421, an entropy coding technique used for compressing portions of video data 422, an entropy coding technique used for compressing portions of video data 423, an entropy coding technique used for compressing video data 424, and/or an entropy coding technique used for compressing video data 427, as discussed herein with respect to FIGS. 4 and 6.
As discussed, bitstream 900 may be generated by an encoder such as, for example, encoder 100 and/or received by a decoder 200 for decoding such that video frames may be presented via a display device.
FIG. 10 is a flow diagram illustrating an example process 1000, arranged in accordance with at least some implementations of the present disclosure. Process 1000 may include one or more operations, functions or actions as illustrated by one or more of operations 1002, 1004, 1006, and/or 1008. Process 1000 may form at least part of a next generation video coding process. By way of non-limiting example, process 1000 may form at least part of a next generation video decoding process as undertaken by decoder system 200 of FIG. 2.
Process 1000 may begin at operation 1002, "Receive Entropy Encoded Bitstream", where an entropy encoded bitstream may be received. For example, a bitstream encoded as discussed herein may be received at a video decoder. In some examples, bitstream 100 or 900 or the like may be received via decoder 200.
Process 1000 may continue at operation 1004, "Disassemble the Entropy Encoded Bitstream to First Compressed Video Data and Second Compressed Video Data", where the received bitstream may be disassembled to determine different types of compressed video data. For example, bitstream 201 may be disassembled via bitstream disassembler 508 to generate compressed video data 51 1-517 (and any sub-component video data) as illustrated in FIG. 7. For example, the disassembled compressed video data may include first compressed video and second compressed video data.
Process 1000 may continue at operation 1006, "Entropy Decode the First and Second Compressed Video Data to Generate First Video Data and Second Video Data", where first and second compressed video data may be entropy decoded to generate (decompressed) first and second video data. As discussed, in some examples, one or more of decoder modules 501-507 may determine a coding technique for compressed video data 511-517 from various coding technique options. For example, decoder module 501 may determine a coding technique for compressed video data 511 from one of an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression technique based on an indicator or indicators provided via the input bitstream. Further, decoder module 502 may determine a coding technique for compressed intra-prediction partition data 725 of video data 512 from one of an adaptive symbol-run variable length coding technique and an adaptive proxy variable length coding technique, and so on. A wide range of examples have been provided herein with respect to FIGS. 5 and 7 and will not be repeated here for the sake of brevity. As discussed, in some examples, two ore more types of video data may be received. In some examples, an entropy decoding technique may be determined for two, three, or more types of video data as described herein.
Process 1000 may continue at operation 1008, "Decode the First and Second Video Data to Generate a Video Frame", where the first and second video data (and any other entropy decoded video data) may be decoded to generate video frame(s). The video frames may be suitable for presentment to a user via a display device, for example.
As discussed, in some examples, one or more of the entropy decoded video data may be post-processed or otherwise manipulated prior to further decoding. For example, entropy decoded difference motion vector(s) may be added to prediction motion vector(s) to generate reconstructed motion vector(s), which may be output for use in motion vector prediction (e.g., inter-prediction) via decoder 200.
Various components of the systems described herein may be implemented in software, firmware, and/or hardware and/or any combination thereof. For example, various components of encoder 100 or encoder 200 may be provided, at least in part, by hardware of a computing System-on-a-Chip (SoC) such as may be found in a computing system such as, for example, a smart phone. Those skilled in the art may recognize that systems described herein may include additional components that have not been depicted in the corresponding figures. For example, the systems discussed herein may include additional components such as bit stream multiplexer or de-multiplexer modules and the like that have not been depicted in the interest of clarity. Some additional and/or alternative details related to process 800, 1000 and other processes discussed herein may be illustrated in one or more examples of implementations discussed herein and, in particular, with respect to FIG. 1 1 below.
FIG. 11 is an illustrative diagram of an example video coding system 1610 and video coding process 1 100 in operation, arranged in accordance with at least some
implementations of the present disclosure. In the illustrated implementation, process 1 100 may include one or more operations, functions or actions as illustrated by one or more of actions 1 100-11 12. By way of non-limiting example, process 1100 will be described herein with reference to example video coding system 1610 including encoder 100 of FIG. 1 and decoder 200 of FIG. 2, as is discussed further herein below with respect to FIG. 16. In various examples, process 1 100 may be undertaken by a system including both an encoder and decoder or by separate systems with one system employing an encoder (and optionally a decoder) and another system employing a decoder (and optionally an encoder). It is also noted, as discussed above, that an encoder may include a local decode loop employing a local decoder as a part of the encoder system.
In the illustrated implementation, video coding system 1610 may include logic circuitry 1150, the like, and/or combinations thereof. For example, logic circuitry 1 150, may include encoder 100 and may include any modules as discussed with respect to FIG. 1 and/or FIGS. 3 and 5 and decoder 200 and may include any modules as discussed with respect to FIG. 2 and/or FIGS. 4 and 6. Although video coding system 1610, as shown in FIG. 16, may include one particular set of blocks or actions associated with particular modules, these blocks or actions may be associated with different modules than the particular module illustrated here. Although process 1100, as illustrated, is directed to encoding and decoding, the concepts and/or operations described may be applied to encoding and/or decoding separately, and, more generally, to video coding.
Process 1100 may begin at operation 1 101, "Obtain Video Data of Different Types", where video data of different types may be received for entropy encoding. For example two or more types of video data may be received for entropy encoding. For example, two or more of video data 41 1^417 (and/or any sub-components of video data 411— 4 IV) may be received via adaptive entropy encoder 1 10. As discussed, the first and second (or additional) video data may be of different types such as any of the types or sub-component types as discussed with respect to video data 41 1-417 or elsewhere herein. Process 1100 may continue from operation 1101 to operation 1102, "Determine Entropy Encoding Technique(s) for one or more Types of Video Data", where entropy encoding technique(s) may be determined for one or more of the video data types. As discussed, in some examples, one or more of encoder modules 401^107 may determine a coding technique for video data 41 1— 417 from various coding technique options. For example, encoder module 401 may determine a coding technique for video data 41 1 from one of an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression technique. Further, encoder module 402 may determine a coding technique for intra- prediction partition data 625 of video data 402 from one of an adaptive symbol-run variable length coding technique and an adaptive proxy variable length coding technique, and so on as discussed herein.
Process 1100 may continue at operation 1103, "Entropy Encode the Video Data using the Determined Technique(s)", the video data may be entropy encoded using the determined technique(s). For example, first video data may be entropy encoded using a first entropy encoding technique to generate first compressed video data. In some examples, second, third, or more additional video data may be entropy encoded using second, third or more additional entropy encoding techniques as discussed herein to generate first, second, third, and so on respective compressed video data.
Process 1100 may continue from operation 1 103 to operation 1 104, "Assemble the Compressed Video Data", where the compressed video data of different types may be assembled to generate an output bitstream. For example, bitstream assembler 408 may assemble or multiplex the compressed video data to generate output bitstream 11 1 as discussed herein.
Process 1100 may continue from operation 1104 to operation 1105, "Optionally Multiplex Video Bitstream with Audio Stream", where the video bitstream may be optionally multiplexed with a coded or uncoded audio stream to generate an audio-visual bitstream.
Process 1100 may continue from operation 1105 or operation 1 104 to operation 1 106, "Transmit Bitstream", where the bitstream may be transmitted. For example, video coding system 1610 may transmit output bitstream 11 1 or bitstream 800 or the like via an antenna 1102 (please refer to FIG. 16). Operations 1101-1106 may provide for video encoding and bitstream transmission techniques, which may be employed by an encoder system as discussed herein. The following operations, operations 1 107-1 112 may provide for video decoding and video display techniques, which may be employed by a decoder system as discussed herein.
Process 1100 may continue at operation 1107, "Receive Bitstream", where the encoded bitstream may be received. For example, input bitstream 100, 201, or bitstream 800 or the like may be received via decoder 200. In some examples, the bitstream may include different types of entropy encoded data as discussed herein.
Process 1100 may continue from operation 1107 to operation 1108, "Disassamble Bitstream to Generate Compressed Video Data of Different Types", where the received bitstream may be disassembled to determine different types of compressed video data. For example, bitstream 201 may be disassembled via bitstream disassembler 508 to generate compressed video data 51 1-517 (and any sub-component video data) as illustrated in FIG. 7. For example, the disassembled compressed video data may include first, second, third, or more compressed video data.
Process 1100 may continue from operation 1 108 to operation 1109, "Determine Entropy Decoding Technique(s) for one or more Types of Compressed Video Data", entropy decoding technique(s) may be determined for one or more of the compressed video data types. In some examples, the entropy decoding technique(s) may be determined based on a flag or indicator or the like conveyed via the received bitstream. As discussed, in some examples, one or more of decoder modules 501-507 may determine a coding technique for compressed video data 511-517 from various coding technique options. For example, decoder module 401 may determine a coding technique for compressed video data 511 from one of an adaptive symbol-run variable length coding technique, an adaptive proxy variable length coding technique, or a variable length coding table or tables compression technique based on an indicator or indicators provided via the input bitstream. Further, decoder module 502 may determine a coding technique for compressed intra- prediction partition data 725 of video data 512 from one of an adaptive symbol-run variable length coding technique and an adaptive proxy variable length coding technique, and so on. A wide range of examples have been provided herein and will not be repeated here for the sake of brevity. As discussed, in some examples, two ore more types of video data may be received. In some examples, an entropy decoding technique may be determined for two, three, or more types of video data as described herein. Process 1100 may continue from operation 1 109 to operation 1610, "Entropy Decode the Compressed Video Data", where the compressed video data may be entropy decoded based on the determined entropy decoding techniques. For example, compressed video data 511-517 may be entropy decoded via decode modules 501-507.
Process 1100 may continue from operation 1610 to operation 1 11 1, "Decode the Entropy Decoded Video Data to Generate Video Frame(s)", where the first and second video data (and any other entropy decoded video data) may be decoded to generate video frame(s). The video frames may be suitable for presentment to a user via a display device, for example. For example, the video frame may be determined based on the
implementation of decode techniques discussed with respect to decoder 200.
Process 1100 may continue from operation 1 11 1 to operation 1 1 12, "Transmit Video Frames for Presentment via a Display Device", where generated video frame(s) may be transmitted for presentment via a display device. For example, the video frame(s) may be transmitted to a display device 1 105 (as shown in FIG. 16) for presentment. In some examples, display device 1 105 may display the video frames to a user, for example.
While implementation of the example processes herein may include the undertaking of all operations shown in the order illustrated, the present disclosure is not limited in this regard and, in various examples, implementation of the example processes herein may include the undertaking of only a subset of the operations shown and/or in a different order than illustrated.
In addition, any one or more of the operations discussed herein may be undertaken in response to instructions provided by one or more computer program products. Such program products may include signal bearing media providing instructions that, when executed by, for example, a processor, may provide the functionality described herein. The computer program products may be provided in any form of one or more machine-readable media. Thus, for example, a processor including one or more processor core(s) may undertake one or more of the operations of the example processes herein in response to program code and/or instructions or instruction sets conveyed to the processor by one or more machine-readable media. In general, a machine-readable medium may convey software in the form of program code and/or instructions or instruction sets that may cause any of the devices and/or systems described herein to implement at least portions of the video systems as discussed herein. As used in any implementation described herein, the term "module" refers to any combination of software logic, firmware logic and/or hardware logic configured to provide the functionality described herein. The software may be embodied as a software package, code and/or instruction set or instructions, and "hardware", as used in any implementation described herein, may include, for example, singly or in any combination, hardwired circuitry, programmable circuitry, state machine circuitry, and/or firmware that stores instructions executed by programmable circuitry. The modules may, collectively or individually, be embodied as circuitry that forms part of a larger system, for example, an integrated circuit (IC), system on-chip (SoC), and so forth. For example, a module may be embodied in logic circuitry for the implementation via software, firmware, or hardware of the coding systems discussed herein.
FIG. 12 is a flow diagram illustrating an example process 1200, arranged in accordance with at least some implementations of the present disclosure. Process 1200 may include one or more operations, functions or actions as illustrated by one or more operations. Process 1200 may form at least part of a next generation video coding process. By way of non-limiting example, process 1200 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or any other encoder system or subsystems described herein.
Process 1200 may begin at operation 1202, "Receive Input Video Frames of a Video Sequence", where input video frames of a video sequence may be received via encoder 100 for example.
Process 1200 may continue at operation 1204, "Associate a Picture Type with each Video Frame", where a picture type may be associated with each video frame in a group of pictures via content pre-analyzer module 102 for example. For example, the picture type may be F/B-picture , P-picture, or I-picture, or the like. In some examples, a video sequence may include groups of pictures and the processing described herein (e.g., operations 1203 through 1211) may be performed on a frame or picture of a group of pictures and the processing may be repeated for all frames or pictures of a group and then repeated for all groups of pictures in a video sequence.
Process 1200 may continue at operation 1206, "Divide a Picture into Tiles and/or Super-fragments and Potential Prediction Partitionings", where a picture may be divided into tiles or super-fragments and potential prediction partitions via prediction partitions generator 105 for example. Process 1200 may continue at operation 1210, "For Potential Prediction Partitioning, Determine Potential Prediction Error", where, for each potential prediction partitioning, a potential prediction error may be determined. For example, for each prediction partitioning (and associated prediction partitions, prediction(s), and prediction parameters), a prediction error may be determined. For example, determining the potential prediction error may include differencing original pixels (e.g., original pixel data of a prediction partition) with prediction pixels. In some examples, the associated prediction parameters may be stored. As discussed, in some examples, the prediction error data partition may include prediction error data generated based at least in part on a previously decoded frame generated using at least one of a morphing technique or a synthesizing technique.
Process 1200 may continue at operation 1212, "Select Prediction Partitioning and Prediction Type and Save Parameters", where a prediction partitioning and prediction type may be selected and the associated parameters may be saved. In some examples, the potential prediction partitioning with a minimum prediction error may be selected. In some examples, the potential prediction partitioning may be selected based on a rate distortion optimization (RDO).
Process 1200 may continue at operation 1214, "Perform Transforms on Potential Coding Partitionings", where fixed or content adaptive transforms with various block sizes may be performed on various potential coding partitionings of partition prediction error data. For example, partition prediction error data may be partitioned to generate a plurality of coding partitions. For example, the partition prediction error data may be partitioned by a bi-tree coding partitioner module or a k-d tree coding partitioner module of coding partitions generator module 107 as discussed herein. In some examples, partition prediction error data associated with an F/B- or P-picture may be partitioned by a bi-tree coding partitioner module. In some examples, video data associated with an I-picture (e.g., tiles or super- fragments in some examples) may be partitioned by a k-d tree coding partitioner module. In some examples, a coding partitioner module may be chosen or selected via a switch or switches. For example, the partitions may be generated by coding partitions generator module 107.
Process 1200 may continue at operation 1216, "Determine the Best Coding
Partitioning, Transform Block Sizes, and Actual Transform", where the best coding partitioning, transform block sizes, and actual transforms may be determined. For example, various coding partitionings (e.g., having various coding partitions) may be evaluated based on RDO or another basis to determine a selected coding partitioning (which may also include further division of coding partitions into transform blocks when coding partitions to not match a transform block size as discussed). For example, the actual transform (or selected transform) may include any content adaptive transform or fixed transform performed on coding partition or block sizes as described herein.
Process 1200 may continue at operation 1218, "Quantize and Scan Transform Coefficients", where transform coefficients associated with coding partitions (and/or transform blocks) may be quantized and scanned in preparation for entropy coding.
Process 1200 may continue at operation 1222, "Entropy Encode Data associated with Each Tile or Super-fragment Decode, Such As Coding Partition Indicator(s), Block Size Data, Transform Type Data, Quantizer (Qp), and Quantized Transform Coefficients, Motion Vectors and Reference Type Data, Characteristic Parameters (e.g., mop, syp)", where data may be entropy encoded. For example, the entropy encoded data may include the coding partition indicators, block size data, transform type data, quantizer (Qp), quantized transform coefficients, motion vectors and reference type data, characteristic parameters (e.g., mop, syp), the like, and/or combinations thereof. Additionally or alternatively, the entropy encoded data may include prediction partitioning, prediction parameters, the selected coding partitioning, the selected characteristics data, motion vector data, quantized transform coefficients, filter parameters, selection data (such as mode selection data), and indictors.
Process 1200 may continue at operation 1223 "Apply DD/DB Filter, Reconstruct Pixel Data, Assemble into a Picture", where deblock filtering (e.g., DD or DB filters) may be applied, pixel data may be reconstructed, and assembled into a picture. For example, after a local decode loop (e.g., including inverse scan, inverse transform, and assembling coding partitions), prediction error data partitions may be generated. The prediction error data partitions may be added with a prediction partition to generate reconstructed prediction partitions, which may be assembled into tiles or super-fragments. The assembled tiles or super-fragments may be optionally processed via deblock filtering and/or quality restoration filtering and assembled to generate a picture.
Process 1200 may continue at operation 1224 "Apply QR/LF Filter Save in
Reference Picture Buffers", where quality restoration filtering (e.g., QR or LF filtering) may be applied, and the assembled picture may be saved in reference picture buffers. For example, in addition to or in the alternative to the DD/DB filtering, the assembled tiles or super-fragments may be optionally processed via quality restoration filtering and assembled to generate a picture. The picture may be saved in decoded picture buffer 1 19 as a reference picture for prediction of other (e.g., following) pictures.
Process 1200 may continue at operation 1225, "Apply AP/AM Filter, Determine Modifying (e.g., Morphing or Synthesizing) Characteristic Parameters for Generating Morphed or Synthesized Prediction Reference(s) and Perform Prediction(s)", where, modifying (e.g., morphing or synthesizing) characteristic parameters and prediction(s) may be performed and adaptive motion filtering or adaptive precision filtering (e.g., AP/AM Filter) may be applied. For example, modifying (e.g., morphing or synthesizing) characteristic parameters for generating morphed or synthesized prediction reference(s) may be generated and prediction(s) may be performed. Additionally, adaptive motion filtering or adaptive precision filtering may be applied at this point in the process.
As discussed, in some examples, inter-prediction may be performed. In some examples, up to 4 decoded past and/or future pictures and several morphing/synthesis predictions may be used to generate a large number of reference types (e.g., reference pictures). For instance in 'inter' mode, up to nine reference types may be supported in P- pictures, and up to ten reference types may be supported for F/B-pictures. Further, 'multi' mode may provide a type of inter prediction mode in which instead of 1 reference picture, 2 reference pictures may be used and P- and F/B-pictures respectively may allow 3, and up to 8 reference types. For example, prediction may be based on a previously decoded frame generated using at least one of a morphing technique or a synthesizing technique. In such examples, and the bitstream (discussed below with respect to operation 1212) may include a frame reference, morphing parameters, or synthesizing parameters associated with the prediction partition.
Process 1200 may continue at operation 1229 "Optionally Apply EP Filter and/or Optionally apply FI FP Filter", where enhanced predicted partition (e.g., EP Filtering) or FI/FP Filtering (e.g., fusion filtering or fusion improvement filtering) may be optionally applied. In some examples, a decision may be made regarding whether to utilize some form or FI/FP Filter (fusion improvement filtering/fusion filtering) or not to use FI/FP Filtering. When some form or FI FP Filter (e.g., fusion filtering or fusion improvement filtering) is to be applied to the selected predicted partition the selected predicted partition and a second selected predicted partition may be assembled to generate at least a portion of an assembled picture. FI/FP Filtering may be applied to filter the portion of the assembled picture. FI/FP Filtering parameters (e.g., filtering parameters or fusion improvement filtering parameters) associated with the FI/FP Filtering may be generated and sent to the entropy coder subsystem.
In implementations where both EP Filtering or FI/FP Filtering are available, an indicator may be generated that indicates to the decoder system whether to use the enhanced predicted partition (e.g., EP Filtering) or the predicted partition data as the selected predicted partition for the prediction partition.
Operations 1202 through 1229 may provide for video encoding and bitstream transmission techniques, which may be employed by an encoder system as discussed herein.
FIG. 13 illustrates an example bitstream 1300, arranged in accordance with at least some implementations of the present disclosure. In some examples, bitstream 1300 may correspond to output bitstream 11 1 as shown in FIG. 1 and/or input bitstream 201 as shown in FIG. 2. Although not shown in FIG. 29 for the sake of clarity of presentation, in some examples bitstream 1300 may include a header portion and a data portion. In various examples, bitstream 1300 may include data, indicators, index values, mode selection data, or the like associated with encoding a video frame as discussed herein.
As discussed, bitstream 1300 may be generated by an encoder such as, for example, encoder 100 and/or received by a decoder 200 for decoding such that decoded video frames may be presented via a display device.
FIG. 14 is a flow diagram illustrating an example process 1400, arranged in accordance with at least some implementations of the present disclosure. Process 1400 may include one or more operations, functions or actions as illustrated by one or more operations. Process 1400 may form at least part of a next generation video coding process. By way of non-limiting example, process 1400 may form at least part of a next generation video decoding process as undertaken by decoder system 200 and/or any other decoder system or subsystems described herein.
Process 1400 may begin at operation 1402, "Receive Encoded Bitstream", where a bitstream may be received. For example, a bitstream encoded as discussed herein may be received at a video decoder. In some examples, bitstream 900 or 1300 may be received via decoder 200. Process 1400 may continue at operation 1404, "Decode the Entropy Encoded Bitstream to Determine Coding Partition Indicator(s), Block Size Data, Transform Type Data, Quantizer (Qp), Quantized Transform Coefficients, Motion Vectors and Reference Type Data, Characteristic Parameters (e.g., mop, syp)", where the bitstream may be decoded to determine coding partition indicators, block size data, transform type data, quantizer (Qp), quantized transform coefficients, motion vectors and reference type data, characteristic parameters (e.g., mop, syp), the like, and/or combinations thereof.
Additionally or alternatively, the entropy encoded data may include prediction partitioning, prediction parameters, the selected coding partitioning, the selected characteristics data, motion vector data, quantized transform coefficients, filter parameters, selection data (such as mode selection data), and indictors.
Process 1400 may continue at operation 1406, "Apply Quantizer (Qp) on Quantized Coefficients to Generate Inverse Quantized Transform Coefficients", where quantizer (Qp) may be applied to quantized transform coefficients to generate inverse quantized transform coefficients. For example, operation 1406 may be applied via adaptive inverse quantize module 203.
Process 1400 may continue at operation 1408, "On each Decoded Block of
Coefficients in a Coding (or Intra Predicted) Partition Perform Inverse Transform based on Transform Type and Block Size Data to Generate Decoded Prediction Error Partitions", where, on each decode block of transform coefficients in a coding (or intra predicted) partition, an inverse transform based on the transform type and block size data may be performed to generate decoded prediction error partitions. In some examples, the inverse transform may include an inverse fixed transform. In some examples, the inverse transform may include an inverse content adaptive transform. In such examples, performing the inverse content adaptive transform may include determining basis functions associated with the inverse content adaptive transform based on a neighboring block of decoded video data, as discussed herein. Any forward transform used for encoding as discussed herein may be used for decoding using an associated inverse transform. In some examples, the inverse transform may be performed by adaptive inverse transform module 204. In some examples, generating the decoded prediction error partitions may also include assembling coding partitions via coding partitions assembler 205.
Process 1400 may continue at operation 1423 "Apply DD/DB Filter, Reconstruct Pixel Data, Assemble into a Picture", where deblock filtering (e.g., DD or DB filters) may be applied, pixel data may be reconstructed, and assembled into a picture. For example, after inverse scan, inverse transform, and assembling coding partitions, the prediction error data partitions may be added with a prediction partition to generate reconstructed prediction partitions, which may be assembled into tiles or super-fragments. The assembled tiles or super-fragments may be optionally processed via deblock filtering.
Process 1400 may continue at operation 1424 "Apply QR/LF Filter Save in
Reference Picture Buffers", where quality restoration filtering (e.g., QR or LF filtering) may be applied, and the assembled picture may be saved in reference picture buffers. For example, in addition to or in the alternative to the DD/DB filtering, the assembled tiles or super-fragments may be optionally processed via quality restoration filtering and assembled to generate a picture. The picture may be saved in decoded picture buffer 1 19 as a reference picture for prediction of other (e.g., following) pictures.
Process 1400 may continue at operation 1425, "Apply AP/AM Filter, Use Decoded Modifying Characteristics (e.g., mop, syp) to Generate Modified References for Prediction and Use Motion Vectors and Reference Info, Predicted Partition Info, and Modified References to Generate Predicted Partition", where modified references for prediction may be generated and predicted partitions may be generated as well, and where adaptive motion filtering or adaptive precision filtering (e.g., AP/AM Filter ) may be applied. For example, where modified references for prediction may be generated based at least in part on decoded modifying characteristics (e.g., mop, syp) and predicted partitions may be generated based at least in part on motion vectors and reference information, predicted partition information, and modified references. Additionally, adaptive motion filtering or adaptive precision filtering may be applied at this point in the process.
Process 1400 may continue at operation 1429 "Optionally Apply EP Filter and/or Optionally apply FI FP Filter", where enhanced predicted partition (e.g., EP Filtering) or FI/FP Filtering (e.g., fusion filtering or fusion improvement filtering) may be optionally applied. In some examples, a decision may be made regarding whether to utilize some form or FI FP Filter (fusion improvement filtering/fusion filtering) or not to use FI/FP Filtering. When some form or FI/FP Filter (e.g., fusion filtering or fusion improvement filtering) is to be applied to the selected predicted partition the selected predicted partition and a second selected predicted partition may be assembled to generate at least a portion of an assembled picture. FI/FP Filtering may be applied to filter the portion of the assembled picture. FI/FP Filtering parameters (e.g., filtering parameters or fusion improvement filtering parameters) associated with the FI/FP Filtering may be generated and sent to the entropy coder subsystem.
In implementations where both EP Filtering or FI/FP Filtering are available, an indicator may be received from the encoder system that indicates to the decoder system whether to use the enhanced predicted partition (e.g., EP Filtering) or the predicted partition data as the selected predicted partition for the prediction partition.
Process 1400 may continue at operation 1430, "Add Prediction Partition to the Decoded Prediction Error Data Partition to Generate a Reconstructed Partition", where a prediction partition my be added to the decoded prediction error data partition to generate a reconstructed prediction partition. For example, the decoded prediction error data partition may be added to the associated prediction partition via adder 206.
Process 1400 may continue at operation 1432, "Assemble Reconstructed Partitions to Generate a Tile or Super-Fragment", where the reconstructed prediction partitions may be assembled to generate tiles or super-fragments. For example, the reconstructed prediction partitions may be assembled to generate tiles or super- fragments via prediction partitions assembler module 207.
Process 1400 may continue at operation 1434, "Assemble Tiles or Super-Fragments of a Picture to Generate a Full Decoded Picture", where the tiles or super- fragments of a picture may be assembled to generate a full decoded picture. For example, after optional deblock filtering and/or quality restoration filtering, tiles or super- fragments may be assembled to generate a full decoded picture, which may be stored via decoded picture buffer 210 and/or transmitted for presentment via a display device after processing via adaptive picture re-organizer module 217 and content post-restorer module 218.
FIGS. 15(A), 15(B), and 15(C) provide an illustrative diagram of an example video coding system 1600 and video coding process 1500 in operation, arranged in accordance with at least some implementations of the present disclosure. In the illustrated
implementation, process 1500 may include one or more operations, functions or actions as illustrated by one or more of actions 1501 through 1580. By way of non-limiting example, process 1500 will be described herein with reference to example video coding system 1600 including encoder 100 of FIG. 1 and decoder 200 of FIG. 2, as is discussed further herein below with respect to FIG. 16. In various examples, process 1500 may be undertaken by a system including both an encoder and decoder or by separate systems with one system employing an encoder (and optionally a decoder) and another system employing a decoder (and optionally an encoder). It is also noted, as discussed above, that an encoder may include a local decode loop employing a local decoder as a part of the encoder system.
In the illustrated implementation, video coding system 1600 may include logic circuitry 1650, the like, and/or combinations thereof. For example, logic circuitry 1650 may include encoder system 100 of FIG. 1 and/or decoder system 200 of FIG. 2 and may include any modules as discussed with respect to any of the encoder systems or subsystems described herein and/or decoder systems or subsystems described herein. Although video coding system 1600, as shown in FIGS. 15(A)-(C) may include one particular set of blocks or actions associated with particular modules, these blocks or actions may be associated with different modules than the particular modules illustrated here. Although process 1500, as illustrated, is directed to encoding and decoding, the concepts and/or operations described may be applied to encoding and/or decoding separately, and, more generally, to video coding.
Process 1500 may begin at operation 1501, "Receive Input Video Frames of a Video Sequence", where input video frames of a video sequence may be received via encoder 100 for example.
Process 1500 may continue at operation 1502, "Associate a Picture Type with each Video Frame in a Group of Pictures", where a picture type may be associated with each video frame in a group of pictures via content pre-analyzer module 102 for example. For example, the picture type may be F/B-picture , P-picture, or I-picture, or the like. In some examples, a video sequence may include groups of pictures and the processing described herein (e.g., operations 1503 through 151 1) may be performed on a frame or picture of a group of pictures and the processing may be repeated for all frames or pictures of a group and then repeated for all groups of pictures in a video sequence.
Process 1500 may continue at operation 1503, "Divide a Picture into Tiles and/or Super-fragments and Potential Prediction Partitionings", where a picture may be divided into tiles or super- fragments and potential prediction partitions via prediction partitions generator 105 for example.
Process 1500 may continue at operation 1504, "For Each Potential Prediction Partitioning, Perform Prediction(s) and Determine Prediction Parameters", where, for each potential prediction partitionings, prediction(s) may be performed and prediction parameters may be determined. For example, a range of potential prediction partitionings (each having various prediction partitions) may be generated and the associated prediction(s) and prediction parameters may be determined. For example, the prediction(s) may include prediction(s) using characteristics and motion based multi-reference predictions or intra-predictions.
As discussed, in some examples, inter-prediction may be performed. In some examples, up to 4 decoded past and/or future pictures and several morphing/synthesis predictions may be used to generate a large number of reference types (e.g., reference pictures). For instance in 'inter' mode, up to 9 reference types may be supported in P- pictures, and up to 10 reference types may be supported for F/B-pictures. Further, 'multi' mode may provide a type of inter prediction mode in which instead of 1 reference picture, 2 reference pictures may be used and P- and F/B-pictures respectively may allow 3, and up to 8 reference types. For example, prediction may be based on a previously decoded frame generated using at least one of a morphing technique or a synthesizing technique. In such examples, and the bitstream (discussed below with respect to operation 1512) may include a frame reference, morphing parameters, or synthesizing parameters associated with the prediction partition.
Process 1500 may continue at operation 1505, "For Each Potential Prediction Partitioning, Determine Potential Prediction Error", where, for each potential prediction partitioning, a potential prediction error may be determined. For example, for each prediction partitioning (and associated prediction partitions, prediction(s), and prediction parameters), a prediction error may be determined. For example, determining the potential prediction error may include differencing original pixels (e.g., original pixel data of a prediction partition) with prediction pixels. In some examples, the associated prediction parameters may be stored. As discussed, in some examples, the prediction error data partition may include prediction error data generated based at least in part on a previously decoded frame generated using at least one of a morphing technique or a synthesizing technique.
Process 1500 may continue at operation 1506, "Select Prediction Partitioning and Prediction Type and Save Parameters", where a prediction partitioning and prediction type may be selected and the associated parameters may be saved. In some examples, the potential prediction partitioning with a minimum prediction error may be selected. In some examples, the potential prediction partitioning may be selected based on a rate distortion optimization (RDO).
Process 1500 may continue at operation 1507, "Perform Fixed or Content Adaptive Transforms with Various Block Sizes on Various Potential Coding Partitionings of Partition Prediction Error Data", where fixed or content adaptive transforms with various block sizes may be performed on various potential coding partitionings of partition prediction error data. For example, partition prediction error data may be partitioned to generate a plurality of coding partitions. For example, the partition prediction error data may be partitioned by a bi-tree coding partitioner module or a k-d tree coding partitioner module of coding partitions generator module 107 as discussed herein. In some examples, partition prediction error data associated with an F/B- or P-picture may be partitioned by a bi-tree coding partitioner module. In some examples, video data associated with an I- picture (e.g., tiles or super- fragments in some examples) may be partitioned by a k-d tree coding partitioner module. In some examples, a coding partitioner module may be chosen or selected via a switch or switches. For example, the partitions may be generated by coding partitions generator module 107.
Process 1500 may continue at operation 1508, "Determine the Best Coding
Partitioning, Transform Block Sizes, and Actual Transform", where the best coding partitioning, transform block sizes, and actual transforms may be determined. For example, various coding partitionings (e.g., having various coding partitions) may be evaluated based on RDO or another basis to determine a selected coding partitioning (which may also include further division of coding partitions into transform blocks when coding partitions to not match a transform block size as discussed). For example, the actual transform (or selected transform) may include any content adaptive transform or fixed transform performed on coding partition or block sizes as described herein.
Process 1500 may continue at operation 1509, "Quantize and Scan Transform Coefficients", where transform coefficients associated with coding partitions (and/or transform blocks) may be quantized and scanned in preparation for entropy coding.
Process 1500 may continue at operation 151 1, "Entropy Encode Data associated with Each Tile or Super- fragment", where data associated with each tile or super- fragment may be entropy encoded. For example, data associated with each tile or super-fragment of each picture of each group of pictures of each video sequence may be entropy encoded. The entropy encoded data may include the prediction partitioning, prediction parameters, the selected coding partitioning, the selected characteristics data, motion vector data, quantized transform coefficients, filter parameters, selection data (such as mode selection data), and indictors.
Process 1500 may continue at operation 1512, "Generate Bitstream" where a bitstream may be generated based on the entropy encoded data. As discussed, in some examples, the bitstream may include a frame or picture reference, morphing parameters, or synthesizing parameters associated with a prediction partition.
Process 1500 may continue at operation 1513, "Transmit Bitstream", where the bitstream may be transmitted. For example, video coding system 2400 may transmit output bitstream 11 1, bitstream 2100, or the like via an antenna 2402 (please refer to FIG. 34).
Process 1500 may continue at operation 1520, "Reconstruct Pixel Data, Assemble into a Picture, and Save in Reference Picture Buffers", where pixel data may be reconstructed, assembled into a picture, and saved in reference picture buffers. For example, after a local decode loop (e.g., including inverse scan, inverse transform, and assembling coding partitions), prediction error data partitions may be generated. The prediction error data partitions may be added with a prediction partition to generate reconstructed prediction partitions, which may be assembled into tiles or super-fragments. The assembled tiles or super-fragments may be optionally processed via deblock filtering and/or quality restoration filtering and assembled to generate a picture. The picture may be saved in decoded picture buffer 1 19 as a reference picture for prediction of other (e.g., following) pictures.
Process 1500 may continue at operation 1523 "Apply DD/DB Filter, Reconstruct Pixel Data, Assemble into a Picture", where deblock filtering (e.g., DD or DB filters) may be applied, pixel data may be reconstructed, and assembled into a picture. For example, after a local decode loop (e.g., including inverse scan, inverse transform, and assembling coding partitions), prediction error data partitions may be generated. The prediction error data partitions may be added with a prediction partition to generate reconstructed prediction partitions, which may be assembled into tiles or super-fragments. The assembled tiles or super-fragments may be optionally processed via deblock filtering and/or quality restoration filtering and assembled to generate a picture.
Process 1500 may continue at operation 1524 "Apply QR/LF Filter Save in
Reference Picture Buffers", where quality restoration filtering (e.g., QR or LF filtering) may be applied, and the assembled picture may be saved in reference picture buffers. For example, in addition to or in the alternative to the DD/DB filtering, the assembled tiles or super-fragments may be optionally processed via quality restoration filtering and assembled to generate a picture. The picture may be saved in decoded picture buffer 1 19 as a reference picture for prediction of other (e.g., following) pictures.
Process 1500 may continue at operation 1525, "Generate Modifying Characteristic Parameters", where, modified characteristic parameters may be generated. For example, a second modified prediction reference picture and second modifying characteristic parameters associated with the second modified prediction reference picture may be generated based at least in part on the second decoded prediction reference picture, where the second modified reference picture may be of a different type than the first modified reference picture.
Process 1500 may continue at operation 1526, "Generate Modified Prediction Reference Pictures", where modified prediction reference pictures may be generated, for example, a first modified prediction reference picture and first modifying characteristic parameters associated with the first modified prediction reference picture may be generated based at least in part on the first decoded prediction reference picture.
Process 1500 may continue at operation 1527, "Generate Motion Data", where, motion estimation data may be generated. For example, motion data associated with a prediction partition of a current picture may be generated based at least in part on one of the first modified prediction reference picture or the second modified prediction reference picture.
Process 1500 may continue at operation 1528, "Apply AP/AM Filter Perform Motion Compensation", where, motion compensation may be performed. For example, motion compensation may be performed based at least in part on the motion data and at least one of the first modified prediction reference picture or the second modified prediction reference picture to generate prediction partition data for the prediction partition and adaptive motion filtering or adaptive precision filtering (e.g., AP/AM Filter) may be applied. Process 1500 may feed this information back to operation 1504 where each decoded prediction error partition (e.g., including zero prediction error partitions) may be added to the corresponding prediction partition to generate a reconstructed prediction partition. Additionally, adaptive motion filtering or adaptive precision filtering may be applied at this point in the process. Process 1500 may continue at operation 1529 "Optionally Apply EP", where enhanced predicted partition (e.g., EP Filtering) may be optionally applied. In some examples, where both EP Filtering or FI/FP Filtering are available, an indicator may be generated that indicates to the decoder system whether to use the enhanced predicted partition (e.g., EP Filtering) or the predicted partition data as the selected predicted partition for the prediction partition.
Process 1500 may continue at operation 1530 "Optionally apply FI/FP Filter", where FI/FP Filtering (e.g., fusion filtering or fusion improvement filtering) may be optionally applied. In some examples, a decision may be made regarding whether to utilize some form or FI/FP Filter (fusion improvement filtering/fusion filtering) or not to use FI/FP Filtering. When some form or FI/FP Filter (e.g., fusion filtering or fusion improvement filtering) is to be applied to the selected predicted partition the selected predicted partition and a second selected predicted partition may be assembled to generate at least a portion of an assembled picture. FI/FP Filtering may be applied to filter the portion of the assembled picture. FI FP Filtering parameters (e.g., filtering parameters or fusion improvement filtering parameters) associated with the FI FP Filtering may be generated and sent to the entropy coder subsystem.
Operations 1501 through 1530 may provide for video encoding and bitstream transmission techniques, which may be employed by an encoder system as discussed herein. The following operations, operations 1554 through 1568 may provide for video decoding and video display techniques, which may be employed by a decoder system as discussed herein.
Process 1500 may continue at operation 1554, "Receive Bitstream", where the bitstream may be received. For example, input bitstream 201, bitstream 2100, or the like may be received via decoder 200. In some examples, the bitstream may include data associated with a coding partition, one or more indicators, and/or data defining coding partition(s) as discussed above. In some examples, the bitstream may include the prediction partitioning, prediction parameters, the selected coding partitioning, the selected characteristics data, motion vector data, quantized transform coefficients, filter parameters, selection data (such as mode selection data), and indictors.
Process 1500 may continue at operation 1555, "Decode Bitstream", where the received bitstream may be decoded via adaptive entropy decoder module 202 for example. For example, received bitstream may be entropy decoded to determine the prediction partitioning, prediction parameters, the selected coding partitioning, the selected characteristics data, motion vector data, quantized transform coefficients, filter parameters, selection data (such as mode selection data), and indictors.
Process 1500 may continue at operation 1556, "Perform Inverse Scan and Inverse Quantization on Each Block of Each Coding Partition", where an inverse scan and inverse quantization may be performed on each block of each coding partition for the prediction partition being processed. For example, the inverse scan and inverse quantization may be performed via adaptive inverse quantize module 203.
Process 1500 may continue at operation 1557, "Perform Fixed or Content Adaptive Inverse Transform to Decode Transform Coefficients to Determine Decoded Prediction Error Data Partitions", where a fixed or content adaptive inverse transform may be performed to decode transform coefficients to determine decoded prediction error data partitions. For example, the inverse transform may include an inverse content adaptive transform such as a hybrid parametric Haar inverse transform such that the hybrid parametric Haar inverse transform may include a parametric Haar inverse transform in a direction of the parametric transform direction and a discrete cosine inverse transform in a direction orthogonal to the parametric transform direction. In some examples, the fixed inverse transform may include a discrete cosine inverse transform or a discrete cosine inverse transform approximator. For example, the fixed or content adaptive transform may be performed via adaptive inverse transform module 204. As discussed, the content adaptive inverse transform may be based on other previously decoded data, such as, for example, decoded neighboring partitions or blocks. In some examples, generating the decoded prediction error data partitions may include assembling decoded coding partitions via coding partitions assembler module 205.
Process 1500 may continue at operation 1558, "Generate Prediction Pixel Data for Each Prediction Partition", where prediction pixel data may be generated for each prediction partition. For example, prediction pixel data may be generated using the selected prediction type (e.g., based on characteristics and motion, or intra-, or other types) and associated prediction parameters.
Process 1500 may continue at operation 1559, "Add to Each Decoded Prediction Error Partition the Corresponding Prediction Partition to Generate Reconstructed
Prediction Partition", where each decoded prediction error partition (e.g., including zero prediction error partitions) may be added to the corresponding prediction partition to generated a reconstructed prediction partition. For example, prediction partitions may be generated via the decode loop illustrated in FIG. 2 and added via adder 206 to decoded prediction error partitions.
Process 1500 may continue at operation 1560, "Assemble Reconstructed Prediction Partitions to Generate Decoded Tiles or Super- fragments", where reconstructed prediction partitions may be assembled to generate decoded tiles or super-fragments. For example, prediction partitions may be assembled to generate decoded tiles or super-fragments via prediction partitions assembler module 207.
Process 1500 may continue at operation 1561, "Apply Deblock Filtering and/or QR Filtering to Generate Final Decoded Tiles or Super- fragments", where optional deblock filtering and/or quality restoration filtering may be applied to the decoded tiles or super- fragments to generate final decoded tiles or super-fragments. For example, optional deblock filtering may be applied via deblock filtering module 208 and/or optional quality restoration filtering may be applied via quality restoration filtering module 209.
Process 1500 may continue at operation 1562, "Assemble Decoded Tiles or Super- fragments to Generate a Decoded Video Picture, and Save in Reference Picture Buffers", where decoded (or final decoded) tiles or super- fragments may be assembled to generate a decoded video picture, and the decoded video picture may be saved in reference picture buffers (e.g., decoded picture buffer 210) for use in future prediction.
Process 1500 may continue at operation 1563, "Transmit Decoded Video Frames for Presentment via a Display Device", where decoded video frames may be transmitted for presentment via a display device. For example, decoded video pictures may be further processed via adaptive picture re-organizer 217 and content post restorer module 218 and transmitted to a display device as video frames of display video 219 for presentment to a user. For example, the video frame(s) may be transmitted to a display device 2405 (as shown in FIG. 34) for presentment.
Process 1500 may continue at operation 1573 "Apply DD/DB Filter, Reconstruct Pixel Data, Assemble into a Picture", where deblock filtering (e.g., DD or DB filters) may be applied, pixel data may be reconstructed, and assembled into a picture. For example, after inverse scan, inverse transform, and assembling coding partitions, the prediction error data partitions may be added with a prediction partition to generate reconstructed prediction partitions, which may be assembled into tiles or super-fragments. The assembled tiles or super-fragments may be optionally processed via deblock filtering.
Process 1500 may continue at operation 1574 "Apply QR/LF Filter Save in
Reference Picture Buffers", where quality restoration filtering (e.g., QR or LF filtering) may be applied, and the assembled picture may be saved in reference picture buffers. For example, in addition to or in the alternative to the DD/DB filtering, the assembled tiles or super-fragments may be optionally processed via quality restoration filtering and assembled to generate a picture. The picture may be saved in a picture buffer as a reference picture for prediction of other (e.g., following) pictures.
Process 1500 may continue at operation 1576, "Generate Modified Prediction Reference Pictures", where modified prediction reference pictures may be generated, for example, at least a portion of a third modified prediction reference picture may be generated based at least in part on the third modifying characteristic parameters. Similarly, at least a portion a fourth modified prediction reference picture may be generated based at least in part on the second modifying characteristic parameters associated.
Process 1500 may continue at operation 1577, "Generate Motion Data", where, motion estimation data may be generated. For example, motion data associated with a prediction partition of a current picture may be generated based at least in part on one of the third modified prediction reference picture or the third modified prediction reference picture.
Process 1500 may continue at operation 1578, "Apply AP/AM Filter and Perform Motion Compensation", where, motion compensation may be performed and where adaptive motion filtering or adaptive precision filtering (e.g., AP/AM Filter ) may be applied. For example, motion compensation may be performed based at least in part on the motion data and at least one of the third modified prediction reference picture or the fourth modified prediction reference picture to generate prediction partition data for the prediction partition. Process 1300 may feed this information back to operation 1559 where each decoded prediction error partition (e.g., including zero prediction error partitions) may be added to the corresponding prediction partition to generate a reconstructed prediction partition. Additionally, adaptive motion filtering or adaptive precision filtering may be applied at this point in the process. Process 1500 may continue at operation 1579 "Optionally Apply EP Filter", where enhanced predicted partition (e.g., EP Filtering) may be optionally applied. In some examples, where both EP Filtering or FI/FP Filtering are available, an indicator may be received from the encoder system that indicates to the decoder system whether to use the enhanced predicted partition (e.g., EP Filtering) or the predicted partition data as the selected predicted partition for the prediction partition.
Process 1500 may continue at operation 1580 "Optionally apply FI/FP Filter", where FI/FP Filtering (e.g., fusion filtering or fusion improvement filtering) may be optionally applied. In some examples, a decision may be made regarding whether to utilize some form or FI/FP Filter (fusion improvement filtering/fusion filtering) or not to use FI/FP Filtering. When some form or FI/FP Filter (e.g., fusion filtering or fusion improvement filtering) is to be applied to the selected predicted partition the selected predicted partition and a second selected predicted partition may be assembled to generate at least a portion of an assembled picture. FI/FP Filtering may be applied to filter the portion of the assembled picture. FI FP Filtering parameters (e.g., filtering parameters or fusion improvement filtering parameters) associated with the FI FP Filtering may be generated and sent to the entropy coder subsystem.
Process 1500 may be implemented via any of the coder systems as discussed herein. Further, process 1500 may be repeated either in serial or in parallel on any number of instantiations of video data such as prediction error data partitions, original data partitions, or wavelet data or the like.
While implementation of the example processes herein may include the undertaking of all operations shown in the order illustrated, the present disclosure is not limited in this regard and, in various examples, implementation of the example processes herein may include the undertaking of only a subset of the operations shown and/or in a different order than illustrated.
FIG. 16 is an illustrative diagram of example video coding system 1600, arranged in accordance with at least some implementations of the present disclosure. In the illustrated implementation, video coding system 1600 may include imaging device(s) 1601, video encoder 100 and/or a video encoder implemented via logic circuitry 1650 of processing unit(s) 1620, video decoder 200 and/or a video decoder implemented via logic circuitry 1650 of processing unit(s) 1620, an antenna 1602, one or more processor(s) 1603, one or more memory store(s) 2004, and/or a display device 1605. As illustrated, imaging device(s) 1601, antenna 1602, processing unit(s) 1620, logic circuitry 1650, video encoder 100, video decoder 200, processor(s) 1603, memory store(s) 1604, and/or display device 1605 may be capable of communication with one another. As discussed, although illustrated with both video encoder 100 and video decoder 200, video coding system 1600 may include only video encoder 100 or only video decoder 200 in various examples.
As shown, in some examples, video coding system 1600 may include antenna 1602. Antenna 1602 may be configured to transmit or receive an encoded bitstream of video data, for example. Further, in some examples, video coding system 1600 may include display device 1605. Display device 1605 may be configured to present video data. As shown, in some example, logic circuitry 1650 may be implemented via processing unit(s) 1620. Processing unit(s) 1620 may include application-specific integrated circuit (ASIC) logic, graphics processor(s), general purpose processor(s), or the like. Video coding system 1600 also may include optional processor(s) 1603, which may similarly include application-specific integrated circuit (ASIC) logic, graphics processor(s), general purpose processor(s), or the like. In some examples, logic circuitry 1650 may be implemented via hardware or video coding dedicated hardware or the like, and processor(s) 1603 may implemented general purpose software or operating systems or the like. In addition, memory stores 1604 may be any type of memory such as volatile memory (e.g., Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.), and so forth. In a non-limiting example, memory stores 1604 may be implemented by cache memory. In some examples, logic circuitry 1650 may access memory stores 1604 (for implementation of an image buffer for example). In other examples, logic circuitry 1650 and/or processing unit(s) 1620 may include memory stores (e.g., cache or the like) for the implementation of an image buffer or the like.
In some examples, video encoder 100 implemented via logic circuitry may include an image buffer (e.g., via either processing unit(s) 1620 or memory store(s) 1604)) and a graphics processing unit (e.g., via processing unit(s) 1620). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include video encoder 100 as implemented via logic circuitry 1650 to embody the various modules as discussed with respect to FIG. 1 and FIGS. 3 and 5. For example, the graphics processing unit may include entropy encoder logic circuitry, and so on. The logic circuitry may be configured to perform the various operations as discussed herein. For example, the entropy encoder logic circuitry may be configured to receive first video data and second video data of different types for entropy encoding, determine a first entropy encoding technique for the first video data based at least in part on a parameter associated with the first video data such that the first entropy encoding technique comprises at least one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, entropy encode the first video data using the first encoding technique to generate first compressed video data and the second video data using a second encoding technique to generate second compressed video data, and assemble the first compressed video data and the second compressed video data to generate an output bitstream. Video decoder 200 may be implemented in a similar manner.
In some examples, antenna 1602 of video coding system 1600 may be configured to receive an entropy encoded bitstream of video data. As discussed, the bitstream may include two or more compressed video data types. Video coding system 1600 may also include video decoder 200 coupled to antenna 1602 and configured to decode the encoded bitstream. For example, video decoder 200 may be configured to disassemble the entropy encoded bitstream to determine first compressed video data and second compressed video data, determine a first entropy decoding technique for the first compressed video data, such that the first entropy decoding technique comprises at least one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, entropy decode the first compressed video data based on the first entropy decoding technique to generate first video data and the second compressed video data based on a second entropy decoding technique to generate second video data, and decode the first video data and the second video data to generate a video frame.
In embodiments, features described herein may be undertaken in response to instructions provided by one or more computer program products. Such program products may include signal bearing media providing instructions that, when executed by, for example, a processor, may provide the functionality described herein. The computer program products may be provided in any form of one or more machine-readable media. Thus, for example, a processor including one or more processor core(s) may undertake one or more features described herein in response to program code and/or instructions or instruction sets conveyed to the processor by one or more machine-readable media. In general, a machine-readable medium may convey software in the form of program code and/or instructions or instruction sets that may cause any of the devices and/or systems described herein to implement at least portions of the features described herein. FIG. 17 is an illustrative diagram of an example system 1700, arranged in accordance with at least some implementations of the present disclosure. In various implementations, system 1700 may be a media system although system 1700 is not limited to this context. For example, system 1700 may be incorporated into a personal computer (PC), laptop computer, ultra-laptop computer, tablet, touch pad, portable computer, handheld computer, palmtop computer, personal digital assistant (PDA), cellular telephone, combination cellular telephone/PDA, television, smart device (e.g., smart phone, smart tablet or smart television), mobile internet device (MID), messaging device, data communication device, cameras (e.g. point-and-shoot cameras, super-zoom cameras, digital single-lens reflex (DSLR) cameras), and so forth.
In various implementations, system 1700 includes a platform 1702 coupled to a display 1720. Platform 1702 may receive content from a content device such as content services device(s) 1730 or content delivery device(s) 1740 or other similar content sources. A navigation controller 1750 including one or more navigation features may be used to interact with, for example, platform 1702 and/or display 1720. Each of these components is described in greater detail below.
In various implementations, platform 1702 may include any combination of a chipset 1705, processor 1710, memory 1712, antenna 1713, storage 1714, graphics subsystem 1715, applications 1716 and/or radio 1718. Chipset 1705 may provide intercommunication among processor 1710, memory 1712, storage 1714, graphics subsystem 1715, applications 1716 and/or radio 1718. For example, chipset 1705 may include a storage adapter (not depicted) capable of providing intercommunication with storage 1714.
Processor 1710 may be implemented as a Complex Instruction Set Computer (CISC) or Reduced Instruction Set Computer (RISC) processors, x86 instruction set compatible processors, multi-core, or any other microprocessor or central processing unit (CPU). In various implementations, processor 1710 may be dual-core processor(s), dual-core mobile processor(s), and so forth.
Memory 1712 may be implemented as a volatile memory device such as, but not limited to, a Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), or Static RAM (SRAM).
Storage 1714 may be implemented as a non-volatile storage device such as, but not limited to, a magnetic disk drive, optical disk drive, tape drive, an internal storage device, an attached storage device, flash memory, battery backed-up SDRAM (synchronous DRAM), and/or a network accessible storage device. In various implementations, storage 1714 may include technology to increase the storage performance enhanced protection for valuable digital media when multiple hard drives are included, for example.
Graphics subsystem 1715 may perform processing of images such as still or video for display. Graphics subsystem 1715 may be a graphics processing unit (GPU) or a visual processing unit (VPU), for example. An analog or digital interface may be used to communicatively couple graphics subsystem 1715 and display 1720. For example, the interface may be any of a High-Definition Multimedia Interface, DisplayPort, wireless HDMI, and/or wireless HD compliant techniques. Graphics subsystem 1715 may be integrated into processor 1710 or chipset 1705. In some implementations, graphics subsystem 1715 may be a stand-alone device communicatively coupled to chipset 1705.
The graphics and/or video processing techniques described herein may be implemented in various hardware architectures. For example, graphics and/or video functionality may be integrated within a chipset. Alternatively, a discrete graphics and/or video processor may be used. As still another implementation, the graphics and/or video functions may be provided by a general purpose processor, including a multi-core processor. In further embodiments, the functions may be implemented in a consumer electronics device.
Radio 1718 may include one or more radios capable of transmitting and receiving signals using various suitable wireless communications techniques. Such techniques may involve communications across one or more wireless networks. Example wireless networks include (but are not limited to) wireless local area networks (WLANs), wireless personal area networks (WPANs), wireless metropolitan area network (WMANs), cellular networks, and satellite networks. In communicating across such networks, radio 1718 may operate in accordance with one or more applicable standards in any version.
In various implementations, display 1720 may include any television type monitor or display. Display 1720 may include, for example, a computer display screen, touch screen display, video monitor, television-like device, and/or a television. Display 1720 may be digital and/or analog. In various implementations, display 1720 may be a holographic display. Also, display 1720 may be a transparent surface that may receive a visual projection. Such projections may convey various forms of information, images, and/or objects. For example, such projections may be a visual overlay for a mobile augmented reality (MAR) application. Under the control of one or more software applications 1716, platform 1702 may display user interface 1722 on display 1720.
In various implementations, content services device(s) 1730 may be hosted by any national, international and/or independent service and thus accessible to platform 1702 via the Internet, for example. Content services device(s) 1730 may be coupled to platform 1702 and/or to display 1720. Platform 1702 and/or content services device(s) 1730 may be coupled to a network 1760 to communicate (e.g., send and/or receive) media information to and from network 1760. Content delivery device(s) 1740 also may be coupled to platform 1702 and/or to display 1720.
In various implementations, content services device(s) 1730 may include a cable television box, personal computer, network, telephone, Internet enabled devices or appliance capable of delivering digital information and/or content, and any other similar device capable of unidirectionally or bidirectionally communicating content between content providers and platform 1702 and/display 1720, via network 1760 or directly. It will be appreciated that the content may be communicated unidirectionally and/or
bidirectionally to and from any one of the components in system 1700 and a content provider via network 1760. Examples of content may include any media information including, for example, video, music, medical and gaming information, and so forth.
Content services device(s) 1730 may receive content such as cable television programming including media information, digital information, and/or other content. Examples of content providers may include any cable or satellite television or radio or Internet content providers. The provided examples are not meant to limit implementations in accordance with the present disclosure in any way.
In various implementations, platform 1702 may receive control signals from navigation controller 1750 having one or more navigation features. The navigation features of controller 1750 may be used to interact with user interface 1722, for example. In various embodiments, navigation controller 1750 may be a pointing device that may be a computer hardware component (specifically, a human interface device) that allows a user to input spatial (e.g., continuous and multi-dimensional) data into a computer. Many systems such as graphical user interfaces (GUI), and televisions and monitors allow the user to control and provide data to the computer or television using physical gestures. Movements of the navigation features of controller 1750 may be replicated on a display (e.g., display 1720) by movements of a pointer, cursor, focus ring, or other visual indicators displayed on the display. For example, under the control of software applications 1716, the navigation features located on navigation controller 1750 may be mapped to virtual navigation features displayed on user interface 1722, for example. In various embodiments, controller 1750 may not be a separate component but may be integrated into platform 1702 and/or display 1720. The present disclosure, however, is not limited to the elements or in the context shown or described herein.
In various implementations, drivers (not shown) may include technology to enable users to instantly turn on and off platform 1702 like a television with the touch of a button after initial boot-up, when enabled, for example. Program logic may allow platform 1702 to stream content to media adaptors or other content services device(s) 1730 or content delivery device(s) 1740 even when the platform is turned "off." In addition, chipset 1705 may include hardware and/or software support for 5.1 surround sound audio and/or high definition 7.1 surround sound audio, for example. Drivers may include a graphics driver for integrated graphics platforms. In various embodiments, the graphics driver may comprise a peripheral component interconnect (PCI) Express graphics card.
In various implementations, any one or more of the components shown in system 1700 may be integrated. For example, platform 1702 and content services device(s) 1730 may be integrated, or platform 1702 and content delivery device(s) 1740 may be integrated, or platform 1702, content services device(s) 1730, and content delivery device(s) 1740 may be integrated, for example. In various embodiments, platform 1702 and display 1720 may be an integrated unit. Display 1720 and content service device(s) 1730 may be integrated, or display 1720 and content delivery device(s) 1740 may be integrated, for example. These examples are not meant to limit the present disclosure.
In various embodiments, system 1700 may be implemented as a wireless system, a wired system, or a combination of both. When implemented as a wireless system, system 1700 may include components and interfaces suitable for communicating over a wireless shared media, such as one or more antennas, transmitters, receivers, transceivers, amplifiers, filters, control logic, and so forth. An example of wireless shared media may include portions of a wireless spectrum, such as the RF spectrum and so forth. When implemented as a wired system, system 1700 may include components and interfaces suitable for communicating over wired communications media, such as input/output (I/O) adapters, physical connectors to connect the I/O adapter with a corresponding wired communications medium, a network interface card (NIC), disc controller, video controller, audio controller, and the like. Examples of wired communications media may include a wire, cable, metal leads, printed circuit board (PCB), backplane, switch fabric,
semiconductor material, twisted-pair wire, co-axial cable, fiber optics, and so forth.
Platform 1702 may establish one or more logical or physical channels to
communicate information. The information may include media information and control information. Media information may refer to any data representing content meant for a user. Examples of content may include, for example, data from a voice conversation, videoconference, streaming video, electronic mail ("email") message, voice mail message, alphanumeric symbols, graphics, image, video, text and so forth. Data from a voice conversation may be, for example, speech information, silence periods, background noise, comfort noise, tones and so forth. Control information may refer to any data representing commands, instructions or control words meant for an automated system. For example, control information may be used to route media information through a system, or instruct a node to process the media information in a predetermined manner. The embodiments, however, are not limited to the elements or in the context shown or described in FIG. 17.
As described above, system 1700 may be embodied in varying physical styles or form factors. FIG. 18 illustrates implementations of a small form factor device 1800 in which system 1800 may be embodied. In various embodiments, for example, device 1800 may be implemented as a mobile computing device a having wireless capabilities. A mobile computing device may refer to any device having a processing system and a mobile power source or supply, such as one or more batteries, for example.
As described above, examples of a mobile computing device may include a personal computer (PC), laptop computer, ultra-laptop computer, tablet, touch pad, portable computer, handheld computer, palmtop computer, personal digital assistant (PDA), cellular telephone, combination cellular telephone/PDA, television, smart device (e.g., smart phone, smart tablet or smart television), mobile internet device (MID), messaging device, data communication device, cameras (e.g. point-and-shoot cameras, super-zoom cameras, digital single-lens reflex (DSLR) cameras), and so forth.
Examples of a mobile computing device also may include computers that are arranged to be worn by a person, such as a wrist computer, finger computer, ring computer, eyeglass computer, belt-clip computer, arm-band computer, shoe computers, clothing computers, and other wearable computers. In various embodiments, for example, a mobile computing device may be implemented as a smart phone capable of executing computer applications, as well as voice communications and/or data communications. Although some embodiments may be described with a mobile computing device implemented as a smart phone by way of example, it may be appreciated that other embodiments may be implemented using other wireless mobile computing devices as well. The embodiments are not limited in this context.
As shown in FIG. 18, device 1800 may include a housing 1802, a display 1804, an input/output (I/O) device 1806, and an antenna 1808. Device 1800 also may include navigation features 1812. Display 1804 may include any suitable display unit for displaying information appropriate for a mobile computing device. I/O device 1806 may include any suitable I/O device for entering information into a mobile computing device. Examples for I/O device 1806 may include an alphanumeric keyboard, a numeric keypad, a touch pad, input keys, buttons, switches, rocker switches, microphones, speakers, voice recognition device and software, and so forth. Information also may be entered into device 1800 by way of microphone (not shown). Such information may be digitized by a voice recognition device (not shown). The embodiments are not limited in this context.
Various embodiments may be implemented using hardware elements, software elements, or a combination of both. Examples of hardware elements may include processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, and so forth), integrated circuits, application specific integrated circuits (ASIC), programmable logic devices (PLD), digital signal processors (DSP), field programmable gate array (FPGA), logic gates, registers, semiconductor device, chips, microchips, chip sets, and so forth. Examples of software may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application program interfaces (API), instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. Determining whether an embodiment is implemented using hardware elements and/or software elements may vary in accordance with any number of factors, such as desired computational rate, power levels, heat tolerances, processing cycle budget, input data rates, output data rates, memory resources, data bus speeds and other design or performance constraints. One or more aspects of at least one embodiment may be implemented by
representative instructions stored on a machine-readable medium which represents various logic within the processor, which when read by a machine causes the machine to fabricate logic to perform the techniques described herein. Such representations, known as "IP cores" may be stored on a tangible, machine readable medium and supplied to various customers or manufacturing facilities to load into the fabrication machines that actually make the logic or processor.
As discussed, systems, apparatus, articles, and methods are described herein related to content adaptive entropy coding for video systems. In some examples, systems, apparatus, articles, and methods are described herein related to content adaptive entropy coding for coded / not coded video data (e.g., data indicating whether blocks or partitions or the like include non-zero transform coefficients) for video systems.
In some examples, a selected entropy coding technique may be determined for coded/not-coded video data from a plurality of entropy coding techniques. For example, the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, the plurality of entropy coding techniques may include two or more proxy variable length coding techniques implementing different variable length codes for different sequences (e.g., via proxy variable length coding tables), or the plurality of entropy coding techniques may include two or more symbol-run coding techniques implementing different codes (e.g., via symbol-run coding tables), or the like.
In some examples, the selected entropy coding technique may be based on evaluations of various processed bitstreams associated with the coded/not-coded video data. For example, the processed bitstream(s) may include a pass-through bitstream (e.g., a bitstream that is the coded/not-coded video data), a reversed bitstream (e.g., a reversal of the coded/not-coded video data), an inverted bitstream (e.g., a bit inversion of the coded/not-coded video data), or a differential bitstream (e.g., a bit difference of the coded/not-coded video data). The various processed bitstreams may be evaluated based on various entropy coding techniques (e.g., proxy variable length coding technique(s) and symbol-run coding technique(s) as discussed) to determine a processed bitstream and entropy coding technique having a minimum bit cost (e.g., the fewest bits of the evaluated processed bitstreams and entropy coding techniques). The processed bitstream may be entropy encoded to generate an encoded coded/not-coded video data bitstream and the encoded coded/not-coded video data bitstream may be assembled to generate an output bitstream. In some examples, the output bitstream may also include a coded/not-coded video data header indicating the selected entropy coding technique and the type of the processed bitstream (e.g., pass-through, reversed, inverted, differential, or the like).
As discussed herein, encoder module 404 may include symbol-run VLC and/or proxy VLC encoder for coded/not-coded data 641 (please refer to FIG. 6). Encoder module 404 may receive video data 414, which may include coded/not-coded symbol bits as is discussed further herein, via symbol-run VLC and/or proxy VLC encoder for coded/not- coded data 641. Symbol-run VLC and/or proxy VLC encoder for coded/not-coded data 641 may determine an entropy encoding technique for video data 414 based on a parameter, parameters or characteristics of video data 414 or other system parameters. In some examples, the entropy encoding technique for video data 414 may be one of an adaptive symbol-run variable length coding technique or an adaptive proxy variable length coding technique, or multiple adaptive symbol-run variable length coding techniques or multiple adaptive proxy variable length coding technique, or combinations thereof as described herein. The determined entropy encoding technique may be applied to video data 414 to generate compressed video data 424. As is discussed further herein, encoder module 404 may determine which technique provides the greatest compression efficiency that the parameter(s) associated with video data 414 may be the number of bits needed for each coding technique or the like. In some examples, the parameter associated with video data 314 may be based on a characteristic of the video data such that the determined encoding technique may be adaptive to the received video data.
As discussed, next generation video coding may use 3 picture types, I-, P-, and F- pictures. For example, I- and P-pictures, while they may have differences in details of prediction and coding, may be functionally similar. For example, inter-partitions (e.g., as referred herein to inter-blocks) of P- or F- pictures, after transform and quantization (please refer to FIG. 1), may result in either any non-zero quantized transform coefficients that may need to be sent to decoder or all zero quantized transform coefficients (e.g., no non-zero quantized transform coefficients). If a block (e.g., partition) has any non-zero transform coefficients, the block may be classified as coded. If a block (e.g., partition) has no non-zero transform coefficients, the block may be classified as not-coded. This coded/not-coded (e.g., CNC) data may be communicated to a decoder via output bitstream 1 11 or the like. In some implementations, the coded / not-coded data may be substantial due to the use of variable block size partitioning that may result in lots of small blocks, particularly at higher bitrates.
FIG. 19 is an illustrative diagram of an example picture 1900 of coded/not-coded inter-blocks, arranged in accordance with at least some implementations of the present disclosure. As shown in FIG. 19, picture 1900 may include tiles 1910, which may include one or more blocks, partitions, sub-partitions, or the like, as discussed herein. In the discussion herein, the term block may be used for the sake of clarity; however, it is to be understood the term block may be representative of a prediction partition or sub-partition or the like, which may be substantially any size or shape. For example, picture 1900 may be a P-picture having a tile size of 32x32 for coding using a quantizer value (e.g., quantization parameter, Qp) of 8. For example, a quantizer value may determine a step size in the quantization of transform coefficients (e.g., by adaptive quantize module 109, please see FIG. 1). In FIG. 19, coded inter-blocks 1920 are illustrated in light gray and not-coded inter-blocks 1930 are illustrated in dark gray. Blocks which are illustrated in white may be other types of blocks (e.g., intra, skip, or auto blocks) such that inter-prediction does not apply to such blocks. The prediction of such blocks may not be relevant to the present discussion.
As shown in FIG. 19, for smaller quantizer values (e.g., Qp = 8), the partitioning for coding may result in a significant number of coded inter-blocks 1920 and not-coded inter- blocks 1930. For example, there may be a significant density of coded/not-coded inter- blocks. In some examples, each block (e.g. of coded/not-coded blocks 1920, 1930) may be assigned one bit. In such examples, the coded / not coded (e.g., CNC) data may result in a substantial bitstream of data.
FIG. 20 is an illustrative diagram of an example picture 2000 of coded/not-coded inter-blocks, arranged in accordance with at least some implementations of the present disclosure. As shown in FIG. 20, picture 2000 may include tiles 2010, which may include one or more blocks, partitions, prediction partitions, sub-partitions, or the like, as discussed herein. For example, picture 2000 may be an F-picture having a tile size of 64x64 for coding using a quantizer value of 32. In FIG. 20, coded inter-blocks 2020 are illustrated in light gray and not-coded inter-blocks 2030 are illustrated in dark gray. As discussed with respect to FIG. 19, blocks illustrated in white may be other types of blocks (e.g., intra, skip, or auto blocks) such that inter-prediction does not apply to such blocks and they may not be relevant to the present discussion. As shown in FIG. 20, for larger quantizer values (e.g., Qp = 32), the partitioning for coding may result in a smaller number of coded inter-blocks 2020 and not-coded inter- blocks 2030. For example, there may be a lesser density of coded/not-coded inter-blocks as compared with the example of FIG. 19. Further, in general, F-pictures may have a lesser density of coded/not-coded inter-blocks as compared with P-pictures. Further still, less complicated scenes or video sequences may have lesser density of coded/not-coded inter- blocks as compared with more complicated or more movement oriented video sequences. Also as discussed with respect to FIG. 19, in some examples, each block (e.g. of coded/not-coded blocks 2020, 2030) may be assigned one bit.
In some examples, entropy coding techniques discussed herein may used to encode the coded/not-coded data. In some examples, an adaptive symbol-run variable coding technique may be used. In some examples, an adaptive proxy variable length coding technique may be used. Such techniques have been described herein and will not be repeated for the sake of brevity. In some implementations, various techniques discussed with respect to adaptive symbol-run variable coding and/or adaptive proxy variable length coding as discussed elsewhere herein may be used with respect to encoder module 404, sub-modules, and the various methods and modes discussed herein.
As is discussed further below, the described entropy coding techniques (e.g., adaptive symbol-run variable coding and adaptive proxy variable length coding) may each be modified based on the codes, symbols, or the like used in the entropy coding implementation. Such implementations will be described further herein with respect to symbol run variable length coding tables and proxy variable length coding tables such as Tables 3-12 below.
Further, a selected entropy coding technique may operate on a processed bitstream associated with the coded/not-coded video data. For example, a frame of video data may have a block (or partition) structure. Referring to FIG. 19, picture 1910 may have a block structure as shown via coded/not-coded blocks 1920, 1930. For example, the block structure may include the partitioning of picture 1910 into blocks (or partitions). In some examples, the block structure may be substantially complicated and may be adaptive to a variety of characteristics and/or parameters of picture 1910, adjacent pictures (or frames), coding settings, or the like. In some examples, a block structure associated with a video frame may be read (e.g., by adaptive entropy encoder 110 or a module therein). The block structure may be associated with the coded/not-coded video data such that the bits of the coded/not-coded video data indicate a 1 for blocks of the block structure that are coded (e.g., have non-zero transform coefficients) and a 0 for blocks of the block structure that are not-coded (e.g., have no non-zero transform coefficients).
The coded/not-coded video data may subsequently be processed. For example, for P- pictures, a 1 -dimensional raster scan may be generated based on the coded/not-coded video data and a 1 -dimensional block-based scan may be generated based on the coded/not- coded video data. In some examples, the 1 -dimensional raster scan and the 1 -dimensional block-based scan may be considered processed bitstreams associated with the coded/not- coded video data. Further, the 1 -dimensional raster scan or the 1 -dimensional block-based scan or both may be further processed by performing bit inversions, reversals, bit differentials, or the like. Such further processed bitstreams may also be considered processed bitstreams as described herein.
A processed bitstream associated with the coded/not-coded video data may be entropy encoded using a selected entropy coding technique (e.g., including selected codes, symbols, or the like as described herein using tables). For example, the selected entropy coding technique may be chosen based on evaluating multiple entropy coding techniques with respect to multiple processed bitstreams to determine the selected entropy coding technique and the relevant (e.g., selected) processed bitstream.
FIG. 21 is an illustrative diagram of an example coded/not-coded data encoder module 404, arranged in accordance with at least some implementations of the present disclosure. As shown, coded/not-coded data encoder module 404, which may be described as encoder module 404 herein for the sake of brevity, may include symbol-run VLC and/or proxy VLC encoder for coded/not-coded data 641, as discussed with respect to FIG. 6. In some examples, coded/not-coded data encoder module 404 may not include symbol-run VLC and/or proxy VLC encoder for coded/not-coded data 641. The other modules (e.g., encoder modules 401-403 and 405^107) of adaptive entropy encoder 110 may be implemented as described elsewhere herein and will not be described further with respect to FIG. 21. In some examples, coded/not-coded data encoder module 404 may provide efficiency by incorporating a number of adaptive techniques. Further, coded/not-coded data encoder module 404 may allow coding efficiency while providing substantially low complexity.
Also as illustrated in FIG. 21, coded/not-coded data encoder module 404 may include a coded/not-coded data processing module 2101, a symbol-run generation and variable length coding (VLC) lookup table (LUT) module 211 1, a proxy variable length coding (VLC) lookup table (LUT) module 2120, a coded/not-coded variable length coding (VLC) encoder module 2130, and a minimum bits selector 2140. As shown, symbol-run generation and VLC LUT module 21 11 may include multiple symbol-run generation and VLC LUT modules 21 12, 21 13, 21 14. Symbol-run generation and VLC LUT module 21 11 may include any number of modules such as modules corresponding to the symbol-run VLC tables discussed herein or the like. Further, proxy VLC LUT module 2121 may include multiple proxy VLC LUT modules 2122, 2123, 2124. Proxy VLC LUT module 211 1 may include any number of modules such as modules corresponding to the proxy VLC tables discussed herein or the like.
Also as shown, coded/not-coded data processing module 2101 may include a bits pass-through module 2102, a bits rescanner/reverser module 2103, a bits inverter module 2104, and a bits differencer module 2105. For example, coded/not-coded data processing module 2101 and coded/not-coded VLC encoder module 2130 may receive video data 414. Video data 414 may include any suitable coded/not-coded video data (e.g., including coded/not symb bits as shown). In some examples, video data 414 may include coded/not- coded video data associated with a block structure of a video frame or picture. In some examples, video data 414 may include a representation of a 2-dimensional description of various size blocks (or partitions) that may be coded/not-coded (e.g., 1 may indicate coded and 0 may indicate not-coded).
Video data 414 may be received and processed by coded/not-coded data processing module 2101. For example, coded/not-coded data processing module 2101 may process 414 via one or more of modules 2102, 2103, 2104, 2105. In some examples, coded/not- coded data processing module 2101 may process video data 414 via all of modules 2102, 2103, 2104, 2105 in parallel. As discussed, coded/not-coded data processing module 2101 may include bits pass-through module 2102, bits rescanner/reverser module 2103, bits inverter module 2104, and/or a bits differencer module 2105. In some examples, bits pass- through module 2102 may pass-through one or more bitstreams associated with coded/not- coded video data of video data 414. For example, the coded/not-coded data may be processed or scanned to produce one or more scans associated with the coded/not-coded video data. For example, the coded/not-coded video data may be processed to generate a 1- dimensional raster scan, a 1 -dimensional block-based scan, a 1 -dimensional tile-based scan, or the like. In some examples, generating a 1 -dimensional raster scan may include a frame or picture based scan that processes from the top-left of the frame to the bottom-right of the frame and generates a 1 bit for each coded block and a 0 bit for each not-coded block (or vice versa). For example, the scan may follow a pattern from top-left to bottom-right by moving horizontally from left to right at a top row and then dropping down one row and repeating the left to right scans vertically down the picture. In such a coding, the size and/or shape of the block may be ignored such that each size and shape of block may be given a 1 or 0 as described. Further, as scanning progresses, blocks may be processed when the block's top-left corner is scanned and, if the block's edge is scanned in subsequent horizontal scans, the block may be skipped as previously scanned. Other portions of the described codec may indicate the size, shape, and location of each block (e.g., via partitioning and the like) such that that information may not be needed in the coded/not-coded data scan. In some examples, generating a 1 -dimensional tile-based scan may include performing a scan (e.g. a scan from top-left to bottom-right as discussed) within a top-left tile and then moving to the next tile and repeating the scan within each tile. In such examples, the ordering of the tiles may also follow a pattern from top-left to bottom-right of the frame or picture by moving horizontally from left to right and then dropping down one row and repeating the left to right scans progressing vertically down the picture until the picture is complete. In some examples, generating a 1 -dimensional block-based scan may include performing a scan (again from top-left to bottom-right as discussed) within a block (not corresponding to the blocks being coded), such that the blocks are portions of a tile. For example, a tile may be divided in half vertically and then each half may be quartered horizontally to define 8 8x16 blocks that may each be scanned. In order, the scan may be performed from top-left to bottom-right within a block, then within a tile, and then within a picture. In some examples, prior to the dividing steps described, a tile may be flipped vertically about a vertical centerline prior to dividing vertically into halves and quartering each half horizontally. As will be appreciated, other block sizes and/or divisions of tiles may be used.
Bits pass-through module 2102 may pass-through the generated or processed bitstream, for example, without further processing. In some examples, the generated or processed bitstream may be generated by coded/not-coded data processing module 2101 or another module of coded/not-coded data encoder module 404.
In some examples, bits rescanner/reverser module 2103 may reverse the order of the scan or bitstream to generate a reversed scan such as, for example, 1 -dimensional reversed raster scan, a 1 -dimensional reversed block-based scan, a 1 -dimensional reversed tile- based scan, or the like. Further, in some examples, bits inverter module 2104 may invert the bits (e.g., bit invert) of the scan or bitstream (e.g., change Os to Is and vice versa) to generate an inverted bits scan such as, for example, a 1 -dimensional inverted bits raster scan, a 1 -dimensional inverted bits block-based scan, a 1 -dimensional inverted bits tile- based scan, or the like. Further, in some examples, bits differencer module 2105 may difference the bits (e.g., bit difference) of the scan or bitstream (e.g., take a difference along sequential bits) to generate a differential bits scan such as, for example, a 1- dimensional differential bits raster scan, a 1 -dimensional differential bits block-based scan, a 1 -dimensional differential bits tile-based scan, or the like. Further, in some examples, bitstreams or scans may be passed between modules 2102, 2103, 2104, 2105. For example, an inverted scan generated by bits inverter module 2104 may be operated on by bits differencer module 2105 to generate an inverted differential scan such as, for example, a 1- dimensional inverted differential bits raster scan, a 1 -dimensional inverted differential bits block-based scan, a 1 -dimensional inverted differential bits tile-based scan, or the like.
As shown in FIG. 21, one or more or all of the generated processed bitstreams as discussed may be transmitted to one or both of symbol-run generation and VLC LUT module 21 1 1 and/or proxy VLC LUT module 2121. In some examples, the same processed bitstreams may be transmitted to both-run generation and VLC LUT module 211 1 and/or proxy VLC LUT module 2121. In some examples, different processed bitstreams may be transmitted to symbol-run generation and VLC LUT module 211 1 and/or proxy VLC LUT module 2121.
For example, the processed bitstream(s) received by symbol-run generation and VLC LUT module 211 1 may be processed to determine bit cost(s) associated with coding the processed bitstream(s) using symbol-run coding with one or more symbol / code combinations. For example, a bit cost may be determined for a processed bitstream using a first and second or a first, second, and third (or the like) symbol-run coding LUT. For example, symbol-run generation and VLC LUT module 211 1 may allow for the counting of the bit cost(s) of events (e.g., symbol-run combination(s)). Similarly the processed bitstream(s) received by proxy VLC LUT module 2121 may be processed to determine bit cost(s) associated with coding the processed bitstream(s) using symbol-run coding with one or more symbol / code combinations. For example, a bit cost may be determined for a processed bitstream using one, a first and second, or a first, second, and third (or the like) proxy VLC LUT. For example, proxy VLC LUT module 2121 may allow for the counting of bits by trying any number of different proxy VLC codes that may replace groups of bits/patterns with shorter codes.
The processing by symbol-run generation and VLC LUT module 211 1 and/or proxy VLC LUT module 2121 may be performed via a bit counting by look up or the like (e.g., actual encoding and bit counting may not be required). Such processing may substantially reduce the complexity of encoder module 404, for example.
A wide range of bit costs may thereby be determined based on combinations of processed bitstream(s), encoding technique (e.g., symbol-run or proxy VLC), and encoding technique implementation (e.g., look up table for the pertinent encoding technique). As will be further discussed with respect to FIGS. 23 and 24, example bit costs are provided for P-pictures and F-pictures, respectively, herein. However, such example
implementations are not meant to be limiting.
As shown in FIG. 21 , the determined bit costs may be transmitted to minimum bits selector 2140 and/or coded/not-coded VLC encoder module 2130. Minimum bits selector
2140 may determine the minimum bit cost based on the received bit cost to determine the selected entropy coding technique and the selected processed bitstream. Minimum bits selector 2140 may transmit the selected entropy coding technique and the selected processed bitstream to coded/not-coded VLC encoder module 2130. For example, minimum bits selector 2140 may determine a best solution (e.g., fewest number of bits) for representing the coded/not-coded data for a current frame. For example, the output of minimum bits selector 2140 may be an index to the entropy coding technique and table that produces the highest compression.
Coded/not-coded VLC encoder module 2130 may generate the selected processed bitstream associated with the coded/not-coded video data, as discussed, and entropy encode the processed bitstream using the selected entropy coding technique to generate an encoded coded/not-coded video data bitstream, which may be a part of or implemented as compressed video data 324. For example, coded/not-coded VLC encoder module 2130 may use the index provided via minimum bits selector 2140 to access the selected entropy coding technique and table to generate a compressed representation of the coded/not-coded video data as, for example compressed video data 424. As discussed herein, compressed video data 424 may be transmitted to bitstream assembler 408, which may assemble compressed video data and other compressed video data (e.g., compressed video data 401- 403 and 405-407) to generate output bitstream 11 1.
Due to the dependence (e.g., statistical dependence) of coded/not coded data on the quantizer (as discussed above) used for transform coding (e.g., a larger quantizer may provide more 'not coded' blocks while a smaller quantizer may provide more 'coded' blocks) and on frame type, P-pictures may have more 'coded' blocks with respect to F- pictures, which may have more 'not coded' blocks. In such examples, it may be advantageous to evaluate symbol-run VLC coding techniques with many tables and to evaluate proxy VLC techniques with many tables on an original stream (e.g., an original scan such as a 1 -dimensional raster scan, a 1 -dimensional block-based scan, or a 1- dimensional tile-based scan) and further processed stream(s) to determine a solution that is most efficient in compression (e.g., a minimum bit cost).
The use of a substantial number of tables may not substantially increase the complexity of coded/not-coded data encoder module 404 since, as discussed, actual encoding of the processed bitstream(s) by many methods may not be needed. Instead, only bit counting by look up may be required to estimate the bit cost for a processed bitstream and entropy coding technique combination.
FIG. 22 is a flow diagram illustrating an example process 2200, arranged in accordance with at least some implementations of the present disclosure. Process 2200 may include one or more operations, functions or actions as illustrated by one or more of operations 2202, 2204, and/or 2206. Process 2200 may form at least part of a next generation video coding process. By way of non-limiting example, process 2200 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or entropy encoder module 1 10 as discussed herein.
In some examples, process 2200 may include reading a block structure associated with a frame of video data as discussed with respect to FIGS. 19 and 20. The block structure may be associated with coded/not-coded video data as discussed such that the coded/not-coded video data has a bit associated with each block of the block structure indicting whether the block is coded (e.g., is associated with a bit of 1) or not-coded (e.g., is associated with a bit of 0).
Process 2200 may begin (or continue from reading the block structure and associating the block structure with coded/not-coded data) at operation 2202, "Determine a Selected Entropy Coding Technique for Coded/Not-Coded Video Data", where a selected entropy coding technique may be determined for coded/not-coded video data from multiple entropy coding techniques as discussed herein. For example the entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, a single proxy variable length coding technique and multiple symbol-run coding techniques (e.g., implemented via different tables), multiple proxy variable length coding techniques (e.g., implemented via different tables) and a single symbol-run coding technique, multiple proxy variable length coding techniques, or multiple symbol-run coding techniques, or the like. The selected entropy coding technique may be determined based on estimating bit costs for the multiple entropy coding techniques and with various processed bitstreams (e.g., a pass-through of the coded/not-coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, 1 -dimensional scans, bit inversions, bit differences, bits reversals, or the like), as discussed herein, and selecting the entropy coding technique and processed bitstream having the lowest bit cost. For example, the selected entropy coding technique may be determined via coded/not-coded data encoder module 304.
Process 2200 may continue at operation 2204, "Entropy Encode a Processed Bitstream Associated with the Coded/Not-Coded Video Data Using the Selected Entropy Coding Technique", where a processed bitstream associated with the coded/not-coded video data may be entropy encoded using the selected entropy coding technique to generate an encoded coded/not-coded video data bitstream. For example, the processed bitstream may be a selected bitstream such that the selected bitstream and selected entropy coding technique provided the lowest bit cost. For example, the processed bitstream may be entropy encoded via coded/not-coded VLC encoder module 2130.
Process 2200 may continue at operation 2206, "Assemble the Encoded Coded/Not- Coded Video into an Output Bitstream", where the encoded coded/not-coded video data may be assembled into an output bitstream. For example, bitstream assembler 308 may assemble the encoded coded/not-coded video data (e.g., compressed video data 424) and other compressed video data (e.g., compressed video data 421-423 and 425-427) into output bitstream 11 1. As discussed, in some examples, a coded/not-coded video data header associated with the selected entropy coding technique and/or the selected processed bitstream may be assembled into the output bitstream. For example, compressed video data 424 may include the coded/not-coded video data header, which may be assembled as discussed into output bitstream 11 1.
As discussed, in some examples, output bitstream 1 11 may be transmitted via encoder system 100 as implemented via video coding system 1100 or the like to one or more video decoders.
FIG. 23 is a flow diagram illustrating an example process 2300, arranged in accordance with at least some implementations of the present disclosure. Process 2300 may include one or more operations, functions or actions as illustrated by one or more of operations 2302, 2304, 2306, and/or 230. Process 2300 may form at least part of a next generation video coding process. By way of non-limiting example, process 2300 may form at least part of a next generation video encoding process as undertaken by decoder system 200 of FIG. 2 and/or entropy decoder module 202 as discussed herein.
Process 2300 may begin at operation 2302, "Receive an Entropy Encoded
Bitstream", where an entropy encoded bitstream may be received. For example, output bitstream 1 1 1, input bitstream 201, or bitstream 800 or the like may be received via video decoder 200 as implemented via video coding system 1 100 or the like.
Process 2300 may continue at operation 2304, "Disassemble the Entropy Encoded Bitstream to Determine a Coded/Not-Coded Video Data Header and an Encoded
Coded/Not-Coded Video Data Bitstream", where the entropy encoded bitstream may be disassembled to determine a coded/not-coded video data header and encoded coded/not- coded video data. For example, the entropy encoded bitstream may be disassembled via bitstream disassembler 408.
Process 2300 may continue at operation 2306, "Determine a Selected Entropy Coding Technique Based on the Coded/Not-Coded Video Data Header", where a selected entropy coding technique may be determined for the encoded coded/not-coded video data from multiple entropy coding techniques based at least in part on the coded/not-coded video data header. For example, the coded/not-coded video data header may include a code indicating a selected entropy coding technique (and/or selected table for the entropy coding technique) for the encoded coded/not-coded video data and a format or type of the bitstream of the encoded coded/not-coded video data. For example, the selected entropy coding technique may include any technique as discussed herein (e.g., symbol-run coding technique(s) as implemented via various tables and/or proxy variable length coding technique(s) as implemented via various tables) and the bitstream may include any bitstream type or processing as discussed herein (e.g., 1 -dimensional scans, bit inversions, bit differences, bits reversals, or the like) encoded or compressed via the selected entropy coding technique.
Process 2300 may continue at operation 2308, "Entropy Decode the Encoded Coded/Not-Coded Video Data Bitstream Using the Selected Entropy Coding Technique to Determine Coded/Not-Coded Video Data", where the encoded coded/not-coded video data may be entropy decoded using the selected entropy coding technique to determine coded/not-coded video data. For example, the encoded coded/not-coded video data may be decompressed to using the selected entropy coding technique to generate a processed bitstream. For example, the processed bitstream may include any processed bitstream (e.g., 1 -dimensional scans, bit inversions, bit differences, bits reversals, or the like) as discussed herein. The processed bitstream may be decoded via the inverse of the operation used to generate the processed bitstream (e.g., by pass-through, bit inversion, bit reversal, or bit differential as discussed) to generate the coded/not-coded video data.
Process 2300 may continue at operation 2310, "Generate a Video Frame Based on the Coded/Not-Coded Video Data", where the coded/not-coded video data may be used to generate a video frame as discussed herein with respect to decoder system 200. Further, the generated video frame may be transmitted to a display device for presentment to a viewer or user.
As discussed, multiple combinations of entropy coding techniques and processed bitstreams may be evaluated to determine a minimum bit cost combination. In some examples, a subset of such combinations may be chosen for evaluation. For example, those combinations likely to produce minimum bit costs or efficient coding may be selected for evaluation. In some examples, the chosen combinations may be based on the frame type (e.g., F- or P-picture) of the video data.
FIG. 24 provides a flow diagram illustrating an example process 2400, arranged in accordance with at least some implementations of the present disclosure. Process A1900 may include one or more operations, functions or actions as illustrated by one or more of operations 2400-2415. Process 2400 may form at least part of a next generation video coding process. By way of non-limiting example, process 2400 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or entropy encoder module 1 10 as discussed herein. In some examples, process 2400 may be applied to P-pictures.
Process 2400 may begin at operation 2401 "Input Frame Level CNC Data", where frame-level tile/block structure and coded/not-coded video data (e.g., binary information; CNC data) for inter mode blocks may be read. For example, a block structure associated with a frame of video data may be read. Process 2400 may continue at operation 2402 "Number of Inter Blocks <Thr (25)?", where, if the number of inter blocks in the frame is less than a threshold value (e.g., 25 in the example), then the CNC data may be coded with a bitmap method (e.g., bit by bit) as shown at operation 2414 "Code CNC Data with a Bitmap Method" and process 2400 may terminate as shown at operation 2415 "Finish Processing Current Frame". In some examples, operation 2402 may check for a simple case such that determining a compression technique may not be advantageous and the frame may be processed with bitmap coding. If the number of inter blocks in the frame is greater than (or equal to) the threshold, process 2400 may continue at operations 2403 and 2404.
At operation 2403 "Create ID Raster Scan of CNC Data", a 1 -dimensional raster scan of the coded/not-coded data may be generated. At operation 2404 "Create ID 8x16 Block-Based Scan of CNC Data", a 1-dimensional block based scan of the coded/not- coded data may be generated. For example, the 1 -dimensional raster scan may include a normal raster scan of blocks (e.g., frame-wise) and the 1-dimensional block based scan may include a 8x16 block-based scan (e.g., a scan of blocks in raster order within the 8x16 areas while processing 8x16 areas in raster order) or an equivalent 2x4 scale scan.
Process 2400 may continue at operations 2405 "Invert Bits" and 2406 "Invert Bits", where the respective scans may be bit inverted. For example, at operation 2405, the 1- dimensional raster scan may be bit inverted to generate an inverted bits 1-dimensional raster scan. At operation 2406, the 1-dimensional block-based scan may be bit inverted to generate an inverted bits 1-dimensional block-based scan. As discussed herein, operations 2403-2406 may generated processed bitstreams.
Process 2400 may continue at operations 2407 "Estimate Cost of Symbol-Run Coding (Table SO)", 2408 "Estimate Cost of Proxy VLC Coding (Tables V2,V3,V6,V1)", 2409 "Estimate Cost of Proxy VLC Coding (Tables V3 & V6)", and 2410 "Estimate Cost of Symbol-Run Coding (Table S2)", where the bit costs of the illustrated processed bitstream and entropy coding technique combinations may be estimated. As discussed, in some examples, the bit costs estimations may be determined based on a table look up or the like (e.g., and not based on an actual encoding and bit counting of the processed bitstreams). For example, at operation 2407, a first bit cost associated with a symbol run coding technique (e.g., using Table SO) of the inverted bits 1 -dimensional raster scan may be estimated. For example, at operation 2408, a second, a third, a fourth, and a fifth bit cost associated with multiple proxy variable length coding techniques (e.g., one each using Tables V2, V3, V6, and VI) of the inverted bits 1 -dimensional raster scan may be estimated. For example, at operation 2409, a sixth and a seventh bit cost associated with a proxy variable length coding techniques (e.g., one each using Tables V3 and V6) of the 1- dimensional raster scan may be estimated. For example, at operation 2410, an eighth bit cost associated with a proxy variable length coding technique (e.g., using Table S2) of the 1 -dimensional raster scan may be estimated.
Operations 2407-1910 may provide advantageous coding technique (including coding table) and processed bitstream combinations for P-pictures. In some examples, some of the provided combinations may be eliminated. In some examples, other available combinations may be added. For example, a difference bits operation may be added to process 2400 to provide additional processed bitstreams for bit cost estimation(s).
Process 2400 may continue at operation 241 1 "Select Method with the Smallest Cost (get_min_ind)", where a minimum (e.g., smallest) bit cost based on the first through eighth bit costs may be determined (e.g., as illustrated via action "get_min_ind"). For example, the estimated bits for all methods may be compared to determine the one with the smallest cost. Process 2400 may continue at operation 2412 "Code the Selected Method Index (See Table 1)", where a header code associated with the selected entropy coding technique (and table if applicable) may be encoded. For example, Table 1 as follows illustrated example header codes:
Method / Entropy Coding H c
Technique and Processed eader ode
Bitstream Code Length
Symbol-run on inverted bits 1 3 with table SO on raster-scan 00
ordered blocks
Symbol-run on inverted bits 1 4 with table S2 on 2x4-scan ordered 100
blocks
Proxy VLC coder on 0 2 inverted bits with table V2 on 0
raster-scan ordered blocks Proxy VLC coder on 1 4 inverted bits with table V3 on 101
raster-scan ordered blocks
Proxy VLC coder on 1 3 inverted bits with table V6 on 01
raster-scan ordered blocks
Proxy VLC coder on 1 4 inverted bits with table V 1 on 1 10
raster-scan ordered blocks
Proxy VLC coder with table 1 4 V3 on raster-scan ordered blocks 1 1 1
Proxy VLC coder with table 0 2 V6 on raster-scan ordered blocks 1
Table 1 : Symbol-run/ Proxy VLC methods in P-frames
Process 2400 may continue at operation 2413 "Code CNC Data with the Selected Method", where the processed bitstream associated with the coded/not-coded video data may be entropy encoded using the selected entropy coding technique to generate encoded coded/not-coded video data. Further, as discussed, the encoded coded/not-coded video data bitstream may be assembled into an output bitstream. For example, the selected method may be applied to the CNC data and the result may be written to bitstream. Process 2400 may terminate at operation 2415 "Finish Processing Current Frame". Process 2400 may be repeated as needed to encode any number of frames such as, for example, P-pictures or P- pictures.
FIG. 25 provides a flow diagram illustrating an example process 2500, arranged in accordance with at least some implementations of the present disclosure. Process A2000 may include one or more operations, functions or actions as illustrated by one or more of operations 2500-2514. Process 2500 may form at least part of a next generation video coding process. By way of non-limiting example, process 2500 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or entropy encoder module 1 10 as discussed herein. In some examples, process 2500 may be applied to F-pictures.
Process 2500 may begin at operation 2501 "Input Frame Level CNC Data", where frame-level tile/block structure and coded/not-coded video data (e.g., binary information; CNC data) for inter mode blocks may be read. For example, a block structure associated with a frame of video data may be read. Process 2500 may continue at operation 2502 "Number of Inter Blocks <Thr (25)?", where, if the number of inter blocks in the frame is less than a threshold value (e.g., 25 in the example), then the CNC data may be coded with a bitmap method (e.g., bit by bit) as shown at operation 2513 "Code CNC Data with a Bitmap Method" and process 2500 may terminate as shown at operation 2514 "Finish Processing Current Frame". In some examples, operation 2502 may check for a simple case such that determining a compression technique may not be advantageous and the frame may be processed with bitmap coding. If the number of inter blocks in the frame is greater than (or equal to) the threshold, process 2500 may continue at operations 2503 and 2504.
At operation 2503 "Create ID Tile-Based of CNC Data", a 1-dimensional tile-based scan of the coded/not-coded data may be generated. At operation 2504 "Create ID Raster Scan of CNC Data", a 1-dimensional raster scan of the coded/not-coded data may be generated. For example, the 1-dimensional raster scan may include a normal raster scan of blocks (e.g., frame-wise) and the 1-dimensional tile-based scan may include a scan of blocks in raster order within the tile areas while processing tiles in raster order).
Process 2500 may continue at operations 2505 "Invert Bits" and 2506 "Invert Bits", where the respective scans may be bit inverted. For example, at operation 2505, the 1- dimensional tile-based scan may be bit inverted to generate an inverted bits 1-dimensional tile-based scan. At operation 2506, the 1-dimensional raster scan may be bit inverted to generate an inverted bits 1-dimensional raster scan. Process 2500 may continue at operations 2507 "Difference Bits" and 2508 "Difference Bits", where the respective scans may be bit differenced. For example, at operation 2507, the inverted bits 1-dimensional tile-based scan may be bit differenced to generate an inverted differential bits 1- dimensional tile-based scan. For example, at operation 2508, the inverted bits 1- dimensional raster scan may be bit differenced to generate an inverted differential bits 1- dimensional raster scan. As discussed herein, operations 2503-2508 may generated processed bitstreams.
Process 2500 may continue at operations set 2509, which may include operations "Estimate Cost of Proxy VLC Coding (Tables V3,V8,V5,V10)" (first instance), "Estimate Cost of Symbol-Run Coding (Tables SO & S2)", "Estimate Cost of Proxy VLC Coding (Tables V3 & V5)", "Estimate Cost of Symbol-Run Coding (Table S2)", "Estimate Cost of Proxy VLC Coding (Table V8)" (first instance), "Estimate Cost of Proxy VLC Coding (Tables V3,V8,V5,V10)" (second instance), "Estimate Cost of Proxy VLC Coding (Tables V0 & V3)", "Estimate Cost of Proxy VLC Coding (Table V8)" (second instance), where the bit costs of the illustrated processed bitstream and entropy coding technique combinations may be estimated. As discussed, in some examples, the bit costs estimations may be determined based on a table look up or the like (e.g., and not based on an actual encoding and bit counting of the processed bitstreams).
For example, at the operation labeled "Estimate Cost of Proxy VLC Coding (Tables V3,V8,V5,V10)" (first instance), a first, a second, a third, and a fourth bit cost associated with multiple proxy variable length codings (e.g., one each using Tables V3, V8, V5, and V10) of the inverted bits 1 -dimensional tile-based scan may be estimated. For example, at the operation labeled "Estimate Cost of Symbol-Run Coding (Tables SO & S2)", a fifth and a sixth bit cost associated with multiple symbol-run codings (e.g., one each using Tables SO and VS2) of the 1 -dimensional tile-based scan may be estimated. For example, at the operation labeled "Estimate Cost of Proxy VLC Coding (Tables V3 & V5)", a seventh and an eighth bit cost associated with multiple proxy variable length codings (e.g., one each using Tables V3 and V5) of the 1 -dimensional tile-based scan may be estimated. For example, at the operation labeled "Estimate Cost of Symbol-Run Coding (Table S2)", a ninth bit cost associated with a symbol-run coding (e.g., using Table S2) of the inverted differential bits 1 -dimensional tile-based scan may be estimated. For example, at the operation labeled "Estimate Cost of Proxy VLC Coding (Table V8)" (first instance), a tenth bit cost associated with a proxy variable length coding (e.g., using Table V8) of the inverted differential bits 1 -dimensional tile-based scan may be estimated. For example, at the operation labeled "Estimate Cost of Proxy VLC Coding (Tables V3,V8,V5,V10)" (second instance), an eleventh, a twelfth, a thirteenth, and a fourteenth bit cost associated with multiple proxy variable length codings (e.g., one each using Tables V3, V8, V5, and V10) of the inverted bits 1 -dimensional raster scan may be estimated. For example, at the operation labeled "Estimate Cost of Proxy VLC Coding (Tables V0 & V3)", a fifteenth and a sixteenth bit cost associated with multiple proxy variable length codings (e.g., one each using Tables V0 and V3) of the 1 -dimensional raster scan may be estimated. For example, at the operation labeled "Estimate Cost of Proxy VLC Coding (Table V8)" (second instance), a seventeenth bit cost associated with a proxy variable length coding (e.g., using Table V8) of the inverted differential bits 1 -dimensional tile-based scan may be estimated.
Operations set 2509 may provide advantageous coding technique (including coding table) and processed bitstream combinations for F-pictures. In some examples, some of the provided combinations may be eliminated. In some examples, other available
combinations may be added.
Process 2500 may continue at operation 2510 "Select Method with the Smallest Cost", where a minimum (e.g., smallest) bit cost based on the first through seventeenth bit costs may be determined. For example, the estimated bits for all methods may be compared to determine the one with the smallest cost. Process 2500 may continue at operation 2511 "Code the Selected Method Index (See Table 2)", where a header code associated with the selected entropy coding technique (and table if applicable) may be encoded. For example, Table 2 as follows illustrated example header codes:
Method
ea 0 d d er e
C
0 1 d e e n
g
t h
Proxy VLC coder on inverted bits
with table V3 on default-scan ordered 0
blocks 0
0
Proxy VLC coder on inverted bits
with table V8 on default-scan ordered 0
blocks 0
1
Proxy VLC coder on inverted bits
with table V5 on default-scan ordered 0
blocks 1
0
Symbol-run on inverted bits with table
SO on default-scan ordered blocks 0
1
1
Symbol-run on inverted bits with table
S2 on default-scan ordered blocks 1
0
0
Proxy VLC coder on inverted bits
with table V3 on default-scan ordered 1
blocks 0
1
Proxy VLC coder on inverted bits
with table V5 on default-scan ordered 1 blocks 1
0
Symbol-run on inverted differential
bits with table S2 on default-scan ordered 1 blocks 1
1
Proxy VLC coder on inverted
differential bits with table V8 on default- 0 scan ordered blocks 0
0
Proxy VLC coder on inverted bits
with table V3 on raster-scan ordered blocks 0
0
1
Proxy VLC coder on inverted bits
with table V8 on raster-scan ordered blocks 0
1
0
Proxy VLC coder on inverted bits
with table V5 on raster-scan ordered blocks 0
1
1
Proxy VLC coder on inverted bits
with table V3 on raster-scan ordered blocks 1
0
0
Proxy VLC coder on inverted bits
with table V0 on raster-scan ordered blocks 1
0
1
Proxy VLC coder on inverted
differential bits with table V8 on raster-scan 1 ordered blocks 1
0
Proxy VLC coder on inverted bits
with table V10 on default-scan ordered 1 blocks 1
1
0
Proxy VLC coder on inverted bits
with table V10 on raster-scan ordered blocks 1
1
1
1
Table 2: Symbol-run/Proxy VLC methods in F-frames
Process 2500 may continue at operation 2512 "Code CNC Data with the Selected Method", where the processed bitstream associated with the coded/not-coded video data may be entropy encoded using the selected entropy coding technique to generate encoded coded/not-coded video data. Further, as discussed, the encoded coded/not-coded video data bitstream may be assembled into an output bitstream. For example, the selected method may be applied to the CNC data and the result may be written to bitstream. Process 2500 may terminate at operation 2514 "Finish Processing Current Frame". Process 2500 may be repeated as needed to encode any number of frames such as, for example, P-pictures or P- pictures.
As discussed, the entropy encoding technique(s) may use various tables. The tables as referenced herein are presented as follows and labeled in accordance with the previous discussion and, in particular the discussion presented with respect to FIGS. 24 and 25 and Tables 1 and 2 above.
Figure imgf000103_0001
000100 5 010000 2 0
0 1 0000 3 1
7
000011 6 010001 3 0
1 1 1000 E
7 6
000101 7 01 SC
0000 E
- - - 5
1 OB
Table 3 : Symbol-run VLC table SO
Figure imgf000104_0001
Figure imgf000105_0001
Table 4: Symbol-run VLC table S2
Figure imgf000105_0003
Table 5: Proxy VLC table V0
Figure imgf000105_0002
Table 6: Proxy VLC table VI
Figure imgf000106_0002
Table 7: Proxy VLC table V2
Figure imgf000106_0003
Table 8: Proxy VLC table V3
Figure imgf000106_0001
Figure imgf000107_0001
Table 9: Proxy VLC table V5
Figure imgf000107_0002
Table 10: Proxy VLC table V6
Figure imgf000107_0003
Table 11 : Proxy VLC table V8 Bit c
s LC ode
Sequence Length
00 1
01 2
0
10 3
10
11 3
1 1
Table 12: Proxy VLC table V10
As discussed with respect to FIGS. 24 and 25, techniques discussed herein may include estimating bit cost(s) of proxy variable length coding and symbol-run coding in various scenarios. In some examples, estimating bit cost(s) associated with proxy VLC may include a look up operation. In some examples, estimating bit cost(s) associated with symbol-run coding may implement the process as illustrated in FIG. 26.
FIG. 26 is a flow diagram illustrating an example process 2600, arranged in accordance with at least some implementations of the present disclosure. Process 2600 may include one or more operations, functions or actions as illustrated by one or more of operations 2302, 2304, 2306, and/or 230. Process 2600 may form at least part of a next generation video coding process. By way of non-limiting example, process 2600 may form at least part of a next generation video encoding process as undertaken by encoder system 100 of FIG. 1 and/or entropy decoder module 1 10 as discussed herein.
Process 2600 may begin at operation 2601 "Input Binary Mask A and Symbol-Run Table T", where a binary mask (e.g., any processed bitstream as discussed herein) and a symbol-run table (e.g., a symbol-run table as illustrated herein or the like) may be input. Process 2600 may continue at operation 2602 "set cost = c = 0", where a running cost (e.g., bit cost), labeled c, may be set to zero. Process 2600 may continue at operation 2603 "Set i = 0", where a counter variable i may be set to zero. Process 2600 may continue at decision operation 2604 "c>=size(A)", where it may be determined whether running cost c is greater than or equal to the size of binary mask A.
If running cost c is not greater than or equal to the size of binary mask A, process 2600 may continue at decision operation 2605 "A[c]==l?", where it may be determined whether binary mask A is 1 at the position of running cost c. If such evaluation is true, process 2600 may continue at operation 2606 "c = c + size(codeword[i] in T), running cost c may be increased by the size of a codeword associated with i in table T. Process 2600 may continue at operation 2612 "c = c + 1", where running cost c may be increased by one. Process 2600 may continue at operation 2613 "c==size(A)", where it may be determined whether running cost c is the same as the size of binary mask A. If so, process 2600 may continue at operation 261 1 "Return Cost as Estimated Cost of Symbol-Run Coding of A with Table T", where the cost as estimated with respect to symbol-run coding of binary mask A with Table may be returned as running cost c. If not, process 2600 may continue at operation 2603, as discussed above.
If operation 2605 was determined to be not true, process 2600 may continue at decision operation 2607 "i > 33 ?" where it may be determined whether counter i is greater than 33. If so, process 2600 may continue at operation 2608 "c = c + size (ESC in T)" where running cost c may be increased by the size of ESC (e.g., "escape", an indicator that a run as gone beyond an upper limit) in table T, process 2600 may continue at operation 2614 "c = c +1", where running cost may be increased by 1, and process 2600 may continue at operation 2603 as discussed. If not, process 2600 may continue at operation 2609 "c = c+1, i = i+1", where the running cost and the index value may be increased by one, and process 2600 may continue at decision operation 2604 as discussed above.
If, at decision operation 2604, running cost c is greater than or equal to the size of binary mask A, process 2600 may continue at operation 2610 "c = c + size(EOB in T)", where running cost c may be increased by the size of EOB (e.g., "end of block") in table T. Process 2600 may continue at operation 2611, where the cost as estimated with respect to symbol-run coding of binary mask A with Table may be returned as running cost c as discussed.
As discussed, process 2600 may provide bit cost(s) estimates associated with symbol-run coding techniques as discussed herein.
Both in P-pictures and F-pictures, for coding of coded/not-coded data for each of the combinations of operations or bitstream processing (e.g., none, pass-through, scanning, bit inversion, bit differencing, or the like), coding techniques (e.g., symbol-run or proxy VLC), and options within each technique (e.g., multiple tables as described herein), the best combination (e.g., minimum bit cost) that yields the most compression may be selected as the coding method (e.g., entropy coding technique) for the given video data (e.g., frame, picture, or slice, or the like). Such various techniques, options, and specific tables have been provided herein and may advantageously provide adaptive coding for coded/not-coded data
As discussed with respect to FIG. 16, example video coding system 1600 may include imaging device(s) 1601, video encoder 100 and/or a video encoder implemented via logic circuitry 1650 of processing unit(s) 1620, video decoder 200 and/or a video decoder implemented via logic circuitry 1650 of processing unit(s) 1620, an antenna 1602, one or more processor(s) 1603, one or more memory store(s) 2004, and/or a display device 2005.
In some examples, video encoder 100 implemented via logic circuitry 1650 may include an image buffer (e.g., via either processing unit(s) 1620 or memory store(s) 1604)) and a graphics processing unit (e.g., via processing unit(s) 1620). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include video encoder 100 as implemented via logic circuitry 1650 to embody the various modules as discussed with respect to FIG. 1 and FIGS. 4, 6 and A 16. For example, the graphics processing unit may include entropy encoder logic circuitry, and so on. The logic circuitry may be configured to perform the various operations as discussed herein. For example, the entropy encoder logic circuitry may be configured to determine a selected entropy coding technique for coded/not-coded video data from a plurality of entropy coding techniques such that the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, entropy encode a processed bitstream associated with the coded/not-coded video data using the selected entropy coding technique to generate encoded coded/not-coded video data, and assemble the encoded coded/not-coded video data into an output bitstream. Video decoder 200 may be implemented in a similar manner.
In some examples, antenna 1602 of video coding system 1600 may be configured to receive an entropy encoded bitstream of video data. As discussed, the bitstream may include compressed video data of various types. Video coding system 1600 may also include video decoder 200 coupled to antenna 1602 and configured to decode the encoded bitstream. For example, video decoder 200 may be configured to disassemble the entropy encoded bitstream to determine a coded/not-coded video data header and encoded coded/not-coded video data, determine a selected entropy coding technique for the encoded coded/not-coded video data from a plurality of entropy coding techniques based at least in part on the coded/not-coded video data header such that the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, entropy decode the encoded coded/not-coded video data using the selected entropy coding technique to determine coded/not-coded video data, and generate a video frame based at least in part on the coded/not-coded video data. The video frame may be configured to be presented to a user via a display device, for example.
As discussed, video data of different types may be entropy coded to generate compressed (e.g.,. coded) video data. The compressed video data may be assembled to generate an output bitstream. With reference to FIG. 8, example bitstream 800 may include indicator data 801, compressed video data 421, compressed video data 422, compressed video data 423, compressed video data 424, compressed video data 425, compressed video data 426, and/or compressed video data 427. For example, compressed video data 424 may include coded coded/not-coded video data, or the like. Further, indicator data 801 and/or compressed video data 424 may include a coded/not-coded video data header associated with the selected entropy coding technique or the processed bitstream (e.g. the coded/not- coded video data header may indicate a selected entropy coding technique or a processed bitstream type— a pass-through of the coded/not-coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, a 1 -dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block-based scan based on the coded/not- coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data, or the like).
While certain features set forth herein have been described with reference to various implementations, this description is not intended to be construed in a limiting sense.
Hence, various modifications of the implementations described herein, as well as other implementations, which are apparent to persons skilled in the art to which the present disclosure pertains are deemed to lie within the spirit and scope of the present disclosure.
The following examples pertain to further embodiments.
In one example, a computer-implemented method for video coding may include determining a selected entropy coding technique for coded/not-coded video data from a plurality of entropy coding techniques such that the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, entropy encoding a processed bitstream associated with the coded/not-coded video data - I l l - using the selected entropy coding technique to generate encoded coded/not-coded video data, assembling the encoded coded/not-coded video data into an output bitstream.
In another example, a computer-implemented method for video coding may further include reading a block structure associated with a frame of video data, associating the block structure and the coded/not-coded video data, encoding a coded/not-coded video data header associated with at least one of the selected entropy coding technique or the processed bitstream, transmitting the output bitstream, receiving the output bitstream, disassembling the output bitstream to determine the coded/not-coded video data header and the encoded coded/not-coded video data, determining the selected entropy coding technique based at least in part on the coded/not-coded video data header, entropy decoding the encoded coded/not-coded video data bitstream using the selected entropy coding technique to determine the coded/not-coded video data, and generating a video frame based at least in part on the coded/not-coded video data. The coded/not-coded video data may include a plurality of bits such that each bit may indicate whether an associated block of the block structure associated with the frame of video data is coded or not-coded, such that a bit of 1 may indicate the associated block is coded and a bit of 0 indicates the associated block is not-coded, such that the coded/not-coded video data ma include symbol bits, and such that a mode associated with the frame of video data may be an inter mode. The processed bitstream associated with the coded/not-coded video data may include at least one of a pass-through of the coded/not-coded video data, a reverse of the coded/not- coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, a 1 -dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block-based scan based on the coded/not-coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data. Assembling the encoded coded/not-coded video into the output bitstream may include assembling the coded/not-coded video data header into the output bitstream. The frame of video data may be a P-picture and determining the selected entropy coding technique may include determining a number of inter-blocks in the frame of video data is greater than a threshold such that the threshold may be 25, generating a 1 -dimensional raster scan based on the coded/not-coded video data, generating a 1 -dimensional block-based scan based on the coded/not-coded video data such that the 1 -dimensional block-based scan may include a 2x4 scale scan, bit inverting the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan, bit inverting the 1 -dimensional block-based scan to generate an inverted bits 1 -dimensional block-based scan, estimating a first bit cost associated with the symbol run coding technique based on the inverted bits 1 -dimensional raster scan, estimating a second bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan, estimating a third bit cost associated with a second proxy variable length coding technique based on the inverted bits 1- dimensional raster scan, estimating a fourth bit cost associated with a third proxy variable length coding technique based on the 1 -dimensional raster scan, estimating a fifth bit cost associated with a fourth proxy variable length coding technique based on the 1- dimensional raster scan, estimating a sixth bit cost associated with a second symbol run coding technique based on the inverted bits 1 -dimensional block-based scan, determining a minimum bit cost based on the first, second, third, fourth, fifth, and sixth bit costs, and determining the selected entropy coding technique as a coding technique associated with the minimum bit cost such that the processed bitstream may include at least one of the inverted bits 1 -dimensional raster scan, the 1 -dimensional raster scan, or the inverted bits 1 -dimensional block-based scan, such that the symbol run coding technique may be associated with a first symbol-run variable length code table and the second symbol run coding technique may be associated with a second symbol-run variable length code table, and such that the proxy variable length coding technique may be associated with a first proxy variable length code table and the second proxy variable length coding technique may be associated with a second proxy variable length code table. The frame of video data may be an F-picture and determining the selected entropy coding technique may include determining a number of inter-blocks in the frame of video data is greater than a threshold such that the threshold may be 25, generating a 1 -dimensional tile-based scan based on the coded/not-coded video data, generating a 1 -dimensional raster scan based on the coded/not-coded video data, bit inverting the 1 -dimensional tile-based scan to generate an inverted bits 1 -dimensional tile-based scan, bit inverting the 1 -dimensional raster scan to generate an inverted bits 1 -dimensional raster scan, bit differencing the inverted bits 1- dimensional tile-based scan to generate an inverted differential bits 1 -dimensional tile- based scan, bit differencing the inverted bits 1 -dimensional raster scan to generate an inverted differential bits 1 -dimensional raster scan, estimating a first bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional tile-based scan, estimating a second bit cost associated with the symbol-run coding technique based on the 1 -dimensional tile-based scan, estimating a third bit cost associated with a second proxy variable length coding technique based on the 1 -dimensional tile- based scan, estimating a fourth bit cost associated with a second symbol-run coding technique based on the inverted differential bits 1 -dimensional tile-based scan, estimating a fifth bit cost associated with a third proxy variable length coding technique based on the inverted differential bits 1 -dimensional tile-based scan, estimating a sixth bit cost associated with a fourth proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan, estimating a seventh bit cost associated with a fifth proxy variable length coding technique based on the 1 -dimensional raster scan, estimating an eighth bit cost associated with a sixth proxy variable length coding technique based on the inverted differential bits 1 -dimensional raster scan, determining a minimum bit cost based on the first, second, third, fourth, fifth, sixth, seventh, and eighth bit costs, and determining the selected entropy coding technique as a coding technique associated with the minimum bit cost such that the processed bitstream may include at least one of the 1 -dimensional tile-based scan, the 1 -dimensional raster scan, the inverted bits 1 -dimensional tile-based scan, the inverted bits 1 -dimensional raster scan, the inverted differential bits 1 - dimensional tile-based scan, or the inverted differential bits 1 -dimensional raster scan, such that the first symbol run coding technique may be associated with a first symbol-run variable length code table and the second symbol run coding technique may be associated with a second symbol-run variable length code table, and such that the first proxy variable length coding technique may be associated with a first proxy variable length code table and the second proxy variable length coding technique may be associated with a second proxy variable length code table.
In other examples, a video encoder may include an image buffer and a graphics processing unit having entropy encoder logic circuitry. The graphics processing unit may be communicatively coupled to the image buffer and the entropy encoder logic circuitry may be configured to determine a selected entropy coding technique for coded/not-coded video data from a plurality of entropy coding techniques such that the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, entropy encode a processed bitstream associated with the coded/not- coded video data using the selected entropy coding technique to generate encoded coded/not-coded video data, and assemble the encoded coded/not-coded video data into an output bitstream.
In a further example video encoder, the entropy encoder logic circuitry may be further configured to read a block structure associated with a frame of video data, associate the block structure and the coded/not-coded video data, and encode a coded/not-coded video data header associated with at least one of the selected entropy coding technique or the processed bitstream. The coded/not-coded video data may include a plurality of bits such that each bit may indicate whether an associated block of the block structure associated with the frame of video data is coded or not-coded, such that a bit of 1 may indicate the associated block is coded and a bit of 0 indicates the associated block is not- coded, such that the coded/not-coded video data ma include symbol bits, and such that a mode associated with the frame of video data may be an inter mode. The processed bitstream associated with the coded/not-coded video data may include at least one of a pass-through of the coded/not-coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not- coded video data, a 1 -dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block-based scan based on the coded/not-coded video data, or a 1- dimensional tile-based scan based on the coded/not-coded video data. To assemble the encoded coded/not-coded video into the output bitstream may include the entropy encoder logic circuitry being configured to assemble the coded/not-coded video data header into the output bitstream. The frame of video data may be a P-picture and to determine the selected entropy coding technique may include the entropy encoder logic circuitry being configured to determine a number of inter-blocks in the frame of video data is greater than a threshold such that the threshold may be 25, generate a 1 -dimensional raster scan based on the coded/not-coded video data, generate a 1 -dimensional block-based scan based on the coded/not-coded video data such that the 1 -dimensional block-based scan may include a 2x4 scale scan, bit invert the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan, bit invert the 1 -dimensional block-based scan to generate an inverted bits 1 -dimensional block-based scan, estimate a first bit cost associated with the symbol run coding technique based on the inverted bits 1 -dimensional raster scan, estimate a second bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan, estimate a third bit cost associated with a second proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan, estimate a fourth bit cost associated with a third proxy variable length coding technique based on the 1 -dimensional raster scan, estimate a fifth bit cost associated with a fourth proxy variable length coding technique based on the 1 -dimensional raster scan, estimate a sixth bit cost associated with a second symbol run coding technique based on the inverted bits 1 -dimensional block-based scan, determine a minimum bit cost based on the first, second, third, fourth, fifth, and sixth bit costs, and determine the selected entropy coding technique as a coding technique associated with the minimum bit cost such that the processed bitstream may include at least one of the inverted bits 1 -dimensional raster scan, the 1 -dimensional raster scan, or the inverted bits 1 -dimensional block-based scan, such that the symbol run coding technique may be associated with a first symbol-run variable length code table and the second symbol run coding technique may be associated with a second symbol-run variable length code table, and such that the proxy variable length coding technique may be associated with a first proxy variable length code table and the second proxy variable length coding technique may be associated with a second proxy variable length code table. The frame of video data may be an F-picture and to determine the selected entropy coding technique may include the entropy encoder logic circuitry being configured to determine a number of inter-blocks in the frame of video data is greater than a threshold such that the threshold may be 25, generate a 1 -dimensional tile- based scan based on the coded/not-coded video data, generate a 1 -dimensional raster scan based on the coded/not-coded video data, bit invert the 1 -dimensional tile-based scan to generate an inverted bits 1 -dimensional tile-based scan, bit invert the 1 -dimensional raster scan to generate an inverted bits 1 -dimensional raster scan, bit difference the inverted bits 1 -dimensional tile-based scan to generate an inverted differential bits 1 -dimensional tile- based scan, bit difference the inverted bits 1 -dimensional raster scan to generate an inverted differential bits 1 -dimensional raster scan, estimate a first bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional tile- based scan, estimate a second bit cost associated with the symbol-run coding technique based on the 1 -dimensional tile-based scan, estimate a third bit cost associated with a second proxy variable length coding technique based on the 1 -dimensional tile-based scan, estimate a fourth bit cost associated with a second symbol-run coding technique based on the inverted differential bits 1 -dimensional tile-based scan, estimate a fifth bit cost associated with a third proxy variable length coding technique based on the inverted differential bits 1 -dimensional tile-based scan, estimate a sixth bit cost associated with a fourth proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan, estimate a seventh bit cost associated with a fifth proxy variable length coding technique based on the 1 -dimensional raster scan, estimate an eighth bit cost associated with a sixth proxy variable length coding technique based on the inverted differential bits 1 -dimensional raster scan, determine a minimum bit cost based on the first, second, third, fourth, fifth, sixth, seventh, and eighth bit costs, and determine the selected entropy coding technique as a coding technique associated with the minimum bit cost such that the processed bitstream may include at least one of the 1 -dimensional tile-based scan, the 1- dimensional raster scan, the inverted bits 1 -dimensional tile-based scan, the inverted bits 1- dimensional raster scan, the inverted differential bits 1 -dimensional tile-based scan, or the inverted differential bits 1 -dimensional raster scan, such that the first symbol run coding technique may be associated with a first symbol-run variable length code table and the second symbol run coding technique may be associated with a second symbol-run variable length code table, and such that the first proxy variable length coding technique may be associated with a first proxy variable length code table and the second proxy variable length coding technique may be associated with a second proxy variable length code table.
In yet another example, a system may include an antenna configured to receive an entropy encoded bitstream of video data and a video decoder communicatively coupled to the antenna and configured to decode the encoded bitstream. The video decoder may be configured to disassemble the entropy encoded bitstream to determine a coded/not-coded video data header and encoded coded/not-coded video data, determine a selected entropy coding technique for the encoded coded/not-coded video data from a plurality of entropy coding techniques based at least in part on the coded/not-coded video data header such that the plurality of entropy coding techniques may include a proxy variable length coding technique and a symbol-run coding technique, entropy decode the encoded coded/not- coded video data using the selected entropy coding technique to determine coded/not- coded video data, and generate a video frame based at least in part on the coded/not-coded video data.
In a further example system, the system may also include a display device configured to present video frames. The coded/not-coded video data may include a plurality of bits such that each bit may indicate whether an associated block of the block structure associated with a frame of video data is coded or not-coded and such that a bit of 1 may indicate the associated block is coded and a bit of 0 may indicate the associated block is not-coded. To entropy decode the encoded coded/not-coded video data using the selected entropy coding technique to determine coded/not-coded video data may include the video decoder being configured to decompress the encoded coded/not-coded video data to generate a processed bitstream and decode the processed bitstream to generate the coded/not-coded video data such that the processed bitstream associated with the coded/not-coded video data may include at least one of a pass-through of the coded/not- coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, a 1- dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block- based scan based on the coded/not-coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data. In a further example, at least one machine readable medium may include a plurality of instructions that in response to being executed on a computing device, causes the computing device to perform the method according to any one of the above examples.
In a still further example, an apparatus may include means for performing the methods according to any one of the above examples.
The above examples may include specific combination of features. However, such the above examples are not limited in this regard and, in various implementations, the above examples may include the undertaking only a subset of such features, undertaking a different order of such features, undertaking a different combination of such features, and/or undertaking additional features than those features explicitly listed. For example, all features described with respect to the example methods may be implemented with respect to the example apparatus, the example systems, and/or the example articles, and vice versa.

Claims

WHAT IS CLAIMED:
1. A computer-implemented method for video coding, comprising:
determining a selected entropy coding technique for coded/not-coded video data from a plurality of entropy coding techniques, wherein the plurality of entropy coding techniques comprise a proxy variable length coding technique and a symbol-run coding technique;
entropy encoding a processed bitstream associated with the coded/not-coded video data using the selected entropy coding technique to generate encoded coded/not- coded video data; and
assembling the encoded coded/not-coded video data into an output bitstream.
2. The method of claim 1, wherein the processed bitstream associated with the coded/not-coded video data comprises at least one of a pass-through of the coded/not- coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, or a bit difference of the coded/not-coded video data.
3. The method of claim 1, wherein the processed bitstream associated with the coded/not-coded video data comprises at least one of a 1 -dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block-based scan based on the coded/not- coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data.
4. The method of claim 1, wherein the proxy variable length coding technique is based on a first proxy variable length coding table and wherein the plurality of entropy coding techniques comprise a second proxy variable length coding technique based on a second proxy variable length coding table.
5. The method of claim 1, wherein the coded/not-coded data is associated with a P-picture of video data, and wherein determining the selected entropy coding technique comprises: generating a 1 -dimensional raster scan based on the coded/not-coded video data;
generating a 1 -dimensional block-based scan based on the coded/not-coded video data;
bit inverting the 1 -dimensional block-based scan to generate an inverted bits 1- dimensional block-based scan;
estimating a first bit cost associated with the proxy variable length coding technique based on the 1 -dimensional raster scan;
estimating a second bit cost associated with the symbol run coding technique based on the inverted bits 1 -dimensional block-based scan;
determining a minimum bit cost based on the first and second bit costs; and determining the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the 1 -dimensional raster scan or the inverted bits 1 -dimensional block-based scan.
6. The method of claim 1, wherein the coded/not-coded data is associated with a
P-picture of video data, and wherein determining the selected entropy coding technique comprises:
generating a 1 -dimensional raster scan based on the coded/not-coded video data;
generating a 1 -dimensional block-based scan based on the coded/not-coded video data;
bit inverting the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan;
bit inverting the 1 -dimensional block-based scan to generate an inverted bits 1- dimensional block-based scan; estimating a first bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan;
estimating a second bit cost associated with the symbol run coding technique based on the inverted bits 1 -dimensional block-based scan;
determining a minimum bit cost based on the first and second bit costs; and determining the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the inverted bits 1 -dimensional raster scan or the inverted bits 1 -dimensional block- based scan.
7. The method of claim 1, wherein the coded/not-coded data is associated with a
P-picture of video data, and wherein determining the selected entropy coding technique comprises:
generating a 1 -dimensional raster scan based on the coded/not-coded video data;
generating a 1 -dimensional block-based scan based on the coded/not-coded video data;
bit inverting the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan;
bit inverting the 1 -dimensional block-based scan to generate an inverted bits 1- dimensional block-based scan;
estimating a first bit cost associated with the symbol run coding technique based on the inverted bits 1 -dimensional raster scan;
estimating a second bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan;
estimating a third bit cost associated with a second symbol run coding technique based on the inverted bits 1 -dimensional block-based scan; determining a minimum bit cost based on the first, second, and third bit costs; and
determining the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the inverted bits 1 -dimensional raster scan or the inverted bits 1 -dimensional block- based scan.
8. The method of claim 1, wherein the coded/not-coded data is associated with a P-picture of video data, and wherein determining the selected entropy coding technique comprises:
generating a 1 -dimensional raster scan based on the coded/not-coded video data;
generating a 1 -dimensional block-based scan based on the coded/not-coded video data;
bit inverting the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan;
bit inverting the 1 -dimensional block-based scan to generate an inverted bits 1- dimensional block-based scan;
estimating a first bit cost associated with the symbol run coding technique based on the inverted bits 1 -dimensional raster scan;
estimating a second bit cost associated with the proxy variable length coding technique based on the 1 -dimensional raster scan;
estimating a third bit cost associated with a second symbol run coding technique based on the inverted bits 1 -dimensional block-based scan;
determining a minimum bit cost based on the first, second, and third bit costs; and determining the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the 1 -dimensional raster scan, the inverted bits 1 -dimensional raster scan, or the inverted bits 1 -dimensional block-based scan.
9. The method of claim 1, wherein the coded/not-coded data is associated with an
F-picture of video data, and wherein determining the selected entropy coding technique comprises:
generating a 1 -dimensional tile-based scan based on the coded/not-coded video data;
generating a 1 -dimensional raster scan based on the coded/not-coded video data;
estimating a first bit cost associated with the symbol-run coding technique based on the 1 -dimensional tile-based scan;
estimating a second bit cost associated with the proxy variable length coding technique based on the 1 -dimensional raster scan;
determining a minimum bit cost based on the first and second bit costs; and determining the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the 1 -dimensional tile-based scan or the 1 -dimensional raster scan.
10. The method of claim 1, wherein the coded/not-coded data is associated with an
F-picture of video data, and wherein determining the selected entropy coding technique comprises:
generating a 1 -dimensional tile-based scan based on the coded/not-coded video data;
generating a 1 -dimensional raster scan based on the coded/not-coded video data; bit inverting the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan;
estimating a first bit cost associated with the symbol-run coding technique based on the 1 -dimensional tile-based scan;
estimating a second bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan;
determining a minimum bit cost based on the first and second bit costs; and determining the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the 1 -dimensional tile-based scan or the inverted bits 1 -dimensional raster scan.
1 1. The method of claim 1, wherein the coded/not-coded data is associated with an F-picture of video data, and wherein determining the selected entropy coding technique comprises:
generating a 1 -dimensional tile-based scan based on the coded/not-coded video data;
generating a 1 -dimensional raster scan based on the coded/not-coded video data;
bit inverting the 1 -dimensional tile-based scan to generate an inverted bits 1- dimensional tile-based scan;
bit inverting the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan;
bit differencing the inverted bits 1 -dimensional tile-based scan to generate an inverted differential bits 1 -dimensional tile-based scan;
bit differencing the inverted bits 1 -dimensional raster scan to generate an inverted differential bits 1 -dimensional raster scan; estimating a first bit cost associated with the symbol-run coding technique based on the inverted differential bits 1 -dimensional tile-based scan;
estimating a second bit cost associated with the proxy variable length coding technique based on the inverted differential bits 1 -dimensional raster scan;
estimating a third bit cost associated with a second proxy variable length coding technique based on the inverted bits 1 -dimensional tile-based scan;
determining a minimum bit cost based on the first, second, and third bit costs; and
determining the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the inverted differential bits 1 -dimensional tile-based scan, the inverted differential bits 1 -dimensional raster scan, or the inverted bits 1 -dimensional tile-based scan.
12. The method of claim 1, wherein the coded/not-coded data is associated with an F-picture of video data, and wherein determining the selected entropy coding technique comprises:
generating a 1 -dimensional tile-based scan based on the coded/not-coded video data;
generating a 1 -dimensional raster scan based on the coded/not-coded video data;
bit inverting the 1 -dimensional tile-based scan to generate an inverted bits 1- dimensional tile-based scan;
bit inverting the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan;
bit differencing the inverted bits 1 -dimensional tile-based scan to generate an inverted differential bits 1 -dimensional tile-based scan; bit differencing the inverted bits 1 -dimensional raster scan to generate an inverted differential bits 1 -dimensional raster scan;
estimating a first bit cost associated with the symbol-run coding technique based on the inverted differential bits 1 -dimensional tile-based scan;
estimating a second bit cost associated with the proxy variable length coding technique based on the inverted differential bits 1 -dimensional raster scan;
estimating a third bit cost associated with a second proxy variable length coding technique based on the inverted differential bits 1 -dimensional tile-based scan;
determining a minimum bit cost based on the first, second, and third bit costs; and
determining the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the inverted differential bits 1 -dimensional tile-based scan or the inverted differential bits 1 -dimensional raster scan.
13. The method of claim 1, wherein the coded/not-coded video data comprises a plurality of bits, wherein each bit indicates whether an associated block of a block structure associated with a frame of video data is coded or not-coded.
14. The method of claim 1, further comprising:
encoding a coded/not-coded video data header associated with at least one of the selected entropy coding technique or the processed bitstream.
15. The method of claim 1, further comprising:
reading a block structure associated with a frame of video data;
associating the block structure and the coded/not-coded video data; encoding a coded/not-coded video data header associated with at least one of the selected entropy coding technique or the processed bitstream;
transmitting the output bitstream; receiving the output bitstream;
disassembling the output bitstream to determine the coded/not-coded video data header and the encoded coded/not-coded video data;
determining the selected entropy coding technique based at least in part on the coded/not-coded video data header;
entropy decoding the encoded coded/not-coded video data bitstream using the selected entropy coding technique to determine the coded/not-coded video data; and
generating a video frame based at least in part on the coded/not-coded video data,
wherein the coded/not-coded video data comprises a plurality of bits, wherein each bit indicates whether an associated block of the block structure associated with the frame of video data is coded or not-coded, wherein a bit of 1 indicates the associated block is coded and a bit of 0 indicates the associated block is not-coded, wherein the coded/not- coded video data comprises symbol bits, and wherein a mode associated with the frame of video data is an inter mode,
wherein the processed bitstream associated with the coded/not-coded video data comprises at least one of a pass-through of the coded/not-coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, a 1 -dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block-based scan based on the coded/not- coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data,
wherein assembling the encoded coded/not-coded video into the output bitstream comprises assembling the coded/not-coded video data header into the output bitstream, wherein the frame of video data is a P-picture, and wherein determining the selected entropy coding technique comprises:
determining a number of inter-blocks in the frame of video data is greater than a threshold, wherein the threshold comprises 25;
generating a 1 -dimensional raster scan based on the coded/not-coded video data;
generating a 1 -dimensional block-based scan based on the coded/not-coded video data, wherein the 1 -dimensional block-based scan comprises a 2x4 scale scan;
bit inverting the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan;
bit inverting the 1 -dimensional block-based scan to generate an inverted bits 1- dimensional block-based scan;
estimating a first bit cost associated with the symbol run coding technique based on the inverted bits 1 -dimensional raster scan;
estimating a second bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan;
estimating a third bit cost associated with a second proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan;
estimating a fourth bit cost associated with a third proxy variable length coding technique based on the 1 -dimensional raster scan;
estimating a fifth bit cost associated with a fourth proxy variable length coding technique based on the 1 -dimensional raster scan;
estimating a sixth bit cost associated with a second symbol run coding technique based on the inverted bits 1 -dimensional block-based scan;
determining a minimum bit cost based on the first, second, third, fourth, fifth, and sixth bit costs; and determining the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the inverted bits 1 -dimensional raster scan, the 1 -dimensional raster scan, or the inverted bits 1 -dimensional block-based scan, wherein the symbol run coding technique is associated with a first symbol-run variable length code table and the second symbol run coding technique is associated with a second symbol-run variable length code table, and wherein the proxy variable length coding technique is associated with a first proxy variable length code table and the second proxy variable length coding technique is associated with a second proxy variable length code table,
wherein the frame of video data is an F-picture, and wherein determining the selected entropy coding technique comprises:
determining a number of inter-blocks in the frame of video data is greater than a threshold, wherein the threshold comprises 25;
generating a 1 -dimensional tile-based scan based on the coded/not-coded video data;
generating a 1 -dimensional raster scan based on the coded/not-coded video data;
bit inverting the 1 -dimensional tile-based scan to generate an inverted bits 1- dimensional tile-based scan;
bit inverting the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan;
bit differencing the inverted bits 1 -dimensional tile-based scan to generate an inverted differential bits 1 -dimensional tile-based scan;
bit differencing the inverted bits 1 -dimensional raster scan to generate an inverted differential bits 1 -dimensional raster scan; estimating a first bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional tile-based scan;
estimating a second bit cost associated with the symbol-run coding technique based on the 1 -dimensional tile-based scan;
estimating a third bit cost associated with a second proxy variable length coding technique based on the 1 -dimensional tile-based scan;
estimating a fourth bit cost associated with a second symbol-run coding technique based on the inverted differential bits 1 -dimensional tile-based scan;
estimating a fifth bit cost associated with a third proxy variable length coding technique based on the inverted differential bits 1 -dimensional tile-based scan;
estimating a sixth bit cost associated with a fourth proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan;
estimating a seventh bit cost associated with a fifth proxy variable length coding technique based on the 1 -dimensional raster scan;
estimating an eighth bit cost associated with a sixth proxy variable length coding technique based on the inverted differential bits 1 -dimensional raster scan;
determining a minimum bit cost based on the first, second, third, fourth, fifth, sixth, seventh, and eighth bit costs; and
determining the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the 1 -dimensional tile-based scan, the 1 -dimensional raster scan, the inverted bits 1- dimensional tile-based scan, the inverted bits 1 -dimensional raster scan, the inverted differential bits 1 -dimensional tile-based scan, or the inverted differential bits 1- dimensional raster scan, wherein the first symbol run coding technique is associated with a first symbol-run variable length code table and the second symbol run coding technique is associated with a second symbol-run variable length code table, and wherein the first proxy variable length coding technique is associated with a first proxy variable length code table and the second proxy variable length coding technique is associated with a second proxy variable length code table.
16. A video encoder comprising:
an image buffer;
a graphics processing unit comprising entropy encoder logic circuitry, wherein the graphics processing unit is communicatively coupled to the image buffer and wherein the entropy encoder logic circuitry is configured to:
determine a selected entropy coding technique for coded/not-coded video data from a plurality of entropy coding techniques, wherein the plurality of entropy coding techniques comprise a proxy variable length coding technique and a symbol-run coding technique;
entropy encode a processed bitstream associated with the coded/not-coded video data using the selected entropy coding technique to generate encoded coded/not- coded video data; and
assemble the encoded coded/not-coded video data into an output bitstream.
17. The video encoder of claim 16, wherein the plurality of entropy coding techniques comprise a second proxy variable length coding technique, wherein the proxy variable length coding technique is associated with a first proxy variable length code table and the second proxy variable length coding technique is associated with a second proxy variable length code table.
18. The video encoder of claim 16, wherein the plurality of entropy coding techniques comprise a second symbol-run coding technique, wherein the symbol run coding technique is associated with a first symbol-run variable length code table and the second symbol run coding technique is associated with a second symbol-run variable length code table.
19. The video encoder of claim 16, wherein the processed bitstream associated with the coded/not-coded video data comprises at least one of a pass-through of the coded/not-coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, a 1- dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block- based scan based on the coded/not-coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data.
20. The video encoder of claim 16, wherein the entropy encoder logic circuitry is further configured to:
read a block structure associated with a frame of video data;
associate the block structure and the coded/not-coded video data; and encode a coded/not-coded video data header associated with at least one of the selected entropy coding technique or the processed bitstream;
wherein the coded/not-coded video data comprises a plurality of bits, wherein each bit indicates whether an associated block of the block structure associated with the frame of video data is coded or not-coded, wherein a bit of 1 indicates the associated block is coded and a bit of 0 indicates the associated block is not-coded, wherein the coded/not- coded video data comprises symbol bits, and wherein a mode associated with the frame of video data is an inter mode,
wherein the processed bitstream associated with the coded/not-coded video data comprises at least one of a pass-through of the coded/not-coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, a 1 -dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block-based scan based on the coded/not- coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data, wherein to assemble the encoded coded/not-coded video into the output bitstream comprises the entropy encoder logic circuitry being configured to assemble the coded/not-coded video data header into the output bitstream,
wherein the frame of video data is a P-picture, and wherein to determine the selected entropy coding technique comprises the entropy encoder logic circuitry being configured to:
determine a number of inter-blocks in the frame of video data is greater than a threshold, wherein the threshold comprises 25;
generate a 1 -dimensional raster scan based on the coded/not-coded video data; generate a 1 -dimensional block-based scan based on the coded/not-coded video data, wherein the 1 -dimensional block-based scan comprises a 2x4 scale scan;
bit invert the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan;
bit invert the 1 -dimensional block-based scan to generate an inverted bits 1- dimensional block-based scan;
estimate a first bit cost associated with the symbol run coding technique based on the inverted bits 1 -dimensional raster scan;
estimate a second bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan;
estimate a third bit cost associated with a second proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan;
estimate a fourth bit cost associated with a third proxy variable length coding technique based on the 1 -dimensional raster scan;
estimate a fifth bit cost associated with a fourth proxy variable length coding technique based on the 1 -dimensional raster scan; estimate a sixth bit cost associated with a second symbol run coding technique based on the inverted bits 1 -dimensional block-based scan;
determine a minimum bit cost based on the first, second, third, fourth, fifth, and sixth bit costs; and
determine the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the inverted bits 1 -dimensional raster scan, the 1 -dimensional raster scan, or the inverted bits 1 -dimensional block-based scan, wherein the symbol run coding technique is associated with a first symbol-run variable length code table and the second symbol run coding technique is associated with a second symbol-run variable length code table, and wherein the proxy variable length coding technique is associated with a first proxy variable length code table and the second proxy variable length coding technique is associated with a second proxy variable length code table,
wherein the frame of video data is an F-picture, and wherein to determine the selected entropy coding technique comprises the entropy encoder logic circuitry being configured to:
determine a number of inter-blocks in the frame of video data is greater than a threshold, wherein the threshold comprises 25;
generate a 1 -dimensional tile-based scan based on the coded/not-coded video data;
generate a 1 -dimensional raster scan based on the coded/not-coded video data; bit invert the 1 -dimensional tile-based scan to generate an inverted bits 1- dimensional tile-based scan;
bit invert the 1 -dimensional raster scan to generate an inverted bits 1- dimensional raster scan; bit difference the inverted bits 1 -dimensional tile-based scan to generate an inverted differential bits 1 -dimensional tile-based scan;
bit difference the inverted bits 1 -dimensional raster scan to generate an inverted differential bits 1 -dimensional raster scan;
estimate a first bit cost associated with the proxy variable length coding technique based on the inverted bits 1 -dimensional tile-based scan;
estimate a second bit cost associated with the symbol-run coding technique based on the 1 -dimensional tile-based scan;
estimate a third bit cost associated with a second proxy variable length coding technique based on the 1 -dimensional tile-based scan;
estimate a fourth bit cost associated with a second symbol-run coding technique based on the inverted differential bits 1 -dimensional tile-based scan;
estimate a fifth bit cost associated with a third proxy variable length coding technique based on the inverted differential bits 1 -dimensional tile-based scan;
estimate a sixth bit cost associated with a fourth proxy variable length coding technique based on the inverted bits 1 -dimensional raster scan;
estimate a seventh bit cost associated with a fifth proxy variable length coding technique based on the 1 -dimensional raster scan;
estimate an eighth bit cost associated with a sixth proxy variable length coding technique based on the inverted differential bits 1 -dimensional raster scan;
determine a minimum bit cost based on the first, second, third, fourth, fifth, sixth, seventh, and eighth bit costs; and
determine the selected entropy coding technique as a coding technique associated with the minimum bit cost, wherein the processed bitstream comprises at least one of the 1 -dimensional tile-based scan, the 1 -dimensional raster scan, the inverted bits 1- dimensional tile-based scan, the inverted bits 1 -dimensional raster scan, the inverted differential bits 1 -dimensional tile-based scan, or the inverted differential bits 1- dimensional raster scan, wherein the symbol run coding technique is associated with a first symbol-run variable length code table and the second symbol run coding technique is associated with a second symbol-run variable length code table, and wherein the proxy variable length coding technique is associated with a first proxy variable length code table and the second proxy variable length coding technique is associated with a second proxy variable length code table.
21. A system comprising:
an antenna configured to receive an entropy encoded bitstream of video data; and
a video decoder communicatively coupled to the antenna and configured to decode the encoded bitstream, wherein the video decoder is configured to:
disassemble the entropy encoded bitstream to determine a coded/not-coded video data header and encoded coded/not-coded video data;
determine a selected entropy coding technique for the encoded coded/not- coded video data from a plurality of entropy coding techniques based at least in part on the coded/not-coded video data header, wherein the plurality of entropy coding techniques comprise a proxy variable length coding technique and a symbol-run coding technique;
entropy decode the encoded coded/not-coded video data using the selected entropy coding technique to determine coded/not-coded video data; and
generate a video frame based at least in part on the coded/not-coded video data.
22. The system of claim 21 , further comprising:
a display device configured to present the video frame.
23. The system of claim 21, further comprising:
a display device configured to present the video frame, wherein the coded/not-coded video data comprises a plurality of bits, wherein each bit indicates whether an associated block of the block structure associated with a frame of video data is coded or not-coded, and wherein a bit of 1 indicates the associated block is coded and a bit of 0 indicates the associated block is not-coded,
wherein to entropy decode the encoded coded/not-coded video data using the selected entropy coding technique to determine coded/not-coded video data comprises the video decoder is configured to:
decompress the encoded coded/not-coded video data to generate a processed bitstream; and
decode the processed bitstream to generate the coded/not-coded video data, wherein the processed bitstream associated with the coded/not-coded video data comprises at least one of a pass-through of the coded/not-coded video data, a reverse of the coded/not-coded video data, a bit inversion of the coded/not-coded video data, a bit difference of the coded/not-coded video data, a 1 -dimensional raster scan based on the coded/not-coded video data, a 1 -dimensional block-based scan based on the coded/not- coded video data, or a 1 -dimensional tile-based scan based on the coded/not-coded video data.
24. At least one machine readable medium comprising:
a plurality of instructions that in response to being executed on a computing device, causes the computing device to perform the method according to any one of claims 1-15.
25. An apparatus, comprising:
means for performing the methods according to any one of claims 1-15.
PCT/US2013/077967 2013-01-30 2013-12-27 Content adaptive entropy coding of coded/not-coded data for next generation video WO2014120373A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US14/375,690 US9794568B2 (en) 2013-01-30 2013-12-27 Content adaptive entropy coding of coded/not-coded data for next generation video
EP13873217.7A EP2952001A4 (en) 2013-01-30 2013-12-27 Content adaptive entropy coding of coded/not-coded data for next generation video
KR1020157016934A KR20150090178A (en) 2013-01-30 2013-12-27 Content adaptive entropy coding of coded/not-coded data for next generation video
CN201380068583.5A CN104885471B (en) 2013-01-30 2013-12-27 Content adaptive entropy coding of coded/uncoded data for next generation video

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201361758314P 2013-01-30 2013-01-30
US61/758,314 2013-01-30

Publications (1)

Publication Number Publication Date
WO2014120373A1 true WO2014120373A1 (en) 2014-08-07

Family

ID=51262831

Family Applications (11)

Application Number Title Priority Date Filing Date
PCT/US2013/077692 WO2014120367A1 (en) 2013-01-30 2013-12-24 Content adaptive parametric transforms for coding for next generation video
PCT/US2013/077698 WO2014120368A1 (en) 2013-01-30 2013-12-24 Content adaptive entropy coding for next generation video
PCT/US2013/077702 WO2014120369A1 (en) 2013-01-30 2013-12-24 Content adaptive partitioning for prediction and coding for next generation video
PCT/US2013/078114 WO2014120374A1 (en) 2012-11-13 2013-12-27 Content adaptive predictive and functionally predictive pictures with modified references for next generation video coding
PCT/US2013/077967 WO2014120373A1 (en) 2013-01-30 2013-12-27 Content adaptive entropy coding of coded/not-coded data for next generation video
PCT/US2013/078157 WO2014120375A2 (en) 2013-01-30 2013-12-28 Content adaptive bi-directional or functionally predictive multi-pass pictures for high efficiency next generation video coding
PCT/US2014/013027 WO2014120575A1 (en) 2013-01-30 2014-01-24 Content adaptive entropy coding of partitions data for next generation video
PCT/US2014/013333 WO2014120656A1 (en) 2013-01-30 2014-01-28 Content adaptive entropy coding of modes and reference types data for next generation video
PCT/US2014/013965 WO2014120987A1 (en) 2013-01-30 2014-01-30 Content adaptive prediction distance analyzer and hierarchical motion estimation system for next generation video coding
PCT/US2014/013915 WO2014120960A1 (en) 2013-01-30 2014-01-30 Content adaptive bitrate and quality control by using frame hierarchy sensitive quantization for high efficiency next generation video coding
PCT/US2014/018077 WO2015099814A1 (en) 2013-01-30 2014-02-24 Content adaptive gain compensated prediction for next generation video coding

Family Applications Before (4)

Application Number Title Priority Date Filing Date
PCT/US2013/077692 WO2014120367A1 (en) 2013-01-30 2013-12-24 Content adaptive parametric transforms for coding for next generation video
PCT/US2013/077698 WO2014120368A1 (en) 2013-01-30 2013-12-24 Content adaptive entropy coding for next generation video
PCT/US2013/077702 WO2014120369A1 (en) 2013-01-30 2013-12-24 Content adaptive partitioning for prediction and coding for next generation video
PCT/US2013/078114 WO2014120374A1 (en) 2012-11-13 2013-12-27 Content adaptive predictive and functionally predictive pictures with modified references for next generation video coding

Family Applications After (6)

Application Number Title Priority Date Filing Date
PCT/US2013/078157 WO2014120375A2 (en) 2013-01-30 2013-12-28 Content adaptive bi-directional or functionally predictive multi-pass pictures for high efficiency next generation video coding
PCT/US2014/013027 WO2014120575A1 (en) 2013-01-30 2014-01-24 Content adaptive entropy coding of partitions data for next generation video
PCT/US2014/013333 WO2014120656A1 (en) 2013-01-30 2014-01-28 Content adaptive entropy coding of modes and reference types data for next generation video
PCT/US2014/013965 WO2014120987A1 (en) 2013-01-30 2014-01-30 Content adaptive prediction distance analyzer and hierarchical motion estimation system for next generation video coding
PCT/US2014/013915 WO2014120960A1 (en) 2013-01-30 2014-01-30 Content adaptive bitrate and quality control by using frame hierarchy sensitive quantization for high efficiency next generation video coding
PCT/US2014/018077 WO2015099814A1 (en) 2013-01-30 2014-02-24 Content adaptive gain compensated prediction for next generation video coding

Country Status (8)

Country Link
US (14) US9787990B2 (en)
EP (12) EP2952002A4 (en)
JP (3) JP6163674B2 (en)
KR (6) KR20150058324A (en)
CN (10) CN104737542B (en)
BR (1) BR112015015575A2 (en)
RU (1) RU2612600C2 (en)
WO (11) WO2014120367A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9609330B2 (en) 2013-01-30 2017-03-28 Intel Corporation Content adaptive entropy coding of modes and reference types data for next generation video
US9819965B2 (en) 2012-11-13 2017-11-14 Intel Corporation Content adaptive transform coding for next generation video
US10924757B2 (en) 2016-10-19 2021-02-16 Sk Telecom Co., Ltd. Apparatus and method for encoding or decoding video

Families Citing this family (259)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101083770B (en) * 2002-08-08 2010-06-16 松下电器产业株式会社 Image decoding method and apparatus
KR101215152B1 (en) 2011-04-21 2012-12-24 한양대학교 산학협력단 Video encoding/decoding method and apparatus using prediction based on in-loop filtering
US9380327B2 (en) * 2011-12-15 2016-06-28 Comcast Cable Communications, Llc System and method for synchronizing timing across multiple streams
US9591302B2 (en) * 2012-07-02 2017-03-07 Microsoft Technology Licensing, Llc Use of chroma quantization parameter offsets in deblocking
US9414054B2 (en) 2012-07-02 2016-08-09 Microsoft Technology Licensing, Llc Control and use of chroma quantization parameter values
CN102883163B (en) * 2012-10-08 2014-05-28 华为技术有限公司 Method and device for building motion vector lists for prediction of motion vectors
US11438609B2 (en) 2013-04-08 2022-09-06 Qualcomm Incorporated Inter-layer picture signaling and related processes
US10057599B2 (en) * 2014-02-19 2018-08-21 Mediatek Inc. Method for performing image processing control with aid of predetermined tile packing, associated apparatus and associated non-transitory computer readable medium
CN103929642B (en) * 2014-04-24 2017-04-12 北京航空航天大学 Method for rapidly calculating deviation value of entropy coding context model of HEVC transformation coefficients
US9723377B2 (en) 2014-04-28 2017-08-01 Comcast Cable Communications, Llc Video management
US10397574B2 (en) * 2014-05-12 2019-08-27 Intel Corporation Video coding quantization parameter determination suitable for video conferencing
US11064204B2 (en) 2014-05-15 2021-07-13 Arris Enterprises Llc Automatic video comparison of the output of a video decoder
US20170249521A1 (en) * 2014-05-15 2017-08-31 Arris Enterprises, Inc. Automatic video comparison of the output of a video decoder
US10057593B2 (en) * 2014-07-08 2018-08-21 Brain Corporation Apparatus and methods for distance estimation using stereo imagery
US9699464B2 (en) * 2014-07-10 2017-07-04 Intel Corporation Adaptive bitrate streaming for wireless video
US9549188B2 (en) * 2014-07-30 2017-01-17 Intel Corporation Golden frame selection in video coding
CN104301724B (en) * 2014-10-17 2017-12-01 华为技术有限公司 Method for processing video frequency, encoding device and decoding device
TWI536799B (en) * 2014-10-29 2016-06-01 虹光精密工業股份有限公司 Smart copy apparatus
WO2016090568A1 (en) * 2014-12-10 2016-06-16 Mediatek Singapore Pte. Ltd. Binary tree block partitioning structure
US10165272B2 (en) * 2015-01-29 2018-12-25 Arris Enterprises Llc Picture-level QP rate control performance improvements for HEVC encoding
US10410398B2 (en) * 2015-02-20 2019-09-10 Qualcomm Incorporated Systems and methods for reducing memory bandwidth using low quality tiles
CN104780377B (en) * 2015-03-18 2017-12-15 同济大学 A kind of parallel HEVC coded systems and method based on Distributed Computer System
CN108028942B (en) * 2015-06-04 2020-06-26 清华大学 Pixel prediction method, encoding method, decoding method, device thereof, and storage medium
GB2539198B (en) * 2015-06-08 2019-09-25 Imagination Tech Ltd Motion estimation using collocated blocks
US9942552B2 (en) * 2015-06-12 2018-04-10 Intel Corporation Low bitrate video coding
US10354290B2 (en) * 2015-06-16 2019-07-16 Adobe, Inc. Generating a shoppable video
WO2016206748A1 (en) * 2015-06-25 2016-12-29 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Refinement of a low-pel resolution motion estimation vector
US10764574B2 (en) * 2015-07-01 2020-09-01 Panasonic Intellectual Property Management Co., Ltd. Encoding method, decoding method, encoding apparatus, decoding apparatus, and encoding and decoding apparatus
US10419512B2 (en) * 2015-07-27 2019-09-17 Samsung Display Co., Ltd. System and method of transmitting display data
WO2017023829A1 (en) * 2015-07-31 2017-02-09 Stc.Unm System and methods for joint and adaptive control of rate, quality, and computational complexity for video coding and video delivery
US10743059B2 (en) * 2015-08-30 2020-08-11 EVA Automation, Inc. Displaying HDMI content at an arbitrary location
US10430031B2 (en) * 2015-08-30 2019-10-01 EVA Automation, Inc. Displaying HDMI content in a tiled window
FR3040849A1 (en) * 2015-09-04 2017-03-10 Stmicroelectronics (Grenoble 2) Sas METHOD FOR COMPRESSING A VIDEO DATA STREAM
US9971791B2 (en) * 2015-09-16 2018-05-15 Adobe Systems Incorporated Method and apparatus for clustering product media files
WO2017052250A1 (en) * 2015-09-23 2017-03-30 엘지전자(주) Image encoding/decoding method and device for same
WO2017057953A1 (en) * 2015-09-30 2017-04-06 엘지전자 주식회사 Method and device for coding residual signal in video coding system
US10805627B2 (en) 2015-10-15 2020-10-13 Cisco Technology, Inc. Low-complexity method for generating synthetic reference frames in video coding
US9998745B2 (en) * 2015-10-29 2018-06-12 Microsoft Technology Licensing, Llc Transforming video bit streams for parallel processing
WO2017073360A1 (en) * 2015-10-30 2017-05-04 ソニー株式会社 Image processing device and method
EP3166313A1 (en) 2015-11-09 2017-05-10 Thomson Licensing Encoding and decoding method and corresponding devices
US10283094B1 (en) * 2015-11-09 2019-05-07 Marvell International Ltd. Run-length compression and decompression of media tiles
US10972731B2 (en) * 2015-11-10 2021-04-06 Interdigital Madison Patent Holdings, Sas Systems and methods for coding in super-block based video coding framework
US20170155905A1 (en) * 2015-11-30 2017-06-01 Intel Corporation Efficient intra video/image coding using wavelets and variable size transform coding
EP3392840A4 (en) * 2015-12-14 2019-02-06 Panasonic Intellectual Property Corporation of America Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
WO2017114450A1 (en) * 2015-12-31 2017-07-06 Mediatek Inc. Method and apparatus of prediction binary tree structure for video and image coding
US10212444B2 (en) * 2016-01-15 2019-02-19 Qualcomm Incorporated Multi-type-tree framework for video coding
US10445862B1 (en) * 2016-01-25 2019-10-15 National Technology & Engineering Solutions Of Sandia, Llc Efficient track-before detect algorithm with minimal prior knowledge
WO2017135663A2 (en) * 2016-02-01 2017-08-10 엘지전자 주식회사 Method and device for performing transformation using row-column transforms
KR20170096088A (en) * 2016-02-15 2017-08-23 삼성전자주식회사 Image processing apparatus, image processing method thereof and recording medium
CN116506600A (en) 2016-03-11 2023-07-28 数字洞察力有限公司 Video coding method and device
US11223852B2 (en) 2016-03-21 2022-01-11 Qualcomm Incorporated Coding video data using a two-level multi-type-tree framework
US20170280139A1 (en) * 2016-03-22 2017-09-28 Qualcomm Incorporated Apparatus and methods for adaptive calculation of quantization parameters in display stream compression
WO2017178782A1 (en) 2016-04-15 2017-10-19 Magic Pony Technology Limited Motion compensation using temporal picture interpolation
WO2017178827A1 (en) * 2016-04-15 2017-10-19 Magic Pony Technology Limited In-loop post filtering for video encoding and decoding
US10574999B2 (en) * 2016-05-05 2020-02-25 Intel Corporation Method and system of video coding with a multi-pass prediction mode decision pipeline
KR102571062B1 (en) 2016-05-10 2023-08-25 삼성전자주식회사 A method of encoding/decoding an image, and an apparatus therefor
JP2019519148A (en) 2016-05-13 2019-07-04 ヴィド スケール インコーポレイテッド System and method for generalized multi-hypothesis prediction for video coding
TWI689148B (en) * 2016-05-17 2020-03-21 英屬開曼群島商鴻騰精密科技股份有限公司 Electrical connector assembly
US10456672B2 (en) 2016-05-19 2019-10-29 Google Llc Methods and systems for facilitating participation in a game session
CA3025334C (en) * 2016-05-25 2021-07-13 Arris Enterprises Llc Binary ternary quad tree partitioning for jvet coding of video data
US10284845B2 (en) 2016-05-25 2019-05-07 Arris Enterprises Llc JVET quadtree plus binary tree (QTBT) structure with multiple asymmetrical partitioning
CA3025477C (en) 2016-05-25 2023-10-03 Arris Enterprises Llc Jvet coding block structure with asymmetrical partitioning
PT3446483T (en) * 2016-05-25 2021-06-23 Arris Entpr Llc Binary, ternary and quad tree partitioning for jvet coding of video data
MX2018014491A (en) 2016-05-25 2019-08-14 Arris Entpr Llc General block partitioning method.
KR102445668B1 (en) * 2016-06-01 2022-09-21 삼성전자주식회사 Method and Apparatus for video encoding and Method and Apparatus for video decoding
US10743210B2 (en) * 2016-06-01 2020-08-11 Intel Corporation Using uplink buffer status to improve video stream adaptation control
US10880548B2 (en) 2016-06-01 2020-12-29 Samsung Electronics Co., Ltd. Methods and apparatuses for encoding and decoding video according to coding order
EP3472806A4 (en) 2016-06-17 2020-02-26 Immersive Robotics Pty Ltd Image compression method and apparatus
US10169362B2 (en) 2016-07-07 2019-01-01 Cross Commerce Media, Inc. High-density compression method and computing system
US11503305B2 (en) 2016-08-03 2022-11-15 Kt Corporation Method and apparatus for processing video signal
WO2018023554A1 (en) * 2016-08-04 2018-02-08 SZ DJI Technology Co., Ltd. System and methods for bit rate control
ES2883258T3 (en) * 2016-08-04 2021-12-07 Sz Dji Technology Co Ltd Parallel video encoding
US10715818B2 (en) * 2016-08-04 2020-07-14 Intel Corporation Techniques for hardware video encoding
WO2018030746A1 (en) * 2016-08-08 2018-02-15 엘지전자(주) Method for processing image and apparatus therefor
US10609423B2 (en) 2016-09-07 2020-03-31 Qualcomm Incorporated Tree-type coding for video coding
CN107872669B (en) * 2016-09-27 2019-05-24 腾讯科技(深圳)有限公司 Video code rate treating method and apparatus
US10567775B2 (en) * 2016-10-01 2020-02-18 Intel Corporation Method and system of hardware accelerated video coding with per-frame parameter control
KR20200062365A (en) * 2016-10-04 2020-06-03 김기백 Image data encoding/decoding method and apparatus
CN117176948A (en) * 2016-10-04 2023-12-05 有限公司B1影像技术研究所 Image encoding/decoding method, recording medium, and method of transmitting bit stream
CN115022626A (en) 2016-10-04 2022-09-06 有限公司B1影像技术研究所 Image data encoding/decoding method and computer-readable recording medium
EP3306927A1 (en) * 2016-10-05 2018-04-11 Thomson Licensing Encoding and decoding methods and corresponding devices
EP3306938A1 (en) 2016-10-05 2018-04-11 Thomson Licensing Method and apparatus for binary-tree split mode coding
US12035049B2 (en) 2016-10-06 2024-07-09 B1 Institute Of Image Technology, Inc. Image data encoding/decoding method and apparatus
CN116248873A (en) 2016-10-10 2023-06-09 三星电子株式会社 Method and apparatus for encoding or decoding image through block mapping
KR20180039323A (en) 2016-10-10 2018-04-18 디지털인사이트 주식회사 Video coding method and apparatus using combination of various block partitioning structure
US20180103271A1 (en) * 2016-10-10 2018-04-12 Qualcomm Incorporated Systems and methods for signaling missing or corrupted video data
US10187178B2 (en) * 2016-10-11 2019-01-22 Microsoft Technology Licensing, Llc Dynamically partitioning media streams
US10779004B2 (en) 2016-10-12 2020-09-15 Mediatek Inc. Methods and apparatuses of constrained multi-type-tree block partition for video coding
WO2018072675A1 (en) * 2016-10-18 2018-04-26 Zhejiang Dahua Technology Co., Ltd. Methods and systems for video processing
WO2018074825A1 (en) * 2016-10-19 2018-04-26 에스케이텔레콤 주식회사 Device and method for encoding or decoding image
CN106454355B (en) * 2016-10-24 2019-03-01 西南科技大学 A kind of method for video coding and device
CN109983771A (en) 2016-11-21 2019-07-05 松下电器(美国)知识产权公司 Code device, decoding apparatus, coding method and coding/decoding method
CN116347077A (en) 2016-11-21 2023-06-27 松下电器(美国)知识产权公司 Computer readable medium
KR20180057564A (en) * 2016-11-22 2018-05-30 한국전자통신연구원 Method and apparatus for encoding/decoding image and recording medium for storing bitstream
GB2556923B (en) * 2016-11-25 2020-04-15 Canon Kk Generation of VCA Reference results for VCA Auto-setting
CN117528108A (en) * 2016-11-28 2024-02-06 英迪股份有限公司 Image encoding method, image decoding method, and method for transmitting bit stream
US10694202B2 (en) * 2016-12-01 2020-06-23 Qualcomm Incorporated Indication of bilateral filter usage in video coding
US10631012B2 (en) * 2016-12-02 2020-04-21 Centurylink Intellectual Property Llc Method and system for implementing detection and visual enhancement of video encoding artifacts
US10397590B2 (en) * 2016-12-05 2019-08-27 Nice-Systems Ltd. System and method for enabling seek in a video recording
JP6565885B2 (en) * 2016-12-06 2019-08-28 株式会社Jvcケンウッド Image encoding device, image encoding method, image encoding program, image decoding device, image decoding method, and image decoding program
CN116567236A (en) * 2016-12-16 2023-08-08 夏普株式会社 Image decoding method
CN110115034B (en) * 2016-12-23 2023-01-13 华为技术有限公司 Intra-frame prediction device for expanding preset directional intra-frame prediction mode set
RU2018131580A (en) * 2016-12-26 2021-01-26 Нек Корпорейшн VIDEO ENCODING METHOD, VIDEO DECODING METHOD, VIDEO ENCODING DEVICE, VIDEO DECODING DEVICE AND PROGRAM
RU2018131583A (en) * 2016-12-26 2021-01-26 Нек Корпорейшн VIDEO ENCODING METHOD, VIDEO DECODING METHOD, VIDEO ENCODING DEVICE, VIDEO DECODING DEVICE AND PROGRAM
WO2018125944A1 (en) * 2016-12-28 2018-07-05 Arris Enterprises Llc Improved video bitstream coding
US10681370B2 (en) * 2016-12-29 2020-06-09 Qualcomm Incorporated Motion vector generation for affine motion model for video coding
TWI617181B (en) * 2017-01-04 2018-03-01 晨星半導體股份有限公司 Scheduling method for high efficiency video coding apparatus
KR102708338B1 (en) * 2017-01-05 2024-09-23 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. Block-based predictive coding and decoding of a picture
US10848788B2 (en) 2017-01-06 2020-11-24 Qualcomm Incorporated Multi-type-tree framework for video coding
EP3349454A1 (en) 2017-01-11 2018-07-18 Thomson Licensing Method and device for coding a block of video data, method and device for decoding a block of video data
KR20180088188A (en) * 2017-01-26 2018-08-03 삼성전자주식회사 Method and apparatus for adaptive image transformation
WO2018143687A1 (en) * 2017-02-01 2018-08-09 엘지전자(주) Method and apparatus for performing transformation by using row-column transform
CN109845260B (en) 2017-02-06 2021-06-22 华为技术有限公司 Encoding and decoding method and device
CN110495107A (en) 2017-02-08 2019-11-22 因默希弗机器人私人有限公司 Day line traffic control for mobile device communication
US10362332B2 (en) * 2017-03-14 2019-07-23 Google Llc Multi-level compound prediction
US10820017B2 (en) * 2017-03-15 2020-10-27 Mediatek Inc. Method and apparatus of video coding
WO2018169571A1 (en) * 2017-03-15 2018-09-20 Google Llc Segmentation-based parameterized motion models
US11496747B2 (en) 2017-03-22 2022-11-08 Qualcomm Incorporated Intra-prediction mode propagation
US10904531B2 (en) * 2017-03-23 2021-01-26 Qualcomm Incorporated Adaptive parameters for coding of 360-degree video
CN108665410B (en) * 2017-03-31 2021-11-26 杭州海康威视数字技术股份有限公司 Image super-resolution reconstruction method, device and system
US11551067B2 (en) * 2017-04-06 2023-01-10 Shanghai Cambricon Information Technology Co., Ltd Neural network processor and neural network computation method
KR102302643B1 (en) 2017-04-21 2021-09-14 제니맥스 미디어 인크. Systems and Methods for Player Input Motion Compensation by Motion Vectors Prediction
CA3059740A1 (en) 2017-04-21 2018-10-25 Zenimax Media Inc. Systems and methods for game-generated motion vectors
KR102326456B1 (en) 2017-04-21 2021-11-12 제니맥스 미디어 인크. Systems and Methods for Encoder-Guided Adaptive-Quality Rendering
AU2018254570B2 (en) 2017-04-21 2021-08-05 Zenimax Media Inc. Systems and methods for deferred post-processes in video encoding
GB2595029B (en) 2017-04-21 2022-02-09 Zenimax Media Inc Systems and methods for rendering & pre-encoded load estimation based encoder hinting
US10979728B2 (en) * 2017-04-24 2021-04-13 Intel Corporation Intelligent video frame grouping based on predicted performance
CN108810556B (en) * 2017-04-28 2021-12-24 炬芯科技股份有限公司 Method, device and chip for compressing reference frame
US10638127B2 (en) * 2017-05-15 2020-04-28 Qualcomm Incorporated Adaptive anchor frame and quantization parameter decision in video coding
EP3635952B1 (en) * 2017-06-05 2024-07-03 Immersive Robotics Pty Ltd Digital content stream compression
KR102315524B1 (en) * 2017-06-16 2021-10-21 한화테크윈 주식회사 A method for adjusting bitrate of the image and image capture apparatus
CN107222751B (en) * 2017-06-23 2019-05-10 宁波大学科学技术学院 3D-HEVC deep video information concealing method based on multi-view point video feature
CN118233624A (en) 2017-06-26 2024-06-21 交互数字Vc控股公司 Method and apparatus for video encoding and decoding
CN112601084A (en) 2017-06-28 2021-04-02 华为技术有限公司 Image data encoding and decoding methods and devices
US10848761B2 (en) 2017-06-29 2020-11-24 Qualcomm Incorporated Reducing seam artifacts in 360-degree video
CN109218748B (en) * 2017-06-30 2020-11-27 京东方科技集团股份有限公司 Video transmission method, device and computer readable storage medium
CN107360419B (en) * 2017-07-18 2019-09-24 成都图必优科技有限公司 A kind of movement forward sight video interprediction encoding method based on perspective model
CN114554221B (en) 2017-07-19 2023-08-01 三星电子株式会社 Encoding method and apparatus therefor, decoding method and apparatus therefor
US11146608B2 (en) * 2017-07-20 2021-10-12 Disney Enterprises, Inc. Frame-accurate video seeking via web browsers
CN107483949A (en) * 2017-07-26 2017-12-15 千目聚云数码科技(上海)有限公司 Increase the method and system of SVAC SVC practicality
CN107295334B (en) * 2017-08-15 2019-12-03 电子科技大学 Adaptive reference picture chooses method
CN107483960B (en) * 2017-09-15 2020-06-02 信阳师范学院 Motion compensation frame rate up-conversion method based on spatial prediction
US10645408B2 (en) * 2017-09-17 2020-05-05 Google Llc Dual deblocking filter thresholds
EP3457695A1 (en) 2017-09-18 2019-03-20 Thomson Licensing Method and apparatus for motion vector predictor adaptation for omnidirectional video
US10609384B2 (en) * 2017-09-21 2020-03-31 Futurewei Technologies, Inc. Restriction on sub-block size derivation for affine inter prediction
CN116489354A (en) * 2017-09-28 2023-07-25 Lg 电子株式会社 Image decoding method and apparatus according to block separation structure in image coding system
US10623744B2 (en) * 2017-10-04 2020-04-14 Apple Inc. Scene based rate control for video compression and video streaming
EP4336800A3 (en) 2017-10-10 2024-05-29 Google LLC Distributed sample-based game profiling with game metadata and metrics and gaming api platform supporting third-party content
EP3714598A4 (en) 2017-11-21 2021-03-31 Immersive Robotics Pty Ltd Frequency component selection for image compression
AU2018372561B2 (en) 2017-11-21 2023-01-05 Immersive Robotics Pty Ltd Image compression for digital reality
CN111164980A (en) 2017-11-30 2020-05-15 深圳市大疆创新科技有限公司 System and method for controlling video encoding within image frames
CN110832856A (en) 2017-11-30 2020-02-21 深圳市大疆创新科技有限公司 System and method for reducing video coding fluctuations
WO2019104635A1 (en) * 2017-11-30 2019-06-06 SZ DJI Technology Co., Ltd. System and method for controlling video coding at frame level
CN109949217B (en) * 2017-12-20 2023-04-11 四川大学 Video super-resolution reconstruction method based on residual learning and implicit motion compensation
US11140207B2 (en) 2017-12-21 2021-10-05 Google Llc Network impairment simulation framework for verification of real time interactive media streaming systems
KR101975404B1 (en) * 2017-12-27 2019-08-28 세종대학교산학협력단 Apparatus and method for generating procedural content
TWI680675B (en) * 2017-12-29 2019-12-21 聯發科技股份有限公司 Image processing device and associated image processing method
WO2019147628A1 (en) 2018-01-24 2019-08-01 Vid Scale, Inc. Generalized bi-prediction for video coding with reduced coding complexity
KR102465206B1 (en) * 2018-01-26 2022-11-09 삼성전자주식회사 Image processing device
US11095876B2 (en) * 2018-01-26 2021-08-17 Samsung Electronics Co., Ltd. Image processing device
KR102543449B1 (en) * 2018-01-26 2023-06-14 삼성전자주식회사 Image processing device and method for operating image processing device
EP4221201A1 (en) 2018-01-29 2023-08-02 InterDigital VC Holdings, Inc. Encoding and decoding with refinement of the reconstructed picture
US11563970B2 (en) * 2018-02-26 2023-01-24 Interdigital Vc Holdings, Inc. Method and apparatus for generalized OBMC
WO2019167673A1 (en) * 2018-03-01 2019-09-06 ソニー株式会社 Image processing device and method, imaging element, and imaging device
US11004178B2 (en) 2018-03-01 2021-05-11 Nvidia Corporation Enhancing high-resolution images with data from low-resolution images
US10432962B1 (en) * 2018-03-13 2019-10-01 Pixelworks, Inc. Accuracy and local smoothness of motion vector fields using motion-model fitting
CN110309328B (en) * 2018-03-14 2022-03-25 深圳云天励飞技术有限公司 Data storage method and device, electronic equipment and storage medium
EP3700640B1 (en) * 2018-03-22 2021-05-26 Google LLC Methods and systems for rendering and encoding content for online interactive gaming sessions
CN108449599B (en) * 2018-03-23 2021-05-18 安徽大学 Video coding and decoding method based on surface transmission transformation
KR102694451B1 (en) 2018-04-02 2024-08-13 구글 엘엘씨 Methods, devices, and systems for interactive cloud gaming
US11077364B2 (en) 2018-04-02 2021-08-03 Google Llc Resolution-based scaling of real-time interactive graphics
CN111886057B (en) 2018-04-02 2024-09-20 谷歌有限责任公司 Input device for electronic system
TWI700922B (en) * 2018-04-02 2020-08-01 聯發科技股份有限公司 Video processing methods and apparatuses for sub-block motion compensation in video coding systems
EP3701489B1 (en) 2018-04-10 2022-10-26 Google LLC Memory management in gaming rendering
US10491897B2 (en) * 2018-04-13 2019-11-26 Google Llc Spatially adaptive quantization-aware deblocking filter
US10798382B2 (en) * 2018-04-26 2020-10-06 Tencent America LLC Sub-block transform
EP3562162A1 (en) 2018-04-27 2019-10-30 InterDigital VC Holdings, Inc. Method and apparatus for video encoding and decoding based on neural network implementation of cabac
EP3804314B1 (en) * 2018-05-29 2024-01-17 InterDigital VC Holdings, Inc. Method and apparatus for video encoding and decoding with partially shared luma and chroma coding trees
TWI746994B (en) * 2018-06-19 2021-11-21 大陸商北京字節跳動網絡技術有限公司 Different precisions for different reference list
CN108848380B (en) * 2018-06-20 2021-11-30 腾讯科技(深圳)有限公司 Video encoding and decoding method, device, computer device and storage medium
US10602147B2 (en) * 2018-07-10 2020-03-24 Samsung Display Co., Ltd. Efficient entropy coding group grouping methodology for transform mode
US11039315B2 (en) 2018-08-01 2021-06-15 At&T Intellectual Property I, L.P. On-demand super slice instantiation and orchestration
CN118694935A (en) * 2018-08-20 2024-09-24 日本放送协会 Encoding device, decoding device, and program
CN115250350B (en) * 2018-09-03 2024-04-09 华为技术有限公司 Motion vector acquisition method, motion vector acquisition device, computer equipment and storage medium
US10863179B1 (en) * 2018-09-05 2020-12-08 Amazon Technologies, Inc. Overlapped rate control for high-quality segmented video encoding
US11665365B2 (en) * 2018-09-14 2023-05-30 Google Llc Motion prediction coding with coframe motion vectors
EP4325859A3 (en) 2018-09-19 2024-05-15 Beijing Bytedance Network Technology Co., Ltd. Syntax reuse for affine mode with adaptive motion vector resolution
US11218694B2 (en) * 2018-09-24 2022-01-04 Qualcomm Incorporated Adaptive multiple transform coding
US11606569B2 (en) * 2018-09-25 2023-03-14 Apple Inc. Extending supported components for encoding image data
US10893281B2 (en) 2018-10-12 2021-01-12 International Business Machines Corporation Compression of a video stream having frames with relatively heightened quality parameters on blocks on an identified point of interest (PoI)
US10764589B2 (en) * 2018-10-18 2020-09-01 Trisys Co., Ltd. Method and module for processing image data
CN111083491B (en) 2018-10-22 2024-09-20 北京字节跳动网络技术有限公司 Use of refined motion vectors
WO2020084476A1 (en) * 2018-10-22 2020-04-30 Beijing Bytedance Network Technology Co., Ltd. Sub-block based prediction
CN109640082B (en) * 2018-10-26 2021-02-12 浙江鼎越电子有限公司 Audio and video multimedia data processing method and equipment thereof
US11412260B2 (en) * 2018-10-29 2022-08-09 Google Llc Geometric transforms for image compression
US10939102B2 (en) * 2018-11-01 2021-03-02 Mediatek Inc. Post processing apparatus with super-resolution filter and loop restoration filter in block-level pipeline and associated post processing method
WO2020098644A1 (en) 2018-11-12 2020-05-22 Beijing Bytedance Network Technology Co., Ltd. Bandwidth control methods for inter prediction
EP3807766B1 (en) 2018-11-16 2021-10-27 Google LLC Shadow tracking of real-time interactive simulations for complex system analysis
US10666985B1 (en) * 2018-11-18 2020-05-26 Sony Corporation Sub-block based entropy coding for image coding
CN113170171B (en) 2018-11-20 2024-04-12 北京字节跳动网络技术有限公司 Prediction refinement combining inter intra prediction modes
CN117319644A (en) 2018-11-20 2023-12-29 北京字节跳动网络技术有限公司 Partial position based difference calculation
WO2020103935A1 (en) 2018-11-22 2020-05-28 Beijing Bytedance Network Technology Co., Ltd. Blending method for inter prediction with geometry partition
US11184633B2 (en) 2018-11-27 2021-11-23 Qualcomm Incorporated Simplification of history-based motion vector prediction
JP7040422B2 (en) * 2018-11-28 2022-03-23 日本電信電話株式会社 Motion vector generator, projection image generator, motion vector generator, and program
US11025947B2 (en) * 2018-11-29 2021-06-01 Mediatek Inc. Method and apparatus for generating motion field motion vectors for blocks of current frame in on-the-fly manner
US10855988B2 (en) 2018-12-19 2020-12-01 Qualcomm Incorporated Adaptive prediction structures
CN113273189B (en) * 2018-12-31 2024-08-16 北京字节跳动网络技术有限公司 Interaction between Merge and AMVR with MVD
US10645386B1 (en) * 2019-01-03 2020-05-05 Sony Corporation Embedded codec circuitry for multiple reconstruction points based quantization
WO2020156517A1 (en) 2019-01-31 2020-08-06 Beijing Bytedance Network Technology Co., Ltd. Fast algorithms for symmetric motion vector difference coding mode
KR20210121021A (en) 2019-01-31 2021-10-07 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 Affine Mode Adaptive Motion Vector Resolution Coding Context
US10778972B1 (en) * 2019-02-27 2020-09-15 Google Llc Adaptive filter intra prediction modes in image/video compression
WO2020177755A1 (en) 2019-03-06 2020-09-10 Beijing Bytedance Network Technology Co., Ltd. Usage of converted uni-prediction candidate
JP7252016B2 (en) * 2019-03-13 2023-04-04 日本放送協会 Video encoding device, video decoding device and program
WO2020182207A1 (en) * 2019-03-13 2020-09-17 Beijing Bytedance Network Technology Co., Ltd. Partitions on sub-block transform mode
US10924625B2 (en) * 2019-03-20 2021-02-16 Xerox Corporation Dynamic compression acceleration using real-time image data entropy analysis
EP3942475B1 (en) * 2019-03-21 2024-07-10 Google LLC Using rate distortion cost as a loss function for deep learning
US11310496B2 (en) * 2019-03-27 2022-04-19 Ati Technologies Ulc Determining quality values for blocks of encoded video
KR102610709B1 (en) 2019-04-02 2023-12-05 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 Decoder side motion vector derivation
CN111901593B (en) * 2019-05-04 2024-01-02 华为技术有限公司 Image dividing method, device and equipment
US11223838B2 (en) * 2019-05-26 2022-01-11 Alibaba Group Holding Limited AI-assisted programmable hardware video codec
US10949982B1 (en) * 2019-06-21 2021-03-16 Amazon Technologies, Inc. Moving object recognition, speed estimation, and tagging
EP3981155A4 (en) 2019-06-24 2022-08-17 Alibaba Group Holding Limited Adaptive resolution change in video processing
CN110339567B (en) * 2019-07-17 2023-05-26 三星电子(中国)研发中心 System resource allocation and scene prediction model training method and device
CN114503581A (en) 2019-08-06 2022-05-13 Op方案有限责任公司 Adaptive block-based resolution management
CN114467305A (en) 2019-08-06 2022-05-10 Op方案有限责任公司 Adaptive resolution management predictive rescaling
EP4011068A4 (en) 2019-08-06 2023-08-09 OP Solutions, LLC Implicit signaling of adaptive resolution management based on frame type
WO2021026363A1 (en) * 2019-08-06 2021-02-11 Op Solutions, Llc Implicit signaling of adaptive resolution management based on frame type
GB2586517B (en) * 2019-08-23 2022-06-15 Imagination Tech Ltd Methods and decompression units for decompressing a compressed block of image data
CN112449140B (en) * 2019-08-29 2021-09-14 华为技术有限公司 Video super-resolution processing method and device
EP4035367A1 (en) * 2019-09-23 2022-08-03 InterDigital VC Holdings France, SAS Video encoding and decoding using block area based quantization matrices
CN112702603A (en) * 2019-10-22 2021-04-23 腾讯科技(深圳)有限公司 Video encoding method, video encoding device, computer equipment and storage medium
EP3820138A1 (en) * 2019-11-06 2021-05-12 Koninklijke Philips N.V. A system for performing image motion compensation
CA3223625A1 (en) 2019-11-08 2021-05-14 Op Solutions, Llc Methods and systems for adaptive cropping
US11843772B2 (en) * 2019-12-06 2023-12-12 Ati Technologies Ulc Video encode pre-analysis bit budgeting based on context and features
US11308093B1 (en) * 2019-12-13 2022-04-19 Amazon Technologies, Inc. Encoding scheme for numeric-like data types
CN111814844B (en) * 2020-03-17 2023-07-11 同济大学 Intensive video description method based on position coding fusion
EP3902244B1 (en) * 2020-04-23 2022-03-23 Axis AB Controlling a pan-tilt-zoom camera
US11589052B2 (en) * 2020-06-10 2023-02-21 Tencent America LLC Techniques for bitstream extraction for subpicture in coded video stream
CN111757107B (en) * 2020-06-29 2024-03-05 北京百度网讯科技有限公司 Video coding method, device, equipment and medium
US11790533B2 (en) * 2020-07-02 2023-10-17 Sony Group Corporation Machine learning based image segmentation training with contour accuracy evaluation
US11436699B1 (en) * 2020-08-03 2022-09-06 Meta Platforms, Inc. Predicting encoding parameters for convex hull video encoding
US11388413B1 (en) 2020-08-03 2022-07-12 Meta Platforms, Inc. Fast encoding parameter selection for convex hull video encoding
US20210144377A1 (en) * 2020-11-19 2021-05-13 Intel Corporation Method and system of video coding with content adaptive quantization
CN114630120B (en) * 2020-12-14 2024-03-29 瑞昱半导体股份有限公司 Video compression method and circuit system based on self-adaptive compression rate
US11363262B1 (en) * 2020-12-14 2022-06-14 Google Llc Adaptive GOP structure using temporal dependencies likelihood
KR20220090887A (en) * 2020-12-23 2022-06-30 삼성전자주식회사 Image processing device and method for operating image processing device
WO2022174118A1 (en) * 2021-02-13 2022-08-18 Meta Platforms, Inc. Video coding efficiency improvement using dynamic frame boost scaling
US20220264111A1 (en) * 2021-02-13 2022-08-18 Meta Platforms, Inc. Video coding efficiency improvement using dynamic frame boost scaling
US11638025B2 (en) 2021-03-19 2023-04-25 Qualcomm Incorporated Multi-scale optical flow for learned video compression
US11523117B2 (en) * 2021-03-22 2022-12-06 Realtek Semiconductor Corp. Encoder using coding tree unit level adaptive quantization mode to adjust frame level quantization parameter and associated signal processing method
US11871061B1 (en) 2021-03-31 2024-01-09 Amazon Technologies, Inc. Automated adaptive bitrate encoding
GB2608107B (en) * 2021-06-18 2023-06-14 Imagination Tech Ltd Mapping multi-dimensional coordinates to a 1D space
KR102562380B1 (en) * 2021-10-18 2023-08-02 (주)카이미디어 8K scalable video encoding method
US11991424B2 (en) * 2021-11-05 2024-05-21 Tencent America LLC Immersive media data complexity analyzer for transformation of asset formats
CN115002482B (en) * 2022-04-27 2024-04-16 电子科技大学 End-to-end video compression method and system using structural preserving motion estimation
WO2023219600A1 (en) * 2022-05-07 2023-11-16 Google Llc Motion vector coding using a motion vector precision
US20230412812A1 (en) * 2022-06-15 2023-12-21 Tencent America LLC Systems and methods for joint signaling of transform coefficient signs
US20240080483A1 (en) * 2022-08-26 2024-03-07 Tencent America LLC Systems and methods for warp sample selection and grouping
CN115272083B (en) * 2022-09-27 2022-12-02 中国人民解放军国防科技大学 Image super-resolution method, device, equipment and medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030113026A1 (en) 2001-12-17 2003-06-19 Microsoft Corporation Skip macroblock coding
US20080112489A1 (en) * 2006-11-09 2008-05-15 Calista Technologies System and method for effectively encoding and decoding electronic information
KR20110112171A (en) * 2010-04-05 2011-10-12 삼성전자주식회사 Method and apparatus for video encoding, and method and apparatus for video decoding using adaptive coefficient scan order
KR20110129493A (en) * 2007-06-15 2011-12-01 콸콤 인코포레이티드 Adaptive coefficient scanning in video coding
US20120128070A1 (en) * 2009-02-23 2012-05-24 Korean Broadcasting System Video Encoding Method for Encoding Division Block, Video Decoding Method for Decoding Division Block, and Recording Medium for Implementing the Same

Family Cites Families (162)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR0151011B1 (en) 1994-11-30 1998-10-01 김광호 A bipolar transistor and method of manufacturing the same
US6542547B1 (en) * 1995-05-30 2003-04-01 Texas Instruments Incorporated Efficient heuristic based motion estimation method for video compression
US5864711A (en) 1995-07-05 1999-01-26 Microsoft Corporation System for determining more accurate translation between first and second translator, and providing translated data to second computer if first translator is more accurate
US5729691A (en) * 1995-09-29 1998-03-17 Intel Corporation Two-stage transform for video signals
US6160846A (en) 1995-10-25 2000-12-12 Sarnoff Corporation Apparatus and method for optimizing the rate control in a coding system
JP3929492B2 (en) 1995-10-25 2007-06-13 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Segmented image encoding method and system, and decoding method and system thereof
JP3743837B2 (en) * 1996-06-14 2006-02-08 株式会社大宇エレクトロニクス Run-length encoder
US5953506A (en) 1996-12-17 1999-09-14 Adaptive Media Technologies Method and apparatus that provides a scalable media delivery system
CN1134172C (en) * 1997-01-13 2004-01-07 皇家菲利浦电子有限公司 Emboding supplemental data in digital video signal
US6208693B1 (en) * 1997-02-14 2001-03-27 At&T Corp Chroma-key for efficient and low complexity shape representation of coded arbitrary video objects
JP3843581B2 (en) * 1998-03-05 2006-11-08 富士ゼロックス株式会社 Image encoding device, image decoding device, image processing device, image encoding method, image decoding method, and image processing method
JP2000050263A (en) * 1998-07-28 2000-02-18 Hitachi Ltd Image coder, decoder and image-pickup device using them
US6895051B2 (en) 1998-10-15 2005-05-17 Nokia Mobile Phones Limited Video data encoder and decoder
US6223162B1 (en) * 1998-12-14 2001-04-24 Microsoft Corporation Multi-level run length coding for frequency-domain audio coding
US7065253B2 (en) 1999-09-03 2006-06-20 Intel Corporation Wavelet zerotree coding of ordered bits
AU2002236659A1 (en) 2000-12-19 2002-07-01 Pulsent Corporation Adaptive transforms
US20020122491A1 (en) 2001-01-03 2002-09-05 Marta Karczewicz Video decoder architecture and method for using same
ES2665693T3 (en) 2001-11-06 2018-04-26 Panasonic Intellectual Property Corporation Of America Encoding method of moving images and decoding method of moving images
US7453936B2 (en) * 2001-11-09 2008-11-18 Sony Corporation Transmitting apparatus and method, receiving apparatus and method, program and recording medium, and transmitting/receiving system
GB2382940A (en) 2001-11-27 2003-06-11 Nokia Corp Encoding objects and background blocks
CN101448162B (en) * 2001-12-17 2013-01-02 微软公司 Method for processing video image
JP2004088722A (en) 2002-03-04 2004-03-18 Matsushita Electric Ind Co Ltd Motion picture encoding method and motion picture decoding method
AU2003285850A1 (en) 2002-04-23 2004-04-30 Nokia Corporation Method and device for indicating quantizer parameters in a video coding system
JP2003324731A (en) 2002-04-26 2003-11-14 Sony Corp Encoder, decoder, image processing apparatus, method and program for them
US20040001546A1 (en) 2002-06-03 2004-01-01 Alexandros Tourapis Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multi-picture reference motion compensation
JP4767992B2 (en) 2002-06-06 2011-09-07 パナソニック株式会社 Variable length encoding method and variable length decoding method
US7729563B2 (en) 2002-08-28 2010-06-01 Fujifilm Corporation Method and device for video image processing, calculating the similarity between video frames, and acquiring a synthesized frame by synthesizing a plurality of contiguous sampled frames
JP3997171B2 (en) 2003-03-27 2007-10-24 株式会社エヌ・ティ・ティ・ドコモ Moving picture encoding apparatus, moving picture encoding method, moving picture encoding program, moving picture decoding apparatus, moving picture decoding method, and moving picture decoding program
HUP0301368A3 (en) * 2003-05-20 2005-09-28 Amt Advanced Multimedia Techno Method and equipment for compressing motion picture data
JP2004007778A (en) 2003-07-14 2004-01-08 Victor Co Of Japan Ltd Motion compensating and decoding method
US20050094729A1 (en) 2003-08-08 2005-05-05 Visionflow, Inc. Software and hardware partitioning for multi-standard video compression and decompression
CN1332563C (en) * 2003-12-31 2007-08-15 中国科学院计算技术研究所 Coding method of video frequency image jump over macro block
US7492820B2 (en) * 2004-02-06 2009-02-17 Apple Inc. Rate control for video coder employing adaptive linear regression bits modeling
EP1571850A3 (en) 2004-03-05 2006-12-13 Samsung Electronics Co., Ltd. Apparatus and method for encoding and decoding image containing grayscale alpha channel image
KR100654431B1 (en) * 2004-03-08 2006-12-06 삼성전자주식회사 Method for scalable video coding with variable GOP size, and scalable video coding encoder for the same
US7461525B2 (en) * 2004-03-13 2008-12-09 Wilson Rodney W Tile sponge washing and conditioning apparatus
JP2005301457A (en) 2004-04-07 2005-10-27 Fuji Xerox Co Ltd Image processor, program, and recording medium
US7689051B2 (en) * 2004-04-15 2010-03-30 Microsoft Corporation Predictive lossless coding of images and video
KR100909541B1 (en) * 2004-06-27 2009-07-27 애플 인크. Multi-pass video encoding method
KR100664932B1 (en) 2004-10-21 2007-01-04 삼성전자주식회사 Video coding method and apparatus thereof
EP1675402A1 (en) * 2004-12-22 2006-06-28 Thomson Licensing Optimisation of a quantisation matrix for image and video coding
JP2006270301A (en) * 2005-03-23 2006-10-05 Nippon Hoso Kyokai <Nhk> Scene change detecting apparatus and scene change detection program
WO2006109974A1 (en) 2005-04-13 2006-10-19 Samsung Electronics Co., Ltd. Method for entropy coding and decoding having improved coding efficiency and apparatus for providing the same
KR100703773B1 (en) 2005-04-13 2007-04-06 삼성전자주식회사 Method and apparatus for entropy coding and decoding, with improved coding efficiency, and method and apparatus for video coding and decoding including the same
US7397933B2 (en) 2005-05-27 2008-07-08 Microsoft Corporation Collusion resistant desynchronization for digital video fingerprinting
US8064516B2 (en) * 2005-06-02 2011-11-22 Broadcom Corporation Text recognition during video compression
KR20070006445A (en) * 2005-07-08 2007-01-11 삼성전자주식회사 Method and apparatus for hybrid entropy encoding and decoding
WO2007011851A2 (en) * 2005-07-15 2007-01-25 Texas Instruments Incorporated Filtered and warped motion compensation
CN101263513A (en) 2005-07-15 2008-09-10 德克萨斯仪器股份有限公司 Filtered and warpped motion compensation
US9077960B2 (en) 2005-08-12 2015-07-07 Microsoft Corporation Non-zero coefficient block pattern coding
US9258519B2 (en) 2005-09-27 2016-02-09 Qualcomm Incorporated Encoder assisted frame rate up conversion using various motion models
KR100977101B1 (en) 2005-11-30 2010-08-23 가부시끼가이샤 도시바 Image encoding/image decoding method and image encoding/image decoding apparatus
US8243804B2 (en) 2005-12-01 2012-08-14 Lsi Corporation Hierarchical motion estimation for images with varying horizontal and/or vertical dimensions
US8265145B1 (en) * 2006-01-13 2012-09-11 Vbrick Systems, Inc. Management and selection of reference frames for long term prediction in motion estimation
US8279928B2 (en) * 2006-05-09 2012-10-02 Canon Kabushiki Kaisha Image encoding apparatus and encoding method, image decoding apparatus and decoding method
US8275045B2 (en) * 2006-07-12 2012-09-25 Qualcomm Incorporated Video compression using adaptive variable length codes
KR101382101B1 (en) * 2006-08-25 2014-04-07 톰슨 라이센싱 Methods and apparatus for reduced resolution partitioning
US8532178B2 (en) 2006-08-25 2013-09-10 Lg Electronics Inc. Method and apparatus for decoding/encoding a video signal with inter-view reference picture list construction
US20080075173A1 (en) * 2006-09-22 2008-03-27 Texas Instruments Incorporated Systems and Methods for Context Adaptive Video Data Preparation
US20100040146A1 (en) 2006-09-22 2010-02-18 Beibei Wang Method and apparatus for multiple pass video coding and decoding
EP4224853A1 (en) 2006-11-08 2023-08-09 InterDigital VC Holdings, Inc. Methods and apparatus for in-loop de-artifact filtering
US8875199B2 (en) * 2006-11-13 2014-10-28 Cisco Technology, Inc. Indicating picture usefulness for playback optimization
EP1926321A1 (en) 2006-11-27 2008-05-28 Matsushita Electric Industrial Co., Ltd. Hybrid texture representation
US8396118B2 (en) * 2007-03-19 2013-03-12 Sony Corporation System and method to control compressed video picture quality for a given average bit rate
TW200840366A (en) 2007-03-28 2008-10-01 Univ Nat Central Complexity control method of video encoder
KR101336445B1 (en) * 2007-06-08 2013-12-04 삼성전자주식회사 Method for rate control in video encoding
US8437564B2 (en) 2007-08-07 2013-05-07 Ntt Docomo, Inc. Image and video compression using sparse orthonormal transforms
GB0716158D0 (en) 2007-08-17 2007-09-26 Imagination Tech Ltd Data compression
US8526489B2 (en) * 2007-09-14 2013-09-03 General Instrument Corporation Personal video recorder
WO2009045682A2 (en) * 2007-09-28 2009-04-09 Athanasios Leontaris Treating video information
US9445110B2 (en) * 2007-09-28 2016-09-13 Dolby Laboratories Licensing Corporation Video compression and transmission techniques
US20090135901A1 (en) 2007-11-28 2009-05-28 The Hong Kong University Of Science And Technology Complexity adaptive video encoding using multiple reference frames
US8149915B1 (en) 2007-11-29 2012-04-03 Lsi Corporation Refinement of motion vectors in hierarchical motion estimation
US20090154567A1 (en) 2007-12-13 2009-06-18 Shaw-Min Lei In-loop fidelity enhancement for video compression
JP5203717B2 (en) * 2007-12-19 2013-06-05 パナソニック株式会社 Encoder, decoder, encoding method, and decoding method
JP4309453B2 (en) 2007-12-26 2009-08-05 株式会社東芝 Interpolated frame generating apparatus, interpolated frame generating method, and broadcast receiving apparatus
US20090167775A1 (en) 2007-12-30 2009-07-02 Ning Lu Motion estimation compatible with multiple standards
US8126054B2 (en) 2008-01-09 2012-02-28 Motorola Mobility, Inc. Method and apparatus for highly scalable intraframe video coding
CN101272494B (en) * 2008-01-25 2011-06-08 浙江大学 Video encoding/decoding method and device using synthesized reference frame
US8798137B2 (en) 2008-02-29 2014-08-05 City University Of Hong Kong Bit rate estimation in data or video compression
US8254469B2 (en) 2008-05-07 2012-08-28 Kiu Sha Management Liability Company Error concealment for frame loss in multiple description coding
EP2277314A1 (en) * 2008-05-22 2011-01-26 Telefonaktiebolaget LM Ericsson (publ) Content adaptive video encoder and coding method
US8897359B2 (en) * 2008-06-03 2014-11-25 Microsoft Corporation Adaptive quantization for enhancement layer video coding
JP2010016454A (en) * 2008-07-01 2010-01-21 Sony Corp Image encoding apparatus and method, image decoding apparatus and method, and program
TWI359617B (en) 2008-07-03 2012-03-01 Univ Nat Taiwan Low-complexity and high-quality error concealment
US8625681B2 (en) * 2008-07-09 2014-01-07 Intel Corporation Rate-distortion cost reducing video encoding techniques
US8385404B2 (en) 2008-09-11 2013-02-26 Google Inc. System and method for video encoding using constructed reference frame
US8503527B2 (en) * 2008-10-03 2013-08-06 Qualcomm Incorporated Video coding with large macroblocks
KR101441903B1 (en) 2008-10-16 2014-09-24 에스케이텔레콤 주식회사 Reference Frame Creating Method and Apparatus and Video Encoding/Decoding Method and Apparatus Using Same
JP4427600B1 (en) 2008-11-28 2010-03-10 株式会社東芝 Video analysis apparatus and program
US8774559B2 (en) * 2009-01-19 2014-07-08 Sharp Laboratories Of America, Inc. Stereoscopic dynamic range image sequence
TWI498003B (en) 2009-02-02 2015-08-21 Thomson Licensing Method for decoding a stream representative of a sequence of pictures, method for coding a sequence of pictures and coded data structure
US9110849B2 (en) * 2009-04-15 2015-08-18 Qualcomm Incorporated Computing even-sized discrete cosine transforms
EP2271102A1 (en) 2009-06-29 2011-01-05 Thomson Licensing Adaptive residual image coding
US20120127003A1 (en) 2009-08-06 2012-05-24 Youji Shibahara Coding method, decoding method, coding apparatus, and decoding apparatus
US8379718B2 (en) 2009-09-02 2013-02-19 Sony Computer Entertainment Inc. Parallel digital picture encoding
US8711938B2 (en) * 2009-09-04 2014-04-29 Sharp Laboratories Of America, Inc. Methods and systems for motion estimation with nonlinear motion-field smoothing
JP2011066844A (en) 2009-09-18 2011-03-31 Toshiba Corp Parallel decoding device, program, and parallel decoding method of coded data
JPWO2011039931A1 (en) 2009-09-30 2013-02-21 三菱電機株式会社 Image encoding device, image decoding device, image encoding method, and image decoding method
US8705623B2 (en) * 2009-10-02 2014-04-22 Texas Instruments Incorporated Line-based compression for digital image data
EP2312854A1 (en) * 2009-10-15 2011-04-20 Siemens Aktiengesellschaft Method for coding symbols from a digital image sequence
CN102939749B (en) * 2009-10-29 2016-12-28 韦斯特尔电子行业和贸易有限公司 For the method and apparatus processing video sequence
US20110109721A1 (en) 2009-11-06 2011-05-12 Sony Corporation Dynamic reference frame reordering for frame sequential stereoscopic video encoding
US9473792B2 (en) * 2009-11-06 2016-10-18 Texas Instruments Incorporated Method and system to improve the performance of a video encoder
KR20120086232A (en) * 2011-01-25 2012-08-02 (주)휴맥스 Method for encoding/decoding video for rate-distortion optimization and apparatus for performing the same
US9819358B2 (en) 2010-02-19 2017-11-14 Skype Entropy encoding based on observed frequency
US8559511B2 (en) 2010-03-30 2013-10-15 Hong Kong Applied Science and Technology Research Institute Company Limited Method and apparatus for video coding by ABT-based just noticeable difference model
US20120281759A1 (en) * 2010-03-31 2012-11-08 Lidong Xu Power efficient motion estimation techniques for video encoding
JP5377395B2 (en) * 2010-04-02 2013-12-25 日本放送協会 Encoding device, decoding device, and program
JP5393573B2 (en) * 2010-04-08 2014-01-22 株式会社Nttドコモ Moving picture predictive coding apparatus, moving picture predictive decoding apparatus, moving picture predictive coding method, moving picture predictive decoding method, moving picture predictive coding program, and moving picture predictive decoding program
KR101752418B1 (en) 2010-04-09 2017-06-29 엘지전자 주식회사 A method and an apparatus for processing a video signal
US8410959B2 (en) * 2010-04-09 2013-04-02 Qualcomm, Incorporated Variable length codes for coding of video data
US8942282B2 (en) 2010-04-12 2015-01-27 Qualcomm Incorporated Variable length coding of coded block pattern (CBP) in video compression
CN106231337B (en) 2010-04-13 2020-06-19 Ge视频压缩有限责任公司 Decoder, decoding method, encoder, and encoding method
CN103119849B (en) * 2010-04-13 2017-06-16 弗劳恩霍夫应用研究促进协会 Probability interval partition encoding device and decoder
WO2011127961A1 (en) 2010-04-13 2011-10-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e. V. Adaptive image filtering method and apparatus
US20110255594A1 (en) * 2010-04-15 2011-10-20 Soyeb Nagori Rate Control in Video Coding
KR101444691B1 (en) * 2010-05-17 2014-09-30 에스케이텔레콤 주식회사 Reference Frame Composing and Indexing Apparatus and Method
WO2011150109A1 (en) 2010-05-26 2011-12-01 Qualcomm Incorporated Camera parameter- assisted video frame rate up conversion
KR20110135787A (en) 2010-06-11 2011-12-19 삼성전자주식회사 Image/video coding and decoding system and method using edge-adaptive transform
US20110310976A1 (en) * 2010-06-17 2011-12-22 Qualcomm Incorporated Joint Coding of Partition Information in Video Coding
WO2011158225A1 (en) * 2010-06-17 2011-12-22 Mirtemis Ltd. System and method for enhancing images
US8934540B2 (en) 2010-07-20 2015-01-13 Cisco Technology, Inc. Video compression using multiple variable length coding methods for multiple types of transform coefficient blocks
US20130148717A1 (en) 2010-08-26 2013-06-13 Freescale Semiconductor, Inc. Video processing system and method for parallel processing of video data
ES2868133T3 (en) * 2010-09-27 2021-10-21 Lg Electronics Inc Method for block partition and decoding device
JP2012080213A (en) 2010-09-30 2012-04-19 Mitsubishi Electric Corp Moving image encoding apparatus, moving image decoding apparatus, moving image encoding method and moving image decoding method
US9628821B2 (en) * 2010-10-01 2017-04-18 Apple Inc. Motion compensation using decoder-defined vector quantized interpolation filters
US8885704B2 (en) * 2010-10-01 2014-11-11 Qualcomm Incorporated Coding prediction modes in video coding
US8913666B2 (en) 2010-10-01 2014-12-16 Qualcomm Incorporated Entropy coding coefficients using a joint context model
WO2012047304A1 (en) * 2010-10-04 2012-04-12 Vidyo, Inc. Delay aware rate control in the context of hierarchical p picture coding
US9055305B2 (en) * 2011-01-09 2015-06-09 Mediatek Inc. Apparatus and method of sample adaptive offset for video coding
EP2630799A4 (en) * 2010-10-20 2014-07-02 Nokia Corp Method and device for video coding and decoding
US8873627B2 (en) * 2010-12-07 2014-10-28 Mediatek Inc Method and apparatus of video coding using picture structure with low-delay hierarchical B group
US9462280B2 (en) 2010-12-21 2016-10-04 Intel Corporation Content adaptive quality restoration filtering for high efficiency video coding
US8761245B2 (en) * 2010-12-21 2014-06-24 Intel Corporation Content adaptive motion compensation filtering for high efficiency video coding
CN102572419B (en) * 2010-12-28 2014-09-03 深圳市云宙多媒体技术有限公司 Interframe predicting method and device
CN103416062A (en) 2011-01-07 2013-11-27 三星电子株式会社 Video prediction method capable of performing bilateral prediction and unilateral prediction and a device thereof, video encoding method and device thereof, and video decoding method and device thereof
WO2012096176A1 (en) * 2011-01-12 2012-07-19 Panasonic Corporation Methods and apparatuses for encoding and decoding video using multiple reference pictures
US9602819B2 (en) * 2011-01-31 2017-03-21 Apple Inc. Display quality in a variable resolution video coder/decoder system
US8953690B2 (en) * 2011-02-16 2015-02-10 Google Technology Holdings LLC Method and system for processing video data
GB2488159B (en) 2011-02-18 2017-08-16 Advanced Risc Mach Ltd Parallel video decoding
JP2012186763A (en) 2011-03-08 2012-09-27 Mitsubishi Electric Corp Video encoding device, video decoding device, video encoding method, and video decoding method
US9848197B2 (en) 2011-03-10 2017-12-19 Qualcomm Incorporated Transforms in video coding
US9154799B2 (en) * 2011-04-07 2015-10-06 Google Inc. Encoding and decoding motion via image segmentation
US20120262545A1 (en) * 2011-04-18 2012-10-18 Paul Kerbiriou Method for coding and decoding a 3d video signal and corresponding devices
US9247249B2 (en) 2011-04-20 2016-01-26 Qualcomm Incorporated Motion vector prediction in video coding
KR101215152B1 (en) 2011-04-21 2012-12-24 한양대학교 산학협력단 Video encoding/decoding method and apparatus using prediction based on in-loop filtering
US8494290B2 (en) 2011-05-05 2013-07-23 Mitsubishi Electric Research Laboratories, Inc. Method for coding pictures using hierarchical transform units
US8989270B2 (en) * 2011-06-23 2015-03-24 Apple Inc. Optimized search for reference frames in predictive video coding system
NO335667B1 (en) 2011-06-29 2015-01-19 Cisco Systems Int Sarl Method of video compression
EP2727355A1 (en) 2011-06-29 2014-05-07 Motorola Mobility LLC Methods and system for using a scan coding pattern during intra coding
AU2012278484B2 (en) 2011-07-01 2016-05-05 Samsung Electronics Co., Ltd. Method and apparatus for entropy encoding using hierarchical data unit, and method and apparatus for decoding
KR101362696B1 (en) 2011-10-19 2014-02-17 전북대학교산학협력단 Signal transformation apparatus applied hybrid architecture, signal transformation method, and recording medium
KR20130045425A (en) 2011-10-25 2013-05-06 (주)케이테크 Expert recommendation search method based on social ontology
US10452743B2 (en) 2011-11-07 2019-10-22 Vid Scale, Inc. Video and data processing using even-odd integer transforms
US20130223524A1 (en) * 2012-02-29 2013-08-29 Microsoft Corporation Dynamic insertion of synchronization predicted video frames
US9912944B2 (en) * 2012-04-16 2018-03-06 Qualcomm Incorporated Simplified non-square quadtree transforms for video coding
KR101327078B1 (en) * 2012-04-27 2013-11-07 엘지이노텍 주식회사 Camera and method for processing image
KR101677406B1 (en) 2012-11-13 2016-11-29 인텔 코포레이션 Video codec architecture for next generation video
WO2014078068A1 (en) 2012-11-13 2014-05-22 Intel Corporation Content adaptive transform coding for next generation video
US9743091B2 (en) * 2012-12-17 2017-08-22 Lg Electronics Inc. Method for encoding/decoding image, and device using same
WO2014120367A1 (en) 2013-01-30 2014-08-07 Intel Corporation Content adaptive parametric transforms for coding for next generation video

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030113026A1 (en) 2001-12-17 2003-06-19 Microsoft Corporation Skip macroblock coding
US20080112489A1 (en) * 2006-11-09 2008-05-15 Calista Technologies System and method for effectively encoding and decoding electronic information
KR20110129493A (en) * 2007-06-15 2011-12-01 콸콤 인코포레이티드 Adaptive coefficient scanning in video coding
US20120128070A1 (en) * 2009-02-23 2012-05-24 Korean Broadcasting System Video Encoding Method for Encoding Division Block, Video Decoding Method for Decoding Division Block, and Recording Medium for Implementing the Same
KR20110112171A (en) * 2010-04-05 2011-10-12 삼성전자주식회사 Method and apparatus for video encoding, and method and apparatus for video decoding using adaptive coefficient scan order
KR20110112180A (en) * 2010-04-05 2011-10-12 삼성전자주식회사 Method and apparatus for low complexity entropy encoding/decoding

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2952001A4 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9819965B2 (en) 2012-11-13 2017-11-14 Intel Corporation Content adaptive transform coding for next generation video
US9609330B2 (en) 2013-01-30 2017-03-28 Intel Corporation Content adaptive entropy coding of modes and reference types data for next generation video
US9686551B2 (en) 2013-01-30 2017-06-20 Intel Corporation Content adaptive entropy coding of partitions data for next generation video
US9762911B2 (en) 2013-01-30 2017-09-12 Intel Corporation Content adaptive prediction and entropy coding of motion vectors for next generation video
US9787990B2 (en) 2013-01-30 2017-10-10 Intel Corporation Content adaptive parametric transforms for coding for next generation video
US9794569B2 (en) 2013-01-30 2017-10-17 Intel Corporation Content adaptive partitioning for prediction and coding for next generation video
US9794568B2 (en) 2013-01-30 2017-10-17 Intel Corporation Content adaptive entropy coding of coded/not-coded data for next generation video
US10009610B2 (en) 2013-01-30 2018-06-26 Intel Corporation Content adaptive prediction and entropy coding of motion vectors for next generation video
US10924757B2 (en) 2016-10-19 2021-02-16 Sk Telecom Co., Ltd. Apparatus and method for encoding or decoding video
US11962798B2 (en) 2016-10-19 2024-04-16 Sk Telecom Co., Ltd. Apparatus and method for encoding or decoding video using adaptive motion vector resolution

Also Published As

Publication number Publication date
WO2014120367A1 (en) 2014-08-07
CN105052140B (en) 2019-01-15
CN105191309B (en) 2018-08-10
CN104885471A (en) 2015-09-02
RU2015126241A (en) 2017-01-11
US9787990B2 (en) 2017-10-10
CN105453570A (en) 2016-03-30
EP2952001A4 (en) 2016-09-14
WO2015099814A1 (en) 2015-07-02
WO2014120375A3 (en) 2016-03-17
CN104885455B (en) 2019-02-22
EP2996338A2 (en) 2016-03-16
KR20150058324A (en) 2015-05-28
EP3008900A2 (en) 2016-04-20
US9794569B2 (en) 2017-10-17
JP6339099B2 (en) 2018-06-06
WO2014120374A1 (en) 2014-08-07
US20150319442A1 (en) 2015-11-05
US20150229926A1 (en) 2015-08-13
US20160127741A1 (en) 2016-05-05
US9686551B2 (en) 2017-06-20
WO2014120375A2 (en) 2014-08-07
US20150016523A1 (en) 2015-01-15
EP2952001A1 (en) 2015-12-09
US9609330B2 (en) 2017-03-28
EP3013053A2 (en) 2016-04-27
CN105191309A (en) 2015-12-23
CN104737542A (en) 2015-06-24
CN105052140A (en) 2015-11-11
CN105453570B (en) 2020-03-17
US10284853B2 (en) 2019-05-07
CN104885470B (en) 2018-08-07
EP2952003B1 (en) 2019-07-17
EP2952002A4 (en) 2016-09-21
EP2952003A1 (en) 2015-12-09
EP2952004A1 (en) 2015-12-09
CN104718756B (en) 2019-02-15
US20140362922A1 (en) 2014-12-11
KR101770047B1 (en) 2017-08-21
US20180176577A1 (en) 2018-06-21
EP2952003A4 (en) 2016-09-14
EP2951993A4 (en) 2016-09-14
US20170006284A1 (en) 2017-01-05
US20150319441A1 (en) 2015-11-05
EP2952004A4 (en) 2016-09-14
US9794568B2 (en) 2017-10-17
EP3008900A4 (en) 2017-03-15
CN104885467A (en) 2015-09-02
CN104885467B (en) 2018-08-17
WO2014120960A1 (en) 2014-08-07
KR20150055005A (en) 2015-05-20
WO2014120368A1 (en) 2014-08-07
WO2014120987A1 (en) 2014-08-07
US20160277738A1 (en) 2016-09-22
US20150373328A1 (en) 2015-12-24
CN105556964B (en) 2019-02-01
US20150010062A1 (en) 2015-01-08
WO2014120369A1 (en) 2014-08-07
CN104885471B (en) 2019-06-28
US10284852B2 (en) 2019-05-07
KR20170066712A (en) 2017-06-14
JP6286718B2 (en) 2018-03-07
CN105556964A (en) 2016-05-04
EP2951999A4 (en) 2016-07-20
CN104737542B (en) 2018-09-25
US20150036737A1 (en) 2015-02-05
US20160277739A1 (en) 2016-09-22
WO2014120575A1 (en) 2014-08-07
EP2951998A1 (en) 2015-12-09
CN104885470A (en) 2015-09-02
EP2951995A1 (en) 2015-12-09
EP2951994A1 (en) 2015-12-09
US10021392B2 (en) 2018-07-10
US9973757B2 (en) 2018-05-15
WO2014120656A1 (en) 2014-08-07
JP2016506187A (en) 2016-02-25
RU2612600C2 (en) 2017-03-09
BR112015015575A2 (en) 2020-02-04
JP2016514378A (en) 2016-05-19
JP6163674B2 (en) 2017-07-19
EP2951993A1 (en) 2015-12-09
EP2996338A3 (en) 2016-07-06
US9762911B2 (en) 2017-09-12
EP2951994A4 (en) 2016-10-12
KR20150090178A (en) 2015-08-05
US9973758B2 (en) 2018-05-15
EP2951998A4 (en) 2016-09-28
US10009610B2 (en) 2018-06-26
KR20150090206A (en) 2015-08-05
EP2952002A1 (en) 2015-12-09
CN104718756A (en) 2015-06-17
EP2951995A4 (en) 2016-09-21
KR102063385B1 (en) 2020-01-07
JP2016508327A (en) 2016-03-17
CN104885455A (en) 2015-09-02
US20170318297A1 (en) 2017-11-02
KR20150090194A (en) 2015-08-05
EP2951999A1 (en) 2015-12-09
EP3013053A3 (en) 2016-07-20

Similar Documents

Publication Publication Date Title
US10009610B2 (en) Content adaptive prediction and entropy coding of motion vectors for next generation video
US9883198B2 (en) Video codec architecture for next generation video
US9819965B2 (en) Content adaptive transform coding for next generation video

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 14375690

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13873217

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20157016934

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2013873217

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE