OA18315A - Enhanced multiple transforms for prediction residual. - Google Patents

Abstract

Example techniques are described to determine transforms to be used during video encoding and video decoding. A video encoder and a video decoder may select transform subsets that each identify one or more candidate transforms. The video encoder and the video decoder may determine transforms from the selected transform subsets.

Description

[0002] Thîs disclosure relates to video encodîng and decoding.

BACKGROUND [0003] Digital video capabilities can be incorporated into a wide range ofdevices, including digital télévisions, digital direct broadcast Systems, wireless broadcast

1S Systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital caméras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite radio téléphonés, so-called “smart phones,” video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video compression techniques, 20 such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265, High Efiîciency Video Coding (HEVC), and extensions of such standards. The video devices may transmit, receive, encode, décodé, and/or store digital video information more efficiently by implementing such video compression techniques.

[0004] Video compression techniques perform spatial (intra-picture) prédiction and/or temporal (înter-pîcture) prédiction to reduce or remove redundancy inhérent in video sequences. For block-based video coding, a video slice (i.e., a video frame or a portion of a video frame) may be partitioned into video blocks. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prédiction with respect to référencé samples in neighboring blocks in the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prédiction with respect to référencé samples in neighboring blocks in the same picture or temporal prédiction with respect to référencé samples in other référencé pictures. Spatial or temporal prédiction results in a prédictive block for a block to be coded. Residual data represents pixel différences

between the original block to be coded and the prédictive block. An inter-coded block is encoded according to a motion vector that points to a block of référencé sam pies forming the prédictive block, and the residual data indicates the différence between the coded block and the prédictive block. An intra-coded block is encoded according to an intra-coding mode and the residual data. For fûrther compression, the residual data may be transformed from the pixel domain to a transform domain, resulting in residual coefficients, which then may be quantized.

SUMMARY [0005] This disclosure describes techniques for determining transforme to use for generatïng a coefficient block from a transform block as part of video encoding and transforms to use for generatïng a transform block from a coefficient block as part of video decoding. In some examples, a video encoder may détermine a plurality of transform subsets. Lîkewise, a video décoder may détermine a plurality of transform subsets. The video encoder and the video décoder may select a transform subset for the plurality of transform subsets using implîcit techniques that do not necessarily require additional signaling and détermine transforms from the selected transform subsets. In this way, the video encoder and the video décoder may select from a relatively large set of transforms wîth a minimal increase in the amount of information that needs to be signaled.

[0006] In one example, the disclosure describes a method of decoding video data, the method comprising determining a plurality of transform subsets, each subset tdentifÿing one or more candidate transforms, wherein at least one transform subset identifies a plurality of candidate transforms, selecting a first transform subset from the plurality of 25 transform subsets for a left transform for a current coefficient block of the video data, selecting a second transform subset from the plurality of transform subsets fora right transform for the current coefficient block of the video data, determining the left transform from the selected first transform subset, determining the right transform from the selected second transform subset, determining a current transform block based on 30 the left transform, right transform, and the current coefficient block, and reconstructing a video block based on the current transform block and a prédictive block.

[0007] In one example, the disclosure describes a method of encoding video data, the method comprising determining a plurality of transform subsets, each subset identifying

one or more candidate transforms, wherein at least one transform subset identifies a plurality of candidate transforms, selecting a first transform subset from the plurality of transform subsets for a left transform for a current transform block of a video block of the video data, selecting a second transform subset from the plurality of transform subsets for a right transform for the transform block of the video block of the video data, determining the left transform from the selected first transform subset, determining the right transform from the selected second transform subset, determining a current coefficient block based on the left transform, right transform, and the current transform block, and generating a video bitstream that includes information indicative of coefficients of the current coefficient block used for reconstruction of the video block.

[0008] In one example, the disclosure describes a device for video decoding video data, the device comprising a video data memory configured to store the video data and transform subsets, each subset îdentifyîng one or more candidate transforms, wherein at least one transform subset identifies a plurality of candidate transforms, and a video décoder comprising integrated circuitry, the video décoder configured to détermine a plurality of transform subsets from the stored transform subsets, select a first transform subset from the plurality of transform subsets for a left transform for a current coefficient block of the video data, select a second transform subset from the plurality of transform subsets for a right transform for the current coefficient block of the video data, détermine the left transform from the selected first transform subset, détermine the right transform from the selected second transform subset, détermine a current transform block based on the left transform, right transform, and the current coefficient block, and reconstruct a video block based on the current transform block and a prédictive block.

[0009] In one example, the disclosure describes a device for encoding video data, the device comprising a video data memory configured to store the video data and transform subsets, each subset identifying one or more candidate transforms, wherein at least one transform subset identifies a plurality of candidate transforms, and a video encoder configured to détermine a plurality of transform subsets from the stored transform subsets, select a first transform subset from the plurality of transform subsets for a left transform for a current transform block of a video block of the video data, select a second transform subset from the plurality of transform subsets for a right transform for the transform block of the video block of the video data, détermine the left transform from the selected first transform subset, détermine the right transform from

the selected second transform subset, détermine a current coefficient block based on the left transform, right transform, and the current transform block, and generate a video bitstream that includes information indicative ofcoefficients ofthe current coefficient block used for reconstruction of the video block.

[0010] In one example, the disclosure describes a device for decoding video data, the device comprising means for determinîng a plurality of transform subsets, each subset îdentifying one or more candidate transforme, wherein at least one transform subset identifies a plurality of candidate transforme, means for selecting a first transform subset from the plurality of transform subsets for a left transform for a current coefficient block 10 of the video data, means for selecting a second transform subset from the plurality of transform subsets for a right transform for the current coefficient block of the video data, means for determinîng the left transform from the selected first transform subset, means for determinîng the right transform from the selected second transform subset, means for determinîng a current transform block based on the left transform, right transform, and the current coefficient block, and means for reconstructîng a video block based on the current transform block and a prédictive block.

[0011] In one example, the disclosure describes a non-transitory computer-readable storage medium storing instructions that when executed cause a video décoder of a device for video decoding to déterminé a plurality of transform subsets, each subset 20 îdentifying one or more candidate transforms, wherein at least one transform subset identifies a plurality of candidate transforms, select a first transform subset from the plurality of transform subsets for a left transform for a current coefficient block of the video data, select a second transform subset from the plurality of transform subsets for a right transform for the current coefficient block of the video data, détermine the left 25 transform from the selected first transform subset, détermine the right transform from the selected second transform subset, détermine a current transform block based on the left transform, right transform, and the current coefficient block, and reconstruct a video block based on the current transform block and a prédictive block.

[0012] In one example, the disclosure describes a device for encoding video data, the 30 device comprising means for determinîng a plurality of transform subsets, each subset îdentifying one or more candidate transforms, wherein at least one transform subset identifies a plurality of candidate transforms, means for selecting a first transform subset from the plurality of transform subsets for a left transform for a current transform block of a video block ofthe video data, means for selecting a second transform subset from

the plurality of transform subsets for a right transform for the transform block of the video block of the video data, means for determining the ïeft transform from the selected fîrst transform subset, means for determining the right transform from the selected second transform subset, means for determining a current coefficient block 5 based on the left transform, right transform, and the current transform block, and means for generating a video bitstream that includes information indicative of coefficients of the current coefficient block used for reconstruction of the video block.

[0013] In one example the disclosure describes, a non-transitory computer-readable storage medium storing instructions that when executed cause a video encoder of a 10 device for video encoding to détermine a pluralîty of transform subsets, each subset identifying one or more candidate transforme, wherein at least one transform subset identifies a pluralîty of candidate transforms, select a fîrst transform subset from the pluralîty of transform subsets for a left transform for a current transform block of a video block of the video data, select a second transform subset from the pluralîty of 15 transform subsets for a right transform for the transform block of the video block of the video data, détermine the left transform from the selected first transform subset, détermine the right transform from the selected second transform subset, détermine a current coefficient block based on the left transform, right transform, and the current transform block, and generate a video bitstream that includes information indicative of 20 coefficients of the current coefficient block used for reconstruction of the video block.

[0014] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the daims.

BRIEF DESCRIPTION OF DRAWINGS [0015] FIGS. 1A-1E are tables illustrating examples of transform types.

[0016] FIG. 2 is a block diagram illustrating an example video coding System that may utilize the techniques described in this disclosure.

[0017] FIG. 3 is a conceptual diagram illustrating an example of a transform scheme 30 based on residual quadtree in high efficiency video coding (HEVC).

[0018] FIG. 4 is a conceptual diagram illustrating an example of a coefficient scan based on coding group in HEVC.

[0019] FIG. 5 is a block diagram illustrâting an exemple video encoder that may implement the techniques described in this disclosure.

[0020] FIG. 6 Îs a block diagram illustrating an example video décoder that may implement the techniques described in this disclosure.

[0021] FIG. 7 is a conceptual diagram illustrâting an example of overiapped block motion compensation (OBMC) in accordance with the video coding process defined in the ITU-T H.263 standard.

[0022] FIGS. SA and 8B are conceptual diagrams illustratîng portions of a block for OBMC.

[0023] FIG. 9 is a flowchart illustratîng an example method of decoding video data.

[0024] FIG. 10 is a flowchart illustratîng an example method of encoding video data.

DETAILED DESCRIPTION [0025] This disclosure is related to multiple transforme applied for Intra or Inter 15 prédiction residual. The techniques may be used in the context of advanced video codées, such as extensions of the high efificiency video coding (HEVC) standard or the next génération of video coding standards.

[0026] In video coding, a video encoder generates a residual block by subtracting sample values of a current block from sample values of a prédictive block. The video 20 encoder divides the residual block into one or more transform blocks and applies a transform (e.g., a discrète frequency transform such as a discrète cosine transform (DCT)) to the one or more transform blocks to transform the residual values in the one or more transform blocks from the pixel domain to the frequency domain. In the frequency domain, the transformed blocks are referred to as coefficient blocks that 25 indude one or more transform coefficient values.

[0027] During decoding, a video décoder performs the reciprocal process. For instance, the video décoder applies an inverse-transform to a coefficient block to transform the coefficient block to a transform block (e.g., transform from frequency domain to pixel domain). The transform block is one block of a residual block, and the video décoder 30 adds residual values of the residual block to the sample values of the prédictive block to reconstruct the current block.

[0028] Only for ease of description, this disclosure describes the video encoder and the video décoder as determining a transform used for the encoding and decoding process, respectively. However, it should be understood that the video encoder applies the

transform to a transform block to generate a coefficient block and that the video décoder applies an inverse of the transform to the coefficient block to reconstruct the transform block. Accordingly, the transform that the video décoder applies is the inverse of the transform that the video encoder applies. Therefore, in this disclosure, when the video 5 décoder is described as determining a transform and/or applying a transform, it should be understood that the video décoder is determining a transform that is the inverse of the transform determined by the video encoder and/or that the video décoder is applying a transform that is the inverse of the transform applied by the video encoder.

[0029] Thîs disclosure describes example techniques for determining the transform that 10 is applied to a transform block of residual values for encoding transform coefficients or applied to a coefficient block of transform coefficients for decodîng residual values.

For instance, the video encoder and the video décoder may each construct a plurality of transform subsets, each transform subset identifies a plurality of candidate transforme. Candidate transforme refer to different types of transforme such as different types of 15 DCTs and different types of discrète sine transforme (DSTs). The video encoder and the video décoder select transform subset(s) and détermine transforms from the selected transform subset(s) that are used for determining a coefficient block from a transform block for video encoding or a transform block from a coefficient block for video decodîng.

[0030] In this way, the video encoder and the video décoder may détermine which transforms to use from a larger set of candidate transforms, allowing for better adaptation to the varying statist ics of the transform block without overly burdening the bitstream bandwidth. For instance, some techniques constraîn how many transforms are available, which may resuit in poor coding performance because the statistics of the transform block are such that none of the available transforms perform well. There may be other better transforms but these transforms are unavailable due to the constraints. [0031] In the techniques described in this disclosure, because more transforms are available, the video encoder and the video décoder may use a transform that provides better coding performance than would be possible with a limited set of transforms.

Furthermore, as described in more detail, signaling overhead, used to indicate which transform is to be used, is kept low so that coding gains can be achieved while having more transforms available and keeping the impact on bandwidth low.

[0032] For example, rather than relying on signaled information in the bitstream, the video décoder may select which transform subset(s) to use based on împlicit techniques

such as based on intra-predictîon mode, location of transform block, etc. The video décoder may then détermine which transform (s) to use from the selected transform subset(s) based possîbly on one or more transform subset indices, for respective ones of the selected transform subset(s), signaled in the bitstream or other factors including but 5 not limited to number of nonzero coefficients, sum of nonzero coefficients, or position of nonzero coefficients in a coefficient block.

[0033] Even where the transform subset index is signaled for respective transform subset(s), the signaling overhead may be kept low because the index value spans only the range of the transform subset rather than spanning ail possible transforms. For 10 instance, assume there are up to 16 possible transforms, and that a transform subset includes three candidate transforms. In this case, the index value will range from 0 to 2, whereas an index into a list of ail transforms would range from 0 to 15. Signaling smaller values such as 0 to 2 may require fewer bits than signaling larger values.

[0034] Prior to describîng the manner in which transform subsets are constructed and selected, the following describes video coding standards, DCTs and DSTs in general, different types of DCTs and DSTs, and some existing DCT and DST techniques. The disclosure then describes some problems in existing techniques, followed by example techniques that may overcome the problems.

[0035] Video coding standards include ITU-T H.261, ISO/ŒC MPEG-1 Visual, ITU-T 20 H.262 or ISO/IEC MPEG-2 Visual, ITU-T H.263, ISO/IEC MPEG-4 Visual and ITU-T

H.264 (also known as ISO/IEC MPEG-4 AVC), including its Scalable Video Coding (SVC) and Multi-view Video Coding (MVC) extensions. In addition, a new video coding standard, namely High Efficiency Video Coding (HEVC), has recently been developed by the Joint Collaboration Team on Video Coding (JCT-VC) of ITU-T Video 25 Coding Experts Group (VCEG) and ISO/IEC Motion Picture Experts Group (MPEG).

The final HEVC draft spécification, and referred to as HEVC WD hereinafter, is available from http://phenix.intevry.fr/jct/doc_end_user/documents/14_Vienna/wgll/JCTVC-N1003-vl.zip. The final draft of the HEVC standard is: ITU-T H.265, Sériés H: Audiovisual and Multimedia

Systems, Infrastructure of audiovisual services - Coding of moving video, Advanced video coding for generic audiovisual services, The International Télécommunication Union, October 2014, and is available from http://www.itu.int/rec/T-REC-H.265201410-I/en.

[0036] The following is a description of discrète sine and cosîne transforme. Transform indicates the process of deriving an alternative représentation of the input signal. For example, the transform converts values from the pixel domain to the frequency domain (e.g., in video encodîng) or from frequency domain to pixel domain (e.g., in video decoding). Given an Appoint vectorx=[xo,xi,..., xn-i]^t and a set ofgiven vectors {¢0, ¢1,.... ÿM-i}.xcan be approximated or exactly represented using a linear combination of ÿo, ¢1,..., ¢^4-1, which can be formulated as follows,

M-l ΐ = Σ/<·Φ< l»O where î can be an approximation or équivalent ofx, vector f = ..,/vf-i] is called the transform coefficient vector and {¢0, ¢1,..., ¢54-1} are the transform basis vectors.

[0037] In the scénario of video coding, transform coefficients are roughly noncorrelated and sparse, i.e., the energy of the input vector x is compacted only on a few transform coefficients, and the remaining majority transform coefficients are typically close to 0. For instance, when a video encoder transforme a transform block to a coefficient block, the nonzero coefficient values in the coefficient block tend to be grouped together at a top-left corner of the coefficient block, and a majority of the coefficient values are zéro. The nonzero coefficients grouped near the top-left comer of the coefficient block reflect low frequency components, whereas coefficient values near the bottom-right comer of the coefficient block, which tend to be zéro, reflect high frequency components.

[0038] Given the spécifie input data, the optimal transform in terms of energy compaction is the so-called Karhunen-Loeve transform (KLT), which uses the eigen vectors of the covariance matrix of the input data as the transform basis vectors. Therefore, KLT is actually a data-dependent transform and does not hâve a general mathematical formulation. However, under certain assumptions, e.g., the input data forms a fïrst-order stationary Markov process, it has been proven in the literature that the corresponding KLT is actually a member of the sinusoïdalfamily of unitary transforms, which is described in Jaîn, A.K., A sinusoïdal family of unitary transforms, IEEE Trans. on Pattern Analysis and Machine Intelligence, 1,356,1979. The sinusoïdalfamily of unitary transforms indicates transforms usîng transform basis vectors formulated as follows:

^)=Λ·«^Λ,+Β·β·^Λ·

ΙΟ where e is the base ofthe naturel logarithm approximately equal to 2.71828, A, B, and o are complex in general, and dépend on the value of m.

[0039] Several well-known transforms including the discrète Fourier, cosine, sine, and the KLT (for first-order stationary Markov processes) are members of thîs sinusoïdal family ofunitary transforms. According to S. A. Martucci, Symmetric convolution and the discrète sine and cosine transforms,** IEEE Trans. Sig. Processing SP-42, 1038-1051 (1994), the complété discrète cosine transform (DCT) and discrète sine transform (DST) families include totally 16 transforms based on different types, i.e., different values of J, B, and o, and a complété définition of the different types of DCT and DST are given below.

[0040] Assume the înput jV-point vector is denoted as x=[ xo, xi,..., xn-i]^t, and it is transformed to another Appoint transform coefficient vector denoted asy=[yt),yi,...,yNt]^T by multiplying a matrix, the process of which can be further illustrated according to one of the following transform formulation, wherein k ranges from 0 through N-l, inclusive:

DCT Type-I (DCT-1):

DCT Type-Π (DCT-2):

DCT Type-III (DCT-3):

DCT Type-IV (DCT-4):

DCT Type-V (DCT-5):

Λ = ^Σ-·» ^5^ros (Sî) · «Ό ' *« *· · where w₀ = fe' ^^{n = 0} .Wj =[vr ^lf^{k = 0} t1, otheriwse l1, otheriwse

OCT Type-VI (DCT-6):

y»=^{Σ ;}° J^^{cos w}° ·^w* ^x» · where>v₀ = fè· = ^V* = °

11, otheriwse t1, otheriwse

DUT Type-VII (DCT-7):

y*

ifn = 0 ifk-N — 1 ' ,_W1 = ]V2' otheriwse 11, otheriwse where w₀ = < V2’

11.

DCT Type-VIII (DCT-8):

N+0.5

DST Type-I (DST-1):

(n+l)(fc+l) N+l

DST Type-II (DST-2):

where w₀

ifk = N-l otherlwse

DSTType-III (DST<3):

wbe^.-fè·

11, otheriwse

DST Type-IV (DST-4):

DST Type-V (DST-5):

DST Type-VI (DST-6):

DST Type-VII (DST-7):

DST Type-VIII (DST-8):

y* . /ir (n+0.5)-(k+0.5n ...

^sinl------J-^o-wrXn, . (4=, ifn = N — l where w₀ = < V2 ' ,

11, otheriwse = ià- ⁱf^k=N~¹

11, otheriwse [0041] The above provides examples of different DCT and DST types, all-in-all there are 16 transform types. The transform type is specified by the mathematica! formulation ofthe transform basîs fonction. The transform type and the transform size should not be confused. The transform type refers to basis fonction, whereas the

transform size refers to the size of the transform. For instance, a 4-point DST-VII and 8-point DST-VII hâve the same transform type, regardless ofthe value of N (e.g., 4point or 8-point).

[0042] Without loss of generality, ail the above transform types can be represented using the below generalized formulation:

Ym ⁼ Ση·0 ^m.n ’ ^n» where T is the transform matrix specîfied by the définition of one certain transform, e.g., DOT Type-I - DCT Type-VIII, or DST Type-I - DST Type-VIII, and the row vectors ofT, e.g., [Ti.o, Ti.i, Ty,..., Tî.n-i] are the i**¹ transform basîs vectors. A transform applied on the Y-point input vector is called an TV-point transform. [0043] It is also noted that, the above transform formulations, which are applied on the 1-D input datax, can be represented in matrix multiplication form as below y = T-x where Tindîcates the transform matrix, x indicates the input data vector, and .y indicates 15 the output transform coefficients vector.

[0044] For instance, the video encoder may perform the matrix multiplication y = T · x to generate the transform coefficient vector. The video décoder may perform the inverse matrix multiplication to generate the transform vector from the transform coefficient vector.

[0045] The transforms as întroduced above are applied on 1 -D input data, and transforms can be also extended for 2-D input data sources. Supposing X is an input MxN data array. The typical methods of applying transform on 2-D input data include the separable and non-separable 2-D transforms.

[0046] A separable 2-D transform applies 1-D transforms for the horizontal and vertical 25 vectors of X sequentially, formulated as below:

Y = C-X-R^T where C and R dénotés the given MxM and NxN transform matrices, respectively. [0047] From the formulation, it can be seen that C applies 1-D transforms for the column vectors ofX, while R applies 1-D transforms for the row vectors ofX. In the later part of this disclosure, for simplicity dénoté C and R as left (vertical) and right (horizontal) transforms and they both form a transform pair. There are cases when C is equal to R and is an orthogonal matrix. In such a case, the separable 2-D transform is determined by just one transform matrix.

[0048] A non-separable 2-D transform first rcorganized ail the éléments of X into a single vector, namely X’, by doing the following mathematical mapping as an example: ^(i-W+Λ ⁼ Xij [0049] Then a l-D transform T* is applied for X’ as below:

Y = T'-X where T’is an (M*N)x(M*N) transform matrix.

[0050] In video coding, separable 2-D transforms are always applied since it requires much less operation (addition, multiplication) counts as compared to l-D transform. As described in more detail below, this disclosure describes example techniques with which 10 a video encoder and a video décoder select the left and right transforms.

[0051] For instance, the video encoder and the video décoder may détermine a plurality of transform subsets, each transform subset identi fying a plurality of candidate transforms. As an example ofthe 16 possible transforms (e.g., DCT-I to DCT-8 and DST-1 to DST-8), the video encoder and the video décoder may détermine three 15 transform subsets and each of the transform subsets includes two or more of the 16 transforms. The video encoder and the video décoder may select one of the three transform subsets and détermine the left transform (e.g., C) from the selected transform subset and select one of the three transform subsets and détermine the right transform (e.g., R) from the selected transform subset. The selected transform subsets may be 20 different subsets or the same subsets.

[0052] The following is a description of transform types applied in HEVC. In conventional video codées, such as H.264/AVC, an integer approximation ofthe 4-point and 8-point Discrète Cosine Transform (DCT) Type-Π is always applied for both Intra and Inter prédiction residual. Intra prédiction residual refers to the residual from intra25 prédiction and Inter prédiction residual refers to the residual from inter-prediction. The residual, inter-predication, and intra-prediction are ail described in more detail below. In general, the residual block is divided into a plurality of transform blocks. In video encoding, the transforms are applied to each ofthe transform blocks to generate coefficient blocks. In video decodîng, the transforms are applied to each of the 30 coefficient blocks to generate the transform blocks and reconstruct the residual block.

[0053] To better accommodate the various statistics of residual samples, more flexible types of transforms other than DCT Type-II are utilized in the new génération video codée. For example, in HEVC, an integer approximation of the 4-point Type-VII

Discrète Sine Transform (DST) is utilized for Intra prédiction residual, which is both theoretically proved and experimentally validated that DST Type-VII is more efficient than DCT Type-II for residuals vectors generated along the Intra prédiction directions, e.g., DST Type-VII is more efficient than DCT Type-II for row residual vectors generated by the horizontal Intra prédiction direction. See, for example, J. Han, A.

Saxena and K. Rose, “Towards jointly optimal spatial prédiction and adaptive transform in video/image coding,” IEEE International Conférence on Acoustics, Speech and Signal Processing (ICASSP), March 2010, pp. 726-729.

[0054] In HEVC, an integer approximation of4-point DST Type-VII is applied only for 4x4 luma Intra prédiction residual blocks (luma intra prédiction residual blocks are described in more detail below). The 4-poînt DST-VII used in HEVC is shown in FIG. IA.

[0055] In HEVC, for residual blocks that are not 4x4 luma Intra prédiction residual blocks, integer approximations of the 4-point, 8-poînt, 16-point and 32-point DCT

Type-II are also applied. FIG. IB illustrâtes an example of the 4-point DCT-Π; FIG. IC illustrâtes an example of the 8-point DCT-Π; FIG. 1D illustrâtes an example of the 16point DCT-II; and FIG. 1E illustrâtes an example of the 32-point DCT-II. FIGS. 1A-1E îllustrate examples ofdifferently sized DCTs of type II, and like FIGS. 1A-1E, there are examples of N-point DCTs and DSTs of different types.

[0056] FIG. 2 is a block diagram illustrating an example video coding System 10 that may utilize the techniques of this disclosure. As used herein, the term “video coder” refers generically to both video encoders and video decoders. In this disclosure, the ternis “video coding” or “coding” may refer generically to video encoding or video decoding. Video encoder 20 and video décoder 30 of video coding System 10 represent examples of devices that may be configured to perform techniques for enhanced multiple transforms for prédiction residual in accordance with various examples described in this disclosure.

[0057] As shown in FIG. 1, video coding System 10 includes a source device 12 and a destination device 14. Source device 12 generates encoded video data. Accordingly, 30 source device 12 may be referred to as a video encoding device or a video encoding apparatus. Destination device 14 may décodé the encoded video data generated by source device 12. Accordingly, destination device 14 may be referred to as a video decoding device or a video decoding apparatus. Source device 12 and destination device 14 may be examples of video coding devices or video coding apparatuses.

[0058] Source device 12 and destination device 14 may comprise a wide range of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, téléphoné handsets such as so-ca!led “smart” phones, télévisions, caméras, display devices, digital media players, video gam ing consoles, in-car computers, or the l ike.

[0059] Destination device 14 may receive encoded video data from source device 12 via a channel 16. Channel 16 may comprise one or more media or devices capable of moving the encoded video data from source device 12 to destination device 14. In one example, channel 16 may comprise one or more communication media that enable source device 12 to transmit encoded video data directly to destination device 14 in realtime. In this example, source device 12 may modulate the encoded video data according to a communication standard, such as a wireless communication protocol, and may transmit the modulated video data to destination device 14. The one or more communication media may include wireless and/or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part ofa packet-based network, such as a local area network, a wïde-area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other equipment that facilitate communication from source device 12 to destination device 14.

[0060] In another example, channel 16 may include a storage medium that stores encoded video data generated by source device 12. In this example, destination device 14 may access the storage medium, e.g., via disk access orcard access. The storage medium may include a variety of local ly-accessed data storage media such as Blu-ray dises, DVDs, CD-ROMs, flash memory, or other suitable digital storage media for storing encoded video data.

[0061] In a fnrther example, channel 16 may include a file server or another întermediate storage device that stores encoded video data generated by source device 12. In this example, destination device 14 may access encoded video data stored at the file server or other întermediate storage device via streaming or download. The file server may be a type of server capable of storing encoded video data and transmitting the encoded video data to destination device 14. Example file servers include web servers (e.g., for a website), file transfer protocol (FTP) servers, network attached storage (NAS) devices, and local disk drives.

[0062] Destination device 14 may access the encoded video data through a standard data connection, such as an Internet connection. Example types of data connections may include wireless channels (e.g., Wi-Fi connections), wired connections (e.g., DSL, cable modem, etc.), or combinations of both that are suitable for accessing encoded 5 video data stored on a file server. The transmission of encoded video data from the file server may be a streaming transmission, a download transmission, or a combination of both.

[0063] The techniques of this disclosure are not limited to wireless applications or settings. The techniques may be applied to video coding in support of a variety of 10 multimedia applications, such as over-the-air télévision broadcasts, cable télévision transmissions, satellite télévision transmissions, streaming video transmissions, e.g., via the Internet, encoding of video data for storage on a data storage medium, decoding of video data stored on a data storage medium, or other applications. In some examples, video coding System 10 may be configured to support one-way or two-way video 15 transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.

[0064] Video coding System 10 illustrated in FIG. 2 is merely an example and the techniques of this disclosure may apply to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between the 20 encoding and decoding devices. In other examples, data is retrieved from a local memory, streamed over a network, or the like. A video encoding device may encode and store data to memory, and/or a video decoding device may retrieve and décodé data from memory. In many examples, the encoding and decoding is performed by devices that do not communicate with one another, but simply encode data to memory and/or 25 retrieve and décodé data from memory.

[0065] In the example of FIG. 2, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. In some examples, output interface 22 may include a modulator/demodulator (modem) and/or a transmitter. Video source 18 may include a video capture device (e.g., a video caméra), a video archive containing prevîously-captured video data, a video feed interface to receive video data from a video content provider, and/or a computer graphies System for generatïng video data, or a combination of such sources of video data.

[0066] Video encoder 20 may encode video data from video source 18. In some examples, source device 12 directly transmits the encoded video data to destination

device 14 vta output interface 22. In other examples, the encoded video data may also be stored onto a storage medium or a file server for later access by destination device 14 for decoding and/or playback.

[0067] In the example of FIG. 2, destination device 14 includes an input interface 28, a 5 video décoder 30, and a display device 32. In some examples, input interface 28 includes a receiver and/or a modem. Input interface 28 may receive encoded video data over channel 16. Display device 32 may be integrated with or may be extemal to destination device 14. In general, display devîce 32 displays decoded video data. Display device 32 may comprise a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0068] Video encoder 20 and video décoder 30 each may be implemented as any of a varîety of suîtable circuîtry, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable 15 gâte arrays (FPGAs), discrète logic, hardware, or any combinations thereof. Ifthe techniques are implemented partially in software, a device may store instructions for the software in a suitable, non-transitory computer-readable storage medium and may execute the instructions in hardware using one or more processors to perform the techniques ofthîs disclosure. Any ofthe foregoing (including hardware, software, a 20 combination of hardware and software, etc.) may be considered to be one or more processors. Each of video encoder 20 and video décoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (CODEC) in a respective device.

[0069] Thîs disclosure may generally refer to video encoder 20 “signaling” or 25 “transmitting” certain information to another device, such as video décoder 30. The term “signaling” or “transmitting” may generally refer to the communication of syntax éléments and/or other data used to décodé the compressed video data. Such communication may occur in real- or near-real-time. Altemately, such communication may occur over a span of time, such as might occur when storing syntax éléments to a 30 computer-readable storage medium in an encoded bitstream at the time of encoding, which then may be retrieved by a decoding device at any time after being stored to this medium.

[0070] In some examples, video encoder 20 and video décoder 30 operate according to a video compression standard, such as HEVC standard mentioned above, extensions of

HEVC, or possibly a next génération of video coding standards in development. For ease of understandîng only, the following provides some information regarding the HEVC standard. However, the techniques described in this disclosure should not be consîdered limited to the HEVC standard.

[0071] In HEVC and other video coding standards, a video sequence typically includes a sériés of pictures. Pictures may also be referred to as “frames.” A picture may include three sample arrays, denoted Su Sa> and Scr. Sl is a two-dimensîonal array (i.e., a block) of luma samples. Scb is a two-dimensional array of Cb chrominance samples. Scr îs a two-dimensional array ofCr chrominance samples. Chrominance samples may also be referred to herein as “chroma” samples. In other instances, a picture may be monochrome and may only include an array of luma samples.

[0072] To generale an encoded représentation of a picture, video encoder 20 may generate a set of coding tree units (CTUs). Each of the CTUs may be a coding tree block of luma samples, two corresponding coding tree blocks of chroma samples, and syntax structures used to code the samples ofthe coding tree blocks. A coding tree block may be an NxN block of samples. A CTU may also be referred to as a “tree block” or a “largest coding unît” (LCU). The CTUs of HEVC may be broadly . analogous to the macroblocks of other standards, such as H.264/AVC. However, a CTU is not necessarily limited to a particular size and may include one or more coding units (CUs). A slice may include an integer number of CTUs ordered consecutively in the raster scan.

[0073] To generate a coded CTU, video encoder 20 may recursively perform quad-tree partitioning on the coding tree blocks of a CTU to divide the coding tree blocks into coding blocks, hence the name “coding tree units.” A coding block is an NxN block of 25 samples. A CU may be a coding block of luma samples and two corresponding coding blocks of chroma samples of a picture that has a luma sample array, a Cb sample array and a Cr sample array, and syntax structures used to code the samples of the coding blocks. Video encoder 20 may partition a coding block of a CU into one or more prédiction blocks. A prédiction block may be a rectangular (i.e., square or non-square) 30 block of samples on which the same prédiction is applied. A prédiction unit (PU) of a

CU may be a prédiction block of luma samples, two corresponding prédiction blocks of chroma samples of a picture, and syntax structures used to predict the prédiction block samples. Video encoder 20 may generate prédictive luma, Cb and Cr blocks for luma, Cb and Cr prédiction blocks of each PU ofthe CU.

[0074] Video encoder 20 may use intra prédiction or inter prédiction to generate (e.g., détermine) the prédictive blocks for a PU. If video encoder 20 uses intra prédiction to generate the prédictive blocks of a PU, video encoder 20 may generate the prédictive blocks of the PU based on decoded samples ofthe picture associated with the PU.

[0075] If video encoder 20 uses inter prédiction to generate (e.g., détermine) the prédictive blocks of a PU, video encoder 20 may generate the prédictive blocks of the PU based on decoded samples of one or more pictures other than the picture associated with the PU. Video encoder 20 may use uni-prediction or bî-prediction to generate the prédictive blocks of a PU. When video encoder 20 uses uni-prediction to generate the prédictive blocks for a PU, the PU may hâve a single motion vector (MV). When video encoder 20 uses bî-prediction to generate the prédictive blocks for a PU, the PU may hâve two MVs.

[0076] After video encoder 20 generates prédictive luma, Cb and Cr blocks for one or more PUs of a CU, video encoder 20 may generate a luma residual block for the CU.

Each sam pie in the CU’s luma residual block indicates a différence between a luma sample in one of the CU’s prédictive luma blocks and a corresponding sample in the CU’s original luma coding block. In addition, video encoder 20 may generate a Cb residual block for the CU. Each sample in the CU’s Cb residual block may indicate a différence between a Cb sample in one of the CU’s prédictive Cb blocks and a corresponding sample in the CU’s original Cb coding block. Video encoder 20 may also generate a Cr residual block for the CU. Each sample in the CU’s Cr residual block may indicate a différence between a Cr sample in one of the CU’s prédictive Cr blocks and a corresponding sample in the CU’s original Cr coding block.

[0077] Furthermore, video encoder 20 may use quad-tree partitioning to décomposé the luma, Cb and Cr residual blocks of a CU into one or more luma, Cb and Cr transform blocks. A transform block may be a rectangular block ofsamples on which the same transform is applîed. A transform unit (TU) of a CU may be a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax structures used to transform the transform block samples. Thus, each TU ofa CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. The luma transform block associated with the TU may be a sub-block of the CU’s luma residual block. The Cb transform block may be a sub-block ofthe CU’s Cb residual block. The Cr transform block may be a sub-block of the CU’s Cr residual block.

[0078] Video encoder 20 may apply one or more transforme to a luma transform block of a TU to generate a luma coefficient block for the TU. A coefficient block may be a two-dimensional array of transform coefficients. A transform coefficient may be a scalar quantity. Video encoder 20 may apply one or more transforms to a Cb transform 5 block of a TU to generate a Cb coefficient block for the TU. Video encoder 20 may apply one or more transforms to a Cr transform block of a TU to generate a Cr coefficient block for the TU. As described in more detail, this disclosure describes example ways in which video encoder 20 détermines the transforms to use for generating the coefficient blocks.

[0079] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block or a Cr coefficient block), video encoder 20 may quantize the coefficient block Quantization generally refers to a process in which transform coefficients are quantized to possibly reduce the amount of data used to represent the transform coefficients, providing further compression. After video encoder 20 quantizes a coefficient block, video encoder 20 may entropy encode syntax éléments indïcating the quantized transform coefficients. For example, video encoder 20 may perform Context-Adaptive Binary Arithmetic Coding (CABAC) on the syntax éléments indicating the quantized transform coefficients. Video encoder 20 may output the entropy-encoded syntax éléments in a bitstream.

[0080] Video encoder 20 may output a bitstream that includes the entropy-encoded syntax éléments. The bitstream may înclude a sequence of bits that forms a représentation of coded pictures and associated data. The bitstream may comprise a sequence of network abstraction layer (NAL) units. Each of the NAL units includes a NAL unit header and encapsulâtes a raw byte sequence payload (RBSP). The NAL unit header may include a syntax element that indicates a NAL unit type code. The NAL unit type code specified by the NAL unit header of a NAL unit indicates the type of the NAL unit. A RBSP may be a syntax structure containing an integer number of bytes that is encapsulated within a NAL unit. In some instances, an RBSP includes zéro bits. [0081] Different types of NAL units may encapsulate different types of RBSPs. For example, a first type of NAL unit may encapsulate an RBSP for a picture parameter set (PPS), a second type of NAL unit may encapsulate an RBSP for a coded slice, a third type of NAL unit may encapsulate an RBSP for SEI, and so on. NAL units that encapsulate RBSPs for video coding data (as opposed to RBSPs for parameter sets and SEI messages) may be referred to as video coding layer (VCL) NAL units.

[0082] Video décoder 30 may receive a bitstream generated by video encoder 20. In addition, video décoder 30 may parse the bitstream to décodé syntax éléments from the bitstream. Video décoder 30 may reconstruct the pictures of the video data based at least in part on the syntax éléments decoded from the bitstream. The process to reconstruct the video data may be generally recîprocal to the process performed by video encoder 20. For instance, video décoder 30 may use MVs of PUs to détermine prédictive blocks for the PUs of a current CU. In addition, video décoder 30 may inverse quantïze transform coefficient blocks associated with TUs of the current CU. [0083] Video décoder 30 may perform inverse transforms on the transform coefficient 10 blocks to reconstruct transform blocks associated with the TUs ofthe current CU. This disclosure describes example techniques for the way in which video décoder 30 détermines the transforms that are used to perform the inverse transforms on the transform coefficient blocks.

[0084] Video décoder 30 may reconstruct the coding blocks of the current CU by t5 adding the samples of the prédictive blocks for PUs of the current CU to corresponding samples of the transform blocks ofthe TUs ofthe current CU. By reconstruct!ng the coding blocks for each CU of a picture, video décoder 30 may reconstruct the picture. [0085] As described above, a CU includes one or more TUs. The following describes transform scheme based on residual quadtree in HEVC. To adapt the various characteristics of the residual blocks, a transform coding structure using the residual quadtree (RQT) is applied in HEVC, which is briefiy described in http://www.hhi.fraunhofer.de/fields-of-competence/image-processing/researchgroups/image-video-coding/hevc-high-efficiency-video-coding/transform-coding-usingthe-residual-quadtree-rqt.html.

[0086] As described above, each picture is divided into CTUs, which are coded in rester scan order for a spécifie tile or slice. A CTU is a square block and représenta the root of a quadtree, i.e., the coding tree. The CTU size may range from 8*8 to 64x64 luma samples, but typically 64*64 is used. Each CTU can be further split into smaller square blocks called coding units (CUs). After the CTU is split recursîvely into CUs, each CU is further divided into prédiction units (PU) and transform units (TU). The partitioning of a CU into TUs is carrîed out recursîvely based on a quadtree approach, therefore the residual signal of each CU is coded by a tree structure namely, the residual quadtree (RQT). The RQT allows TU sizes from 4^X4 up to 32^x32 luma samples.

[0087] FIG. 3 shows an examplc where a CU includes 10 TUs, labeled with the letters “a” to “j, and the corresponding block partitioning. Each node ofthe RQT is actually a transform unît (TU). The individual TUs are processed in depth-first tree traversai order, which îs illustrated in FIG. 3 as alphabetîcal order, whîch follows a recursive Z5 scan with depth-first traversai. The quadtree approach enables the adaptation ofthe transform to the varying space-frequency characteristics ofthe residual signal.

Typically, larger transform block sizes, which hâve larger spatial support, provide better frequency resolution. However, smaller transform block sizes, which hâve smaller spatial support, provide better spatial resolution. The trade-off between the two, spatial 10 and frequency resolutions, is chosen by the encoder mode decision (e.g., by video encoder 20), for example, based on rate-distortion optimization technique. The ratedistortion optimization technique calculâtes a weighted sum of coding bits and reconstruction distortion, i.e., the rate-distortion cost, for each coding mode (e.g., a spécifie RQT splitting structure), and select the coding mode with least rate-distortion 15 cost as the best mode.

[0088] Three parameters are defined in the RQT: the maximum depth of the tree, the minimum allowed transform size, and the maximum allowed transform size. The minimum and maximum transform sizes can vary within the range from 4*4 to 32x32 samples, which correspond to the supported block transforms mentioned in the previous 20 paragraph. The maximum allowed depth ofthe RQT restricts the number ofTUs. A maximum depth equal to zéro means that a CB (coding block) cannot be split any further if each included TB (transform block) reaches the maximum allowed transform size, e.g., 32x32.

[0089] Ail these parameters înteract and influence the RQT structure. Consider a case, 25 in which the root CB size is 64x64, the maximum depth is equal to zéro and the maximum transform size is equal to 32*32. In this case, the CB has to be partitioned at least once, sînee otherwise it would lead to a 64*64 TB, which is not allowed. In HEVC, larger size transforms, e.g., 64x64 transforms, are not adopted maînly due to its limited benefit considering and relatively high complexîty for relatively smaller resolution videos.

[0090] The RQT parameters, i.e., maximum RQT depth, minimum and maximum transform size, are transmitted in the bitstream at the sequence parameter set level. Regarding the RQT depth, different values can be specified and signaled for intra and

inter coded CUs (i.e., intra-predicted encoded CUs or inter-predicted decoded CUs or intra-predicted encoded CUs or inter-predicted CUs).

[0091] The quadtree transform is applied for both Intra and Inter residual blocks. Typically the DCT-II transform of the same size of the current residual quadtree partition is applied for a residual block. However, if the current residual quadtree block is 4x4 and is generated by Intra prédiction, the above 4x4 DST-VII transform is applied. [0092] The following describes coefficient coding in HEVC. Regardless of the TU size, the residual ofthe transform unit is coded with non-overlapped coefficient groups (CG), and each contains the coefficients of a 4x4 block of a TU. For example, a 32x32

TU has totally 64 CGs, and a 16x16 TU has totally 16 CGs. The CGs inside a TU are coded according to a certain pre-defined scan order. When coding each CG, the coefficients inside the current CG are scanned and coded according to a certain predefined scan order for 4x4 block. FIG. 4 illustrâtes the coefficient scan for an 8x8 TU containing 4 CGs.

[0093] For each color component, one flag may be firstly signaled to indicate whether current transform unit has at least one non-zero coefficient. If there is at least one nonzero coefficient, the position ofthe last significant coefficient in the coefficient scan order in a transform unit is then expïicïtly coded with a coordination relative to the topleft comer of the transform unit. The vertical or horizontal component of the coordination is represented by its prefix and suffix, wherein prefix is binarized with truncated rice (TR) and suffix is binarized with fixed length.

[0094] last_sig_coefT_x_preflx spécifiés the prefix ofthe column position ofthe last significant coefficient in scanning order within a transform block. The values of Jast_sig_coefT_x_prefix shall be In the range of0 to ( log2TrafoSîze « I ) - I, 25 inclusive.

[0095] last_sig_coefT_y_prefix spécifiés the prefix ofthe row position ofthe last significant coefficient in scanning order within a transform block. The values of last_sig_coeff_y_prefix shall be in the range of 0 to ( log2TrafoSize « 1 ) - 1, inclusive.

[0096] last_slg_coeff_x_sufïix spécifiés the suffix ofthe column position ofthe last significant coefficient in scanning order within a transform block. The values of last_sig_L_coefT_x_suffix shall be in the range of 0 to (1 « ( ( last_sig_coeff_x_prefix » 1)-1))-1, inclusive.

[0097] The column position of the last signîficant coefficient in scanning order within a transform block LastSignificantCoeffX is derived as follows:

— If last_sig_coeff_x_suffix is not présent, the following applies: LastSignificantCoeffX = last_sîgjcoeff_x_prefix — Otherwise (last_sîg_coeff_x_suffïx is présent), the following applies:

LastSignificantCoeffX = ( 1 « ((last__sig^coeffx_prefix » 1 )- 1 ))* ( 2 + (last_sîg_coeff_x_prefix & 1 ) ) + last_sig_coeff_x_suffix [0098] Iast_sig_coeff_y_sufnx spécifiés the suffix ofthe row position ofthe last signîficant coefficient in scanning order within a transform block. The values of last_sig_coeff_y_suffîx shall be in the range of 0 to (1 « ( ( last_sig_coeff_y_prefix » 1)-1))-1, inclusive.

[0099] The row position of the last signîficant coefficient in scanning order within a transform block LastSignificantCoeffY is derived as follows:

- Iflast_sig_coeff_y_suffix is not présent, the following applies:

LastSignificantCoeffY = last_s ig_coeff_y_prefix

- Otherwise (last_sig_coeff_y_suffix is présent), the following applies:

LastSignificantCoeffY = ( 1 « ( ( last_sig_coeff_y_P^rcfi^{x >:>} O ~ O ) * ( 2 + ( last_sig_coeff_y_prefix & 1 ) ) + last_sig_coeff_y_suffix [0100] When scanldx is equal to 2, the coordinates are swapped as follows:

(LastSignificantCoeffX, LastSignificantCoeffY)=Swap( LastSignificantCoeffX, LastSignificantCoeffY ) [0101] With such a position coded and also the coefficient scanning order ofthe CGs, one flag is further signaled for CGs except the last CG (in scanning order), which 25 indicates whether it contains non-zero coefficients. For those CGs that may contain non-zero coefficients, signîficant flags, absolute values of coefficients and sîgn information may be further coded for each coefficient according to the pre-defined 4x4 coefficient scan order.

[0102] As described above, the techniques described in thîs disclosure describe ways to 30 détermine the transform that video encoder 20 applies to convert a transform block to a coefficient block and ways to détermine the transform that video décoder 30 applies (e.g., as an inverse transform) to convert a coefficient block to a transform block. The following describes multiple transform for intra and inter prédiction residual (e.g.,

different transform types for when the residual block is generated from intra-prediction and for when the residual block is generated from inter-prediction).

(0103] In some cases, despîte the fact that DST Type-VII can efficiently improve the intra coding efïïciency compared to the conventional DCT Type-II, the transform efïïciency is relatively limited because prédiction residuals présent various statistics, and fixed usage of DCT Type-Π and DST Type-VII cannot efficiently adapt to ail the possible cases. Some techniques hâve been proposed to adapt to different cases. [0104] In S.-C. Lim, D.-Y. Kim, S. Jeong, J. S. Choi, H. Choi, and Y.-L. Lee, “Ratedistortion optimized adaptîve transform coding,” Opt. Eng., vol. 48, no. 8, pp. 08700410 1-087004-14, Aug. 2009, a new transform scheme which adaptively employs integer version of DCT or DST for prédiction residue is proposed, for each block it is signaled whether the DCT or DST transform is used for the prédiction residue. In Y. Ye and M. Karczewicz, “Improved H.264 intra coding based on bidîrectîonal intra prédiction, directiona! transform, and adaptive coefficient scanning,” in Proc. 15th IEEE Int. Conf.

Image Process., Oct. 2008, pp. 2116-2119, it has been proposed that each Intra prédiction mode can be mapped to a unique pair of transform (C and R), a pre-defmed as KLT pair, so that mode dépendent transform (MDDT) applies. This way, different KLT transforms can be used for different Intra prédiction modes; however, which transform to be used is predefined and dépendent on the intra prédiction mode.

[0105] In X. Zhao, L. Zhang, S. W. Ma, and W. Gao, “Video coding with rate-distortion optimized transform,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 1, pp. 138-151, Jan. 2012, however, more transforms can be used and an index to the transforms from a pre-defined set of transform candidates which are derived from offline training process is explicitly signaled. Similar to MDDT, each Intra prédiction direction may hâve its unique set of pairs of transforms. An index is signaled to specify which transform pair is chosen from the set. For example, there are up to four vertical KLT transforms and up to four horizontal KLT transforms for smallest block sizes 4x4; therefore 16 combinations may be chosen. For larger block sizes, less number of combinations are used. The proposed method in “Video coding with rate-distortion 30 optimized transform” applies to both Intra and Inter prédiction residual. For Inter prédiction residual, up to 16 combinations of KLT transforms can be chosen and the index to one of the combinations (four for 4x4 and sixteen for 8x8) is signaled for each block.

[0106] In A. Saxena and F. Femandes, “DCT/DST-based transform coding for intra prédiction in image/video coding, IEEE Trans. Image Processing and C. Yeo, Y. H. Tan, Z. Li, and S. Rahardja, “Mode-dependent transforms for coding dîrectional intra prédiction residuals, IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 4, pp.

545-554,2012, multiple transforms are used; however, instead of using KLT transforms (which typically need to be trained), eîther DCT (DCT-II) or DST (DST-VII) is used for a transform unit (with both Ieft and right transforms (e.g., C and R) being the same) and which one to be used is determined by a signaled flag. In F. Zou, O. C. Au, C. Pang, J.

Dai, and F. Lu, “Rate-Distortion Optimized Transforms Based on the Lloyd-Type

Algorithm for Intra Block Coding,” IEEE Journal of Selected Topics in Signal Processing, Volume:7, Issue: 6, Nov. 2013, several pre-defined KLT transform pairs are used, and an index to a transform pair is signaled (instead of derived) for a coding unît, so that each transform unit of the coding unit uses the same pair of transforms.

[0107] In J. An, X. Zhao, X. Guo and S. Lei, “Non-CE7: Boundary-Dependent

Transform for Inter-Predicted Residue,” JCTVC-G281, multiple transforms are chosen for inter predicted resîdual of TUs accordîng to theîr locations wîthin a CU. Both the C and R transforms are chosen from DST-VII and the flipped version of DST-VII. Therefore, up to four combinations are possible for the TUs within a CU. However, since the combination is fùlly determined by the location of the PUs, there is no need to 20 s ignat wh ich comb ination is be ing used.

[0108] There may be certain problems with techniques related to transforms for residuals (e.g., problems for intra-predicted residuals that resuit from intra-predication, but may be applicable to inter-predicted residuals that resuit from inter-predictîon as well). The exïsting methods may use a couple of DST or DCT transforms for Intra 25 predicted resîdual. However, those transforms cannot cover ail the possible distributions ofthe resîdual signal.

[0109] For example, only DCT Type-II is applied for Intra prédiction resîdual blocks larger than or equal to 8x8 in HEVC, which cannot adapt to the varying statistics of Intra prédiction resîdual. Only DCT Type-II is applied for Inter prédiction resîdual in 30 HEVC, which cannot adapt to the varying statistics of Inter prédiction resîdual. Simply choosîng the transform depending on transform block size or Intra prédiction modes is not very efficient because the resîdual statistics may still hâve large variation even under the same Intra prédiction mode or same transform size.

[0110] This disclosure describes the following techniques. In some examples, one or more ofthe following techniques may address one or more ofthe above mentioned problems. However, it is not a requirement that the following techniques address one or more ofthe above mentioned problems. The following techniques may be applied individually. In some cases, any combination of the example techniques may be applied. For instance, video encoder 20 and video décoder 30 may apply the techniques individually, or, in some cases, apply any combination of the one or more techniques. [OUI] In some examples, in addition to the DCT-II based transform used in HEVC, for each residual block generated by an întra-prediction mode, video encoder 20 and video décoder 30 may select the transforms from two or more candidate transforms from DCT and DST families. As one example, the candidate transforms may belong to the total 16 transforms based on different types ofthe DCT and DST families, and may include, but are not limited to the DCT-I ~ DCT-VIII, DST-I ~ DST-VIII. Altematively or in addition, video encoder 20 and video décoder 30 may use other sinusoïdal unitary transforms, or even other KLT transforms may be used. For each TU, the horizontal and vertical transforms (e.g., right and left transforms) may be the same type. For example, the candidate transforms are DST-VII, DCT-VIII, DST-I and DST-V. [0112] As described above, there are 16 transforms (e.g., DCT-I to DCT-VIII and DSTI to DST-VIII). One way to identify whïch transforms to use is for video encoder 20 and video décoder 30 to construct a list of these 16 transforms. Video encoder 20 may then signal (e.g., generate in the bitstream) a first index into the list to identify the left transform (e.g., transform C for the équation Y = C*X*R^T, where X is the transform block and Y is the resulting coefficient block) and signal (e.g., generate in the bitstream) a second index into the list to identify the right transform (e.g., transform R for the équation Y = C*X*R^T). Video décoder 30 would then receive the first index and the second index from the bitstream and détermine the transforms C and R that video décoder 30 is to use to inverse transform the coefficient block back to a transform block. [0113] In this example, the value ofthe first index may range from 0 to 15, and the value ofthe second index may range from 0 to 15. In general, coding larger numbers requires signaling more bits than coding smaller numbers (e.g., indicating index value requires more bits than indicating index value 2). In the case where the list includes ail 16 transforms, there may be signaling overhead that consumes more bandwidth than désirable. However, limiting options as to which transforms can be used, as done in

HEVC, may reduce signaling overhead but negatïvely impact coding efïïciency as better transforms are not usable.

[0114] In the techniques described in this disclosure, video encoder 20 and video décoder 30 may be able to détermine the left and right transform from a relatively large amount of candidate transforms with low impact on signaling overhead. As one example, video encoder 20 and video décoder 30 may each détermine a plurality of transform subsets, where each transform subset identifies a plurality of candidate transforms.

[0115] For example, video encoder 20 and video décoder 30 may each construct the 10 following three transform subsets and store the transform subsets in memory: transform subset 0: (DST-VII, DCT-VIII}, transform subset I: [DST-VII, DST-I}, and transform subset 2: {DST-VII, DCT-V). In some examples, these three transforms may be prestored in memory of video encoder 20 and video décoder 30. In any event, video encoder 20 and video décoder 30 may be considered as determining these three transform subsets, where each of the three transform subset identifies a plurality of candidate transforms (e.g., two transforms in this example). The plurality oftransform subsets may include more than or less than three transform subsets, and generally includes two or more transform subsets. Each transform subset may include one or more candidate transforms, but at least one identifies a plurality of candidate transforms.

For instance, some transform subsets may identi fy only one transform, and others may identify two or more transforms. In some examples, each transform subset may identify a relatively small number of transforms (e.g., less than or equal to 5).

[0116] In the techniques described in this disclosure, video encoder 20 and video décoder 30 may détermine corresponding transform subsets. For instance, if the stored 25 transform subsets in video encoder 20 are transform subset 0: {DST-VII, DCT-VIII}, transform subset 1: {DST-VII, DST-I}, and transform subset 2: {DST-VII, DCT-V}, then video décoder 30 may store inverse transform subsets: inverse-transform subset 0: {(DST-VII, IDCT-VIII}, inverse-transform subset 1: {IDST-VII, IDST-I}, and inversetransform subset 2: {IDST-VII, IDCT-V}. As another example, video décoder 30 may 30 store the same transforms as video encoder 20 and may invert them prior to applying the inverse-transform. In either example, video encoder 20 and video décoder 30 may be considered as storing corresponding transform subsets (e.g., the same subsets or subsets having inverse transforms of each other).

[0117] Video encoder 20 and video décoder 30 may utilize implicit techniques to select transform subsets for the left and right transforms. Implicit techniques mcans that video encoder 20 does not need to signal information to video décoder 30 instructing video décoder 30 about which transform subsets to select. Video encoder 20 and video décoder 30 may be configured to perform the same implicit technique to select transform subsets resultîng in video encoder 20 and video décoder 30 seiecting the same transform subsets without any increase in the amount of information that needs to be sîgnaled.

[0118] As one example, if the transform block is generated from intra-prediction, video 10 encoder 20 and video décoder 30 may détermine which transform subsets to select based on the intra-prediction mode. For instance, video encoder 20 and video décoder 30 may each store a table that maps an intra-prediction mode to a transform subset from which the left transform is to be determined and to a transform subset from which the right transform is to be determined.

[0119] As an example, video encoder 20 may intra-prediction encode a current block tn intra-prediction mode X. In thîs example, video encoder 20 generates a transform block from the residual block generated from intra-prediction encoding the current block in intra-prediction mode X. Video encoder 20 may select a transform subset for the left transform based on the intra-prediction mode X and select a transform subset for the right transform based on the intra-prediction mode X. Video encoder 20 may détermine left and right transforms from respective selected transform subsets, as described in more detail below, and apply the transforms to generate the coefficient block. [0120] Video encoder 20 may generate a video bitstream that includes information indicative of the coefficient values off the coefficient block as well as information indicatîng that the transform block that is generated from the coefficient block is for a block that was intra-prediction encoded using intra-prediction mode X. Video décoder 30 may generate the coefficient block from the sîgnaled information and détermine that the Intra-prediction mode was mode X also from the sîgnaled information. Video décoder 30 may select a transform subset for the left transform (which in this case wîl 1 30 be an inverse of the transform applied by video encoder 20) and a transform subset for the right transform (which in this case will be an inverse ofthe transform applied by video encoder 20) based on the intra-prediction mode beîng mode X.

[0121] The stored mapping indicating which transform subsets map to which intraprediction mode is the same on the video encoder 20 sîde and the video décoder 30 side.

Therefore, video encoder 20 and video décoder 30 select the corresponding transform subsets. Video décoder 30 may détermine left and right transforms from respective selected transform subsets, as described in more detail below, and apply the transforms to generate the transform block.

[0122] Although the above example is described with respect to intra-prediction modes, the techniques described in this disclosure are not so limited. In some examples, rather than intra-prediction modes, video encoder 20 and video décoder 30 may select respective transform subsets based on other information such as RQT depth, quantized coefficients, and the like.

[0123] Also, although the above example is described for intra-prediction, the techniques described in this disclosure may be extended to inter-prediction as well. For example, similar to above, video encoder 20 and video décoder 30 may détermine a plurality of transform subsets. These plurality of transform subsets for the interprediction case may be the same or different than the plurality of transform subsets for the intra-prediction case. In some cases, the plurality of transform subsets for the interprediction case may be the same as some, but not ail, of the plurality of transform subset for the intra-prediction case.

[0124] For inter-prediction, video encoder 20 and video décoder 30 may store mapping between position of the transform block relative to the PU, CU, or LCU with which it is 20 associated. For instance, the mapping may indicate that if the transform block is at left boundary of PU, CU, or LCU, a first group of the transform subsets is selected (e.g., one transform subset for the left transform and one transform subset for the right transform). If the transform block is at a right boundary of a PU, CU, or LCU, a second group of transform subsets is selected, and so forth for the top and bottom boundaries, where in 25 each case, video encoder 20 and video décoder 30 select one transform subset for the left transform and one transform subset for the right transform.

[0125] Video encoder 20 and video décoder 30 may encode and décodé blocks of the picture in a particular order. Accordingly, based on the location ofthe just encoded or decoded block, video encoder 20 and video décoder 30 may détermine the location of 30 the transform block in the PU, CU, or LCU. Again, from the perspective ofvideo décoder 30, video décoder 30 îs generating the transform block from a coefficient block. However, based on the decoding order, video décoder 30 may be able to détermine the location of the transform block that is to be generated from the coefficient block.

[0126] In this way, video encoder 20 and video décoder 30 may détermine respective transform subsets from which the left transform and right transform are to be determtned without any increase in the amount of information that needs to be signaled. In some examples, after video encoder 20 selects the transform subsets, video encoder

20 may signal information (e.g., generate in the video bitstream information) indicating which transform in the selected transform subsets is for the left transform and which transform is for the right transform. Video décoder 30 receîves the signaled information and détermines the left and right transforms.

[0127] For instance, video encoder 20 may signa! (e.g., generate in the bitstream) an index into the transform subset selected for the left transform and signal (e.g., generate in the bitstream) an index into the transform subset selected for the right transform. Video décoder 30 may receîve the respective indices into the respective transform subsets, and détermine the left and right transform.

[0128] In this example, there may be an increase in the information that needs to be signaled (e.g., indices to détermine the left and right transforms are signaled). However, the increase in the information that needs to be signaled may be minimal. As described above, each of the transform subsets may îdentify a relatively small number of transforms. Therefore, the range of the index value may be relatively small (e.g., 0 to 4 if the maximum number of transforms that each transform subset identifies is 5).

[0129] Accordingly, for a relatively small increase in signaling overhead, the techniques described in this disclosure allow for a relatively large increase in the number of transforms that can be selected. For example, because there are a plurality of transform subsets each including one or more transforms, many, and possîbly ail, of the 16 example transforms may be identified in one or more ofthe transforms. Because the transform subsets are selected with implicit techniques there is no increase in signaling overhead and because each transform subset identifies a relatively small number of transforms, identifying a particular transform does not drastically increase the signaling overhead.

[0130] In some exemptes, it may be possible to further reduce the amount of signaling overhead. For instance, in some examples, video encoder 20 and video décoder 30 may select transform subsets as described above, but then be configured to détermine a particular transform from each of the respective transform subsets based on certain conditions. In this case, video encoder 20 may not need to signa! and video décoder 30

may not need to receive information indicatîng which transform within the selected transform subsets to use.

[0131] As an example, during the encoding process, video encoder 20 may use a parttcular transform from the selected transform subset (e.g., the first identified transform in the selected transform subset), and after the transform is applied, détermine that the number of nonzero coefficients in the resulting coefficient block is less than the threshold. In this case, video décoder 30 may receive the information indicatîng the coefficient values of the coefficient block and similarly détermine that the number of nonzero coefficients is less than the threshold. In some examples, if video décoder 30 10 détermines that the number of nonzero coefficients in the coefficient block is less than the threshold (e.g., 1 or 2), then video décoder 30 may détermine that video décoder 30 should use a partïcular transform from the selected transform subset (e.g., the first identified transform in the selected transform subset).

[0132] For instance, assume that based on the intra-prediction mode, video encoder 20 determined that the transform subset for the left transform is subset 0 and for the right transform is subset 1. In this case, video encoder 20 may détermine that if the first identified transform in subset 0 is used as the left transform and if the first identified transform in subset 1 is used as the right transform, the number of nonzero coefficients in the resulting coefficient block is less than a threshold value. In this example, video encoder 20 may not signal information indicatîng that first identified transform in subset 0 and subset 1 is to be used as the left and right transforms, respectïvely. In other cases, ïf the first identified transform in subset 0 (or subset 1) is not used as the left transform (or right transform), the number of nonzero coefficients in the resulting coefficient block is less than a threshold value. In this example, video encoder 20 adds a restriction that the identified transforms in subset 0 and subset 1 cannot be used as the left and right transform.

[0133] Video décoder 30 may receive the intra-prediction mode and like video encoder 20 détermine that transform subset 0 and transform subset 1 are to be selected for the left and right transforms, respectïvely, based on the intra-prediction mode. Also, after 30 generating the coefficient block from the information indicatîng the coefficient values, video décoder 30 may also détermine that the number of nonzero coefficients in the coefficient block is less than the threshold. Video décoder 30 may détermine that a first identified transform in subset 0 and a first identified transform in subset 1 are to be used

as the left and right transforms, respectively, without receiving this information from video encoder 20 because the number of nonzero coefficients is less than the threshold. [0134] In the above examples, the transforms subsets are formed from 16 transforms (i.e., eight DCTs and eight DSTs). However, the techniques described in this disclosure 5 are not so lîmited. Additional examples of transforms include the KLT transforms.

Accordingly, the transform subsets may include one or more transforms from the eight DCTs, eight DSTs, KLT transforms, and other transform examples. Solely for ease of description, the examples are described with respect to the eight DCTs and eight DSTs. [0135] As a summary, in some of the examples described în this disclosure, a pre10 sélection from three or more candidate transforms is performed to formulate a subset of transforms, and the final transform to be used for a current TU is chosen from the subset of transforms. For example, the subset of transforms may composite a subset of left transforms and/or a subset of right transforms. The pre-selection to formulate the subset of transforms (or the subset of left transforms and the subset of right transforms) may be 15 determined by already decoded information such as Intra prédiction modes, RQT depth, quantized coefficients, etc.

[0136] The number of subsets of transform may be limited to a small înteger, e.g., 1,2, 3 or 4, and different subsets of transform contain different type of transforms. In one example, three subsets of transforms, each containtng two transforms, are created.

Based on a given Intra prédiction mode, the subset of left transforms is set to one of the three subsets and the subset of right transforms is also set to one of the three subsets (may or may not be the same as the subset of left transforms). As an example, the three subsets of transforms are: { DST-VII, DCT-VIII}, { DST-VII, DST-I} and { DST-VII, DCT-V}. Either the subset of left transforms or the subset of right transforms can be one of the above three subsets. Therefore, various Intra prédiction modes may correspond to up to 9 different combinations of the subsets for left and right transforms. Altematively or additionally, the subset of left transforms or the subset of right transforms contain only one transform. Altematively or additionally, both the subset of left transforms and the subset of right transforms may contain only one transform.

[0137] In the examples described above, the transform subsets and the transforms identifîed in the transform subsets may be the same regardless ofthe TU size and the number of transforms in the transform subsets may be the same for the different intraprediction modes. However, the techniques described in this disclosure are not so limited.

[0138] In some examples, for different TU sîzes, the number of transforms in a subset of lefVright transforms can be different, a typica! number of may be, but not Iimited to 2,3 and 4. For different Intra prédiction modes, the number of transforms in a subset of left/right transforms can be different; a typical number of transforms may be, but is 5 not Iimited to 2,3 and 4.

[0139] As described above, when a subset of transforms has been pre-selected, the final transform to be used may be sîgnaled by an index to the subset of transforms. When a subset of left transforms (or a subset of right transforms) contains two or more transforms, an index to a transform belonging to the subset of left transforms (or the 10 subset ofright transforms) is signaled. That means, when the number ofthe subset of left or right transforms is equal to I, there is no need to signal the index of transforms.

[0140] The above examples described the case where video encoder 20 and video décoder 30 may pre-select to formulate transform subsets. However, the examples described in this disclosure are not so Iimited. Altematively or additîonally, there may 15 be no need to do the pre-selection to formulate the subsets of transforms and one index to the two or more candidate transforms (as the full set) is directly signaled to indicate the left or the right transform. For example, at video encoder 20, a constraint may be introduced that only some of transforms within the full set may be tested and other transforms are not tested to reduce the encoder complexity. Which transforms to be 20 selected and the indexes of transforms may dépend on the Intra prédiction mode or other information.

[0141] In some examples, for each TU, it may be constrained that for the left transform (right transform), video encoder 20 and video décoder 30 may select the left and right transform from a subset of the candidate transforms. For example, the only one subset 25 of transforms contains DST-VII, DCT-VIII and DCT-Π, and the left transform for each

TU is always selected from (DST-VII, DCT-VIII and DCT-II), and the right transform for each TU is also always selected from (DST-VII, DCT-VIII and DCT-II).

[0142] As described above, the example techniques described in this disclosure may be applicable to both intra-prediction and inter-prediction. In HEVC, for a transform block 30 generated from inter-prediction, only the DCT-Π based transform was available. In some examples, in addition to the conventional DCT-Π based transform as in HEVC, for each residual block generated by an Inter prédiction mode, video encoder 20 and video décoder 30 may select the transforms from two or more candidate transforms methods from DCT and DST families or other transforms, e.g., KLT, in addition to that

a subset of left transforme and a subset of right transforms are created. Similar to the above example for intra-prediction, video encoder 20 may signal (e.g., generate in the bitstream) and video décoder 30 may receive in the bîtstream an index to the subset of left transforms and an index to the subset of right transforms for each TU to détermine 5 the left and right transforms.

[0143] As one example, two transforms, e.g., DST-V1I and DCT-VIII are put in the subset of left transforms and the subset of right transforms. A one-bit index to each of these subsets détermines the final left and right transforms of a current TU. The subsets can be either {DST-VII, DCT-VIII} or (DST-VIII, DCT-VII}.

[0144] Altemativeïy or addîtionally, a pre-selection from three or more candidate transforms is performed to formulate a subset of transforms and the final transform to be used for a current TU is chosen from the subset of transforms. For example, the présélection to formulate the subset of transforms (or the subset of left transforms and the subset of right transforms) may be determined by the relative position of current TU to the belongîng PU, î.e., whether the current TU is located at the top boundary, left boundary, right boundary, bottom boundary or other position of the belongîng PU. [0145] In one example, three subsets of transforms, each containing two transforms, are created. Based on the relative position of the current TU to the belongîng PU, the subset of left transforms is set to one of the three subsets and the subset of right transforms is also set to one of the three subsets (may or may not be the same as the subset of left transforms). Altemativeïy or addîtionally, the subset of left transforms or the subset ofright transforms contain only one transform. Altemativeïy or addîtionally, both the subset of left transforms and the subset of right transforms may contain only one transform.

[0146] In the above examples, vîdeo encoder 20 and video décoder 30 may select transform subsets for each TU of a CU and then détermine left and right transforms for each TU as described above. In this example, the détermination of which transforms to use is considered to be at the TU level. However, the example techniques described in this disclosure are not so limited.

[0147] In some cases, video encoder 20 may détermine that the left and right transforms for each TU of a CU should be a same default transform (e.g., DCT-II as one example, but other transform types are possible as well). Also, there may be a default transform for the left transform and a default transform for the right transform, or the default transform for the left transform and the right transform may be the same. In the

following description, the term “default transform” should be interpreted to include both the case where the default transform for the left and right transforms is different and where the default transform for the left and right transforms is the same. For instance, the default transform for the left and right transform (e.g., where different or same) may 5 be pre-selected and be the same for video encoder 20 and video décoder 30.

[0148] If video encoder 20 détermines that each TU of a CU should hâve the same default transform, video encoder 20 may signal information indicating as such (e.g., generate in the video bitstream information indicating as such). In this example, video encoder 20 may not signal indices into transform subsets, which reduces the amount of 10 information that needs to be signaled, because video décoder 30 may détermine that the default transform is to be used for each TU of the CU based on the received information.

[0149] As an example, video encoder 20 may signal (e.g., generate in the bitstream) a flag indicating whether each TU of the CU is to apply the same default transform. If the 15 flag is of a first value (e.g., a digital high), then each TU of the CU was applied with the same default transform. If the flag is of a second value (e.g., a digital low), then at least one TU ofthe CU was applied with a transform other than the default transform. In the case where at least one TU of the CU was applied with a different transform, video encoder 20 may select transform subsets and signal indices in the transform subsets, if 20 needed (e.g., nonzero coefficients greater than threshold), as described above. In the case where each TU of the CU was applied with the same default transform, video encoder 20 may not signal any indices in any of the transform subsets as video décoder 30 may already détermine which transform to use.

[0150] Video décoder 30 may receive the flag indicating whether each TU of the CU is 25 to apply the same default transform. If the flag is of the first value, video décoder 30 may détermine that no transform subsets are to be selected and no indices into transform subsets are to be parsed (e.g., received) from the bitstream. In this case, video décoder 30 may apply the default transform to each coefficient block ofthe CU. Ifthe flag is of the second value, video décoder 30 may détermine that transform subsets are to be 30 selected, détermine whether indices are to be received (e.g., based on number of nonzero coefficients), and receive the indices in the selected transform subsets based on the détermination that the indices are to be received.

[0151] In the above example, the flag indicating whether each TU is to use the same default transform is at the CU level (e.g., indicating that each TU of the CU use the

same default transform). In some examples, rather than at the CU level, the flag may be at the CTU level or PU level.

[0152] For example, video encoder 20 may signal (e.g., generate in the bitstream) a flag indîcating whether ail transform blocks of a block are transformed using the same transform. In response to receiving the flag indicating that not ail transform blocks of the block are transformed using the same transform, video décoder 30 may select transform subsets and détermine indices in the selected transforms as described above. In response to receiving the flag indicting that ail transform blocks ofthe block are transformed using the same transform, video décoder 30 may use that transform for W each ofthe transform block of the block. In this example, the “block” may be one of a CTU, a CU, or a PU, as a few examples.

[0153] As a summary, signaling of the transforms to be used for each TU can be done in TU level when the current CU utilizes the additional transforms are used, e.g., as described above. For example, video encoder 20 may send one flag for each CU indicating whether the TUs within it are coded with additional transforms (e.g., using transforms other than those in HEVC). Altematively or additionally, such indication may be signaled at LCU level (CTU level), CU level, PU level, TU or any other block level.

[0154] When the flag indicates that no TU within the CU is coded with additional transforms, ail the TUs are coded with one default transform. In one example, the default transform is DCT-II. Altematively or additionally, the default transform may dépend on intra/inter modes, intra prédiction mode, block size, TU position within a PU, or any other statistics of the current TU. For example, as described above, video encoder 20 and video décoder 30 may détermine the same default transform, and the condition for which default transform to use may be based on factors such as intra/inter modes, intra prédiction mode, block size, TU position within a PU, or any other statistics ofthe current TU. In this way, by using default transforms, the amount of information that needs to be signaled may be reduced.

[0155] In addition, indications can be présent in different hiérarchies. For example, video encoder 20 may first signal (e.g., generate in the bitstream) a one-bît flag at LCU (CTU) level, if the one-bît flag Îs 0, video encoder 20 and video décoder 30 may only apply DCT-II for each CU, otherwise if the one-bit flag is 1, video encoder 20 may signal another flag at CU level specifying whether TUs within the CU can use multiple transforms or just a default transform.

[0156] In this example, video décoder 30 may détermine at each hîerarchical level whether al! TUs within a particular hîerarchical level use a default transform. For example, if the flag at the CTU level indicates that ail TUs of the CTU are to use the same default transform, then video décoder 30 may use the same default transform for 5 ail TUs ofthe CTU. If the flag at the CTU level indicates that not ait TUs ofthe CTU are to use the same default transform, then video décoder 30 may select transform subsets and détermine transforms for each TU ofthe CTU as described above.

[0157] In some cases, rather than stoppîng at the CTU level and determinîng transforms for each TU, there may be another flag for each of the CUs of the CTU. For example, video décoder 30 may receive a flag for each CU of the CTU indicating whether ail TUs of the CU use the same default transform or do not use the same default transform. If for a CU, video décoder 30 receives a flag indicating that ail TUs of the CU use the same default transform, video décoder 30 may apply the default transform. If for a CU, video décoder 30 receives a flag indicating that not ail TUs of the CU use the same de fau It trans form, v ideo décoder 3 0 may se lect transform subsets and determ ine transforms for each TU of the CU as described above.

[0158] In some cases, rather than stopping at the CU level and determinîng transforms for each TU, there may be another flag for each of the PUs of the CU. For example, video décoder 30 may receive a flag for each PU of the CU indicating whether ail TUs 20 of the PU use the same default transform or do not use the same default transform. If for a PU, video décoder 30 receives a flag indicating that ail TUs ofthe PU use the same default transform, video décoder 30 may apply the default transform. If for a PU, video décoder 30 receives a flag indicating that not ail TUs of the PU use the same default transform, video décoder 30 may select transform subsets and détermine transforms for 25 each TU of the PU as described above.

[0159] Altematively or additionally, furthermore, when the overlapped block motion compensation (OBMC) flag of a CU is signaled as off, the one-bit flag, which indicates whether only one default transform is applied, is not signaled for the current CU and ts inferred as a default value (e.g., 0) which indicates a default transform (e.g., DCT-II) is 30 applied. Altematively or additionally, the CABAC context modeling ofthe one-bit flag of one block, which indicates whether only one default transform is applied, is dépendent on the OBMC flag of the current block when OBMC is allowed for the current slice (e.g., dépendent on a value ofthe OBMC flag).

[0160] In one example, when the OBMC flag (either implicitly derived or explicitly signaled) is true (i.e., equal to 1), video encoder 20 and video décoder 30 may use one set of context models for CABAC-encodtng or CABAC-decoding the one-bit flag. When the OBMC flag is false (i.e., equal to 0), another set of context models may be used for coding the one-bit flag. Altematively or additionally, furthermore, the initial ïzed probabil ities ofthe two sets of context models may be different. Altematively or additionally, the CABAC context modeling of the one-bit flag of one block, which indicates whether only one default transform is applied, is dépendent on the value ofthe corresponding one-bit flag ofthe spatial neighboring blocks (e.g., a left 10 neighboring block and/or an above neighboring block) or the temporal neighboring blocks (e.g., the co-located block in a référencé picture).

[0161] When a CU has additional transforms enabled (e.g., meaning more the limited choices of HEVC), for each TU, video encoder 20 may signal and video décoder 30 may receive indices to the transforms from candidate transforms (of a set or subset) as 15 described above. Altematively or additionally, video encoder 20 may signal such information and video décoder 30 may receive such information at LCU level, CU level, PU level, or any other block level. When video encoder 20 signais the indicator is at LCU level, CU level, PU level or any other block level, ail the included TUs within that level may use the same pair of transforms.

[0162] For instance, video encoder 20 and video décoder 30 may select transform subsets as described above (e.g., based on intra-prediction mode or based on location of TU for inter-prediction). In some examples, for each transform block, video encoder 20 may signal indices and video décoder 30 may receive indices for each transform block. However, in some examples, rather than receiving indices for each transform block, video encoder 20 may signal one index for the left transform and one index for the right transform for ail TUs of a CTU, ail TUs of a CU, or ail TUs of a PU, as a few examples. In this example, for each transform subset that video décoder 30 selects for the right and left transforms for a TU, video décoder 30 may apply the transform identifîed by the index for ail ofthe TUs of a block (e.g., of a CUT, CU, or PU).

[0163] In other words, in some case, the indices into the transform subsets may be consîdered as more “global. For example, video encoder 20 may signal indices for the left transform and the right transform. In this case, the indices may be global in the sense that the indices are the same for each TU of a block regardless of the particular transform subsets that are selected, where the block is a CTU, CU, or PU. In such

examples, video décoder 30 may détermine the left and right transforms from the selected transform subsets from these global indices. For instance, video décoder 30 may not parse indices for each selected transform subset for each transform block, but rather identify the transform for ail transform blocks of a block (e.g., CTU, CU, or PU) based on the global indices.

[0164] As described above, in some examples, video encoder 20 may not signal indices into selected transform subsets. For example, for certain TUs, the signaling of the additional transforms may be skipped if the energy of the residual signal is limited, e.g., if there is no nonzero coefficient transmitted for the current TU. Similar skipping of additional transform signaling may apply to LCU, CU, PU or any other block level.

[0165] Altematively or addîtîonally, the indicator at a certain block level may be skipped if the total number or the total absolute sum or the sum of squared value of nonzero coefficients transmitted at that certain block level is smaller than a given threshold value. In other words, video encoder 20 may not signal indices into selected 15 transform subsets if the total number or the total absolute sum or the sum of squared value of nonzero coefficients of a coefficient block is smaller than a threshold value. In such examples, if video décoder 30 détermines that the total number or the total absolute sum or the sum of squared value of nonzero coefficients is smaller than a given threshold value, then video décoder 30 may détermine that indices into the selected transforms subsets is not be received (e.g., parsed) from the bitstream.

[0166] In one example, the threshold value of total number of nonzero coefficients is 2.

Altematively or additionally, the threshold value for total number of nonzero coefficients may be different for different block sizes or different Intra prédiction modes.

[0167] In some examples, when the size of LCU, CU, PU or block is larger than or smaller than a pre-defined threshold value, or within a given threshold value range, video encoder 20 may skip signaling of the indicator (e.g., indices into transform subsets) and video encoder 20 and video décoder 30 may apply only a default transform type. In one example, the default transform is DCT-II. Furthermore, when the CU size is larger than 32x32, video encoder 20 may not signal the indicator and video encoder and video décoder 30 may only apply the DCT-II for each TU.

[0168] Video encoder 20 and video décoder 30 may binarize the indicators (e.g., indices into the transform subsets) using, for example, fixed-length code, truncated unary code or exponentîal Golomb code. Video encoder 20 and video décoder 30 may entropy

code (e.g., encode or décodé, respectively) indîcators usîng CABC with contexte, and for each bin, one context is applied. In one example, the context mode! is selected based on the bin index. In another example, furthermore, the intra prédiction mode or TU size or TU depth is also considered when selectîng the context models.

Altematively or additionally, partial of b ins are coded with context models and remaining bins are coded with bypass mode. Altematively or additionally, indicators may be bypass coded, i.e., no context modeling is applied.

[0169] In the example techniques, video encoder 20 may signal various information in the bitstream and video décoder 30 may receive such information from the bitstream.

Video encoder 20 may signal such information and video décoder 30 may receive such information from different locations.

[0170] As one example, syntax related to the multiple transforms can be présent in high-level syntax. Video encoder 20 may signal (e.g., generate in the bitstream) and video décoder 30 may receive the number of candidate transforms, as descrîbed above 15 with respect to the transforms being selected from two or more candidate transforms, to be used in the picture parameter set (PPS), sequence parameter set (SPS) or any other places, including even at the slice header. Video encoder 20 may signal (e.g., generate in the bitstream) and video décoder 30 may receive the number of candidate transforms in each subset, as descrîbed above with respect to the pre-selection from three or more 20 candidate transforms, at slice header, picture parameter set (PPS), sequence parameter set (SPS) or any other places.

[0171] A flag or index can be signaled at slice header, PPS, SPS or any other places to indicate whether the above mentioned multiple transform is applied at block level. As descrîbed above, one dedicated value of this flag or index may indicate that ail the TUs 25 are coded with one default transform. Additionally or altematively, one dedicated value of this flag or index may indicate that a flag/index or flags/indexes may be signaled at block level for transform sélection at block level. Also, the size of the block for which multiple transforms do not apply (when a size is larger than a signaled size or smaller than a signaled size or in a range of two signaled sïzes) may be présent in a parameter 30 set, e.g., picture parameter set or sequence parameter set.

[0172] To reiterate, the above description utilizes the term “transform.” However, it should be understood that video encoder 20 utilizes a transform to generate a transform block of transform coefficient values from a resîdual block. Video décoder 30, on the other hand, utilizes an inverse-transform to generate a resîdual block of resîdual values

from the transform block. Accord ingly, in the above description, it should be understood that the description of a transform is equally applicable to video décoder 30; however, video décoder 30 utilizes an inverse-transform.

[0173] FIG. 5 is a block diagram illustrating an exemple video encoder 20 that may implement the techniques of this disclosure. FIG. 5 is provided for purposes of explanation and should not be considered limiting of the techniques as broadly exemplified and descrîbed in this disclosure. For purposes ofexplanation, this disclosure describes video encoder 20 in the context of HEVC coding. However, the techniques of this disclosure may be applicable to other coding standards or methods.

For example, video encoder 20 may be configured to apply more transforms to the transform block than the limited options provided in HEVC.

[0174] In the example of FIG. 5, video encoder 20 includes a prédiction processing unit 100, video data memory 101, a residual génération unit 102, a transform processing unit 104, a quantization unit 106, an inverse quantization unit 108, an inverse transform processing unît 110, a reconstruction unit 112, a filter unit 114, a decoded picture buffer 116, and an entropy encoding unit 118. Prédiction processing unit 100 includes an inter-prediction processing unit 120 and an intra-predîction processing unit 126. Interprédiction processing unit 120 includes a motion estimation unit and a motion compensation unit (not shown). In other examples, video encoder 20 may include more, 20 fewer, or different functional components.

[0175] Video data memory 101 may store video data to be encoded by the components of video encoder 20. The video data stored in video data memory 101 may be obtained, for example, from video source 18. Decoded picture buffer 116 may be a référencé picture memory that stores référencé video data for use in encoding video data by video 25 encoder 20, e.g., in intra- or inter-coding modes. Video data memory 101 and decoded picture buffer 116 may be formed by any of a variety of memory devices, such as dynamïc random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), résistive RAM (RRAM), or other types of memory devices. Video data memory 101 and decoded picture buffer 116 may be provided by 30 the same memory device or separate memory devices. In various examples, video data memory 101 may be on-chîp with other components of video encoder 20, or off-chip relative to those components.

[0176] Video encoder 20 may receive video data. Video encoder 20 may encode each CTU in a slice of a picture of the video data. Each of the CTUs may be associated with

equally-sized luma coding trec blocks (CTBs) and corresponding CTBs of the picture. As part of encoding a CTU, prédiction processîng unit 100 may perform quad-tree partitioning to divide the CTBs ofthe CTU into progressively-smaller blocks. The smaller block may be coding blocks of CUs. For example, prédiction processîng unit 5 100 may partition a CTB associated with a CTU into four equally-sized sub-blocks, partition one or more of the sub-blocks into four equally-sized sub-blocks, and so on. [0177] Video encoder 20 may encode CUs of a CTU to generate encoded représentations of the CUs (i.e., coded CUs). As part of encoding a CU, prédiction processîng unit 100 may partition the coding blocks associated with the CU among one 10 or more PUs of the CU. Thus, each PU may be associated with a luma prédiction block and corresponding chroma prédiction blocks. Video encoder 20 and video décoder 30 may support PUs having various sizes. As indicated above, the size of a CU may refer to the size of the luma coding block of the CU and the size of a PU may refer to the size of a luma prédiction block of the PU. Assuming that the size of a particular CU is 15 2Nx2N, video encoder 20 and video décoder 30 may support PU sizes of 2Nx2N or

NxN for intra prédiction, and symmetric PU sizes of 2Nx2N, 2NxN, Nx2N, NxN, or similar for inter prédiction. Video encoder 20 and video décoder 30 may also support asymmetric partitioning for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter prédiction.

[0178] Inter-prediction processîng unit 120 may generate prédictive data for a PU by performing inter prédiction on each PU of a CU. The prédictive data for the PU may include prédictive blocks of the PU and motion information for the PU. Inter-prediction unit 121 may perform different operations for a PU of a CU depending on whether the PU is in an I slice, a P slice, or a B slice. In an I slice, ail PUs are intra predicted.

Hence, if the PU is in an I slice, inter-prediction unit 121 does not perform inter prédiction on the PU. Thus, for blocks encoded in I-mode, the predicted block is formed using spatial prédiction from previously-encoded neighboring blocks within the same frame.

[0179] If a PU is in a P slice, the motion estimation unit of inter-prediction processîng 30 unit 120 may search the référencé pictures in a list of référencé pictures (e.g., “ReiPicListO”) for a référencé région for the PU. The référencé région for the PU may be a région, within a référencé picture, that contains sample blocks that most closely corresponds to the sample blocks of the PU. The motion estimation unit may generate a référencé index that indicates a position in RefPicListO of the référencé picture

contaîning the référencé région for the PU. In addition, the motion estimation unit may generate an MV that indicates a spatial displacement between a coding block ofthe PU and a référencé location associated wîth the référencé région. For instance, the MV may be a two-dimenstonal vector that provides an offset from the coordinates in the current decoded picture to coordinates in a référencé picture. The motion estimation unît may output the référencé index and the MV as the motion information of the PU. The motion compensation unit of inter-prediction processing unit 120 may generate the prédictive blocks ofthe PU based on actual or interpolated samples at the référencé location indicated by the motion vector of the PU.

i o [0180] If a PU is in a B slice, the motion estimation unit of inter-prediction processing unît 120 may perform uni-prcdiction or bi-prediction for the PU. To perform uniprediction for the PU, the motion estimation unit may search the référencé pictures of RefPicListO or a second référencé picture list (“RefPÎcListl”) for a référencé région for the PU. The motion estimation unît may output, as the motion information of the PU, a 15 référencé index that indicates a position in RefPicListO or RefPicListl ofthe référencé picture that contains the référencé région, an MV that indicates a spatial displacement between a prédiction block of the PU and a référencé location associated with the référencé région, and one or more prédiction direction indicators that indicate whether the référencé picture is in RefPicListO or RefPicListl. The motion compensation unit of 20 inter-prediction processing unit 120 may generate the prédictive blocks of the PU based at least in part on actual or interpolated samples at the référencé région indicated by the motion vector of the PU.

[0181] To perform bi-dîrectional inter prédiction for a PU, the motion estimation unit may search the référencé pictures in RefPicListO for a référencé région for the PU and 25 may also search the référencé pictures in RefPicListl for another référencé région for the PU. The motion estimation unit may generate référencé picture indexes that indicate positions în RefPicListO and RefPicListl of the référencé pictures that contain the référencé régions. In addition, the motion estimation unit may generate MVs that indicate spatial displacements between the référencé location associated with the référencé régions and a sample block of the PU. The motion information of the PU may include the référencé indexes and the MVs ofthe PU. The motion compensation unit of inter-prediction processing unit 120 may generate the prédictive blocks ofthe PU based at least in part on actual or interpolated samples at the référencé régions indicated by the motion vectors of the PU.

[0182] Intra-prediction processing unit 126 may generate prédictive data for a PU by performing intra prédiction on the PU. The prédictive data for the PU may include prédictive blocks for the PU and various syntax éléments. Intra-prediction processing unit 126 may perform intra prédiction on PUs in I slices, P slices, and B slices.

[0183] To perform intra prédiction on a PU, intra-prediction processing unit 126 may use multiple intra prédiction modes to generate multiple sets of prédictive data for the PU. Intra-prediction processing unit 126 may use samples from sample blocks of neighboring PUs to generate a prédictive block for a PU. The neighboring PUs may be above, above and to the right, above and to the left, or to the left of the PU, assumîng a 10 left-to-right, top-to-bottom encoding order for PUs, CUs, and CTUs. Intra-prediction processing unit 126 may use various numbers of intra prédiction modes, e.g., 35 directional intra prédiction modes. In some examples, the number of intra prédiction modes may dépend on the size of the région associated with the PU.

[0184] Prédiction processing unit 100 may select the prédictive data for PUs of a CU from among the prédictive data generated by inter-prediction processing unit 120 for the PUs or the prédictive data generated by intra-prediction processing unît 126 for the PUs. In some examples, prédiction processing unit 100 selects the prédictive data for the PUs of the CU based on rate/dîstortion metrics of the sets of prédictive data. The prédictive blocks of the selected prédictive data may be referred to herein as the selected prédictive 20 blocks.

[0185] In the examples described in this disclosure, the techniques are applicable to when a video block is intra-predicted or intra-predîcted. For example, when a block is intra-predicted, the intra-prediction mode may be used to détermine transform subsets. When a block is înter-predîcted, its position may be used to détermine transform 25 subsets. Accordingly, the example techniques apply to a video block that is intrapredîcted in any ofthe intra-prediction modes or inter-predîcted in uni-direction or bidirection.

[0186] Furthermore, the example techniques are not limited to intra-prediction or interprediction, and may be extended to intra-block copy (IBC) mode as well. In IBC mode, 30 a prédictive block is in the same picture as the video block being encoded, and is identified by a block vector. In IBC mode, transform subsets may be selected from a position of the video block, position of the prédictive block, or the block vector as a few examples.

[0187] Residual génération unit 102 may generate, based on the luma, Cb and Cr coding block of a CU and the selected prédictive luma, Cb and Cr blocks of the PUs of the CU, a luma, Cb and Cr residual blocks of the CU. For instance, residual génération unit 102 may generate the residual blocks of the CU such that each sample in the residual blocks has a value equal to a différence between a sample in a coding block of the CU and a corresponding sample in a corresponding selected prédictive block of a PU of the CU. [0188] Transform processing unit 104 may perform quad-tree partitioning to partition the residual blocks associated with a CU into transform blocks associated with TUs of the CU. Thus, a TU may be associated with a luma transform block and two chroma transform blocks. The sïzes and positions of the luma and chroma transform blocks of TUs of a CU may or may not be based on the sîzes and positions of prédiction blocks of the PUs of the CU. A quad-tree structure known as a “residual quad-tree” (RQT) may include nodes associated with each of the régions. The TUs of a CU may correspond to leaf nodes of the RQT.

[0189] Transform processing unit 104 may generate transform coefficient blocks for each TU of a CU by applying one or more transforms to the transform blocks of the TU. Transform processing unit 104 may apply various transforms to a transform block associated with a TU. For example, transform processing unit 104 may apply a discrète cosine transform (DCT), a directional transform, or a conceptual 1 y similar transform to a transform block. In some examples, transform processing unit 104 does not apply transforms to a transform block. In such examples, the transform block may be treated as a transform coefficient block.

[0190] In the techniques described in this disclosure, transform processing unit 104 may apply a left transform and a right transform to a transform block of the TU. In some 25 examples, prédiction processing unit 100 may détermine which transforms to apply using the techniques described in this disclosure.

[0191] For example, prédiction processing unit 100 may détermine a pluralîty of transform subsets, each subset identifying one or more candidate transforms, where at least one transform subset identifies a pluralîty of candidate transforms. The candidate 30 transforms are of different transform types, and in some examples, prédiction processing unit 100 détermines the pluralîty of transform subsets based on a size of a video block being encoded.

[0192] In some examples, video data memory 101 stores the pluralîty of transform subsets, and prédiction processing unit 100 may détermine the pluralîty of transform

subsets from the stored transform subsets. In some examples, video data memory 101 may store ail of the transforms, and prédiction processing unit 100 may construct the transform subsets in a predefïned manner. Examples of the candidate transforms include DCT-I to DCT-VIII, DST-I to DST-VIII, KLT transforms, and the like. In some examples, the plurality of transform subsets includes three or more transform subsets.

[0193] Prédiction processing unit 100 may select a first transform subset from the plurality of transform subsets for a left transform for a current transform block of a video block ofthe video data and select a second transform subset from the plurality of 10 transform subsets for a right transform for the transform block of the video block of the video data. The current transform block may be the transform block that transform processing unit 104 generates and on which transform processing unit 104 is to apply the transforms.

[0194] Prédiction processing unit 100 may détermine the left transform from the selected first transform subset and détermine the right transform from the selected second transform subset. For instance, prédiction processing unit I00 may test each of the transforms in the selected transform subsets and détermine which transform provides the best video coding. Prédiction processing unit 100 may détermine the respective transforms that provide the best video coding as the left transform and the right transform.

[0195] Transform processing unit 104 may détermine a current coefficient block based on the left transform, right transform, and the current transform block. For instance, transform processing unit 104 may perform the following équation: Y = C*X*R^T, where C is the left transform, R. is the right transform, X is the current transform block, and Y 25 is the resulting current coefficient block.

[0196] If the video block (e.g., CU or PU) is intra-prediction encoded, prédiction processing unit 100 may détermine the intra-prediction mode ofthe video block. Prédiction processing unit 100 may select the first transform subset based on the determined intra-prediction mode, and select the second transform subset based on the 30 determined intra-prediction mode.

[0197] If the video block (e.g., CU or PU) is inter-prediction encoded, prédiction processing unit 100 may détermine a location ofthe current transform block in the video block (e.g., détermine whether transform block is for the residual generated from a partïcular location In the video block). Prédiction processing unit 100 may select the

first transform subset based on the determined location ofthe current transform block, and select the second transform subset based on the determined location ofthe current transform block.

[0198] For intra-prediction or inter-prediction, in some examples, prédiction processing unit 100 may cause entropy encoding unit 118 to signal (e.g., generate in the bitstream) a first transform subset index into the first transform subset to identify a transform in the first transform subset used to détermine the current coefficient block, and signal (e.g., generate in the bitstream) a second transform subset index into the second transform subset to identify a transform in the second transform subset used to détermine the current coefficient block. In some examples, prédiction processing unît 100 may détermine a number of nonzero coefficients in the current coefficient block. In these examples, prédiction processing unit 100 may cause entropy encoding unît 118 to signal the first transform subset index based on the number of nonzero coefficients being greater than a threshold, and signal the second transform subset index based on the number of nonzero coefficients being greater than the threshold. Ifthe number of nonzero coefficients is less than the threshold, prédiction processing unit 100 may not cause entropy encoding unit 118 to signal indices in the first and second transform subsets.

[0199] In some examples, at least one ofthe first transform subset or the second transform subset includes a transform that is different than a discrète cosine transform (DCT)-II transform and a discrète sine transform (DST)-VII transform. In some examples, the first transform subset and the second transform subset include different transforms (e.g., at least one transform in the first transform subset is not in the second transform subset or vice-versa).

[0200] Quantization unit 106 may quantize the transform coefficients in a coefficient block. The quantization process may reduce the bit depth associated with some or ail of the transform coefficients. For example, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. Quantization unit 106 may quantize a coefficient block associated with a TU of a CU based on a quantization parameter (QP) value associated with the CU. Video encoder 20 may adjust the degree of quantization applied to the coefficient blocks associated with a CU by adjusting the QP value associated with the CU. Quantization may introduce loss of information; thus quantized transform coefficients may hâve lower précision than the original ones.

[0201] Inverse quantization unit 108 and inverse transform processing unit 110 may apply inverse quantization and inverse transforms to a coefficient block, respectively, to reconstruct a residual block from the coefficient block. Reconstruction unit 112 may add the reconstructed residual block to corresponding samples from one or more prédictive blocks generated by prédiction processing unît 100 to produce a reconstructed transform block associated with a TU. By reconstructîng transform blocks for each TU of a CU în this way, video encoder 20 may reconstruct the coding blocks of the CU. [0202] Filter unit 114 may perform one or more deblocking operations to reduce blocking artifacts in the coding blocks associated with a CU. Decoded picture buffer

116 may store the reconstructed coding blocks after filter unit 114 performs the one or more deblocking operations on the reconstructed coding blocks. Inter-predictîon processing unît 120 may use a référencé picture that contains the reconstructed coding blocks to perform inter prédiction on PUs of other pictures. In addition, intra-prediction processing unit 126 may use reconstructed coding blocks in decoded picture buffer 116 to perform intra prédiction on other PUs in the same picture as the CU.

[0203] Entropy encoding unit 118 may receive data from other functional components of video encoder 20. For exemple, entropy encoding unit 118 may receive coefficient blocks from quantization unit 106 (e.g., information indicative of coefficients of the current coefficient block used to reconstruct the video block) and may receive syntax éléments from prédiction processing unit 100 (e.g., indices into the first and second transform subsets). Entropy encoding unit 118 may perform one or more entropy encoding operations on the data to generate entropy-encoded data. For example, entropy encoding unit 118 may perform a context-adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable-to-variable (V2V) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a Probability Interval Partitîoning Entropy (PIPE) coding operation, an ExponentialGolomb encoding operation, or another type of entropy encoding operation on the data. Video encoder 20 may output a bitstream that includes entropy-encoded data generated by entropy encoding unit 118. For instance, the bitstream may include data that represents a RQT for a CU.

[0204] In the example techniques, prédiction processing unit 100 détermines a prédictive block, and generates in the video bitstream, that entropy encoding unit 118 outputs, information indicative of a prédiction mode of the video block based on the prédictive block. The prédiction mode indicates whether the video block is intra

predîcted or inter-predîcted. For example, the prédictive block is a block in the same picture as the video block based on the video block being intra-predicted or in a picture different than the picture that includes the video block based on the video block being inter-predicted. Residual génération unit 102 may détermine the current transform block as a residual between the video block and the prédictive block.

[0205] FIG. 6 is a block dîagram illustrating an example video décoder 30 that is confîgured to implement the techniques of this disclosure. FIG. 6 is provided for purposes of explanatîon and is not limiting on the techniques as broadly exemplified and described in this disclosure. For purposes of explanatîon, this disclosure describes video décoder 30 in the context of HEVC coding. However, the techniques of this disclosure may be applicable to other coding standards or methods.

[0206] Video décoder 30 représente an example of a device that may be confîgured to perform techniques in accordance with various examples described in this disclosure. In the example of FIG. 6, video décoder 30 includes an entropy decodîng unit 150, video data memory 151, a prédiction processing unît 152, an inverse quantization unit 154, an inverse transform processing unit 156, a reconstruction unit 158, a fïlter unit 160, and a decoded picture buffer 162. Prédiction processing unit 152 includes a motion compensation unit 164 and an intra-prediction processing unît 166. In other examples, video décoder 30 may include more, fewer, or different functional components.

[0207] Video data memory 151 may store video data, such as an encoded video bitstream, to be decoded by the components of video décoder 30. The video data stored in video data memory 151 may be obtained, for example, from computer-readable medium 16, e.g., from a local video source, such as a caméra, via wired or wireless network communication of video data, or by accessing physîcal data storage media. Video data memory 151 may form a coded picture buffer (CPB) that stores encoded video data from an encoded video bitstream. Decoded picture buffer 162 may be a reference picture memory that stores reference video data for use in decodîng video data by video décoder 30, e.g., in intra- or inter-coding modes. Video data memory 151 and decoded picture buffer 162 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistîve RAM (MRAM), résistive RAM (RRAM), or other types of memory devices. Video data memory 151 and decoded picture buffer 162 may be provided by the same memory device or separate memory devices. In various

examples, video data memory 151 may be on-chip with other components of video décoder 30, or off-chip relative to those components.

[0208] A coded picture butter (CPB) may receive and store encoded video data (e.g., NAL units) of a bitstream. Entropy decoding unit 150 may receive encoded video data (e.g., NAL units) from the CPB and parse the NAL units to décodé syntax éléments.

Entropy decoding unit 150 may entropy décodé entropy-encoded syntax éléments in the NAL units. Prédiction processing unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filter unit 160 may generate decoded video data based on the syntax éléments extracted from the bitstream.

[0209] The NAL units of the bitstream may include coded slice NAL units. As part of decoding the bitstream, entropy decoding unit 150 may extract and entropy décodé syntax éléments from the coded slice NAL units. Each ofthe coded slices may include a slice header and slice data. The slice header may contain syntax éléments pertaîning to a slice. The syntax éléments in the slice header may include a syntax element that identifies a PPS associated with a picture that contains the slice.

[0210] In addition to decoding syntax éléments from the bitstream, video décoder 30 may perform a reconstruction operation on a non-partitioned CU. To perform the reconstruction operation on a non-partitioned CU, video décoder 30 may perform a reconstruction operation on each TU of the CU. By performîng the reconstruction operation for each TU of the CU, video décoder 30 may reconstruct residual blocks of theCU.

[0211] As part of performîng a reconstruction operation on a TU of a CU, inverse quantization unit 154 may inverse quantize, i.e., de-quantize, coefficient blocks associated with the TU. Inverse quantization unît 154 may use a QP value associated with the CU of the TU to détermine a degree of quantization and, lîkewise, a degree of inverse quantization for inverse quantization unit 154 to apply. That is, the compression ratio, i.e., the ratio of the number of bits used to represent original sequence and the compressed one, may be controlled by adjusting the value of the QP used when quantizing transform coefficients. The compression ratio may also dépend on the method of entropy coding employed.

[0212] Aller inverse quantization unit 154 inverse quantizes a coefficient block, inverse transform processing unit 156 may apply one or more inverse transforms to the coefficient block in order to generate a residual block associated with the TU. For example, inverse transform processing unit 156 may apply an inverse DCT, an inverse

înteger transform, an inverse Karhunen-Lœve transform (KLT), an inverse rotationa! transform, an inverse directional transform, or another inverse transform to the coefficient block.

[0213] In the techniques described in this disclosure, prédiction processing unit 152 may détermine the left and right transforms that inverse transform processing unît 156 is to apply. For example, prédiction processing unit 152 may détermine a pluralîty of transform subsets, each subset identifyîng one or more candidate transforms, where at least one transform subset identifies a pluralîty of candidate transforms. The candidate transforms are of different transform types, and in some examples, prédiction processing to unit 152 détermines the pluralîty of transform subsets based on a size of a video block being decoded.

[0214] In some examples, video data memory 151 stores the pluralîty of transform subsets, and prédiction processing unit 152 may détermine the pluralîty of transform subsets from the stored transform subsets. In some examples, video data memory 151 15 may store ail of the transforms, and prédiction processing unit 152 may construct the transform subsets in a predefined mariner. In some examples, prédiction processing unit 152 may receive information identifyîng the transform subsets from the bitstream. Examples ofthe candidate transforms include DCT-I to DCT-VIII, DST-I to DST-VIII, KLT transforms, and the like. In some examples, the pluralîty oftransform subsets 20 includes three or more transform subsets.

[0215] Prédiction processing unit 152 may select a first transform subset from the pluralîty of transform subsets for a left transform for a current coefficient block of the video data, and select a second transform subset from the pluralîty of transform subsets for a right transform for the current coefficient block of the video data. Prédiction processing unit 152 may détermine the left transform from the selected first transform subset, and détermine the right transform from the selected second transform subset. [0216] Inverse transform processing unit 156 may détermine a current transform block based on the left transform, right transform, and the current coefficient block. For instance, inverse transform processing unit 156 may perform the inverse ofthe following équation: Y - C*X*R^T, where Y is the coefficient block, C is the left transform, X is the transform block, and R is the right transform. Again, in this disclosure, it should be understood that inverse transform processing unit 156 applies the inverse of the transform that video encoder 20 applied, but for ease video décoder 30 is described as applymg a transform.

[0217] Prédiction processïng unit 152 may reconstruct (e.g., intra-prediction or interprediction décodé) a video block based on the current transform block and a prédictive block. For example, if a PU is encoded using intra prédiction, intra-prediction processïng unit 166 may perform intra prédiction to generate prédictive blocks for the

PU. Intra-prediction processïng unit 166 may use an intra prédiction mode to generate the prédictive luma, Cb and Cr blocks for the PU based on the prédiction blocks of spatially-neighboring PUs. Intra-prediction processïng unit 166 may détermine the intra prédiction mode for the PU based on one or more syntax éléments decoded from the bitstream.

[0218] Prédiction processïng unit 152 may construct a first référencé picture list (RefPicListO) and a second référencé picture list (RefPicListl) based on syntax cléments extracted from the bitstream. Furthermore, if a PU is encoded using inter prédiction, entropy decoding unit 150 may extract motion information for the PU. Motion compensation unit 164 may détermine, based on the motion information ofthe PU, one or more référencé régions for the PU. Motion compensation unit 164 may generate, based on samples blocks at the one or more référencé blocks for the PU, prédictive luma, Cb and Cr blocks for the PU.

[0219] In the examples described in this disclosure, the techniques are applicable to when a video block is intra-predicted or intra-predicted. For example, when a block is 20 intra-predicted, the intra-prediction mode may be used to détermine transform subsets.

When a block is inter-predicted, its position may be used to détermine transform subsets. Accordingly, the example techniques apply to a video block that is intrapredicted in any of the intra-prediction modes or inter-predicted in uni-direction or bidirectton.

[0220] Furthermore, the example techniques are not limited to intra-prediction or interprediction, and may be extended to intra-block copy (IBC) mode as well. In IBC mode, a référencé block used to form a prédictive block îs in the same picture as the video block being encoded, and is identifled by a block vector. In IBC mode, transform subsets may be selected from position ofthe video block, position ofthe référencé block, or the block vector as a few examples.

[0221] Reconstruction unit 158 may use the luma, Cb and Cr transform blocks associated with TUs of a CU and the prédictive luma, Cb and Cr blocks of the PUs of the CU, i.e., either intra-prediction data or inter-prediction data, as applicable, to reconstruct the luma, Cb and Cr coding blocks of the CU. For example, reconstruction

unit 158 may add samples of the luma, Cb and Cr transform blocks to corresponding samples of the prédictive luma, Cb and Cr blocks to reconstruct the luma, Cb and Cr coding blocks of the CU.

[0222] Ftlter unit 160 may perform a deblocking operation to reduce blocking artifacts associated with the luma, Cb and Cr coding blocks of the CU. Video décoder 30 may store the luma, Cb and Cr coding blocks of the CU in decoded picture buffer 162. Decoded picture buffer 162 may provide reference pictures for subséquent motion compensation, intra prédiction, and présentation on a display device, such as display device 32 of FIG. 1. For instance, video décoder 30 may perform, based on the luma,

Cb, and Cr blocks in decoded picture buffer 162, intra prédiction or inter prédiction operations on PUs of other CUs.

[0223] In some examples, where the video block is to be intra-prediction decoded, prédiction processing unit 152 may détermine an intra-prediction mode ofthe video block. Prédiction processing unit 152 may select the first transform subset based on the 15 determined intra-prediction mode, and select the second transform subset based on the determined intra-prediction mode.

[0224] Where the video block is to be inter-prediction decoded, prédiction processing unit 152 may détermine a location of the current transform block in the video block (e.g., détermine whether coefficient block is for the residual generated from a particular 20 location in the video block). Prédiction processing unit 152 may select the first transform subset based on the determined location of the current transform block, and select the second transform subset based on the determined location ofthe current transform block.

[0225] In some examples, prédiction processing unit 152 may receive a first transform 25 subset index into the first transform subset, and receive a second transform subset index into the second transform subset. In these examples, prédiction processing unit 152 may détermine the left transform based on a transform in the first transform subset identified by the first transform subset index, and détermine the right transform based on a transform in the second transform subset identified by the second transform subset 30 index.

[0226] However, prédiction processing unit 152 may not need to receive indices in the first and second transform subsets. For instance, prédiction processing unit 152 may détermine that a number of nonzero coefficients in the current coefficient block is less than a threshold. In such cases, prédiction processing unît 152 may détermine that a

first transform identified in the first transform subset is the left transform without receiving a transform subset index into the first transform subset, in response to determining that the number of nonzero coefficients in the current coefficient block is less than the threshold, and détermine that a first transform identified in the second transform subset is the right transform without receiving a transform subset index into the second transform subset, in response to determining that the number of nonzero coefficients in the current coefficient block is less than the threshold.

[0227] Also, prédiction processing unit 152 may not necessarily détermine transforms from transform subsets in ail cases. In some examples, prédiction processing unit 152 to may receive a flag indicating that not ail transform blocks of a block that includes the current transform block are transformed using the same transform. In such examples, prédiction processing unit 152 may select the first and second transforms, and détermine the left and right transforms from the respective first and second transforms in response to receiving the flag indicating that not ail transform blocks of the block that includes 15 the current transform block are transformed using the same transform. Examples ofthe block include a coding tree unit (CTU), a coding unit (CU), or a prédiction unit (PU).

[0228] In some examples, at least one ofthe first transform subset or the second transform subset includes a transform that is different than a discrète cosine transform (DCT)-II transform and a discrète sine transform (DST)-VII transform. In some 20 examples, the first transform subset and the second transform subset include different transforms (e.g., at least one transform in the first transform subset is not in the second transform subset or vice-versa).

[0229] In the example techniques, video décoder 30 may receive from a bitstream information indicating a prédiction mode (e.g., whether a video block is intra-predicted 25 or inter-predicted), and receive from the bitstream information indicating coefficients of the current coefficient block. Prédiction processing unit 152 may détermine the prédictive block based on the prédiction mode, and inverse transform unît 156 or prédiction processing unit 152 may construct the coefficient block based on the received information indicating the coefficients. The prédiction mode is one of an inter30 prédiction mode or an intra-prediction mode, and the current transform block is a resîdual ofthe video block and the prédictive block.

[0230] The techniques descrîbed above may be performed by video encoder 20 (FIGS.

and 5) and/or video décoder 30 (FIGS. 4 and 6), both of which may be generally referred to as a video coder. Likewise, video coding may refer to video encoding or

video decoding, as applicable. In addition, video encoding and video decoding may be generically referred to as “processing” video data.

[0231] In following subsections, examples of above techniques will be provided. In practice, any combination of any part of the examples may be used as new example technique.

[0232] The following describes examples of constructing an additional candidate transform list. Besides a default transform method which always applies DCT-II for ail the included TUs, for each TU, additional candidate transform methods can be constructed given selected transform sets. In one example, the additional candidate transform list for Intra and Inter prédiction residuals is constructed as follows: Firstly, a transform set is defïned as a collection of transform types, for example, an example transform set can be defïned as [DCT-II, DST-VII], which includes two types of transforms, i.e., DCT-II and DST-VII. Based on two given transform sets, différent transform methods can be generated by selectîng one transform type from the fîrst transform set as the horizontal transform, and another transform type from the second transform set as the vertical transform. For example, when the transform set 0 {DCTII, DST-VII} is used for horizontal transform, and the transform set 1 { DCT-VIIÏ, DST-VII} is used for vertical transform, totally four transform methods can be generated as:

Table 1: Four transform methods based on the transform set (DCT-II,

DST-VII]

	Transform Method 1	Transform Method 2	Transform Method 3	Transform Method 4
Horizontal transform	DCT-II	DCT-II	DST- VII	VII	DST-
Vertical	DCT-	DST-	DCT-		DST-
transform	VIII	VII	VIII	VII

[0233] For Intra prédiction residual, totally three transform subsets are defïned, încluding: Transform Subset 0: {DST-VII, DCT-VIII}, Transform Subset 1: { DST25 VII, DST-I}, and Transform Subset 2: { DST-VII, DCT-V}. The sélection on the transform set for horizontal and vertical transforms is dépendent on the Intra prédiction mode, as shown in Table 2 below,

Table 2: Mapping table between Intra prédiction mode and Transform set

Intra Msde	0	1	2	3	4	S	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33
RlghtTransSubsetldx	2	1	0	1	0	1	0	1	0	0	0	0	0	1	0	1	0	1	0	1	0	1	0	1	2	2	2	2	2	1	0	1	0	1
LcftTraniSubteddx	2	1	0	1	0	1	0	1	2	2	2	2	2	1	0	1	0	1	0	1	0	1	0	1	0	0	0	0	0	1	0	1	0	1

[0234] For example, for Intra mode 10, the candidate transform types for horizontal(right) transform are from transform set 0 including DST-VII and DCT-VIII, and the candidate transform types for vertical(left) transform are from transform set 2 5 including DST-VII and DCT-V, Therefore, the additional candidate transform list for intra mode 10 is finally constructed as shown in Table 3 in which totally four transform methods are generated

Table 3: Example of four transform methods for Intra prédiction mode 10 based on Table 2

	Transform Method 1	Transform Method 2	Transform Method 3	Transform Method 4
Horizontal transform	VII	DST-	DST- VII	DCT- VIII	DCT- VIII
Vertical transform	VII	DST-	DCT-V	DST- VII	DCT-V

[0235] According to Table 2, for each TU, given an Intra prédiction mode, totally four transform methods can be generated. For Inter prédiction residual, the same transform set {DST-VII, DCT-VIII} is used for both horizontal and vertical transforms. Therefore, for each TU, the additional candidate transform list is constructed as shown in Table 4.

Table 4: Four transform methods for Inter prédiction residual

	Transform Method 1	Transform Method 2	Transform Method 3	Transform Method 4
Horizontal transform	DST- VII	DST- VII	DCT- VIII	DCT- VIII
Vertical transform	DST- VII	DCT- VIII	DST- VII	DCT- VIII

[0236] The following description is an example of indicator signaling. To adaptively fît for different contents, indicators on the transform usage are signaled at both CU-level and TU-level. The CU-level indicator is a one-bit flag indicating whether the default 20 DCT-II is applied for ail the TUs included in the current CU. If the one-bit fiag is 0,

only the default DCT-Ιί can applïed for ail the TUs included in the current CU; otherwîse, a two-bit TU-level indîcator may be further signaled for each TU, and the first bit spécifiés which transform type from the given transform set is appiied as the horizontal transform, and the second bit spécifiés which transform type from the gîven 5 transform set is appiied as the vertical transform.

[0237] For Intra prédiction residual, the TU-level indicators are signaled after the coefficients, and the TU-level indicator is not signaled and derived as 0 when the total number of nonzero coefficients of the TU is no larger than 2. Otherwîse, the TU-level indicator is explicitly signaled. For Inter prédiction residual, the TU-level indicators to can be signaled either before or after the coefficients, and the TU-level indicator is not signaled when there is no nonzero coefficients in the TU.

[0238] The syntax, semantics of the proposed examples based on HEVC are provided below. In the below syntax, graying of éléments is used to indicate potential changes in the syntax or to otherwîse assist with understanding.

Transform tree syntax

transform tree( xO, yO, xBase, y Base, log2TrafoSize, trafoDepth, blkldx ) {	Descriptor
ίίζ log2TrafoSize <= Log2MaxTrafoSize && log2TrafoSize > Log2MînTrafoSize && trafoDepth < MaxTrafoDepth && !( IntraSplitFlag && (trafoDepth = = 0)))
split transform flag[ xO ][ yO ][ trafoDepth ]	ae(v)
if( log2TrafoSize > 2 ) {
if( trafoDepth = = 0 11 cbf cb[ xBase ][ yBase ][ trafoDepth - 1 ] )
cbf cb[ xO ][ yO ][ trafoDepth ]	ae(v)
trafoDepth - = 0 11 cbf cr[ xBase ][ yBase ][ trafoDepth - 1 ] )
cbf cr[ xO ][ yO ][ trafoDepth ]	ae(v)
}
if( split transform flag[ xO ][ yÔ ][ trafoDepth ] ) {
xl = xO + ( 1 « ( log2TrafoSize - 1 ) )
y 1 = yO + ( 1 « ( log2TrafoSize -1 ) )
1 iff trafoDepth = 0 )
1 .. add multi transform flag| xO ][y0 ]	ee(v)
transform tree( xO, yO, xO, yO, log2TrafoSize - 1, trafoDepth +1,0)
transform tree( xl, yO, xO, yO, log2TrafoSize - 1, trafoDepth +1,1)
transform tree( xO, yl, xO, yO, log2TrafoSize - 1, trafoDepth + 1,2)
transform tree( xl, yl, xO, yO, log2TrafoSize - 1, trafoDepth + 1,3)
} else {
if( CuPredMode[ xO ][ yO ] = = MODEJNTRA 11 trafoDepth != 0 11 cbf cb[ xO ][ yO ][ trafoDepth ] | [ cbf cr[ xO ]( yO lf trafoDepth 1 )
cbf luma[ xO ][ yO ][ trafoDepth ]	ae(v)
1. ..... if( cbf luma[ xO ][ yO ][ trafoDepth ] && trafoDepth = 0 )
1 add multi transform flagf xO ][ yO J
transform unit( xO, yO, xBase, yBase, log2TrafoSize, trafoDepth, blkldx )
}
}

Altematively, add_multi_transform_flag may be signaled without dependency 5 to cbfjuma.

transform tree( xO, yO, xBase, yBase, log2TrafoSize, trafoDepth, blkldx ) {	Descriptor
Lif( trafoDepth ~0 )
L„add multi transfonn flag[ xO H yO ]	ae(v)
if( log2TrafoSize <=t Log2MaxTrafoSize && log2TrafoSize > Log2MinTrafoSize && trafoDepth<MaxTrafoDepth && !(IntraSplitFlag && (trafoDepth = =0)))
split transform flag[ xO ][ yO ][ trafoDepth ]	ae(v)
··«
}

This is équivalent to sending the flag in the coding unit.

coding^unit( xO, yO, log2CbSize ) {	Descriptor
if( transquant bypass enab!ed flag )
cu transquant bypass flag	ae(v)
if( slice type 1= I)
cu skip flag[ xO ][ yO ]	ae(v)
nCbS ~ ( 1 « log2CbSize )
if( cu skip flag[ xO ][ yO ] )
prediction unit( xO, yO, nCbS, nCbS )
else {
îf( slîce type != I)
pred mode flag	ae(v)
if( CuPredMode[ xO ][ yO ] 1= MODE INTRA || log2CbSize = = MinCbLog2SizeY)
part mode	ae(v)
if( CuPredMode[ xO ][ yO ] = = MODEJNTRA ) {
if(PartMode == PART_2Nx2N && pcm_enabled_flag && log2CbSize >= Log2MinIpcmCbSizeY && log2CbSîze <~ Log2MaxIpcmCbSizeY )
pcm flag[ xO ][ yO ]	ae(v)
if( pcm flag( xO ][ yO ] ) {
while( !byte aligned( ) )
pcm al ign men t zero bi t	f(i)
pcm sample( xO, yO, log2CbSize )
} else {
pbOffset = ( PartMode = = PART NxN ) ? ( nCbS / 2 ) : nCbS
for( j = 0; j < nCbS; j =j + pbOffset )
for( i = 0; i < nCbS; i = i + pbOffset )

prevjntra luma pred flag[ xO + i ][ yO + j ]	ae(v)
forf j = 0; j <nCbS; j =j + pbOffset )
for( i = 0; i < nCbS; i = i + pbOflset )
if( prev intra luma pred flag[ xO + i ][ yO +j ] )
mpm idx[ xO + î ][ yO + j ]	ae(v)
Else
rem intra luma pred mode[ xO + i ][ yO +j ]	ae(v)
intra chroma pred mode[ xO ][ yO ]	ae(v)
}
} else {
if( PartMode == PART 2Nx2N)
prediction unit( xO, yO, nCbS, nCbS )
else if( PartMode == PARTJNxN ) {
prédiction unit( xO, yO, nCbS, nCbS / 2 )
prédiction unit(xO, yO + (nCbS/2),nCbS,nCbS 12)
} else if( PartMode = = PART Nx2N ) {
prediction unit( xO, yO, nCbS / 2, nCbS )
prediction unit( xO + ( nCbS / 2 ), yO, nCbS / 2, nCbS )
} else ίί( PartMode = = PART2NxnU ) {
prediction unit( xO, yO, nCbS, nCbS / 4 )
prédiction unit( xO, yO + ( nCbS / 4 ), nCbS, nCbS * 3 / 4 )
} else iR PartMode = = PART 2NxnD ) {
prediction unit( xO, yO, nCbS, nCbS * 3 / 4 )
prédiction unit(xO,yO + (nCbS * 3 / 4 ), nCbS, nCbS 14)
} else îf( PartMode = = PART nLx2N ) {
prédiction unit( xO, yO, nCbS / 4, nCbS )
prédiction unit( xO + ( nCbS / 4 ), yO, nCbS · 3 / 4, nCbS )
} else PartMode = = PARTnRx2N ) {
prédiction unit( xO, yO, nCbS *3/4, nCbS )
prediction unit( xO + ( nCbS ♦ 3 / 4 ), yO, nCbS / 4, nCbS )
} else { /* PART NxN ·/
prédiction unit( xO, yO, nCbS / 2, nCbS / 2 )
prédiction unit( xO + ( nCbS / 2 ), yO, nCbS ! 2, nCbS / 2 )
prédiction unit( xO, yO + ( nCbS / 2 ), nCbS / 2, nCbS 12)
prédiction unit( xO + ( nCbS / 2 ), yO + ( nCbS / 2 ), nCbS / 2, nCbS / 2 )
î
}
ifl[ Ipcm flagi xO 1[ yO ] ) {
if( CuPredMode[ xO ][ yO ] 1= MODE_JNTRA && !( PartMode = = PART 2Nx2N && merge flag[ xO ][ yO 1 ) )
rqt root cbf	ae(v)
if( rqt root cbf) {
1 add multi transform flag	ae(v)

MaxTrafoDepth = ( CuPredMode[ x0 ][ yO ] = = MODEJNTRA ? ( max_transform_hîerarchy_depth_intra + IntraSplitFlag ) : max transform hïerarchy depth inter)
transform tree( xO, yO, xO, yO, log2CbSize, 0,0 )
}
}
}
}

Transform tree Semantics âdd_multi_transform_flag[ χΟ ][ ÿû ] spécifiés whetherenanhcedmultiplë Îransform isapplîed for each _TU included inthë current CU.whëq

KâdjnütOnmsformlflagTxÔ'jtyO'] isO, DCT-II isalwaysappliéd for eachTU ïncludëd in the'current CU, otherwise, lêftjtrërisform_flag and right_transform_flag piay bë further signaled for each TUtospecify thTîeft transform and thè right transform appiiêd for à TU beloing to the current transform tree. Whëq àdd_multi_transform_fîag[ xO ]( yO ] is not présent,' it is iriferred to be equal to 0.'

Résidu al coding syntax

residual coding( xO, yO, log2TrafoSize, cldx ) {	Descriptor
if( transform_skip_enabled_flag && !cu_transquant_bypass_flag && ( log2TrafoSize ==2))
transform sklp flag[ xO ][ yO ][ cldx ]	ae(v)
last sig coefT x p re fix	ae(v)
last sig coeff y p refix	ae(v)
if( last sig coeff x prefix > 3 )
last sig coeff x suffix	ae(v)
if( last sig coefT y prefix > 3 )
last sig coeff y su fïï x	ae(v)
lastScanPos = 16
lastSubBlock = ( 1 « ( log2TrafoSize -2 ) ) * ( 1 « ( log2TrafoSize - 2 ))-i
do {
if( lastScanPos == 0) {
lastScanPos = 16
lastSubBlock—
}
lastScanPos—

xS = ScanOrderf log2TrafoSize - 2 ][ scanldx ][ lastSubBlock ][ 0 ]
yS = ScanOrder[ log2TrafoSize - 2 ][ scanldx ][ lastSubBlock ][ l ]
xC = ( xS « 2 ) + ScanOrder[ 2 ][ scanldx ][ lastScanPos ][ 0 ]
yC = ( yS « 2 ) + ScanOrder[ 2 ][ scanldx ][ lastScanPos ][ l ]
} while( ( xC I= LastSignificantCoefïX ) 11 ( yC l= LastSignificantCoeflY ))
for( i - lastSubBlock; i >= 0; i— ) {
xS = ScanOrder[ log2TrafoSize - 2 ][ scanldx ][ ί ][ 0 ]
yS = ScanOrderf log2TrafoSize - 2 ][ scanldx ][ i ][ l ]
inferSbDcSigCoefÎFlag = 0
if( ( i < lastSubBlock ) && ( i > 0 ) ) {
coded sub block flag[ xS ][ yS ]	ae(v)
inferSbDcSigCoefIFlag = l
)
for( n = ( i = “ lastSubBlock ) ? lastScanPos - l : 15; n >= 0; n— ) {
xC-(xS « 2 ) + ScanOrder[ 2 ][ scanldx ][ n ][ 0 ]
yC = ( yS « 2 ) + ScanOrder[ 2 ][ scanldx ]( η ][ l ]
îf( coded sub block flag[ xS ][ yS ] && ( n > 0 11 linferSbDcSigCoefïFlag ) )”{
sig coeff flag[ xC ][ yC ]	ae(v)
ift sîg coeiT i]ag[ xC ][ yC ] )
InferSbDcSigCoefÎFlag = 0
}
firstSigScanPos = 16
lastStgScanPos = -1
numGreater 1 Flag = 0
lastGreater 1 ScanPos = -1
for( n = 15; n >= 0; n— ) {
xC = ( xS « 2 ) + ScanOrder[ 2 ][ scanldx ][ n ][ 0 ]
yC = ( yS « 2 ) + ScanOrderf 2 ][ scanldx ][ n ][ 1 ]
if( sig coeff flag[ xC ][ yC ] ) {
ifÎ numGreater 1 Flag < 8 ) {
coeff abs level greaterl flag[ n ]	ae(v)
numGreater 1 Flag++
iflfcoeff abs level_greaterl flag[n] && lastGreater 1 ScanPos = =-l)
lastGreaterl ScanPos = n
}
if{ lastSigScanPos ==-1)
lastSigScanPos = n
firstSigScanPos = n

}
}
signHidden = ( lastSigScanPos - firstSigScanPos > 3 && !cu transquant bypass flag)
ift lastGreaterlScanPos l= -I )
coeff abs level greater2 flag[ lastGreaterlScanPos ]	ae(v)
for(n= I5;n >= 0;n—) {
xC-(xS « 2 ) +ScanOrderf 2 ][ scanïdx ][ n ][ 0 ]
yC = ( yS « 2 ) + ScanOrderf 2 ][ scanïdx ][η][ l ]
iflf sig_coeff_flag[ xC ][ yC ] && (Isign data hiding enabled flag 11 IstgnHidden II (n != firstSigScanPos ) ) )
coefT sign flag[ n ]	ae(v)
}
numSigCoeff= 0
sumAbsLevel = 0
for( n = 15; n >= 0; n— ) {
xC = ( xS « 2 ) + ScanOrderf 2 ][ scanïdx ][ n ][ 0 ]
yC = ( yS « 2 ) + ScanOrderf 2 ][ scanïdx ][ η ][ I ]
ift sig coefT flag[ xC ][ yC ] ) {
baseLevel = l + cocfT_abs_level_greaterl_flag[ n ] + coeff abs level greater2 flagf η l
if( baseLevel = = ( ( numSigCoeff < 8 ) ? ( (n - = lastGreaterl ScanPos) 73:2):1))
coeff abs Ievel remaining[ n ]	ae(v)
TransCoefïLevelf xO ][ yO ][ cldx ]f xC ][ yC ] = ( coefT_abs_level_remaining[ n ] + baseLevel ) * ( 1 - 2 * coefT sign flag[n])
if( sign data hîding enabled flag && signHidden ) {
sumAbsLevel += ( coeff absjevel remaining[ n ] + baseLevel )
if( ( n = = firstSigScanPos ) && ( ( sumAbsLevel % 2 ) = = 1 ) )
TransCoefïLevelf xO ][ yO ][ cldx ][ xC ][ yC ] = -TransCoefïLevelf xO ][ yO ][ cldx lf xC lf yC ]
}
numSigCoefî-H-
}
}
}
[if(add multi transform flag[ xO ][ yO J ) {
j if(numSigCoefE>2 ff(CuPredModef xO]fyO] MODEJNTRA && numSigCoeff>0)) (
L... Ieft transform flag[x0][y0]	ae(v)
L right transform flag[ xO ][ yO ]	ae(v)

LJ
LJ
}

Residual coding semantics |ëft JransfôraTilâg[ xOJ[ ÿO ] spécifiés; the transform index appliéd foîjhë'Jëfi iransformof the current TU, whërï not presented, Ieft_transform2flag[x0 ][ yO ] is 5 infëred as 0] fightj_transform_flag[ xO ][yO ] spécifiés the transformindex applied for the fight transform of the current TU, whërï riot presented, rightjransform_flag[ xO ][ ÿO] }s infered as 0]

Decoding process for deriving the left tranforrm and right transform lIf CûPredMôde[ xO ][ ÿO ] ^MÔDETNTRÀ' given the Intra Mode value IntraPredModeY[ xPb ][ yPb], the valueof LeftTransSubsetldx ând RjghtTransSùbsetldx is’derivéd based ôn the following tablé;

Intra Mode	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34
RlghtTransSubsetldx	1	1	0	1	0	1	0	1	0	0	0	0	0	1	0	1	0	1	0	1	0	1	0	1	2	2	2	2	2	1	0	1	0	1	0
LeftTransSubsetldx	2	1	0	1	0	1	0	1	2	2	2	2	2	1	0	1	0	1	0	1	0	1	0	1	0	0	0	0	0	1	0	1	0	1	0

• Gîvën the value of Le ftTransSubsétldx and left/trari fornTfiagrthe left transform îsdêriÿêd using the following’tablei

		Leftl	fransSubsetldx
		0	1	2
left_transform_flag	0	DST-7	DST-7	DST-7
1	DCT-8	DST-1	DCT-5

•Given theyalue ofRightTransSubsetldx ând right_tranfôrin_flag,thë left transform is derived using thé following table;

		RlghtTransSubsettdx
		0	1	2
rlght_tra nsf orm_f 1 ag	0	DST-7	DST-7	DST-7
1	DCT-8	DST-1	DCT-5

• Otherwise (CuîYëdModêîxÔ j[ÿO~] 1= MÔDEJnTRA),’ the followings are performed;

rT^Giventhe vâlûè left trâhform flagfthë lëft tansfotm Îsdërivëd üsïngthé ïbllowinglablê;

1 eft_tran sf orm_flag	0	DCT-8
1	DST-7

ΓGiventhevalue right tranform flag?thérîght'transform isderivedusinethe following tahlë^

rlght_transform_flag	0	DCT-8
1	DST-7

[0239] The following are examples of applying the enhanced multiple transform and large transform. For the encoding process performed by video encoder 20, in one example, for each CU smaller than (or equal to) 32x32, the current CU is coded twice. In the first pass, the current CU is coded using only DCT-II, The rate-distortion cost for 10 coding the whole CU is recorded as RDcost_CU_DCT; and the rate-distortion cost for coding each Intra prédiction mode of PU is recorded as RDcost_PU_DCT[i][p], where i îndicates the index of the Intra prédiction mode inside the current CU, and p indicates the index of the PU inside the current CU. The optimal Intra prédiction mode based on rate-distortion optïmization for the current PU indexed by p is denoted as IPM[p].

[0240] In the second pass, the current CU is coded again using multiple transforms described as below. For each included PU, indexed by p’, of the current CU, the following is performed. For each candidate Intra prédiction mode i’, if the RDcost_PU_DCT[i’][p’j > RDcost_PU_DCT[IPM[p’]][p’], the Intra prédiction mode i’ is skipped and not selected as the optimal Intra prédiction mode for the current PU.

Otherwise, the following is performed for each TU included in the current PU. [0241] For each TU inside the current PU, given the current candidate Intra prédiction mode, according to above example,, the 2 candidate right (R) transforms and 2 candidate left (L) transforms are selected, so totally 4 different R and L transform combination. Then, each candidate R and L transform combination is tested using rate25 distortion cost.

[0242] During this process, ifone R and L transform combination generates zéro coefficient (e.g., zéro value or no coefficients), the remaining L and R transform combinations are skipped and not selected as the optimal R and L transform combination. The R and L transform combination with the least rate-distortion cost is 30 selected as the actual transforms to encode the current residual block. Moreover, during the above process of selecting combination R and L transforms, if one candidate R and L transform combination generates no more than 2 nonzero coefficients, it is not

selected as the optimal transform combination unless both R and L transform is the DST-VII transform.

[0243] After the above process is done for ail the PUs inside the current CU, then the rate-dîstortion cost for coding the whole CU is recorded as RDcost_CU_EMT. If

RDcost_CU_DCT is smaller than RDcost_CU_EMT, then one flag add_multi_transform_flag is conditionally signaled as 0 as described in above example, and ail the included TUs are encoded using DCT-II. Otherwise, add_multi_transform_flag is conditionally signaled as 1 as described in above example, and for each included TU, a flag left_transform_flag and another flag right_transfonn_flag is conditionally signaled after the coefficients are signaled as described in above example, to indîcate which left transform and right transform are selected for encoding the current TU.

[0244] The following describes examples ofthe decoding process performed by video décoder 30. In one example, for each CU smaller than (or equal to) 32x32, one bit flag add_multi_transform_flag is conditionally signaled as described in above example. This flag is not signaled only when transform depth is 0 and the coded block flag (CBF) value of Luma component is 0, otherwise, the flag is always signaled.

[0245] Ifadd_mu!ti_transform_flag is 0, only DCT-2 is applied for ail the included TUs, otherwise, the following is performed. For each TU, one bit flag left_transform_flag and another flag right_transform_flag is conditionally signaled after the coefficients are signaled as described in above example. The condition of whether left_transform_flag and rîght_transform_flag are signaled are described below. [0246] When current CU ts Intra coded, when the total number of nonzero coefficients are less than (or equal to) 2, left_transform_flag and right_transform_flag arc not signaled. Otherwise, left_transform_flag and right_transform_flag are signaled. [0247] Otherwise, when current CU is not Intra coded, when there îs no nonzero coefficient, left_transform_flag and right_transform_flag are not signaled. Otherwise, left_transform_flag and right_transform_flag are signaled.

[0248] For each TU, given the signaled left_transform_flag and right_transform_flag, the left and right transforms are derived as described in above example. When the current CU is larger than 32x32, for each TU with same size of64x64, the transform is performed as described in more detail below with respect to examples for larger sized TUs.

[0249] The following are exemples of another alternative of constructing an addîtional transform list based on Intra prédiction mode. Besides a default transform method which always applies DCT-II for ail the included TUs, for each TU, an addîtional candidate transform DST-VII can be appiied as follows. Given the intra prédiction mode of current TU, denoted as IPM, the left and right transform appiied on this TU is specified as below. if (IPM & 1) equals 1, DCT-II is appiied as both the left and right transform for current TU; otherwîse ((IPM &l) equals 0), DST-VII is appiied as both left and right transform for current TU.

[0250] The above described example techniques for determining transforms to use. The following describes examples for supporting larger sized transforms. For example, supportîng 64x64 transform is bénéficiai, especially for coding videos with larger resolutions, e.g., 1080p and 4K. To support 64x64 transform while constraîning the complexity for both video encoder 20 and video décoder 30 is important, and although various ways can be done to achieve that, a better solution may be available.

[0251] The actual appiied NxN transform matrices may be an integer-point approximation after the scaling of the original floating-point transform matrices, and the scaling may be larger than 64*log2jV., including but not limited to s’IogiN, where s can be 128 or 256. In one example, the resulting transform coefficients after the horizontal and vertical transforms are kept within the 16-bit représentation by applying addîtional right shift operations. The addîtional right shifting operations include, but are not limited to, right shifting the resulting transform coefficients after vertical and horizontal înverse/forward transform by addîtional log2(s/64) bits.

[0252] Transform sizes larger than 32-point transform may be appiied, including but not limited to 64-point, 128-ροΐηζ 256-point, on residual blocks. When AApoînt and N25 point transforms are appiied for the horizontal and vertical transforms, respectively, where M and N are integers and M could be the same as or different from N, only the top-leftXxF lower frequency coefficients insîde the resulting MxN coefficient block, where X<M and K<N, are signaled and the remaining coefficients are not signaled and derived as 0.

[0253] The position ofthe last nonzero coefficient inside the resulting Nf*Ncoefficient block can be coded by reusîng the same logic of the last nonzero coefficient position coding used for SxT blocks, where X<S<M and T<T<N, in terms ofcontext modeling. To zéro out the coefficients beyond AxK, a constraint may be introduced for LastSignificantCoeffY and LastSignificantCoefïX. For example, the value of

LastSignificantCoefiX(LastSÎgnificantCoefiY) may be smaller than Λ(Κ). The value of X and Y may be constant, e.g., 32, or dépendent on transform size, e.g., X=M/2, Y=N12. A typical value ofXand Y is 32 for 64-point transform.

[0254] In one example, one or more CG (coding group) scan orders for MxN may be pre-defined and used. However, for CGs outsîde the top-left XxYrégion of the resulting MxN coefficient block, the CG-level flags sîgnaling whether there is at least one nonzero coefficient in the each CG are skipped and not coded. Altematively or addîtionally, for ail the WxHCGs, where a typical value of Wand H is 4, the CGs are coded following the scan order for XxY régions which includes, but not Iimited to diagonal, zigzag, horizontal or vertical scan order. Altematively or addîtionally, ail the CGs are grouped in units of JK’x/f', where W‘ is a multiples of IF and H’ is a multiples of H, the W'xH' units are coded following the scan order including, but not Iimited to (MIWyxtNIH') diagonal, zigzag, horizontal or vertical scan order, and the CGs inside each WxH unit are coded following the scan order including, but not Iimited to, (IF7IF)x(H7W) diagonal, zigzag, horizontal or vertical scan order. [0255] To support coding RQT depth corresponding to transform size of64x64, 128x128 or 256x256, the CABAC contexts used for coding RQT splîttîng flags, which dépend on transform sizes, can be shared for RQT depth values corresponding to transform size larger than 32x32. For example, for certain RQT depth cases, including but not Iimited to RQT depth values corresponding to transform size 64x64 and 32x32, the same CABAC context can be applied for coding the RQT splitting flag.

[0256] The following are examples of performing 32x32 zero-out for 64x64 transform. There may be a constraint for the last position, where !ast_sig_coefr_x_sufïtx spécifiés the suffix of the column position of the last significant coefficient in scannîng order within a transform block. The values of lastjsig_coeff_x_suffix shall be in the range of 0 to ( 1 « ( ( last_sig_coefT_x_prefix » 1)-1))-1, inclusive. The column position of the last significant coefficient in scanning order within a transform block LastSignificantCoefiX is derived as follows. IfIast_sig_coeff_x_suffix is not présent, the following applies: LastSignificantCoefiX = last_sig_coeff_x_prefix, otherwise (Iast_sig_coeff_xjsuffix is présent), the following applies LastSignificantCoefiX = ( l « ( (last_sig_coeff_x_prefix » 1 ) - 1 ) ) * ( 2 + (last_sig_coeff_x_prefix & 1 ) ) + last_sig_coefî_x_suffix.

[0257] The syntax element last_sig_coefT_y_suffix spécifiés the suffix of the row position ofthe last significant coefficient in scanning order within a transform block.

The values of last_stg_coeff_y_suffix shall be in the range of 0 to (1 « ( ( last_sig_coeff_y_prefix » 1)-1))-1, inclusive. The row position of the last significant coefficient in scanning order within a transform block.

LastSignificantCoefïY is derived as follows. If last_sig_coeff_y_suffix is not présent, the following applies LastSignificantCoefïY = last_sig_coefT_y_prefix, otherwise (lastjsig_coeff_y_sufïïx is présent), the following applies LastSignificantCoefïY = (1 « ( ( last_sigjcoeff_y_prefix » 1 ) -1 ) ) * ( 2 + ( Iast_sigLCoeff_y_prefix & 1 ) ) + last_sig_coeff_y_suffix [0258] When scanldx is equal to 2, the coordînates are swapped as follows (

LastSignïficantCoefïX, LastSignificantCoefïY ) = Swap( LastSïgnificantCoefïX, LastSignificantCoefïY ). ÿhë value ofQstSÏ^ificahtCoëfïX ôr LastSignificantCoefïY shall be smallér than 32] [0259] The following are conditions for the sîgnaling of significant CG and significant coefficients.

Residual coding syntax

resîdual coding( xO, yO, log2TrafoSize, cldx ) {	Descriptor
if( transform skip enabled flag && leu transquant bypass flag && ( log2TrafoSize = =2))
transform skip flag[ xO ][ yO ][ cldx ]	ae(v)
last sig coeff x prefi x	ae(v)
last sig coeff y prefix	ae(v)
if( last sig coeff x prefix > 3 )
last sig coeff x suffix	ae(v)
if( last sig coeff y prefix > 3 )
last sig^coeff y suffix	ae(v)
lastScanPos = 16
lastSubBlock = ( 1 « ( log2TrafoSize - 2 ) ) * ( 1 « ( log2TrafoSize - 2 ) ) -1
do {
if( lastScanPos = = 0 ) {
lastScanPos = 16
lastSubBlock—
}
lastScanPos—
xS = ScanOrder[ log2TrafoSize — 2 ][ scanldx ][ lastSubBlock ][ 0 ]
yS = ScanOrder[ log2TrafoSîze - 2 ][ scanldx ][ lastSubBlock ][ 1 ]

xC = ( xS « 2 ) + ScanOrder[ 2 ][ scanldx ][ lastScanPos ][ 0 ]
yC = ( yS « 2 ) + ScanOrder[ 2 ][ scanldx ][ lastScanPos ][ 1 ]
} while( ( xC 1= LastSignificantCoeffX ) 11 ( yC 1= LastSîgnificantCoefïY ) )
for( i » lastSubBlock; i >= 0; i— ) {
xS = ScanOrder[ log2TrafoSize - 2 ][ scanldx ][ i ][ 0 ]
yS = ScanOrder[ log2TrafoSize - 2 ][ scanldx ][ i ][ 1 ]
inferSbDcSigCoeftFlag = 0
ift ( i < lastSubBlock ) && ( i > 0 ) && (xS<321| yS<32)j {
coded sub block flag[ xS ][ yS ]	ae(v)
inferSbDcSigCoeftFlag = 1
)
for( n = ( i = = lastSubBlock ) ? lastScanPos - 1 :15; n >= 0; n—){
xC = ( xS « 2 ) + ScanOrder[ 2 ][ scanldx ][ n ][ 0 ]
yC = ( yS « 2 ) + ScanOrderf 2 ][ scanldx ][ n ][ 1 ]
ifi( coded_sub_block_flag[ xS ][ yS ] && ( n > 0 11 1 inferSbDcSigCoeftFlag )) {
sig coefT flag[ xC ][ yC ]	ae(v)
if( sig coeff flag[ xC ][ yC ] )
inferSbDcSigCoeftFlag = 0
}
}
fïrstSigScanPos = 16
lastSigScanPos = -1
numGreaterl Flag = 0
lastGreaterlScanPos = -1
for( n = 15; n >= 0; n— ) {
xC = ( xS « 2 ) + ScanOrder[ 2 ][ scanldx ][ n ][ 0 ]
yC = ( yS « 2 ) + ScanOrder[ 2 ][ scanldx ][ n ][ 1 ]
if( sig coefï flag[ xC ][ yC ] ) {
if( numGreaterl Flag < 8 ) {
coefr abs leve!greaterl flag[ n ]	ae(v)
numGreaterl Flag++
if( coeft_abs_level_greaterl_f!ag[ n ] && lastGreaterlScanPos == -1 )
lastGreater 1 ScanPos = n
}
if( lastSigScanPos == -1 )
lastSigScanPos = n
fïrstSigScanPos = n
ï

}
signHidden = ( lastSigScanPos — firstSigScanPos > 3 && îcutransquantbypassflag )
if( JastGreaterlScanPos != -1 )
coeff abs level greater2 flag[ JastGreaterlScanPos ]	ae(v)
for( n = 15; n >= 0; n— ) {
xC = (xS « 2 ) + ScanOrder[ 2 ][ scanldx ][ n ][ 0 ]
yC = ( yS « 2 ) + ScanOrder[ 2 ][ scanldx ][ n ][ 1 ]
if( sig_coeff_flag[ xC ][ yC ] && ( !sign_data_hiding_enabled_flag 11 IsignHidden 11 ( n != firstSigScanPos ) ) )
cocff slgn flag[ n ]	ae(v)
}
numSigCoeff = 0
sumAbsLevel = 0
for( n = 15; n >= 0; n— ) {
xC = ( xS « 2 ) + ScanOrder[ 2 ][ scanldx ][ n ][ 0 ]
yC = ( yS « 2 ) + ScanOrder[ 2 ][ scanldx ][ n ][ 1 ]
ifr sig coeff flag[ xC ][ yC ] ) {
baseLevel = 1 + coeff_abs_level_greaterl_f!ag[ n ] + coeff abs level greater2 flag[ n ]
if{ baseLevel = = ( ( numSigCoeff <8)7 ( (n = = lastGreater 1 ScanPos) 73:2):1))
coeff abs level remaining[ n ]	ae(v)
TransCoeffLevelf xO ][ yO ][ cldx ][ xC ][ yC ] = ( coeff_abs_level_remaining[ n ] + baseLevel ) · ( 1 - 2 * coeff sign flagr ni)
if( sign data hiding enabled flag && signHidden ) {
sumAbsLevel += ( coeff_abs_level_remaining[ n ] + baseLevel )
if( ( n = = firstSigScanPos ) && ( ( sumAbsLevel % 2 ) == D)
TransCoefiLevel[ xO ][ yO ][ cldx ][ xC ][ yC ] = -TransCoefïLeveI[ xO ][ yO ][ cldx ][ xC ][ yC ]
}
numSigCoeffH-
}
}
}
}

[0260] The followîng describes an example ofoverlapped block motion compensation (OBMC). OBMC was proposed in the development of ITU-T H.263. See Video

Coding for Low Bitrate Communication, document Rec. H.263, ITU-T, Apri! 1995. OBMC is performed on an 8x8 block, and motion vectors oftwo connected neighboring 8x8 blocks are used for the current block as shown in FIGS. 8A and 8B. For example, for the first 8x8 block in current macroblock, besides its own motion vector, the above and left neighboring motion vector are also applied to generate two additional prédiction blocks. In this way, each pixel in the current 8x8 block has three prédiction values and a weighted average of these three prédiction values is used as the final prédiction for the respective pixel.

[0261] When a neighboring block is not coded or coded as intra, Le., the neighboring block does not hâve an available motion vector, the motion vector of the current 8x8 block is used as the neighboring motion vector. Meanwhile, for the third and fourth 8x8 block of a current macroblock (as shown in FIG. 7), the below neighboring block may not be used (e.g., always not used or not always used). In other words, in some examples, for each MB, no motion information from MBs below it will be used to reconstruct the pixels of the current MB during the OBMC.

[0262] The following describes OBMC as proposed in HEVC. In HEVC, OBMC was also proposed to smooth the PU boundary in U.S. Provîsional Application No. 61/561,783, filed November 18,2011, U.S. Application No. 13/678,329, filed November 15,2012, U.S. Provîsional Application No. 61/431,480, filed January 10,

2011, U.S. Provîsional Application No. 61/450,532, filed March 8,2011, and U.S.

Application No. 13/311,834, filed December 6,2011. An example of the method proposed in HEVC is shown in FIGS. 8A and 8B, where the white région is the first prédiction unit (PU) denoted by PU0, and the gray région is the second PU denoted by PU1). When a CU contains two (or more) PUs, lines/columns near the PU boundary are smoothed by OBMC. For pixels marked with “A” or “B” in PU0 or PU1, two prédiction values are generated, i.e., by applying motion vectors of PU0 and PU1, respectively, and a weighted average of them is used as the final prédiction. [0263] Moreover, in U.S. Provîsional Application No. 62/107,964, filed January 26, 2015, and U.S. Provîsional Application No. 62/116,631, filed February 16,2015, a CU30 level flag indicating whether OBMC is applied for a current CU, namely OBMC flag, has been proposed.

[0264] It is observed that, when OBMC is not applied to one coding unit (e.g., the signaled flag is 0), transforms other than DCT-II are not efficient. Therefore, the additional signaling for indication of usage of multiple transforms is redundant.

[0265] As described, video encoder 20 may signal (e.g., generate in the bitstream) a CU-level flag to indicate whether OBMC enabled or not for the current CU. In some examples, when this OBMC flag has been signaled as 1 (ïndïcating OBMC is enabled for the current CU), only the default DCT-II is used for each TU, and therefore video encoder 20 may not signal anything for the transform sélection, i.e., neither the CUlevel flag nor the TU-level index is signaled.

[0266] The following describes examples of optimizing video encoder 20. For instance, the following examples may be for video encoder 20. However, it may be possible for video décoder 30 to perform similar techniques.

[0267] At the encoder (e.g., video encoder 20), when the proposed multiple transforms are applied for the current TU, for transform sizes larger than or equal to MxN, only the M’xN’ low frequency coefficients are calculated and other coefficients are set as 0 wherein (M*<=M and N’ <=N and M’*N’<M*N). In one example, the value ofeach of M and N is 32 and the value of each of M’and N’ is 16. In this example, coefficients positioned at locations greater than M’and/or greater than N’ may be consîdered as higher frequency coefficients. In general, the coefficients further to the right of the TU and further to the bottom of the TU may be consîdered as higher frequency coefficients. [0268] At the encoder, for a certain coding mode, if the coding cost using the default transform, e.g., DCT-II, is larger than the current smallest coding cost multiplied by a given threshold value, then the proposed multiple transforms are skipped. The coding cost can be the rate-distortion cost, sum of absolute prédiction residual, sum of squared prédiction residual or sum of absolute transform différence. The threshold value may dépend on the coding block size. In one example, the value of the threshold is 1.1. [0269] At the encoder, for a certain Intra prédiction direction mode, if the coding cost using the default transform, e.g., DCT-II, is larger than the coding cost of the best Intra prédiction direction mode multiplied by a given threshold value, then the proposed multiple transforms are not applied and skipped for this Intra prédiction mode. The coding cost can be the rate-distortion cost, sum of absolute prédiction residual, sum of squared prédiction residual or sum ofabsolute transform différence. The threshold values may dépend on the coding block size. In one example, the value for the thresholds are 1.47, 1.28, 1.12 and 1.06 for 4x4,8x8,16x16 and 32x32 block sizes, respectively.

[0270] At the encoder, if the coding cost of NxN Intra PU partition using the default transform, e.g., DCT-II, is larger than the coding cost of the 2Nx2N Intra PU partition

multip! ied by a given threshold value, then the proposed multiple transforms are not appiied and skipped for NxN Intra PU partition. The coding cost can be the ratedistortion cost, sum of absolute prédiction residual, sum ofsquared prédiction residual or sum of absolute transform différence. The threshold values may dépend on the coding block size. In one example, the value for the threshold is 12.

[0271] At the encoder, if the coding cost of 2Nx2N Intra PU partition mode using the default transform, e.g., DCT-II, is larger than the coding cost ofthe best Inter coding modes multiplied by a given threshold value, then the proposed multiple transforms are not appiied and skipped for Intra PU modes. The coding cost can be the rate-distortion 10 cost, sum of absolute prédiction residual, sum of squared prédiction residual or sum of absolute transform différence. The threshold values may dépend on the coding block size. In one example, the value for the threshold is 1.4.

[0272] At the encoder, ifusing one ofthe multiple transform candidates is generatïng ail zéro coefficients for the current block, then the remaining transform candidates are 15 not appiied and are skipped for the current block. Altematively or additionally, ifusing the default transform (e.g., DCT-II) is generatïng ail zéro coefficients for the current block, then the multiple transform candidates are not appiied and are skipped for the current block, and only the default transform (e.g., DCT-II) is used for coding the current block.

[0273] At the encoder, when the OBMC flag has been signaled and it is indicating

OBMC off, then the one-bit flag, indicating whether only one default transform is appiied, is still signaled as a default value (e.g., 0), which indicates the default transform (e.g., DCT-II) is appiied, and the multiple transform candidates are not appiied and are skipped for the current block.

[0274] FIG. 9 is a flowchart illustrâting an example method of decoding video data.

Video décoder 30 may détermine a plurality of transform subsets, each subset identifying one or more candidate transforms, where at least one transform subset identifies a plurality of candidate transforms (200). For example, prédiction processing unit 152 may retrieve the plurality of transform subsets from the transform subsets stored in video data memory 151. The plurality of transform subsets may be pre-stored in video data memory 151 or information identifying how to construct the transform subsets may be received from video encoder 20. Video décoder 30 may select a first transform subset from the plurality of transform subsets for a left transform for a current coefficient block ofthe video data, and select a second transform subset from the

plurality of transform subsets for a right transform for the current coefficient block of the video data (202). For example, prédiction processing unit 152 may select the first and second transform subsets based on the intra-prediction mode information signaled in the video bitstream or based on a position of the video block being decoded as a few 5 example ways to détermine the transform subsets.

[0275] Video décoder 30 may détermine the left transform from the selected first transform subset, and détermine the right transform from the selected second transform subset (204). For example, prédiction processing unit 152 may receive information in the bitstream such as indices into the selected transform subsets or may implicitly 10 détermine the transforms based on the number of nonzero coefficients. Video décoder may détermine a current transform block based on the left transform, right transform, and the current coefficient block (206). For example, inverse transform processing unit 156 may détermine the current transform block by applying the left transform and right transform on the coefficient block outputted by inverse quantization unit 154. Video décoder 30 may reconstruct (e.g., intra-prediction or inter-prediction décodé) a video block based on the current transform block and a prédictive block (208). For example, reconstruction unit 158 may add the current transform block (which is a residual between of the video block and the prédictive block) to the prédictive block to reconstruct the video block.

[0276] FIG. 10 is a flowchart illustrating an example method of encoding video data.

Video encoder 20 may détermine a plurality of transform subsets, each subset identifying one or more candidate transforms, where at least one transform subset identifies a plurality of candidate transforms (300). For example, prédiction processing unît 100 may retrieve the plurality of transform subsets from the transform subsets stored in video data memory 101. The plurality of transform subsets may be pre-stored in video data memory 101. Video encoder 20 may select a first transform subset from the plurality of transform subsets for a left transform for a current transform block of a video block of the video data, and select a second transform subset from the plurality of transform subsets for a right transform for the transform block of the video block of the video data (302). For example, prédiction processing unit 100 may select the first and second transform subsets based on the intra-prediction mode information that entropy encoding unit 118 generates in the video bitstream or based on a position of the video block being encoded as a few example ways to détermine the transform subsets.

[0277] Video encoder 20 may détermine the left transform from the selected first transform subset, and détermine the right transform from the selected second transform subset (304). For example, prédiction processing unit 100 may test the various détermine transforms to identify the transform that provides good video coding qualîty.

Video encoder 20 may détermine a current coefficient block based on the left transform, right transform, and the current transform block (306). For example, transform processing unit 104 may détermine the current coefficient block by applying the left transform and right transform on the transform block outputted by reconstruction unît 102. Video encoder 20 may generate a video bitstream with information (e.g., signal information) indicative of coefficients of the current coefficient block used for reconstruction of the video block (308). For example, entropy encoding unit 118 may output information that video décoder 30 uses to reconstruct the video block.

[0278] It should be understood that ail of the techniques described herein may be used individually or in combination. This disclosure includes several signaling methods 15 which may change depending on certain factors such as block size, slice type etc. Such variation in signaling or inferring the syntax éléments may be known to the encoder and décoder a-priori or may be signaled explicitly in the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header, at a tile level or elsewhere.

[0279] It is to be recognized that depending on the example, certain acts or events of any of the techniques described herein can be performed in a different sequence, may be added, merged, or left out altogether (e.g., not ail described acts or events are necessary for the practice of the techniques). Moreover, in certain examples, acts or events may be performed concurrently, e.g., through multi-threaded processing, interrupt processing, or multiple processors, rather than sequentially. In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of thîs disclosure may be performed by a combination of units or modules associated with a video coder.

[0280] While particular combinations of various aspects of the techniques are described 30 above, these combinations are provided merely to illustrate examples of the techniques described in this disclosure. Accordingly, the techniques ofthis disclosure should not be limited to these example combinations and may encompass any conceivable combination ofthe various aspects ofthe techniques described in this disclosure.

[0281] In one or more examples, the fonctions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the fonctions may be stored on or transmitted over, as one or more instructions or code, a computer-readable medium and executed by a hardware-based processing unit.

Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media. In this manner, computer-readable media generally may correspond to tangible computer-readable storage media which is non-transitory. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implémentation of the techniques described in this disclosure. A computer program product may include a computer-readable medium. By way ofexample, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signais, or other transient media, but are instead dîrected to non-transient, tangible storage media. Disk and dise, as used herein, includes compact dise (CD), laser dise, optical dise, digital versatile dise (DVD), floppy disk and Blu-ray dise, where disks usually reproduce data magnetically, whîle dises reproduce data optically with lasers. Combinations of the above should also be included within the scope of computerreadable media.

[0282] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application spécifie integrated circuits (ASICs), field programmable logïc arrays (FPGAs), or other équivalent integrated or discrète logic circuitry. Accordingly, the terni “processor,” as used herein may refer to any of the foregoing structure or any other structure suitable for implémentation ofthe techniques described herein. In addition, in some aspects, the functionalîty described herein may be provided within dedicated hardware and/or software modules confîgured for encoding and decodîng, or incorporated in a combined codée. Also, the techniques could be folly implemented in one or more circuits or logic éléments.

[0283] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chip set). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, various units may be combined in a codée hardware unit or provided by a collection of interoperatîve hardware units, including one or more processors as described above, in conjunction with suitable software and/or firmware. [0284] Various examples hâve been described. These and other examples are within the 10 scope of the following daims.

Claims (60)

1. Claims

   1. A method of decoding video data, the method comprising: determining a pluralîty of transform subsets for transforming a current coefficient block of a video block encoded according to one of a pluralîty of prédiction

   5 modes, the prédiction modes comprising a pluralîty of intra-prediction modes and a pluralîty of inter prédiction modes, each subset identifyîng one or more candidate transforms, wherein at Ieast one transform subset identifies a pluralîty of candidate transforms, wherein determining the pluralîty of transform subsets comprises determining either a firsL same, one of the pluralîty of transform subsets for each of the 10 intra-prediction modes, or a second, same, one of the pluralîty of transform subsets for each of the inter-prediction modes;

   selecting a first transform subset from the determïned pluralîty of transform subsets for a left transform for the current coefficient block of the video data;

   selecting a second transform subset from the determïned pluralîty of transform

   15 subsets for a right transform for the current coefficient block of the video data;

   determining the left transform from the selected first transform subset; determining the right transform from the selected second transform subset; determining a current transform block based on the left transform, right transform, and the current coefficient block; and

   20 reconstructing the video block based on the current transform block and a prédictive block.
2. 2. The method of claim 1, further comprising:

   determining an intra-prediction mode ofthe video block based on a prédiction

   25 mode of the video block being intra-prediction, wherein selecting the first transform subset comprises selecting the first transform subset based on the determïned intra-prediction mode, wherein selecting the second transform subset comprises selecting the second transform subset based on the determïned intra-prediction mode, and

   30 wherein reconstructing the video block comprises reconstructing the video block based on the determïned intra-prediction mode.
3. 3. The method of claim 1, further comprising:

   determinîng a location of the current transform block in the video block based on a prédiction mode ofthe video block being inter-prediction, wherein selecting the first transform subset comprises selecting the first

   5 transform subset based on the determined location of the current transform block, and wherein selecting the second transform subset comprises selecting the second transform subset based on the determined location ofthe current transform block.
4. 4. The method of claim 1, further comprising:

   10 receiving a first transform subset index into the first transform subset; and receiving a second transform subset index into the second transform subseζ wherein determining the left transform comprises determinîng the left transform based on a transform in the first transform subset identified by the first transform subset index, and

   15 wherein determining the right transform comprises determinîng the right transform based on a transform in the second transform subset identified by the second transform subset index.
5. 5. The method of claim 1, further comprising:

   20 determining that a number of nonzero coefficients in the current coefficient block is less than a threshold, wherein determining the left transform comprises determining that a first transform identified in the first transform subset is the left transform without receiving a transform subset index into the first transform subset, in response to determinîng that

   25 the number of nonzero coefficients in the current coefficient block is less than the threshold, and wherein determinîng the right transform comprises determining that a first transform identified in the second transform subset is the right transform without receiving a transform subset index into the second transform subset, in response to

   30 determining that the number of nonzero coefficients in the current coefficient block is less than the threshold.
6. 6. The method of claim 1, further comprising:

   receiving a flag indicating that not ail transform blocks of the video block that includes the current transform block are transformed using the same transform, wherein selecting the first transform subset for the left transform for the current 5 coefficient block of the video data, selecting the second transform subset for the right transform for the current coefficient block ofthe video data, determining the left transform from the selected first transform subseL and determining the right transform from the selected second transform subset comprise selecting the first transform subset for the left transform for the current coefficient block of the video data, selecting the 10 second transform subset for the right transform for the current coefficient block of the video data, determining the left transform from the selected first transform subset, and determining the right transform from the selected second transform subset, in response to receiving the flag indicating that not ail transform blocks of the video block that includes the current transform block are transformed using the same transform.
7. 7. The method of daim 6, wherein the video block comprises one of:

   a coding tree unit (CTU), a coding unit (CU), or a prédiction unit (PU).
8. 8. The method of claim 1, wherein at least one of the first transform subset or the 20 second transform subset includes a transform that is different than a discrète cosine transform (DCT)-II transform and a discrète sine transform (DST)-VII transform.
9. 9. The method of claim 1, wherein the first transform subset and the second transform subset include different transform types.
10. 10. The method of claim 1, wherein the plurality of transform subsets comprises three or more transform subsets.
11. 11. The method of claim 1, whereîn the candidate transforms are different transform 30 types.
12. 12. The method of daim 1, whereîn determining the plurality of transform subsets comprises determining the plurality of transform subsets based on a size of the video block.
13. 13. The method of claim 1, further comprising:

    receiving from a bitstream information indicating a prédiction mode;

    receiving from the bitstream information indicating coefficients ofthe current coefficient block;

    5 constructing the current coefficient block based on the received information indicating the coefficients; and determinîng the prédictive block based on the prédiction mode.
14. 14. The method of claim 13, wherein the prédiction mode comprises one of an inter10 prédiction mode or an ïnter-prediction mode.
15. 15. The method of claim 1, wherein the current transform block is a residual ofthe video block and the prédictive block.

    15
16. 16. A method ofencoding video data, the method comprising:

    determinîng a plurality of transform subsets for transformîng a current transform block of a vîdeo block encoded according to one of a plurality of prédiction mode, the prédiction modes comprising a plurality of intra-prediction modes and a plurality of inter-prediction modes, each subset identifying one or more candidate transforms, 20 wherein at least one transform subset identifies a plurality of candidate transforms, wherein determinîng the plurality of transform subsets comprises determinîng either a first, same, one of the plurality transform subsets for each of the intra-prediction modes, or a second, same, one of the plurality of transform subsets for each of the interprediction modes;

    25 selecting a first transform subset from the determined plurality of transform subsets for a left transform for the current transform block of the video block of the video data;

    selecting a second transform subset from the determined plurality of transform subsets for a right transform for the current transform block of the video block of the 30 video data;

    determinîng the left transform from the selected first transform subset; determinîng the right transform from the selected second transform subset; determinîng a current coefficient block based on the left transform, right transform, and the current transform block; and generating a video bitstream that includes information indicative of coefficients of the current coefficient block used for reconstruction of the video block.
17. 17. The method of c laim 16, further comprising:

    determining an intra-prediction mode of the video block, wherein selecting the first transform subset comprises selecting the first transform subset based on the determined intra-prediction mode, and wherein selecting the second transform subset comprises selecting the second transform subset based on the determined intra-prediction mode.
18. 18. The method of claim 16, further comprising:

    determining a location of the current transform block in the video block based on the video block being inter-prediction encoded, wherein selecting the first transform subset comprises selecting the first transform subset based on the determined location ofthe current transform block, and wherein selecting the second transform subset comprises selecting the second transform subset based on the determined location of the current transform block.
19. 19. The method of claim 16, further comprising:

    generating in the video bitstream a first transform subset index into the first transform subset to identify a transform in the first transform subset used to détermine the current coefficient block; and generating in the video bitstream a second transform subset index into the second transform subset to identify a transform in the second transform subset used to détermine the current coefficient block.
20. 20. The method of claim 19, further comprising:

    determining a number of nonzero coefficients in the current coefficient block, wherein signaling the first transform subset index comprises signafing the first transform subset index based on the number of nonzero coefficients being greater than a threshold, and wherein signaling the second transform subset index comprises signaling the second transform subset index based on the number of nonzero coefficients being greater than the threshold.
21. 21. The method of claim 16, wherein at least one of the first transform subset or the second transform subset includes a transform that is different than a discrète cosine transform (DCT)-II transform and a discrète sine transform (DST)-VII transform.
22. 22. The method of claim 16, wherein the first transform subset and the second transform subset include different transform types.
23. 23. The method of claim 16, wherein the plurality of transform subsets comprises three or more transform subsets.
24. 24. The method of claim 16, wherein the candidate transforms are different transform types.
25. 25. The method of claim 16, wherein determining the plurality of transform subsets comprises determining the plurality of transform subsets based on a size of the video block.
26. 26. The method of claim 16, further comprising:

    determining the prédictive block; and generating in the video bitstream information indicative of a prédiction mode of the video block based on the prédictive block.
27. 27. The method of claim 26, wherein the prédictive block is a block in the same picture as the video block based on the video block beîng intra-predicted or in a picture different that the picture that includes the video block based on the video block being inter-predicted.
28. 28. The method of claim 16, further comprising:

    determining the current transform block as a residual between the video block and the prédictive block.
29. 29. A device for video decoding video data, the device comprising:

    a video data memory configured to store the video data and transform subsets, each subset identifying one or more candidate transforms, wherein at least one transform subset identifies a pluralîty of candidate transforms; and

    5 a video décoder comprising integrated circuitry, the video décoder configured to:

    détermine a pluralîty of transform subsets from the stored transform subsets for transforming a current coefficient block of a video block encoded according to one of a pluralîty of prédiction modes, the prédiction modes comprising a pluralîty of intra-prediction modes and a pluralîty of inter10 prédiction modes, wherein to détermine the pluralîty of transform subsets, the video décoder is configured to détermine either a fîrst, same, one of the pluralîty of transform subsets for each of the intra-prediction modes or a second, same, one of the pluralîty of transform subsets for each of the inter-predictîon modes;

    select a fîrst transform subset from the determined pluralîty of transform 15 subsets for a left transform for the current coefficient block of the video data;

    select a second transform subset from the determined pluralîty of transform subsets for a right transform for the current coefficient block of the video data;

    détermine the left transform from the selected fîrst transform subset;

    20 détermine the right transform from the selected second transform subset;

    détermine a current transform block based on the left transform, right transform, and the current coefficient block; and reconstruct the video block based on the current transform block and a prédictive block.
30. 30. The device of claim 29, wherein the video décoder is configured to détermine an intra-prediction mode of the video block based on a prédiction mode of the video block being intra-prediction, wherein to select the fîrst transform subset, the video décoder is configured to select the fîrst transform subset based on the determined intra-prediction

    30 mode, wherein to select the second transform subset, the video décoder is configured to select the second transform subset based on the determined intra-prediction mode, and wherein to reconstruct the video block, the video décoder is configured to reconstruct the video block based on the determined intra-prediction mode.
31. 31. The device of claim 29, wherein the video décoder is configured to détermine a location of the current transform block in the video block based on a prédiction mode of the video block being inter-prediction, wherein to select the first transform subset, the video décoder is configured to select the first transform subset based on the determined

    5 location of the current transform block, and wherein to select the second transform subset, the video décoder is configured to select the second transform subset based on the determined location of the current transform block.
32. 32. The device of claim 29, wherein the video décoder is configured to:

    10 receive a first transform subset index into the first transform subset; and receive a second transform subset index into the second transform subset, wherein to détermine the left transform, the video décoder is configured to détermine the left transform based on a transform in the first transform subset identified by the first transform subset index, and

    15 wherein to détermine the right transform, the video décoder is configured to détermine the right transform based on a transform in the second transform subset identified by the second transform subset index.
33. 33. The device of claim 29, wherein the video décoder is configured to:

    20 détermine that a number of nonzero coefficients in the current coefficient block is less than a threshold, wherein to détermine the left transform, the video décoder is configured to détermine that a first transform identified in the first transform subset is the left transform without receivîng a transform subset index into the first transform subset, in 25 response to determining that the number of nonzero coefficients in the current coefficient block is less than the threshold, and wherein to détermine the right transform, the video décoder is configured to détermine that a first transform identified in the second transform subset is the right transform without receivîng a transform subset index into the second transform subset, 30 in response to determining that the number of nonzero coefficients in the current coefficient block is less than the threshold.
34. 34. The device of claim 29, wherein the video décoder is configured to: receive a flag indicating that not ail transform blocks of the video block that ïncludes the current transform block are transformed using the same transform, wherein to select the first transform subset for the left transform for the current coefficient block ofthe video data, select the second transform subset for the right transform for the current coefficient block ofthe video data, détermine the left transform from the selected first transform subset, and détermine the right transform from the selected second transform subset, the video décoder is configured to select the first transform subset for the left transform for the current coefficient block of the video data, select the second transform subset for the right transform for the current coefficient block of the video data, détermine the left transform from the selected first transform subset, and détermine the right transform from the selected second transform subset, in response to receiving the flag indicating that not ail transform blocks of the video block that includes the current transform block are transformed using the same transform.
35. 35. The device of claim 34, wherein the video block comprises one of:

    a coding tree unit (CTU), a coding unit (CU), or a prédiction unit (PU).
36. 36. The device of claim 29, wherein at least one of the first transform subset or the second transform subset includes a transform that is different than a discrète cosine transform (DCT)-II transform and a discrète sine transform (DST)-VII transform.
37. 37. The device of claim 29, wherein the first transform subset and the second transform subset tnclude different transform types.
38. 38. The device of claim 29, wherein the plurality of transform subsets comprises three or more transform subsets.
39. 39. The device of claim 29, wherein the candidate transforms are different transform types.
40. 40. The device of claim 29, wherein to détermine the plurality of transform subsets, the video décoder is configured to détermine the plurality of transform subsets based on a size ofthe video block.

    5
41. 41. The device of claim 29, wherein the video décoder is configured to:

    receive from a bitstream information indicating a prédiction mode; receive from the bitstream information indicating coefficients ofthe current coefficient block;

    construct the current coefficient block based on the received information

    10 indicating the coefficients; and détermine the prédictive block based on the prédiction mode.
42. 42. The device of claim 41, wherein the prédiction mode comprises one of an interprediction mode or an inter-prediction mode.
43. 43. The device of claim 29, wherein the current transform block is a residual of the video block and the prédictive block.
44. 44. A device for encoding video data, the device comprising:

    a video data memory confîgured to store the video data and transform subsets, each subset identîfying one or more candidate transforms, wherein at least one transform subset identifies a plurality of candidate transforms; and

    5 a video encoder confîgured to:

    détermine a plurality of transform subsets from the stored transform subsets for transforming a current transform block of a video block encoded according to one of a plurality of prédiction mode, the prédiction modes comprising a plurality of intra-prediction modes and a plurality of inter10 prédiction modes, wherein to détermine the plurality of transform subsets the video encoder is confîgured to détermine either a first, same, one of the plurality transform subsets for each of the intra-prediction modes, or a second, same, one of the plurality of transform subsets for each ofthe inter-prediction modes;

    select a first transform subset from the determined plurality of transform 15 subsets for a left transform for the current transform block of the video block of the video data;

    select a second transform subset from the determined plurality of transform subsets for a right transform for the current transform block of the video block of the video data;

    20 détermine the left transform from the selected first transform subset;

    détermine the right transform from the selected second transform subset;

    détermine a current coefficient block based on the left transform, right transform, and the current transform block; and generate a video bitstream that includes information indicative of 25 coefficients of the current coefficient block used for reconstruction of the video block.
45. 45. The device of claim 44, wherein the video encoder is confîgured to détermine an intra-prediction mode of the video block, wherein to select the first transform subset, the

    30 video encoder is confîgured to select the first transform subset based on the determined intra-prediction mode, and wherein to select the second transform subset, the video encoder is confîgured to select the second transform subset based on the determined intra-prediction mode.
46. 46. The device of claim 44, wherein the video encoder is configured to déterminé a location of the current transform block in the video block based on the video block being inter-prediction encoded, wherein to select the first transform subset, the video encoder is configured to select the first transform subset based on the determined

    5 location of the current transform block, and wherein to select the second transform subset, the video encoder is configured to select the second transform subset based on the determined location ofthe current transform block.
47. 47. The device of claim 44, wherein the video encoder is configured to:

    10 generate in the video bitstream a first transform subset index into the first transform subset to identify a transform in the first transform subset used to détermine the current coefficient block; and generate in the video bitstream a second transform subset index into the second transform subset to identify a transform in the second transform subset used to

    15 détermine the current coefficient block.
48. 48. The device of claim 47, wherein the video encoder is configured to détermine a . number of nonzero coefficients in the current coefficient block, wherein to generate in the video bitstream the first transform subset index, the video encoder is configured to

    20 generate in the video bitstream the first transform subset index based on the number of nonzero coefficients being greater than a threshold, and wherein to generate the second transform subset index, the video encoder is configured to generate in the video bitstream the second transform subset index based on the number of nonzero coefficients being greater than the threshold.
49. 49. The device of claim 44, wherein at least one of the first transform subset or the second transform subset includes a transform that is different than a discrète cosine transform (DCT)-II transform and a discrète sine transform (DST)-VII transform.

    30
50. 50. The device of claim 44, wherein the first transform subset and the second transform subset include different transform types.
51. 51. The device of claim 44, wherein the plurality of transform subsets comprises three or more transform subsets.
52. 52. The device of claim 44, wherein the candidate transforms are different transform types.
53. 53. The device of claim 44, wherein to détermine the plurality of transform subsets, 5 the video encoder is confîgured to détermine the plurality of transform subsets based on a size of the video block.
54. 54. The device of claim 44, wherein the video encoder is confîgured to: détermine the prédictive block; and

    10 generate in the video bitstream information indicative of a prédiction mode of the video block based on the prédictive block.
55. 55. The device of claim 44, wherein the prédictive block îs a block in the same picture as the video block based on the video block being intra-predicted or in a picture

    15 different that the picture that includes the video block based on the video block being inter-predicted.
56. 56. The device of claim 44, wherein the video encoder is confîgured to:

    détermine the current transform block as a residual between the video block and 20 the prédictive block.
57. 57. A device for decodîng video data, the device comprising:

    means for determining a plurality of transform subsets for transforming a current coefficient block of a video block encoded according to one of a plurality of prédiction 25 modes, the prédiction modes comprising a plurality of intra-predîctîon modes and a plurality of inter prédiction modes, each subset identifying one or more candidate transforms, wherein at least one transform subset identifies a plurality of candidate transforms, wherein the means for determining the plurality of transform subsets comprises means for determining either a first, same, one ofthe plurality of transform 30 subsets for each of the intra-prediction modes, or a second, same, one of the plurality of transform subsets for each ofthe inter-prediction modes;

    means for selecting a first transform subset from the determined plurality of transform subsets for a left transform for the current coefficient block of the video data;

    means for selecting a second transform subset from the determined plurality of transform subsets for a right transform for the current coefficient block of the video data;

    means for determining the left transform from the selected first transform subset;

    5 means for determining the right transform from the selected second transform subset;

    means for determining a current transform block based on the left transform, right transform, and the current coefficient block; and means for reconstruct in g the video block based on the current transform block

    10 and a prédictive block
58. 58. A non-transitory computer-readable storage medium storing instructions that when executed cause a video décoder of a device for video decoding to:

    détermine a plurality of transform subsets for transforming a current coefficient

    15 block of a video block encoded accordîng to one of a plurality of prédiction modes, the prédiction modes comprising a plurality of intra-prediction modes and a plurality of inter prédiction modes, each subset identifying one or more candidate transforms, wherein at least one transform subset identifies a plurality of candidate transforms, wherein the instructions that cause the video décoder to détermine the plurality of

    20 transform subsets comprise instructions that cause the video décoder to détermine either a first, same, one ofthe plurality oftransform subsets for each ofthe intra-prediction modes, or a second, same, one of the plurality of transform subsets for each of the interprediction modes;

    select a first transform subset from the determined plurality of transform subsets 25 for a left transform for the current coefficient block of the video data;

    select a second transform subset from the determined plurality of transform subsets for a right transform for the current coefficient block ofthe video data;

    détermine the left transform from the selected first transform subset; détermine the right transform from the selected second transform subset;

    30 détermine a current transform block based on the left transform, right transform, and the current coefficient block; and reconstruct the video block based on the current transform block and a prédictive block.
59. 59. A device for encoding video data, the device comprising:

    means for determining a pluralîty of transform subsets for transforming a current transform block of a video block encoded according to one of a pluralîty of prédiction mode, the prédiction modes comprising a pluralîty of intra-prediction modes and a

    5 pluralîty of inter-prediction modes, each subset identifyîng one or more candidate transforms, wherein at least one transform subset identifies a pluralîty of candidate transforms, wherein the means for determining the pluralîty of transform subsets comprise means for determining either a first, same, one ofthe pluralîty transform subsets for each of the intra-prediction modes, or a second, same, one of the pluralîty of 10 transform subsets for each of the inter-prediction modes;

    means for selecting a first transform subset from the determïned pluralîty of transform subsets for a left transform for the current transform block of the video block ofthe video data;

    means for selecting a second transform subset from the determïned pluralîty of

    15 transform subsets for a right transform for the current transform block of the video block of the video data;

    means for determining the left transform from the selected first transform subset; means for determining the right transform from the selected second transform subset;

    20 means for determining a current coefficient block based on the left transform, right transform, and the current transform block; and means for generating a video bitstream that includes information indicative of coefficients of the current coefficient block used for reconstruction of the video block.
60. 60. A non-transitory computer-readable storage medium storing instructions that when executed cause a video encoder of a device for video encoding to:

    détermine a plurality of transform subsets for transformîng a current transform block of a video block encoded according to one of a plurality of prédiction mode, the

    5 prédiction modes comprising a plurality of intra-prediction modes and a plurality of inter-prediction modes, each subset identifying one or more candidate transforms, wherein at least one transform subset identifies a plurality of candidate transforms, wherein the instructions that cause the video encoder to détermine the plurality of transform subsets comprise instructions that cause the video encoder to détermine either 10 a first, same, one of the plurality transform subsets for each of the intra-prediction modes, or a second, same, one of the plurality of transform subsets for each of the interprediction modes;

    select a first transform subset from the determined plurality of transform subsets for a left transform for the current transform block of the video block of the video data;

    15 select a second transform subset from the determined plurality of transform subsets for a right transform for the current transform block of the video block of the video data;

    détermine the left transform from the selected first transform subset; détermine the right transform from the selected second transform subset;

    20 détermine a current coefficient block based on the left transform, right transform, and the current transform block; and generate a video bitstream that includes information indicative of coefficients of the current coefficient block used for reconstruction ofthe video block.