EP3202146B1

EP3202146B1 - Method and apparatus of alternative transform for video coding

Info

Publication number: EP3202146B1
Application number: EP15862450.2A
Authority: EP
Inventors: Chih-Wei Hsu
Original assignee: HFI Innovation Inc
Current assignee: HFI Innovation Inc
Priority date: 2014-11-28
Filing date: 2015-11-26
Publication date: 2022-08-17
Anticipated expiration: 2035-11-26
Also published as: CN110855988B; WO2016082774A1; US11089332B2; MY190919A; EP4090018A1; US20190327491A1; CN107005695B; US10390045B2; CN110855988A; CA2966862A1; ES2930030T3; EP3202146A4; CN107005695A; US20170366824A1; CA2966862C; EP3202146A1

Description

CROSS REFERENCE TO RELATED APPLICATIONS

The present invention claims priority to U.S. Provisional Patent Application, Serial No. 62/085,358, filed on November 28, 2014 .

TECHNICAL FIELD

The present invention relates video coding. In particular, the present invention relates to using alternative transforms for Inter prediction coded blocks in video coding. More particularly, the present invention relates to a method, an apparatus and a data stream according to the pre-characterizing parts of the appended independent claims, respectively. Such method, apparatus and data stream are shown in JCTVC-G281.

BACKGROUND

US 2014/0140393 A1 discloses a method of decoding video information, wherein the method comprises a step of receiving video information associated with a base layer and an enhancement layer. The method further comprises a step of determining whether the video information associated with the enhancement layer is to be determined based on the video information associated with the base layer. The method further comprises a step of selecting between a first transform and a second transform based at least in part on at least one of a TU size and a color component type of the enhancement layer video information in response to determining that the video information associated with the enhancement layer is to be determined based on the video information associated with the base layer. The method further comprises a step of decoding the video information associated with the enhancement layer using the selected transform.
US 2014/0056361 A1 discloses that only for a coding unit (CU) encoded as intra base layer (Intra BL) mode or generalized residual prediction (GRP) mode with non-zero weighting, a flag is transmitted for the CU. This flag is transmitted for a CU, but may only select the transform for transform unit (TU) size larger than a threshold in this CU. For CU smaller than this threshold, discrete sine transform (DST) or discrete cosine transform (DCT) is always used.
High-Efficiency Video Coding (HEVC) is a new international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on the hybrid block-based motion-compensated DCT-like transform coding architecture. The basic unit for compression, termed coding unit (CU), is a 2Nx2N square block. A CU may begin with a largest CU (LCU), which is also referred as coded tree unit (CTU) in HEVC and each CU can be recursively split into four smaller CUs until the predefined minimum size is reached. Once the splitting of CU hierarchical tree is done, each CU is further split into one or more prediction units (PUs) according to prediction type and PU partition. Each CU or the residual of each CU is divided into a tree of transform units (TUs) to apply 2D transforms such as DCT (discrete cosine transform) or DST (discrete sine transform).
Like many other precedent standards, HEVC adopts Discrete Cosine Transform type II (DCT-II) as its core transform because it has a strong "energy compaction" property. Most of the signal information tends to be concentrated in few low-frequency components of the DCT-II, which approximates the Karhunen-Loève Transform (KLT). As known in the art, KLT is the optimal transform to de-correlate signals that can be modelled by Markov processes. The N-point DCT-II of the signal f[n] is defined in equation (1). ${\hat{ƒ}}_{DCT - II} [k] = λ_{k} \frac{2}{\sqrt{N}} \sum_{n = 0}^{N - 1} ƒ [n] \cos [\frac{kπ}{N} (n + \frac{1}{2})], k = 0, 1,2, \dots, N - 1, λ_{k} = {\begin{array}{l} 2^{- 0.5}, k = 0 \\ 1, k \neq 0 \end{array} .$
For Intra-predicted residue, there are other transforms that are found to be more efficient than DCT-II. In JCTVC-B024 (Yeo, et al., "Mode-Dependent Fast Separable KLT for Block-based Intra Coding", Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 2nd Meeting: Geneva, CH, 21-28 July, 2010, Document: JCTVC-B024),JCTVC-C108 (Saxena, et al., "Jointly optimal intra prediction and adaptive primary transform", Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 3rd Meeting: Guangzhou, CN, 7-15 October, 2010, Document: JCTVC-C108) and JCTVC-E125 (Saxena, et al., "CE7: Mode-dependent DCT/DST without 4∗4 full matrix multiplication for intra prediction", Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 5th Meeting: Geneva, CH, 16-23 March, 2011, Document: JCTVC-E125), Discrete Sine Transform (DST) was introduced as an alternative to DCT for oblique Intra modes. For Inter-predicted residue, DCT-II is the only transform used in the current HEVC. However, the DCT-II is not the optimal transform for all cases. In JCTVC-G281 (An, et al., "Non-CE7: Boundary-Dependent Transform for Inter-Predicted Residue", Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, Document: JCTVC-G281), the Discrete Sine Transform type VII (DST-VII) and Discrete Cosine Transform type IV (DCT-IV) are proposed to replace DCT-II in some cases. When using Inter prediction for a PU, the prediction error (i.e., prediction residue or residue) is usually larger near the PU boundaries than in the middle of the PU. Fig. 1 illustrates an example of the squared residue values for a 4x4 PU. As shown in Fig. 1, the residue (i.e., prediction error) tends to have larger squared values at the PU boundaries.
When a PU is split into multiple TUs as shown in Fig. 2, the prediction error is larger near PU boundaries than near interior TU (non-PU) boundaries. In Fig. 2, PU 210 is split into four TUs (i.e., TU0, TU1, TU2 and TU3). Block 220 corresponds to the squared residue values of the upper-left TU (i.e., TU0), where the residues near the PU boundaries (indicated by thick solid lines) are generally larger than the residues near the interior TU boundaries (indicated by dashed lines). For TU0, a trend can be observed that the squared residue values peak roughly around the upper-left corner and decrease toward the lower-right corner. Similarly, for other TUs, the squared residue values also peak roughly around the respective PU corners and decrease toward the center of the PU.
The reason of this effect may be due to the different motion vectors (MV) between two neighboring PUs. In order to handle this uneven error distribution, alternative transforms such as DST-VII and DCT-IV can be used. Equations (2) and (3) show the N-point DST-VII and DCT-IV of signal f[n], respectively. ${\hat{ƒ}}_{DST - VII} [k] = \frac{2}{\sqrt{2 N + 1}} \sum_{n = 0}^{N - 1} ƒ [n] \sin \frac{(2 k + 1) (n + 1) π}{2 N + 1}, k = 0,1, \dots, N - 1,$
${\hat{ƒ}}_{DCT - IV} [k] = \frac{2}{\sqrt{N}} \sum_{n = 0}^{N - 1} ƒ [n] \cos [\frac{π}{N} (k + \frac{1}{2}) (n + \frac{1}{2})], k = 0,1, \dots, N - 1 .$

Based on the above observation, a method is disclosed in JCTVC-G281 that uses DST-VII or DCT-IV instead of DCT-II if one and only one of the two TU boundaries in the horizontal or vertical direction is a PU boundary. Table 1 shows the mapping from boundary type to transform type by using the DST-VII. Table 2 shows the mapping from boundary type to transform by using the DCT-IV.

Table 1.

TU Boundary		Horizontal Trans.	TU Boundary		Vertical Trans.
Left	Right	Horizontal Trans.	Top	Bottom	Vertical Trans.
non-PU	PU	DST-VII	non-PU	PU	DST-VII
PU	non-PU	F(DST-VII)	PU	non-PU	F(DST-VII)
PU	PU	DCT-II	PU	PU	DCT-II
non-PU	non-PU	DCT-II	non-PU	non-PU	DCT-II

Table 2.

TU Boundary		Horizontal Trans.	TU Boundary		Vertical Trans.
Left	Right	Horizontal Trans.	Top	Bottom	Vertical Trans.
non-PU	PU	F(DCT-IV)	non-PU	PU	F(DCT-IV)
PU	non-PU	DCT-IV	PU	non-PU	DCT-IV
PU	PU	DCT-II	PU	PU	DCT-II
non-PU	non-PU	DCT-II	non-PU	non-PU	DCT-II

The F(DST-VII) in Table 1 means flipping the DST matrix from left to right. The cases of using F(DST-VII) can also be implemented as flipping the input data first and then using DST-VII. It is also the case for F(DCT-IV). In Table 1 and Table 2, the items "non-PU" and "PU" mean non-PU boundary and PU boundary, respectively. In Table 1 and Table 2, when both of the TU boundaries in the horizontal or vertical direction are PU boundary or a non-PU boundary, DCT-II is used.
According to the Table 1, the four TUs in Fig. 2 will use transforms as shown in Table 3. Table 3.

TU Horizontal Trans. Vertical Trans.

TU0 F(DST-VII) F(DST-VII)

TU1 DST-VII F(DST-VII)

TU2 F(DST-VII) DST-VII

TU3 DST-VII DST-VII
While the Boundary Dependent Transform (BDT) method disclosed in JCTVC-G281 may improve the overall performance of the coding system. However, due to the characteristics of individual TU or PU, the BDT method according to JCTVC-G281 may not always achieve the best performance for a given block. It is desirable to develop a method to further improve the performance.

SUMMARY

The above-mentioned objectives are achieved by a method, an apparatus and a data stream according to the appended independent claims, respectively.
A method and apparatus for alternative transforms in a video coding system according to a control flag are disclosed. In the present invention, control flag for the current PU is determined. If the control flag has a first value, a first transform is applied to each current TU (transform unit) at an encoder side or an inverse transform of the first transform is applied to each current TU at a decoder side if the current TU has a first boundary type. Furthermore, a second transform is applied to each current TU at the encoder side or an inverse transform of the second transform is applied to each current TU at the decoder side if the current TU has a second boundary type. The first transform is different from the second transform. On the other hand, if the control flag has a second value, a selected transform is applied to each current TU at the encoder side or an inverse transform of the selected transform is applied to each current TU at the decoder side regardless of boundary type of the current TU.
In the present invention, the control flag can be signaled at a TU, PU or CU (coding unit) level. In the present invention, the control flag can be explicitly signaled only for the current PU, TU or CU having first sizes. In the present invention, the control flag can be explicitly signaled depending on other flag in a sequence, picture or slice header level.
In one embodiment that does not fall under the scope of the appended claims, the control flag is implicitly determined based on the boundary type of the current PU, and the first transform, the second transform, the selected transform or a combination thereof is determined depending on another flag being explicitly signaled.
In the present invention, the second transform corresponds to a flipped version of the first transform. The first transform may correspond to discrete sine transform type V-II (DST-VII) or discrete cosine transform type IV (DCT-IV). The first boundary type includes a left PU boundary plus a right non-PU boundary and a top non-PU boundary plus a bottom PU boundary, and the second boundary type includes a left non-PU boundary plus a right PU boundary and a top PU boundary plus a bottom non-PU boundary. Furthermore, the first boundary type further includes the left PU boundary plus the right PU boundary, the left non-PU boundary plus the right non-PU boundary, the top PU boundary plus the bottom PU boundary and the top non-PU boundary plus the bottom non-PU boundary.
The first transform, the second transform, the selected transform or a combination thereof may be dependent on block size of the current PU. For example, the first transform may correspond to DST-VII (discrete sine transform type V-II) if the block size of the current PU is 4x4, and the first transform may correspond to DCT-IV (discrete cosine transform type IV) if the block size of the current PU is 8x8 or 16x16. The selected transform may correspond to discrete cosine transform type II (DCT-II).
In another aspect that does not fall under the scope of the appended claims, a method and apparatus for alternative transforms in a video coding system according to a control flag are disclosed. According to an embodiment that does not fall under the scope of the appended claims, control flag for the current PU is determined. If the control flag has a first value, a first transform is applied to each current TU (transform unit) at an encoder side or an inverse transform of the first transform is applied to each current TU at a decoder side if the current TU belongs to a first type. Furthermore, a second transform is applied to each current TU at the encoder side or an inverse transform of the second transform is applied to each current TU at the decoder side if the current TU belongs to a second type. The first transform is different from the second transform. On the other hand, if the control flag has a second value, a selected transform is applied to each current TU at the encoder side or an inverse transform of the selected transform is applied to each current TU at the decoder side regardless of boundary type of the current TU. In one embodiment that does not fall under the scope of the appended claims, whether the current TU belongs to the first type or the second type is determined according to a boundary type, block size, mode information, or a combination thereof.

BRIEF DESCRIPTION OF DRAWINGS

Fig. 1 illustrates an example of the squared residue values for a 4x4 PU.
Fig. 2 illustrates an example that the prediction error is larger near PU (prediction unit) boundaries than near interior TU (non-PU) boundaries, where the PU is partitioned into four TUs (transform units).
Fig. 3 illustrates an exemplary flowchart of a coding system incorporating block characteristics-dependent transform based on a control flag according to an embodiment of the present invention.

DETAILED DESCRIPTION

The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.
As mentioned before, it is observed that the Inter prediction error is larger near the PU boundaries than that in the middle of the PU. Accordingly, a Boundary Dependent Transform (BDT) method is disclosed in JCTVC-G281 to adapt this uneven error distribution in a single PU. The BDT method selects transform according to the boundary type (i.e., non-PU boundary or PU boundary), which is summarized in Table 1 and Table 2.
However, for a given boundary type, the BDT method always applies a fixed-type transform to the block such as DST-VII or DCT-II. However, the characteristics of prediction residues may be quite different from block to block. The BDT method may not always achieve the best performance. Accordingly, the present invention discloses a method to use a control flag to control the BDT operation in order to further improve the performance of BDT. With the explicit control flag, an encoder can decide whether to turn on or off the BDT using mode decision methods such as Rate Distortion Optimization (RDO).The encoder then signals the control flag to inform decoder. For example, the control flag may be signaled in a TU, PU or CU level. In one example, the control flag is signaled only for the current PU, TU or CU having a specific size. However, the control flag may also be determined implicitly. For example, the value of the control flag may be dependent on the block size so that the boundary dependent transform is on for some block sizes and off for other block sizes. Alternatively, the value of the control flag may be dependent on mode information so that the boundary dependent transform is on for some prediction modes and off for other prediction modes. Furthermore, the control flag can be explicitly signaled depending on other flag in the sequence, picture or slice header level. For example, the other flag may correspond to present_flag in a sequence level. If present_flag is 1, the control flag exists. If present_flag is 0, the control flag does not exist and its value is inferred to 0. When the control flag is implicitly determined, another explicit flag can be signaled to select transform for the TUs associated with the PU.
The explicit control flag can be used as follows. If the control flag is equal to 0, original DCT in HEVC will be used for the TU. If the control flag is equal to 1, BDT will be used for the TU.

In the present invention, the explicit control flag can be used as follow. If the control flag is equal to 0, original DCT in HEVC will be used for the TU. If the control flag is equal to 1, the BDT will be used for the TU. However, the BDT is modified from that in JCTVC-G281so that the modified BDT only follows the decision regarding whether to use the flipped or non-flipped version of the transform (T). The modified BDT may use a transform different from the original BDT for each boundary type. Table 4illustrates an example of transform selection for each boundary type. The transform selection may be based on the block size of the current transform unit (or current prediction unit). For example, DST-VII is used for 4-point and DCT-IV for 8-point and 16-point, respectively. In Table 4, Flip-T denotes a transform using flipped transform matrix of T. In another example, the transform selection may be based on the mode information of the current prediction unit.

Table 4.

TU Boundary		Horizontal Trans.	TU Boundary		Vertical Trans.
Left	Right	Horizontal Trans.	Top	Bottom	Vertical Trans.
non-PU	PU	Flip-T	non-PU	PU	Flip-T
PU	non-PU	T	PU	non-PU	T
PU	PU	T	PU	PU	T
non-PU	non-PU	T	non-PU	non-PU	T

In the above examples, specific transforms such as DST-VII and DCT-IV are used as exemplary transforms to practice the present invention. However, the present invention is not limited to the specific transforms used. The control flag can be used to control the use of boundary dependent transform regardless the transforms used.
In an embodiment that does not fall under the scope of the appended claims, the control flag may be determined implicitly based on the boundary type, block size or motion information of the current block, and the transform selection is further determined depending on another flag being explicitly signaled. For example, a current transform type is implicitly determined for the current block by selecting from a first transform type or an explicitly signaling type based on block boundary type, block size, or mode information. If the explicitly signaling type is selected, the current transform type is further determined by an explicitly signaled flag.
Fig. 3 illustrates an exemplary flowchart of a coding system incorporating block characteristics-dependent transform based on a control flag according to an embodiment of the present invention. The system receives input data associated with a current PU (prediction unit) coded in Inter or Intra prediction mode, wherein the current PU is divided into one or more TUs (transform units) in step 310. The input data may correspond to residues of the current PU to be coded at the encoder side or the coded residues of the current PU in the decoder side. The current PU may be retrieved from storage such as a computer memory of buffer (RAM or DRAM). The video bitstream may also be received from a processor such as a processing unit or a digital signal. A control flag for the current PU is determined in step 320. Whether the control flag has a first value (e.g., is on) is tested in step 330. If the result is "yes", steps 340 and 350 are performed. If the result is "No", step 360 is performed. In step 340, a first transform is applied to each current TU (transform unit) at an encoder side or an inverse transform of the first transform to each current TU at a decoder side if the current TU belongs to a first type (i.e., the block characteristic thereof belongs to the first type). In step 350, a second transform is applied to each current TU at the encoder side or an inverse transform of the second transform to each current TU at the decoder side if the current TU belongs to a second type(i.e., the block characteristic thereof belongs to the second type), where the first transform is different from the second transform. In the present invention, whether the current TU belongs to the first type or the second type is associated with a boundary type of the current TU. In an embodiment that does not fall under the scope of the appended claims, whether the current TU belongs to the first type or the second type is determined according to a boundary type (e.g., a PU boundary or a non-PU boundary), block size (e.g., larger than a predetermined size or not), mode information (e.g., information related to a specific prediction mode), or a combination thereof. In step 360, a selected transform is applied to each current TU at the encoder side or an inverse transform of the selected transform to each current TU at the decoder side regardless of type of the current TU.
The flowchart shown above is intended to illustrate examples of Intra prediction filtering according an embodiment of the present invention. A person skilled in the art may modify each step, re-arranges the steps, split a step, or combine steps to practice the present invention without departing from the spirit of the present invention.
The above description is presented to enable a person of ordinary skill in the art to practice the present invention as provided in the context of a particular application and its requirement. Various modifications to the described embodiments will be apparent to those with skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the appended claims. In the above detailed description, various specific details are illustrated in order to provide a thorough understanding of the present invention. Nevertheless, it will be understood by those skilled in the art that the present invention may be practiced.
Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be one or more electronic circuits integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware code may be developed in different programming languages and different formats or styles. The software code may also be compiled for different target platforms.

Claims

A method for video coding using alternative transforms, the method comprising:
receiving input data associated with a current PU (prediction unit) coded in Inter or Intra prediction mode, wherein the current PU is divided into one or more current TUs (transform units), and wherein the input data correspond to to-be coded residues of the current PU at an encoder side or to coded residues of the current PU at a decoder side;

determining a control flag for the current PU;

when the control flag has a first value:
applying a first transform to a current TU (transform unit) at the encoder side or an inverse transform of the first transform to a current TU at the decoder side if the current TU has a first boundary type; and

applying a second transform to the current TU at the encoder side or an inverse transform of the second transform to the current TU at the decoder side if the current TU has a second boundary type; wherein the second transform is a flipped version of the first transform; and

when the control flag has a second value:
applying a selected transform to the current TU at the encoder side or an inverse transform of the selected transform to the current TU at the decoder side regardless of type of the current TU;

characterized in that the first boundary type includes a left PU boundary plus a right non-PU boundary, a top PU boundary plus a bottom non-PU boundary, a left PU boundary plus a right PU boundary, a left non-PU boundary plus a right non-PU boundary, a top PU boundary plus a bottom PU boundary, and a top non-PU boundary plus a bottom non-PU boundary, and the second boundary type includes a left non-PU boundary plus a right PU boundary and a top non-PU boundary plus a bottom PU boundary.
An apparatus for video coding using alternative transforms, the apparatus comprising one or more electronic circuits configured to:
receiving input data associated with a current PU (prediction unit) coded in Inter or Intra prediction mode, wherein the current PU is divided into one or more current TUs (transform units), and wherein the input data correspond to to-be coded residues of the current PU at an encoder side or to coded residues of the current PU at a decoder side;

determining a control flag for the current PU;

when the control flag has a first value:
applying a first transform to a current TU (transform unit) at the encoder side or an inverse transform of the first transform to a current TU at the decoder side if the current TU has a first boundary type; and

applying a second transform to the current TU at the encoder side or an inverse transform of the second transform to the current TU at the decoder side if the current TU has a second boundary type; wherein the second transform is a flipped version of the first transform; and

when the control flag has a second value:
applying a selected transform to the current TU at the encoder side or an inverse transform of the selected transform to the current TU at the decoder side regardless of type of the current TU;

characterized in that the first boundary type includes a left PU boundary plus a right non-PU boundary, a top PU boundary plus a bottom non-PU boundary, a left PU boundary plus a right PU boundary, a left non-PU boundary plus a right non-PU boundary, a top PU boundary plus a bottom PU boundary, and a top non-PU boundary plus a bottom non-PU boundary, and the second boundary type includes a left non-PU boundary plus a right PU boundary and a top non-PU boundary plus a bottom PU boundary.
Data stream having encoded therein a control flag for a current PU (prediction unit) coded in Inter or Intra prediction mode, and transform coefficients for one or more current TUs (transform units), wherein the current PU is divided into the one or more current TUs,
when the control flag has a first value:
a first transform is applied to a current TU (transform unit) at an encoder side to generate the transform coefficients for the current TU or an inverse transform of the first transform is applied to a current TU at a decoder side if the current TU has a first boundary type;

a second transform is applied to the current TU at the encoder side to generate the transform coefficients for the current TU or an inverse transform of the second transform is applied to the current TU at the decoder side if the current TU has a second boundary type; wherein the second transform is a flipped version of the first transform;

when the control flag has a second value:
a selected transform is applied to the current TU at the encoder side to generate the transform coefficients for the current TU or an inverse transform of the selected transform is applied to the current TU at the decoder side regardless of type of the current TU;

characterized in that the first boundary type includes a left PU boundary plus a right non-PU boundary, a top PU boundary plus a bottom non-PU boundary, a left PU boundary plus a right PU boundary, a left non-PU boundary plus a right non-PU boundary, a top PU boundary plus a bottom PU boundary, and a top non-PU boundary plus a bottom non-PU boundary, and the second boundary type includes a left non-PU boundary plus a right PU boundary and a top non-PU boundary plus a bottom PU boundary.