WO2017088170A1

WO2017088170A1 - Entropy coding the binary tree block partitioning structure

Info

Publication number: WO2017088170A1
Application number: PCT/CN2015/095775
Authority: WO
Inventors: Shih-Ta Hsiang; Jicheng An
Original assignee: Mediatek Inc.
Priority date: 2015-11-27
Filing date: 2015-11-27
Publication date: 2017-06-01

Abstract

A binary tree block partitioning structure is proposed. The proposed flexible block partitioning structure plays an important role in the substantial performance gains relative the HEVC.

Description

ENTROPY CODING THE BINARY TREE BLOCK PARTITIONING STRUCTURE

TECHNICAL FIELD

The invention relates generally to video processing. In particular, the present invention relates to methods and apparatuses for block partitioning structure in video and image coding.

BACKGROUND

The High Efficiency Video Coding (HEVC) standard is the most recent joint video project of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG) standardization organizations, working together in a partnership known as the Joint Collaborative Team on Video Coding (JCT-VC) .

In HEVC, one slice is partitioned into multiple coding tree units (CTU) . In main profile, the minimum and the maximum sizes of CTU are specified by the syntax elements in the sequence parameter set (SPS) among the sizes of 8x8, 16x16, 32x32, and 64x64. Inside a slice, a raster scan method is used for processing the CTU.

The CTU is further partitioned into multiple coding units (CU) to adapt to various local characteristics. A quadtree denoted as the coding tree is used to partition the CTU into multiple CUs. Let CTU size be MxM where M is one of the values of 64, 32, or 16. The CTU can be a single CU or can be split into four smaller units of equal sizes of M/2xM/2, which are nodes of coding tree. If units are leaf nodes of coding tree, the units become CUs. Otherwise, the quadtree splitting process can be iterated until the size for a node reaches a minimum allowed CU size specified in the SPS. This representation results in a recursive structure specified by a coding tree as shown in Fig. 1. The solid lines indicate CU boundaries. The decision whether to code a picture area using interpicture (temporal) or intrapicture (spatial) prediction is made at the CU level. Since the minimum CU size can be 8x8, the minimum granularity for switching different basic prediction type is 8x8.

One or more prediction units (PU) are specified for each CU. Coupled with the CU, the PU works as a basic representative block for sharing the prediction information. Inside one PU, the same prediction process is applied and the relevant information is transmitted to the decoder on a PU basis. A CU can be split into one, two, or four PUs according to the PU splitting type. HEVC defines eight shapes for splitting a CU into PU as shown in Fig. 2. Unlike the CU, the PU may only be split once.

After obtaining the residual block by prediction process based on PU splitting type, a CU can be partitioned into transform units (TU) according to another quadtree structure which is analogous to the coding tree for the CU as shown in Fig. 1. The solid lines indicate CU boundaries and dotted lines indicate TU boundaries. The TU is a basic representative block having residual or transform coefficients for applying the integer transform and quantization. For each TU, one integer transform having the same size to the TU is applied to obtain residual coefficients. These coefficients are transmitted to the decoder after quantization on a TU basis.

The terms coding tree block (CTB) , coding block (CB) , prediction block (PB) , and transform block (TB) are defined to specify the 2-D sample array of one color component associated with CTU, CU, PU, and TU, respectively. Thus, a CTU consists of one luma CTB, two chroma CTBs, and associated syntax elements. A similar relationship is valide for CU, PU, and TU.

The tree partitioning is generally applied simultaneously to both luma and chroma, although exceptions apply when certain minimum sizes are reached for chroma.

However, due to some restrictions in the current HEVC block partitioning structure, e.g., the quadtree based partitioning from CTU to CU and from CU to TU, the limited depth of PU partitioning, the coding efficiency is still not sufficient to cope with the ever increasing demands for storage and transmission of video content.

SUMMARY

In light of the previously described problems, there exists a need for an apparatus and method, in which a more flexible block partitioning structure is used.

A binary tree structure is proposed for partitioning a unit into multiple smaller units such as partitioning a slice into CTUs, a CTU into CUs, a CU into PUs, or a CU into TUs, and so on.

The binary tree structure can be combined with quadtree structure, which is a quadtree plus binary tree (QTBT) structure. In the QTBT structure, the root unit is firstly partitioned by a quadtree structure, then the leaf nodes of the quadtree is further partitioned by a binary tree structure. Some parameters for restriction of the quadtree and binary tree can be defined in high level such as in SPS.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:

Fig. 1 illustrates the subdivision of a CTU into CUs and TUs (left) and its corresponding quadtree (right) in HEVC.

Fig. 2 illustrates the types for splitting a CU into PUs in HEVC.

Fig. 3 illustrates the possible splitting types in the proposed binary tree.

Fig. 4 illustrates an example of block partitioning (left) and its corresponding binary tree (right) .

Fig. 5 illustrates an example of block partitioning (left) and its corresponding quadtree plus binary tree (right) .

DETAILED DESCRIPTION

The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

A binary tree block partitioning structure is proposed. In the proposed binary tree partitioning structure, a block can be recursively split into two smaller blocks. There are kinds of splitting types as shown in Fig. 3. The most efficient and simplest ones are the symmetric horizontal and vertical split as shown in the top two splitting types in Fig. 3. Therefore, as one embodiment, we only use these two splitting types. For a given block of size MxN, a flag signals whether it is split into two smaller blocks. If yes, another syntax element is signaled to indicate which splitting type is used. If the horizontal splitting is used then it is split into two blocks of size MxN/2, otherwise if the vertical splitting is used then it is split into two blocks of size M/2xN. The binary tree splitting process can be iterated until the size (width or height) for a splitting block reaches a minimum allowed block size (width or height) that can be defined in high level syntax such as SPS. Since the binary tree has two splitting types horizontal and vertical, so the minimum allowed block width and height should be both indicated. Not horizontal splitting is implicit when splitting would result in a block height smaller than the indicated minimum. Not vertical splitting is implicit when splitting would result in a block width smaller than the indicated minimum. Fig. 4 illustrates an example of block partitioning (left) and its corresponding binary tree (right) . In each splitting (i.e., non-leaf) node of the binary tree, one flag indicates which splitting type (horizontal or vertical) is used, 0 indicates horizontal splitting and 1 indicates vertical splitting.

The proposed binary tree structure can be used for partitioning a block into multiple smaller blocks such as partitioning a slice into CTUs, a CTU into CUs, a CU into PUs, or a CU into TUs, and so on. As one embodiment, we use the binary tree for partitioning a CTU into CUs, i.e., the root node of the binary tree is a CTU and the leaf node of the binary tree is CU, the leaf nodes are further processed by prediction and transform coding. And for simplification as another embodiment there is no further partitioning from CU to PU or from CU to TU. That means CU equal to PU equal to TU, therefore, it is also to say that the leaf node of the binary tree is the basic unit for prediction and transform coding.

Binary tree structure is more flexible than quadtree structure, since much more partition shapes can be supported which is also the source of coding efficiency improvement. However, the encoding complexity will also increase in order to select the best partition shape. In order to balance the complexity and coding efficiency, it is proposed to combine the quadtree and binary tree structure, which is called as quadtree plus binary tree (QTBT) structure. In the proposed QTBT structure, a block is firstly partitioned by a quadtree structure, the quadtree splitting can be iterated until the size for a splitting block reaches the minimum allowed quadtree leaf node size. If the leaf quadtree block is not larger than the maximum allowed binary tree root node size, it can be further partitioned by a binary tree structure, the binary tree splitting can be iterated until the size (width or height) for a splitting block reaches the minimum allowed binary tree leaf node size (width or height) or the binary tree depth reaches the maximum allowed binary tree depth. In the QTBT structure, the minimum allowed quadtree leaf node size, the maximum allowed binary tree root node size, the minimum allowed binary tree leaf node width and height, and the maximum allowed binary tree depth can be indicated in the high level syntax such as in SPS. Fig. 5 illustrates an example of block partitioning (left) and its corresponding QTBT (right) . The solid lines indicate quadtree splitting and dotted lines indicate binary tree splitting. In each splitting (i.e., non-leaf) node of the binary tree, one flag indicates which splitting type (horizontal or vertical) is used, 0 indicates horizontal splitting and 1 indicates vertical splitting.

The proposed QTBT structure can be used for partitioning a block into multiple smaller blocks such as partitioning a slice into CTUs, a CTU into CUs, a CU into PUs, or a CU into TUs, and so on. As one embodiment, we use the QTBT for partitioning a CTU into CUs, i.e., the root node of the QTBT is a CTU which is partitioned into multiple CUs by a QTBT structure, and the CUs are further processed by prediction and transform coding. And for simplification as another embodiment there is no further partitioning from CU to PU or from CU to TU. That means CU equal to PU equal to TU, therefore, it is also to say that the leaf node of the QTBT structure is the basic unit for prediction and transform.

For the proposed QTBT structure, we can take a specific example here. We have a CTU with size 128x128, and the minimum allowed quadtree leaf node size equal to 16x16, the maximum allowed binary tree root node size equal to 64x64, the minimum allowed binary tree leaf node width and height both equal to 4, and the maximum allowed binary tree depth equal to 4. Firstly, the CTU is partitioned by a quadtree structure and the leaf quadtree unit may have size from 16x16 (minimum allowed quadtree leaf node size) to 128x128 (equal to CTU size, i.e., no split) , if the leaf quadtree unit is 128x128 it cannot be further split by binary tree since the size exceeds the maximum allowed binary tree root node size 64x64, otherwise the leaf quadtree unit can be further split by binary tree, the leaf quadtree unit which is also the root binary tree unit has binary tree depth as 0. When the binary tree depth reaches 4 (the indicated maximum) , not splitting is implicit, when the binary tree node has width equal to 4, not horizontal splitting is implicit, and when the binary tree node has height equal to 4, not vertical splitting is implicit. The leaf nodes of the QTBT are further processed by prediction (intrapicture or interpicture) and transform coding.

The proposed tree structure is applied separately to luma and chroma for I slice, and applied simultaneously to both luma and chroma (except when certain minimum sizes are reached for chroma) for P and B slice. That is to say that, in I slice, the luma CTB has its QTBT-structured block partitioning, and the two chroma CTBs has another QTBT-structured block partitioning, wherein as another embodiment, the two chroma CTBs can also have their own QTBT-structured block partitionings.

The representation of the proposed binary tree structure can be entropy coded by the context-based adaptive binary arithmetic coding (CABAC) mode. The context modeling can consider information on the binary tree depths of the neighboring blocks and the current block subject to signaling the split decision. The contextual information can further consider the size and shape of the current block. When combined with the quadtree structure, the context modeling can further consider information on the quadtree depths of the neighboring blocks and the current block. A binary syntax element bt_split_flag is coded to signal if the current block is to be further split or not. If the syntax element bt_split_flag is true, a syntax element bt_split_mode is next coded to indicate the split/partitioning mode to be adopted by the current block. In one embodiment for the proposed QTBT structure with the two binary split modes, context modeling for entropy coding the syntax element bt_split_flag is conditioned on the number of the neighboring blocks, consisting of the top block and left block, with the higher combined QTBT depth than the current block. The combined QTBT depth of a block is defined to be equal to the sum of its quadtree depth scaled by 2 and its binary tree depth. Context modeling for entropy coding the syntax element bt_split_mode is conditioned on the shape of the current block. Tables 1 and 2 provide the example mapping tables for context selection for such an embodiment for coding syntax elements bt_split_flag and bt_split_mode, respectively. In another embodiment, the context modeling is further conditioned on the binary tree depth of the current block. Tables 3 and 4 provide the example mapping tables for context selection for such an embodiment for coding syntax elements bt_split_flag and bt_split_mode, respectively, where “W” and “H” denotes the width and height of the current block. In another embodiment, context modeling is further conditioned on the quadtree depth of the current block. In another embodiment, context modeling for different quadtree depths or binary tree depths can share the same set of modeling contexts.

Table 1

Table2

Table 3

Table 4

The methods described above can be used in a video encoder as well as in a video decoder. Embodiments of the methods according to the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program codes to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA) . These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware codes may be developed in different programming languages and different format or style. The software code may also be compiled for different target platform. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art) . Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims

A video or image coding method, the method comprising a binary tree block partitioning structure possibly combined with a quadtree block partitioning structure, an arithmetic coding engine, a plurality of probability models built up from the different contextual conditions, and a entropy coder for coding the said block partitioning structure using context-based adaptive binary arithmetic coding conditioned on the contextual information related to the binary tree depths of the neighboring blocks and the current block and the size and shape of the current block.
The method according to Claim 1, wherein the entropy coder comprising a binary syntax element, indicated by bt_split_flag, to signal if the current block is to be further split or not and entropy coded conditioned on the number of the neighboring blocks with the higher combined QTBT depth than the current block.
The method according to Claim 1, wherein the entropy coder comprising a syntax element, indicated by bt_split_mode, to signal the split/partitioning mode to be adopted by the current block and entropy coded conditioned on the shape of the current block.
The method as claimed in claim 3, wherein the shape of the current block is classified by the relationship between width and height of the block.
The method according to Claim 2, wherein the syntax element bt_split_flag is entropy coded further conditioned on the binary tree depth of the current block.
The method according to Claim 3, wherein the syntax element bt_split_mode is entropy coded further conditioned on the binary tree depth of the current block.
The method as claimed in Claim 2, wherein the combined QTBT depth is replaced by the binary tree depth.
The method as claimed in Claim 5, wherein the binary tree depth is replaced by the combined QTBT depth.
The method as claimed in Claim 6, wherein the binary tree depth is replaced by the combined QTBT depth.