WO2015139183A1

WO2015139183A1 - Method of signaling of depth-based block partitioning mode for three-dimensional and multi-view video coding

Info

Publication number: WO2015139183A1
Application number: PCT/CN2014/073547
Authority: WO
Inventors: Jian-Liang Lin; Yi-Wen Chen; Xianguo Zhang; Kai Zhang; Jicheng An; Han HUANG
Original assignee: Mediatek Inc.
Priority date: 2014-03-17
Filing date: 2014-03-17
Publication date: 2015-09-24

Abstract

They are methods for the DBBP mode signaling in multi-view and 3D video coding. It is proposed to always signal both DBBP flag and partition flag without dependence. It is also proposed to signal the DBBP flag before the partition flag. When the DBBP mode is enabled, the transmission of the partition flag is skipped.

Description

METHOD OF SIGNALING OF DEPTH-BASED BLOCK

PARTITIONING MODE FOR THREE-DIMENSIONAL AND MULTI- VIEW VIDEO CODING

TECHNICAL FIELD

[0001] The invention relates generally to Three-Dimensional (3D) video processing. In particular, the present invention relates to method for depth-based block partitioning (DBBP) signaling in 3D video coding.

BACKGROUND

[0002] Three-dimensional (3D) television has been a technology trend in recent years that is targeted to bring viewers sensational viewing experience. Multi-view video is a technique to capture and render 3D video. The multi-view video is typically created by capturing a scene using multiple cameras simultaneously, where the multiple cameras are properly located so that each camera captures the scene from one viewpoint. The multi-view video with a large number of video sequences associated with the views represents a massive amount data. Accordingly, the multi-view video will require a large storage space to store and/or a high bandwidth to transmit. Therefore, multi-view video coding techniques have been developed in the field to reduce the required storage space and the transmission bandwidth. A straightforward approach may simply apply conventional video coding techniques to each single-view video sequence independently and disregard any correlation among different views. Such straightforward techniques would result in poor coding performance. In order to improve multi-view video coding efficiency, multi-view video coding always exploits inter-view redundancy. The disparity between two views is caused by the locations and angles of the two respective cameras.

[0003] To reduce the inter- view redundancy, disparity-compensated prediction (DCP) has been used as an alternative to motion-compensated prediction (MCP). MCP refers to an inter-picture prediction that uses already coded pictures of the same view in a different access unit, while DCP refers to inter-picture prediction that uses already coded pictures of other views in the same access unit, as illustrated in Fig. 1. The three-dimensional/multi-view data consists of texture pictures (110) and depth maps (120). The motion compensated prediction is applied to texture pictures or depth maps in the temporal direction (i.e., the horizontal direction in Fig. 1). The disparity compensated prediction is applied to texture pictures or depth maps in the view direction (i.e., the vertical direction in Fig. 1). The vector used for DCP is termed disparity vector (DV), which is analog to the motion vector (MV) used in MCP.

[0004]3D-HEVC is an extension of HEVC (High Efficiency Video Coding) that is being developed for encoding/decoding 3D video. One of the views is referred to as the base view or the independent view. The base view is coded independently of the other views as well as the depth data. Furthermore, the base view is coded using a conventional HEVC video coder.

[0005] In 3D-HEVC, a hybrid block-based motion-compensated DCT-like transform coding architecture is still utilized. The basic unit for compression, termed coding unit (CU), is a 2NX 2N square block, and each CU can be recursively split into four smaller CUs until the predefined minimum size is reached. Each CU contains one or multiple prediction units (PUs). The PU size can be 2NX 2N, 2NXN, NX 2N, or NXN. When asymmetric motion partition (AMP) is supported, the PU size can also be 2N X nU, 2N X nD, nL X 2N and nR X 2N.

[0006] In the depth-based block partitioning (DBBP) mode, an arbitrarily shaped block partitioning for the collocated texture block is derived based on a binary segmentation mask computed from the corresponding depth map. Each of the two partitions (resembling foreground and background) is motion compensated and afterwards merged based on the depth-based segmentation mask.

[0007] A single flag is added to the coding syntax to signal to the decoder that a block uses

DBBP for prediction. When current coding unit is coded with the DBBP mode, the corresponding partition size is set to SIZE_2Nx2N and bi-prediction is inherited.

[0008] A disparity vector derived from the Do BDV process is applied to identify a corresponding depth block in a reference view. The corresponding depth block has the same size as current texture block. When the depth block is found, a threshold is calculated based on the average of all depth pixels within the corresponding depth block. Afterwards, a binary segmentation mask m_D (x,y) is generated based on depth values and the threshold. When the depth value located at the relative coordinator (x, y) is larger than the threshold, the binary mask m_D (x,y) is set to 1. Otherwise, m_D (x,y) is set to 0. An example is shown in Fig. 2.

[0009] For each of the two decoded motion parameters a 2Nx2N motion compensation is performed. The resulting prediction signals p_T0 (x,y) and p_Tl (x,y) are combined using the DBBP mask m_D (x,y), as depicted in Fig. 3. The combination process is defined as follows:

P_T (x,y) = P_T0 (x,y), if m_D (x,y)=l = p_Tl (x,y), otherwise

[0010] Whether the DBBP mode is used is signaled on coding unit as shown in Table 1. In currently design, the DBBP flag is conditionally signaled depended on a transmitted partition mode. The flag is signaled only when the transmitted PartMode equals to 2NxN partition.

Table 1

depth_based_blk_part_flag[ layerld ] equal to 0 specifies that depth based block partitioning is not used for the layer with nuh layer id equal to layerld. depth_based_blk_part_flag[ layerld ] equal to 1 specifies that depth based block partitioning might be used for the layer with nuh layer id equal to layerld. When not present, the value of depth_based_blk_part_flag[ layerld ] is inferred to be equal to 0.

SUMMARY

[0011]In the current design (HTM-10.0), the DBBP flag is conditionally signaled depended on a transmitted partition mode. This invention proposes to improve the DBBP mode signaling in 3D video coding.

[0012] Other aspects and features of the invention will become apparent to those with ordinary skill in the art upon review of the following descriptions of specific embodiments.

BRIEF DESCRIPTION OF DRAWINGS

The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:

[0013] Fig. 1 illustrates an example of three-dimensional/multi-view coding, where motion compensated prediction (MCP) and disparity compensated prediction (DCP) are used.

[0014] Fig. 2 illustrates an example of segmentation mask generation.

[0015] Fig. 3 illustrate an example of DBBP merging process.

DETAILED DESCRIPTION

[0016] The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

[0017] The basic concept of the invention is to improve the DBBP mode signaling in 3D video coding. Several methods are illustrated as below:

[0018] When current coding unit is coded with the DBBP mode, the corresponding partition size is set to SIZE 2NX 2N and bi-prediction is inherited. Therefore, in one embodiment proposed in this invention, the DBBP flag is proposed to be signaled only when the transmitted partition mode is 2Nx2N partition mode. An example of signaling the DBBP flag when the partition mode is 2NX 2N is shown in Table 2.

Table 2

[0019] In another embodiment, the DBBP flag is proposed to be signaled only when the transmitted partition mode with the longest binarization codeword is transmitted. As an example shown in Table 3, the DBBP flag is signaled when the transmitted partition mode equal to AMP xxx partition mode, (xxx could be one of the following AMP partitioning modes: PART 2N X nU, PART 2N X nD, PART nL X 2N and PART nR X 2N).

Table 3

if( cu_ski _flag[ xO ] [ yO ] )

prediction_unit( xO, yO, nCbS, nCbS )

else {

if( slice type != I )

pred mode flag ae(v) if( ( CuPredMode[ xO ][ yO ] != MODE INTRA | |

log2CbSize = = MinCbLog2SizeY ) && !predPartModeFlag )

part mode ae(v) if( depth based blk_part flag[ nuh layer id ] && PartMode = =

PART AMPxxx )

dbbp_flag[ xO ][ yO ] ae(v) if( sdcEnableFlag )

sdc_flag[ xO ][ yO ] ae(v)

[0020] In yet another embodiment of the present invention, the transmission of the DBBP flag doesn't not depend on transmitted partition mode. The DBBP flag is always signaled as long as the DBBP mode is enabled in current coding slice or picture. For example, when the flag depth_based_blk_part_flag[ layerld ], which is used to specified whether the depth based block partitioning is used or not for current layer, is true, the DBBP is always signaled as an example shown in Table 4.

Table 4

[0021] Since the mask for motion compensation and the partition for motion storage in DBBP are implicitly derived based on the corresponding depth block, the transmission of partition mode can actually be skipped when it is DBBP mode. In another embodiment of the present invention, the DBBP flag is transmitted before the partition flag. When the DBBP mode is enabled, the transmission of the partition flag is skipped. An example is shown in Table 5.

Table 5

[0022] In yet another embodiment of the present invention, the DBBP flag and the partition flag are both transmitted. When the DBBP mode is enabled, the partition flag is actually used to represent the partition for motion information storage in DBBP and is not used to represent the partition of motion compensation. The examples are shown in Table 4 and Table 6.

Table 6

else {

if( slice type != I )

pred mode flag ae(v) if( depth_based_blk_part_flag[ nuh layer id ] )

dbbp_flag[xO][yO] ae(v) if( ( CuPredMode[ xO ][ yO ] != MODE INTRA | |

log2CbSize = = MinCbLog2SizeY ) && !predPartModeFlag )

part mode ae(v) if( sdcEnableFlag )

sdc_flag[xO][yO] ae(v)

[0023] In yet another embodiment of the present invention, the DBBP flag is transmitted when current CU is not predicted by an intra mode. When the current CU is intra predicted, dbbp flag is not transmitted. The examples are shown in Table 7 and Table 8.

Table 7

coding_unit( xO, yO, log2CbSize , ctDepth) { Descrip

tor

if( transquant bypass enabled flag )

cu transquant bypass flag ae(v) if( slice type != I )

cu_skip_flag[ xO ] [ yO ] ae(v) nCbS = ( 1 « log2CbSize )

if( cu_skip _flag[ xO ] [ yO ] )

prediction_unit( xO, yO, nCbS, nCbS )

else {

if( slice type != I )

pred mode flag ae(v) if( depth based blk_part flag[ nuh layer id ] && CuPredMode[ xO ][ yO ] !=

MODE INTRA )

dbbp_flag[xO][yO] ae(v) if( ( CuPredMode[ xO ][ yO ] != MODE INTRA | |

log2CbSize = = MinCbLog2SizeY ) && !predPartModeFlag )

part mode ae(v) if( sdcEnableFlag )

Table 8

coding_unit( xO, yO, log2CbSize , ctDepth) { Descrip

tor

if( transquant bypass enabled flag )

cu transquant bypass flag ae(v) if( slice type != I )

cu_skip_flag[ xO ] [ yO ] ae(v) nCbS = ( 1 « log2CbSize )

[0024] In the above examples, since the DBBP would implicitly divide current coding blocking into multiple partitions (e.g. 2 partitions or 4 partitions), we need to ensure the PU level syntax will be parsed correctly according to the number of partitions.

[0025] To solve this issue, some semantic constraint could be added to the "part mode" as shown below. One example is given as below, for a CU with dbbp flag equal to 1, its partition mode is set to a partition type with multiple (2 or 4) partitions (e.g. PART 2N X N, PART N X 2N, PART NXN, or AMP partitions ). In the below example, we modify the semantic description of "part mode" in the SPEC of 3D-HEVC and the modified parts are highlighted as bold font.

1.7.4.9.4 Coding quadtree semantics (in 3D-HEVC Draft Text 3, JCT3V-G1001)

part mode specifies partitioning mode of the current coding unit. The semantics of part mode depend on CuPredMode[ xO ][ yO ]. The variables PartMode and IntraSplitFlag are derived from the value of part mode and partPredldc as defined in Table 1-2. When dbbp flag is equal to 1, PartMode is explicitly derived as PART 2N X N.

Table 1-2 - Name association to prediction mode and partitioning type

[0026] Another way to solve this issue is to add condition to the parsing stage. In the below example la-b, when DBBP divides current block into two partitions, it may need to parse two PU level syntax. In example 1-b, instead of combining the dbbp flag with the partition PART 2NX N (else if( PartMode == PART 2N X N || dbbp_flag[ xO ][ yO ]))), the dbbp flag could also be combined with other partition mode with two partitions such as PART N X 2N or one of the AMP partition.

[0027] In the example 2, when DBBP divides current block into four partitions, it may need to parse four PU level syntax.

Example la

Example 2

[0028] The above description is presented to enable a person of ordinary skill in the art to practice the present invention as provided in the context of a particular application and its requirement. Various modifications to the described embodiments will be apparent to those with skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed. In the above detailed description, various specific details are illustrated in order to provide a thorough understanding of the present invention. Nevertheless, it will be understood by those skilled in the art that the present invention may be practiced.

[0029] Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware code may be developed in different programming languages and different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

[0030] The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

1. Method to decide the enabling or disabling of DBBP mode comprising:

receiving input data associated with texture block of a current texture picture in a current dependent view;

receiving a depth block associated with the current texture block;

signaling a flag to indicate the enabling or disabling of the DBBP mode.

2. The method as claimed in claim 1, wherein the DBBP flag is signaled only when the partition mode is 2NX 2N.

3. The method as claimed in claim 1, wherein the DBBP flag is signaled only when the partition mode is AMP xxx. (xxx could be one of the following AMP partitioning modes: PART 2N

X nU, PART 2N X nD, PART nL X 2N and PART nR X 2N)

4. The method as claimed in claim 1, wherein the DBBP flag is always signaled and doesn't depend on the partition mode.

5. The method as claimed in claim 1, wherein the DBBP flag is transmitted before the partition flag, when the DBBP mode is enabled, the transmission of the partition flag is skipped.

6. The method as claimed in claim 1, wherein the DBBP flag and the partition flag are both transmitted, when the DBBP mode is enabled, the partition flag is actually used to represent the partition for motion information storage in DBBP and is not used to represent the partition of motion compensation.

7. The method as claimed in claim 1, wherein the DBBP flag is always signaled when the current CU is one inter-predicted CU and doesn't depend on the partition mode.