WO2016044974A1

WO2016044974A1 - Palette table signalling

Info

Publication number: WO2016044974A1
Application number: PCT/CN2014/087082
Authority: WO
Inventors: Kai Zhang; Tzu-Der Chuang; Shan Liu; Jicheng An; Xianguo Zhang
Original assignee: Mediatek Singapore Pte. Ltd.
Priority date: 2014-09-22
Filing date: 2014-09-22
Publication date: 2016-03-31

Abstract

Methods are proposed to signal the palette table correctly and efficiently.

Description

PALETTE TABLE SIGNALLING

FIELD OF INVENTION

The invention relates generally to video/image coding/processing. Particularly, it is related palette coding.

BACKGROUND OF THE INVENTION

Palette coding [1] [2] is described as follows.

In this first method, proposed by Qualcomm, palette is utilized to represent a given video block (e. g. CU) . The encoding process is as follows [1] :

1. Transmission of the palette: the palette size is first transmitted followed by the palette elements.

2. Transmission of pixel values: the pixels in the CU are encoded in a raster scan order. For each position, a flag is first transmitted to indicate whether the “run mode” or “copy above mode” is being used.

2.1 “Run mode” : In “run mode” , a palette index is first signaled followed by “palette_run” (e.g. , M) . No further information needs to be transmitted for the current position and the following M positions as they have the same palette index as signaled. The palette index (e. g., i) is shared by all three color components, which means that the reconstructed pixel values are (Y, U, V) ＝ (paletteY [i] , palette_U [i] , paletteV [i] ) (assuming the color space is YUV)

2.2 “Copy above mode” : In “copy above mode” , a value “copy_run” (e. g., N) is transmitted to indicate that for the following N positions (including the current one) , the palette index is equal to the palette index of the one that is at the same location in the row above.

3. Transmission of residue: the palette indices transmitted in Stage 2 are converted back to pixel values and used as the prediction. Residue information is transmitted using HEVC residue coding and is added to the prediction for the reconstruction.

In the original version of the work [2] , palette of each component are constructed and transmitted. The palette can be predicted (shared) from its left neighboring CU to reduce the bitrate. Later on Qualcomm proposed a second version of their palette coding technique [1] , in which each element in the palette is a triplet, representing a specific combination of the three color components. The predictive coding of palette across CU is removed. This invention proposes the usage of palette prediction/sharing that can also be applied to the triplet palette format. Again both the palette from the left and/or above CU are utilized, as long as the above CU is within the same CTB (LCU) as the current CU being encoded (reducing line buffer).

Major-color-based (or palette) coding [3] was proposed by Microsoft. Similar to [1], palette of each component are constructed and transmitted. However, instead of predicting the entire palette from the left CU, individual entry in a palette can be predicted from the exact corresponding palette entry in the above CU or left CU.

For transmission of pixel values, a predictive coding method is applied on the indices [3], in which a pixel line can be predicted by different modes. In specific, three kinds of line modes are used for a pixel line, i.e. horizontal mode, vertical mode and normal mode. In horizontal mode, all the pixels in the same line have the same value. If the value is the same as the first pixel of the above pixel line, only line mode signalling bits are transmitted. Otherwise, the index value is also transmitted. In vertical mode, the current pixel line is the same with the above pixel line. Therefore, only line mode signalling bits are transmitted. In normal mode, pixels in a line are predicted individually. For each pixel, the left or above neighbours is used as predictor, and the prediction symbol is transmitted to the decoder.

Furthermore, [3] classified pixels into major color pixels and escape pixel. For major color pixels, the decoder reconstruct pixel value by major color index (palette index in [1] [2] ) and palette. For escape pixel, the encoder would further send the pixel value.

In the reference software of screen content coding (SCC) standard, SCM-2.0, an improved palette scheme based on [4] is integrated.

The syntax table for palette coding is as follows.

In SCM-2.0, the palette table of last coded palette CU is used as palette predictors for current palette table coding. In palette table coding, a palette_share_flag is first signalled. If palette_share_flag is 1, all the palettes in the last coded palette table are reused for current CU. The palette size is also equal to the palette size of last coded palette CU. Otherwise (palette_share_flag is 0) , the current palette table is signalled by choosing which palette in the last coded palette table can be reuse and transmitting the new palette. The size of the palette is set as the size of the predicting palette (numPredPreviousPalette) and the size of the transmitted palette size (num_signalled_palette_entries) . The predicting palette is a palette derived from the previously reconstructed palette coded CUs. When coding the current CU as palette mode, those palette colors which are not predicted by the predicting palette are directly transmitted into the bitstream. For example, if current CU is coded as palette mode with a palette size equal to six. We assume three of the six major colors are predicted from the predicting palette and three are directly transmitted through the bitstream. The transmitted three will be signalled using the sample syntax as given below.

num_signalled_palette_entries ＝ 3

for (cIdx ＝ 0； cIdx < 3； cIdx++ ) //signal colors for different components

for (i ＝ 0； i < num_signalled_palette_entries； i++ )

palette_entries [cIdx] [numPredPreviousPalette + i]

Since the palette size is six in this example, the palette index from 0 to 5 is used to indicate each palette coded pixel could be reconstructed as the major color in the palette color table.

In SCM-2.0, to indicate that a pixel is coded as escape pixel, the color index for the escape pixel is signalled as the value equal to the palette size. With different interpretation, when escape pixels is coded in a palette coded block, the palette size of that block is increased by one and the last major color index is used as the index of escape pixels. In the above example, the major color index 6 indicates that this pixel is an escape pixel. Moreover, in SCM-2.0, one CU-level escape flag, palette_escape_val_present_flag, is signalled for each palette CU to indicate whether escape pixels indexing is coded for this palette CU.

Removing the redundancy of the major color index is another coding tool in SCM-2.0 which also affects the palette size. When the left pixel is coded as normal mode with runs and the left pixel is not escape pixel, it could be assumed that current pixel would not equal to the major color index of the left pixel； otherwise current pixel could be merged to the run coding of the left pixel. In another case, when the left pixel is coded in the run of copy-above mode and the above pixel is not escape pixel, it could be assumed that current pixel would not equal to the major color index of the above pixel； otherwise current pixel could be merged to the copy-above run coding of the left pixel. In SCM-2.0, to remove the redundancy, in the above mentioned cases, the major color of the left pixel (or the major color of the above pixel) could be removed from the palette table when coding current pixel. In the cases, the palette size could be regarded as the original size decreased by one. Note that, when the left pixel is coded as copy-above mode and the above pixel is an escape pixel or when the left pixel is coded as run mode and the left pixel is an escape pixel, the value of the above/left pixel (escape index) cannot be removed from the palette color table for coding current pixel, and the palette size cannot be decreased by one.

For palette [1] [2] and the major color sets [3] , we call it major color table.

For the palette index [1] [2] and color index [3] , we call it color index.

For the color index related to current coding unit, we call it (major color) index map.

When not specified, in the following context, the palette size represents the size of the predicting palette plus the size of the transmitted palette.

In SCM-2.0, an indexMax parameter is defined to indicate the maximum palette index of current CU. The definition of indexMax is as follows.

if (palette_escape_val_present_flag )
if (palette_escape_val_present_flag )		indexMax ＝ palette_size
else		indexMax ＝ palette_size
else		indexMax ＝ palette_size –1

The indexMax depends on palette_escape_val_present_flag and palette_size. The indexMax should be an integer that equal to or larger than 0. If the indexMax is smaller than 0, it means there is no palette for current CU. The decoder cannot parse and reconstruct the current palette coded CU. It’s meaningless. Since the palette_size is an integer that equal to or larger than 0, the indexMax can be smaller than 0 only when palette_size equal to 0 and palette_escape_val_present_flag equal to 0. The palette size is equal to 0 when palette_share_flag, numPredPreviousPalette and num_signalled_palette_entries are all equal to zero or when palette_share_flag is equal to 1 and previousPaletteSize is equal to 0.

In SCM-2.0, palette table coding, a palette_share_flag is first signalled. If palette_share_flag is 1, all the palettes in the last coded palette table are reused for current CU. The palette size is also equal to the palette size of last coded palette CU (previousPaletteSize) . Otherwise (palette_share_flag is 0) , the current palette table is signalled by choosing which palette in the last coded palette table can be reuse and transmitting the new palette.

If previousPaletteSize is equal to 0, there is no available palette for prediction. The signalling of palette_share_flag is redundant because there is no available palette to be reused for current palette table.

Acronyms:

CE: Core Experiments

CU: Coding Unit

CTB (LCU) : Coded tree block (largest coding unit)

HEVC: High Efficiency Video Coding

IntraBC: Intra picture Block Copy

MC: Motion Compensation

MV: Motion Vector

PU: Prediction Unit

RExt: HEVC Range Extensions

WPP: wavefront parallel process

SUMMARY OF THE INVENTION

In light of the previously described problems, methods are proposed to signal the palette table correctly and efficiently.

Other aspects and features of the invention will become apparent to those with ordinary skill in the art upon review of the following descriptions of specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:

Fig. 1 is a diagram illustrating the proposed palette_share_flag signaling method based on previousPaletteSize.

DETAILED DESCRIPTION

The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

In order to constraint the indexMax value to be equal to or larger than 0, we propose two several methods for palette_escape_val_present_flag coding.

Method-1 : encoder constrain for palette_escape_val_present_flag coding. If palette_size is equal to 0, the palette_escape_val_present_flag is constrainted to be 1. Otherwise, it’s not a conformance/legal bitstream.

Method-2: syntax constrain for palette_escape_val_present_flag coding. If palette_size is equal to 0, the palette_escape_val_present_flag is inferred to be 1. The signalling of palette_escape_val_present_flag can be skipped. The syntax table can be modified as below. A palette size checking should be added.

palette_coding (x0, y0, nCbS ) {	Descriptor
palette_coding (x0, y0, nCbS ) {	Descriptor	……
if (palette_size > 0 )		……
if (palette_size > 0 )		palette_escape_val_present_flag	ae (v)
if (palette_escape_val_present_flag )		palette_escape_val_present_flag	ae (v)
if (palette_escape_val_present_flag )		indexMax ＝ palette_size
Else		indexMax ＝ palette_size
Else		indexMax ＝ palette_size –1
……		indexMax ＝ palette_size –1
……		}

palette_escape_val_present_flag is inferred to be equal to 1 when not present.

In SCM-2.0, the escape pixels are signaled by using escape index. However, the escape pixels can be signaled by using an explicit flag instead. For example, before palette index signaling, a pixel-level escape flag is signaled to indicate whether the current pixel is escape pixel. If the flag is 1, the current pixel is escape pixel. The escape values of each component (and run) are signaled. Otherwise (the pixel-level escape flag is 0) , the palette index and run are signaled.

If the pixel-level escape flag is used, and the palette size is zero, the pixel-level escape flag should be all 1.

Therefore, we propose that if the pixel-level escape flag is used and the palette size is equal to zero, all the pixel-level escape flags are not signaled and inferred as 1.

Another proposed method is to add an encoding constraint. If the pixel-level escape flag is used and the palette size is zero, the pixel-level escape flags are constrained to be 1. Otherwise, it’s not a conformance/legal bitstream.

If previousPaletteSize is equal to 0, there is no available palette for prediction. The signalling of palette_share_flag is redundant because there is no available palette to be reused for current palette table. The palette_share_flag should be equal to 0 when previousPaletteSize is equal to 0. Therefore we propose two methods for palette_share_flag signalling.

Method-1 : encoder constrain for palette_share_flag signalling.

If previousPaletteSize is equal to 0, the palette_share_flag is constrainted to be 0. Otherwise, it’s not a conformance/legal bitstream.

Method-2: syntax constrain for palette_share_flag signalling

If previousPaletteSize is equal to 0, the palette_share_flag is inferred to be 0. The signalling of palette_share_flag can be skipped.

The syntax table can be modified as below. A previousPaletteSize checking should be added.

palette_coding (x0, y0, nCbS ) {	Descriptor
palette_coding (x0, y0, nCbS ) {	Descriptor	palette_transpose_flag	ae (v)
if (previousPaletteSize > 0 )		palette_transpose_flag	ae (v)
if (previousPaletteSize > 0 )		palette_share_flag [x0] [y0]	ae (v)
if (palette_share_flag [x0] [y0] ) {		palette_share_flag [x0] [y0]	ae (v)
if (palette_share_flag [x0] [y0] ) {		palette_size ＝ previousPaletteSize
for (n ＝ 0； n < palette_size； n++ )		palette_size ＝ previousPaletteSize
for (n ＝ 0； n < palette_size； n++ )		for (cIdx ＝ 0； cIdx < 3； cIdx++ )
palette_entries [cIdx] [n] ＝ previousPaletteEntries [cIdx] [n]		for (cIdx ＝ 0； cIdx < 3； cIdx++ )
		}else {
……		}else {
……		}

palette_share_flag [x0] [y0] is inferred to be equal to 0 when not present.

Fig. 1 demonstrates the proposed palette_share_flag signaling method based on previousPaletteSize.

We propose that at least one of pixel-level escape flags in a block should be 0. In other words, the case that all pixel-level escape flags in a block is 1 should be forbidden by syntax constrains or semantic constrains.

The methods described above can be used in a video encoder as well as in a video decoder. Embodiments of disparity vector derivation methods according to the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program codes to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA) . These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware codes may be developed in different programming languages and different format or style. The software code may also be compiled for different target platform. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art) . Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

References:

[1] . L. Guo, M. Karczewicz, J. Sole, and R. Joshi, “Evaluation of Palette Mode Coding on HM-12.0+RExt-4.1” , JCTVC-O0218, Geneva, CH, Oct 2013.

[2] . L. Guo, M. Karczewicz, and J. Sole, “RCE3: Results of Test 3.1 on Palette Mode for Screen Content Coding” , JCTVC-N0247, Vienna, AT, July 2013.

[3] . X. Guo, B. Li, J. Xu, Y. Lu, S. Li, and F. Wu, “AHG8: Major-color-based screen content coding” , JCTVC-O0182, Geneva, CH, October 2013.

[4] . P. Onno, X. Xiu, Y. -W. Huang, R. Joshi, “Suggested combined software and text for run-based palette mode” , JCTVC-R0348, Sapporo, JP, July 2014.

Claims

A method of palette table coding, comprising,

A method to signal the CU-level escape flag；

A method to signal the pixel-level escape flag；

A method to signal the palette sharing flag.
The method as claimed in claim 1, wherein if the palette size is equal to zero, the CU-level escape flag is not signaled and inferred as 1.
The method as claimed in claim 1, wherein if the palette size is equal to zero, the CU-level escape flag is constrained to be 1.
The method as claimed in claim 2 and claim 3, wherein the palette size is equal to 0 when palette_share_flag, numPredPreviousPalette and num_signalled_palette_entries are all equal to zero or when palette_share_flag is equal to 1 and previousPaletteSize is equal to 0.
The method as claimed in claim 2 and claim 3, wherein the palette size is equal to 0 when numPredPreviousPalette and num_signalled_palette_entries are both equal to zero.
The method as claimed in claim 1, wherein if the pixel-level escape flag is used and the palette size is equal to zero, the pixel-level escape flag should be all 1.
The method as claimed in claim 1, wherein if the pixel-level escape flag is used and the palette size is equal to zero, all the pixel-level escape flags are not signalled and inferred as 1.
The method as claimed in claim 1, wherein if the pixel-level escape flag is used and the palette size is equal to zero, the pixel-level escape flags are constrained to be 1.
The method as claimed in claim 6-claim 8, wherein the palette size is equal to 0 when palette_share_flag, numPredPreviousPalette and num_signalled_palette_entries are all equal to zero or when palette_share_flag is equal to 1 and previousPaletteSize is equal to 0.
The method as claimed in claim 6-claim 8, wherein the palette size is equal to 0 when numPredPreviousPalette and num_signalled_palette_entries are all equal to zero.
The method as claimed in claim 1, wherein if the palette size of last coded palette CU (previousPaletteSize) is equal to zero, the palette sharing flag is not signaled and inferred as 0.
The method as claimed in claim 1, wherein if the palette size of last coded palette CU (previousPaletteSize) is equal to zero, the palette sharing flag is constrained to be 0.
The method as claimed in claim 1, wherein at least one of pixel-level escape flags in a block should be 0.