GB2521410A - Method and apparatus for encoding or decoding blocks of pixel - Google Patents

Abstract

Processing a current block of pixels of an image using a palette coding mode in which a current palette 2003 is predicted from a palette predictor 2001. The current colour palette comprises a set of colour values associated to palette indices, and the palette coding mode involves predicting the pixels of a block by a set of palette indices. Preferably the palette predictor may be a previously processed predictor, such as that associated to the previous block, or an adjacent processed block. Alternatively, the palette predictor is derived from the values of previously processed pixels adjacent to the current block. The current palette may be an ordered table of colour values, with each entry being predicted from the previous entry in the palette. The palette may be derived from the palette predictor plus a residual corresponding to the difference between the current and predictor palette entries. In the embodiment videos are encoded according to HEVC RExt range extension and the palette indices representing the pixels are run length encoded.

Description

METHOD AND APPARATUS FOR ENCODING OR DECODING BLOCKS OF PIXEL

FIELD OF THE INVENTION

The present invention concerns a method and a device for processing, e.g. encoding or decoding, a current block of pixels of an image using a palette prediction mode. It particularly concerns the palette mode encoding in HEVC Range Extension.

BACKGROUND OF THE INVENTION

It applies more particularly to a mode of coding where a current block of pixels is predictively encoded based on a predictor block encoded with or built from a so-called palette.

A palette in this document is defined as a look up table having entries associating an index with a value of a pixel. Typically, but not necessary, the value of a pixel is constituted by the value of each colour component associated to the pixel, resulting in a colour palette. On the other hand, the value of a pixel may be made of a single pixel component, resulting in a monochrome palette.

This mode of encoding a block of pixel is generally referred to as Palette coding mode. It is contemplated to adopt this mode, for example, in the Range Extension of the High Efficiency Video Coding (HEVC: ISO/IEC 23008-2 MPEG-H Part 2/ ITU-T H.265) international standard.

Wien encoding an image in a video sequence, the image is first divided into coding entities of pixels of equal size referred to as Coding Tree Block (CTB). The size of a Coding Tree Block is typically 64 by 64 pixels. Each Coding Tree Block may then be broken down into a hierarchical tree of smaller blocks which size may vary and which are the actual blocks of pixels to encode. These smaller blocks to encode are referred to as Coding Unit (CU).

The encoding of a particular Coding Unit is typically predictive. This means that a predictor block is first determined. Next, the difference between the predictor block and the Coding Unit is calculated. This difference is called the residue. Next, this residue is compressed. The actual encoded information of the Coding Unit is made of some information to indicate the way of determining the predictor block and the compressed residue. Best predictor blocks are blocks as similar as possible to the Coding Unit in order to get a small residue that could be efficiently compressed.

The coding mode is defined based on the method used to determine the predictor block for the predictive encoding method of a Coding Unit.

A first coding mode is referred to as INTRA mode. According to INTRA mode, the predictor block is built based on the value of pixels immediately surrounding the Coding Unit within the current image. It is worth noting that the predictor block is not a block of the current image but a construction. A direction is used to determine which pixels of the border are actually used to build the predictor block and how they are used. The idea behind INTRA mode is that, due to the general coherence of natural images, the pixels immediately surrounding the Coding Unit are likely to be similar to pixels of the current Coding Unit. Therefore, it is possible to get a good prediction of the value of pixels of the Coding Unit using a predictor block based on these surrounding pixels.

A second coding mode is referred to as INTER mode. According to INTER mode, the predictor block is a block of another image. The idea behind the INTER mode is that successive images in a sequence are generally very similar. The main difference comes typically from a motion between these images due to the scrolling of the camera or due to moving objects in the scene. The predictor block is determined by a vector giving its location in a reference image relatively to the location of the Coding Unit within the current image. This vector is referred to as a motion vector. According to this mode, the encoding of such Coding Unit using this mode comprises motion information comprising the motion vector and the compressed residue.

We focus in this document on a third coding mode called Palette mode.

According to the Palette mode, it is possible to define a predictor block for a given Coding Unit as a block of indexes from a palette: for each pixel location in the predictor block, the predictor block contains the index associated with the pixel value in the Palette which is the closest to the value of the pixel having the same location (i.e. colocated) in the coding unit. A residue representing the difference between the predictor block and the coding unit is then calculated and encoded. Entry indexes in the Palette are also known as "levels".

When using the Palette mode, the palette and the predictor block of indexes or "levels" have to be transmitted in the bitstream encoding the image. This represents a high cost of signalling because a palette, that may comprise tenths of entries, needs to be transmitted at each Coding Unit.

SUMMARY OF THE INVENTION

The present invention has been devised to improve the encoding using the Palette mode, in particular to substantially decrease the signalling costs.

It proposes to improve the encoding of the Palette mode by predicting the palette used when encoding a block of pixels. This results in having less information about the palette to be transmitted from the encoder to the decoder: the signalling costs are substantially reduced.

In this context, according to a first aspect of the invention there is provided a method for processing a current block of pixels of an image using a palette coding mode, the palette coding mode using a current palette to build a predictor block of indexes to predict the current block of pixels, wherein the current palette comprises a set of entries associating respective entry indexes with corresponding pixel values, the method comprising the step of predicting the current palette using a palette predictor.

Accordingly, as mentioned above, less information about the current palette are transmitted to the decoder, thus reducing the signalling in the bitstream. As explained below in this document, all or part of the current palette may be predicted using the palette predictor.

Correspondingly, according to a first aspect of the invention there is provided a device for processing, i.e. an encoder or a decoder depending on the case, a current block of pixels of an image using a palette coding mode, the palette coding mode using a current palette to build a predictor block of indexes to predict the current block of pixels, wherein the current palette comprises a set of entries associating respective entry indexes with corresponding pixel values, the device comprising a prediction module configured to predicting the current palette using a palette predictor.

Optional features of embodiments of the invention are defined in the appended claims. Some of these features are explained here below with reference to a method, while they can be transposed into system features dedicated to a device according to embodiments of the invention.

First referring to the obtaining of the palette predictor, several implementations may be contemplated.

In embodiments of the invention, blocks of pixels of the image are processed according to a predefined scanning order; and the palette predictor for the current block of pixels is selected from a set of palettes used to predict blocks of pixels previously processed according to the palette coding mode. This corresponds to an Inter prediction of the current palette since it is based on past palettes. This configuration is liable to save a great amount of signalling costs since past palettes with a lot of information are already available at both sides (encoder and decoder) when processing the current block of pixels. Efficient prediction of palette can thus be conducted.

In specific embodiments, the palette predictor for the current block of pixels is a palette used for the last block of pixels processed according to the palette coding mode. In other words, it is the last used (e.g. decoded) palette that is selected as a palette predictor. This provision takes advantage of high pixel redundancy between block of pixels that are processed successively, because they are often spatially closed. It also saves memory costs to store previously used palettes, since only the last one needs to be stored. In addition, it may save bits in the encoded bitstream, since there is no need to designate which previously used palette is used as a palette predictor for the current block of pixels.

In other specific embodiments, the palette predictor for the current block of pixels is selected from the set of palettes used for blocks of pixels previously processed according to the palette coding mode and are contiguous to the current block of pixels.

In practice, this is generally the pixels blocks that are above or on the left of the current pixel block. This provision also takes advantage of high pixel redundancy between adjacent block of pixels.

In yet other specific embodiments, the set of palettes used for previously processed blocks of pixels is reset when the current block of pixels starts a new coding entity made of blocks of pixels or starts a new line of coding entities, each made of blocks of pixels. In a variant, the reset may occur at each new image or frame. This provision may speed up the processing (encoding or decoding) of the image, since the coding entities or the lines of coding entities (e.g. Coding Tree Blocks in HEVC) can be processed in parallel. Furthermore, the reset at each new line of coding entities proves to provide a more efficient coding of pixel blocks than a reset at each coding entity.

According to a specific feature, the set of palettes is reset to a null set.

In a variant, the reset set of palettes comprises a by-default palette. This configuration may thus provide palette prediction even in case of reset, i.e. when a new coding entity is processed. This lets it possible to again save signalling costs for the first coding unit in a new coding entity.

In this variant, the by-default palette may comprise a set of predetermined entries corresponding to pixel values equally distributed over a colour space. For example, for pixels represented in the YUV colour space (the same would apply for a RGB colour space), the pixel values may be equally distributed over the Y component, while the U and V component values may be fixed at half the bit-depth used to code each colour component (i.e. U and V take the median value of the corresponding components in the colour space).

In other embodiments of the invention, reference palette predictors are associated with respective coding entities of blocks of pixels that form the image, and the palette predictor for the current block of pixels is the reference palette predictor associated with the coding entity that includes the current block of pixels. These embodiments of the invention make it possible to define very efficient palette predictors at the coding entity (e.g. CTB) level. As an example, a single reference palette predictor may be defined for a single line of coding entities, in the same spirit as the reset above. This is to reduce the costs of signalling the reference palette predictors to the decoder.

In specific embodiments, the reference palette predictor associated with a coding entity is used as the palette predictor for each block of pixels composing the coding entity. Thanks to this provision, a restricted amount of information, namely the reference palette predictor, is needed for the whole coding entity. This aims at substantially saving signalling costs compared to conventional Palette mode.

Preferably, the reference palette predictors are inserted in a bitstream coding the image. This makes it possible for the decoder to have these reference palette predictors to efficiently perform palette prediction for each block of pixels in the coding entity.

In specific embodiments regarding the encoder, the method may further comprise selecting, as a reference palette predictor for a coding entity, a palette that minimizes a rate-distortion criterion from the palettes used to predict all the blocks of pixels of the coding entity. Then the encoder should signal the selected reference palette predictor in the bitstream to the decoder. This offers to select an optimal Palette for all palettes of the coding entity.

In a variant regarding the encoder, the method may further comprise selecting, as a reference palette predictor for a coding entity, the palette used to predict the largest block of pixels of the coding entity. Then the encoder should signal the selected reference palette predictor in the bitstream to the decoder. The implementation of this selection of the reference palette predictor is very simple and this implementation has a low complexity.

In yet other embodiments of the invention, the palette predictor for the current block of pixels includes entries corresponding to values of pixels neighbouring the current block of pixels. This is to take advantage of high pixel redundancies between spatially closed pixels.

In specific embodiments, the pixels neighbouring the current block of pixels are selected from pixels contiguous to the upper and left sides of the current block of pixels. This provision increases the above-specified advantage while keeping dependency on pixels that are always available at the encoder and at the decoder given the causal effect of the block-by-block HEVC coding. Indeed, conventional scanning orders make that the blocks contiguous to the upper and left sides of the current pixel blocks have already been reconstructed.

According to a specific feature, the pixels neighbouring the current block of pixels include the pixels that are, relative to the contiguous current block of pixels, top-left, top-right and boftom-left. Of course, these three specific pixels may form the selected neighbouring pixels. These pixels are those used for conventional Intra prediction mode. They provide relevant information on pixels of the current pixel block with small information.

In other specific embodiments, a predefined number of pixels is selected from a predetermined set of pixels neighbouring the current block of pixels, as entries of the palette predictor, the selected pixels being those having the highest spatial distances between them within the predetermined set of pixels. This creates diversity and avoids duplicate pixels.

In yet other specific embodiments, a pixel class is associated with each neighbouring pixel, and the neighbouring pixels are ordered according to a number of occurrences of their associated class within a predetermined set of pixels neighbouring the current block of pixels, to give lower entry indexes of the palette predictor to neighbouring pixels having more frequent pixel classes. This is to reduce the costs when encoding the palette predictor when appropriate. Note that the classes associated with a given pixel may be defined by the pixel values surrounding the value of the given pixel, for example given a predefined margin.

In yet other embodiments of the invention, the current palette has ordered entries, and predicting the current palette using the palette predictor comprises predicting an entry of the current palette from a preceding entry of the same current palette. In other words, the palette predictor when processing a given entry of the palette is made of (includes) the entries that are prior to the given entry in the colour palette. This sounds like an Intra prediction of the colour palette.

In specific embodiments, a given entry of the current palette is predicted from the entry directly preceding the given entry in the same current palette. This provision reduces signalling in the bitstream (and thus saves coding costs) since it is no longer necessary to send (to the decoder) information on which preceding entry is used to predict the current entry.

In a similar manner, all the entries of the current palette, except the first one, may be predicted from the entry directly preceding them in the same current palette. This also reduces signalling in the bitstream. This is because all the entries are predicted and thus it no longer necessary to identify which entries are predicted and which are not.

The above various embodiments to obtain the palette predictor may be partly or all combined. Taking into account such combination, the palette predictor used to predict the current palette thus combines two or more of: a palette used to predict a previously processed block of pixels; a reference palette predictor associated with a coding entity that includes the current block of pixels; entries corresponding to values of pixels neighbouring the current block of pixels; and at least one entry of the current palette that precedes a current entry to be predicted in the current palette.

Turning now to the prediction of the current palette from the palette predictor, several implementations may be contemplated.

In embodiments of the invention, predicting the current palette using the palette predictor comprises obtaining a bitmap of flags, each of which defining whether or not a corresponding entry in the palette predictor is selected as an entry of the current palette. This configuration has low (memory and coding) costs to define the actual prediction from the palette predictor. The costs are limited to a bitmap.

In specific embodiments, the bitmap of flags comprises the same number of bits as the number of entries in the palette predictor, and each bit at a position in the bitmap defines whether or not the entry having the corresponding position in the palette predictor is selected as an entry of the current palette. This configuration improves the coding efficiency. A variant that may further reduce the size of the bitmap may consider stopping the bitmap at the last entry that is selected as an entry of the current palette.

This is particularly advantageous since, as suggested above, the entries in the palette predictor are ordered according to their occurrences. It results that, in some

B

embodiments, the last entries of the palette predictor are statistically not often used for the current palette.

In other specific embodiments, the method may further comprise adding additional entries at the end of the current palette having the selected entries from the palette predictor. These additional entries may be entries for additional pixels decoded (at both the decoder and the encoder using a decoding loop) and entries from a predetermined palette that is for example built by the encoder and transmitted (in the bitstream) to the decoder (as in the conventional Palette coding mode). This provision is to increase the coding efficiency of the current palette.

In other embodiments of the invention, predicting the current palette using the palette predictor comprises obtaining at least one (possibly two or more) entry residual corresponding to the difference between at least one corresponding entry of the current palette and an entry of the palette predictor. That means that the residuals from palette prediction have to be sent to the decoder. This configuration makes it possible to obtain a finer palette compared to the previous embodiments (based on a copy of entries from the predictor to the current palette).

In specific embodiments, the current palette and the palette predictor have respective ordered entries, and each entry residual corresponds to the difference between an entry of the current palette and an entry of the palette predictor that have the same entry index. This provision reduces signalling in the bitstream (and thus saves coding costs) since it is no longer necessary to send (to the decoder) information on which entry of the predictor is used to predict the current entry.

In a similar manner, a residual may be obtained for every entry of the current palette that has a corresponding entry with the same entry index in the palette predictor. This also reduces signalling in the bitstream since it additionally makes it no longer necessary to identify which entries need prediction and which do not need prediction.

Note that both predictions based on copy of entries or on residuals may be used with the palette predictors obtained using any of the methods described above.

Exception is made for the case where the current palette is intra predicted in which case only the residual approach can be used (otherwise two entries would be the same in the palette).

In embodiments, the pixel values of the entries of the current palette have colour components, and only a subpart of the colour components are predicted using the palette predictor. In practice, one or two colour components out of three may be predicted. This provision reduces processing and signalling in the bitstream.

Another aspect of the invention relates to a device for processing a current block of pixels of an image using a palette coding mode, the palette coding mode using a current palette to build a predictor block of indexes to predict the current block of pixels, wherein the current palette comprises a set of entries associating respective entry indexes with corresponding pixel values, the device being configured to implement any embodiment of the processing method as defined above.

Another aspect of the invention relates to a non-transitory computer-readable medium storing a program which, when executed by a microprocessor or computer system in a device for processing a current block of pixels of an image using a palette coding mode, the palette coding mode using a current palette to build a predictor block of indexes to predict the current block of pixels, wherein the current palette comprises a set of entries associating respective entry indexes with corresponding pixel values, causes the device to perform the step of predicting the current palette using a palette predictor.

The non-transitory computer-readable medium may have features and advantages that are analogous to those set out above and below in relation to the method and device, in particular that of improving coding efficiency of the Palette prediction mode.

Another aspect of the invention relates to a method for processing a current block of pixels of an image using a palette coding mode, the palette coding mode using a current palette to build a predictor block of indexes to predict the current block of pixels, wherein the current palette comprises a set of entries associating respective entry indexes with corresponding pixel values, substantially as herein described with reference to, and as shown in, Figure 13 or 14 or 15 or 17 or 19 or 21 or 22 of the accompanying drawings.

At least parts of the methods according to the invention may be computer implemented. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system".

Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

Since the present invention can be implemented in software, the present invention can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RE signal.

BRIEF DESCRIPTION OF THE DRAW NGS

Embodiments of the invention will now be described, by way of example only, and with reference to the following drawings in which: Figure 1 illustrates the HEVC encoder architecture; Figure 2 illustrates the HEVC decoder architecture; Figure 3 illustrates the concept of the causal area; Figure 4 illustrates Chroma formats supported by HEVC RExt; Figure 5 illustrates the Coding Tree Block splitting in Coding Units and the scan order decoding of these Coding Unit; Figure 6 illustrates the Golomb based binary coding of a syntax element in HEVC; Figure 7 illustrates the principle of Palette mode prediction at the decoder side under investigation in the Range Extension of HEVC; Figure 8 illustrates an example of coding unit with its corresponding block of levels and the associated palette; Figure 9 illustrates the same block of levels and the set of syntax elements used for the encoding of this block of levels; Figure 10 illustrates the decoding process of the syntax elements related to the Palette mode; Figure 11 illustrates the reconstruction process to build the block of levels at the decoding side; Figure 12 illustrates an exemplary palette determination algorithm at the encoder; Figure 13 illustrates the selection of the Pred mode, Level and Run syntax elements at the encoder for the Palette mode; Figure 14 illustrates the principle of using the palette prediction; Figure 14a shows a frame with several CTBs arranged in rows; Figure 15 illustrates the decoding process based on reference palette predictors transmitted in the bitstream according to embodiments of the invention; Figure 16 illustrates the current coding unit with neighboring pixels used as predictors in embodiments of the invention; Figure 17 illustrates the generation of the palette predictor for the current coding unit, based on the neighbouring pixels according to embodiments of the invention; Figure 18 shows an example of the building of a palette predictor according to embodiments of the invention; Figure 19 illustrates the decoding of the palette syntax based on a bitmap of flags according to embodiments of the invention; Figure 20 illustrates the process of Figure 19 on one example; Figure 21 illustrates a decoding process based on having residuals between palette elements and element predictors according to embodiments of the invention; Figure 22 illustrates the intra prediction of the palette according to embodiments of the invention; and.

Figure 23 is a schematic block diagram of a computing device for implementation of one or more embodiments of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

Figure 1 illustrates the HEVC encoder architecture. In the video encoder, an original sequence 101 is divided into blocks of pixels 102. A coding mode is then affected to each block. There are two families of coding modes typically used in HEVC: the modes based on spatial prediction or INTRA modes 103 and the modes based on temporal prediction or INTER modes based on motion estimation 104 and motion compensation 105. An extension of HEVC being currently designed, known as HEVC RExt, adds an additional coding mode, namely the Palette coding mode, that compete with INTRA and INTER coding modes to encode blocks of pixels. This Palette coding mode is described with more details below, in particular with reference to Figures 7 to 13.

An INTRA Coding Unit is generally predicted from the encoded pixels at its causal boundary by a process called INTRA prediction.

Temporal prediction of INTER coding mode first consists in finding in a previous or future frame called the reference frame 116 the reference area which is the closest to the Coding Unit in a motion estimation step 104. This reference area constitutes the predictor block. Next this Coding Unit is predicted using the predictor block to compute the residue in a motion compensation step 105.

In both cases, spatial and temporal prediction, a residual is computed by subtracting the Coding Unit from the original predictor block.

In the INTRA prediction, a prediction direction is encoded. In the temporal prediction, at least one motion vector is encoded. However, in order to further reduce the bitrate cost related to motion vector encoding, a motion vector is not directly encoded. Indeed, assuming that motion is homogeneous, it is particularly interesting to encode a motion vector as a difference between this motion vector, and a motion vector in its surrounding. In H.264/AVC coding standard for instance, motion vectors are encoded with respect to a median vector computed between 3 blocks located above and on the left of the current block. Only a difference, also called residual motion vector, computed between the median vector and the current block motion vector is encoded in the bitstream. This is processed in module "Mv prediction and coding" 117.

The value of each encoded vector is stored in the motion vector field 118. The neighboring motion vectors, used for the prediction, are extracted from the motion

vector field 118.

Then, the mode optimizing the rate distortion performance is selected in module 106. In order to further reduce the redundancies, a transform, typically a OCT, is applied to the residual block in module 107, and a quantization is applied to the coefficients in module 108. The quantized block of coefficients is then entropy coded in module 109 and the result is inserted in the bitstream 110.

The encoder then performs a decoding of the encoded frame for the future motion estimation in modules 111 to 116. This is a decoding loop at the encoder.

These steps allow the encoder and the decoder to have the same reference frames. To reconstruct the coded frame, the residual is inverse quantized in module 111 and inverse transformed in module 112 in order to provide the "reconstructed" residual in the pixel domain. According to the encoding mode (INTER or INTRA), this residual is added to the INTER predictor 114 or to the INTRA predictor 113.

Then, this first reconstruction is filtered in module 115 by one or several kinds of post filtering. These post filters are integrated in the decoding loop. It means that they need to be applied on the reconstructed frame at the encoder and decoder in order to use the same reference frames at the encoder and decoder. The aim of this post filtering is to remove compression artifacts.

The principle of an HEVC decoder has been represented in Figure 2. The video stream 201 is first entropy decoded in a module 202. The residual data are then inverse quantized in a module 203 and inverse transformed in a module 204 to obtain pixel values. The mode data are also entropy decoded and in function of the mode, an INTRA type decoding or an INTER type decoding is performed. In the case of INTRA mode, the INTRA prediction direction is decoded from the bitstream. The prediction direction is then used to locate the reference area 205. If the mode is INTER, the motion information is decoded from the bitstream 202. This is composed of the reference frame index and the motion vector residual. The motion vector predictor is added to the motion vector residual to obtain the motion vector 210. The motion vector is then used to locate the reference area in the reference frame 206. Note that the motion vector field data 211 is updated with the decoded motion vector in order to be used for the prediction of the next decoded motion vectors. This first reconstruction of the decoded frame is then post filtered 207 with exactly the same post filter as used at encoder side. The output of the decoder is the de-compressed video 209.

Figure 3 illustrates the causal principle resulting from block-by-block encoding as in HEVC.

At a high-level, an image is divided into Coding Units that are encoded in raster scan order. Thus, when coding block 3.1, all the blocks of area 3.3 have already been encoded, and can be considered available to the encoder. Similarly, when decoding block 3.1 at the decoder, all the blocks of area 3.3 have already been decoded and thus reconstructed, and can be considered as available at the decoder.

Area 3.3 is called the causal area of the Coding Unit 3.1. Once Coding Unit 3.1 is encoded, it will belong to the causal area for the next Coding Unit. This next Coding Unit, as well as all the next ones, belongs to area 3.4 illustrated as doted area, and cannot be used for coding the current Coding Unit 3.1. It is worth noting that the causal area is constituted by reconstructed blocks. The information used to encode a given Coding Unit is not the original blocks of the image for the reason that this information is not available at decoding. The only information available at decoding is the reconstructed version of the blocks of pixels in the causal area, namely the decoded version of these blocks. For this reason, at encoding, previously encoded blocks of the causal area are decoded to provide this reconstructed version of these blocks.

It is possible to use information from a block 3.2 in the causal area when encoding a block 3.1. In the HEVC Range Extension draft specifications, a displacement vector 3.5, which can be transmitted in the bitstream, may indicate this block 3.2.

Figure 5 illustrates a splitting of a Coding Tree Block into Coding Units and an exemplary scan order to sequentially process of these Coding Units. In the HEVC standard, the block structure is organized by Coding Tree Blocks (CTBs). A frame contains several non-overlapped and square Coding Tree Block. The size of a Coding Tree Block can be equal to 64x64 to 16x16. This size is determined at sequence level.

The most efficient size, in term of coding efficiency, is the largest one: 64x64. Please note that all Coding Tree Blocks have the same size except for the image border, meaning that they are arranged in rows. The size of the boundary CTBs is adapted according to the amount of remaining pixels.

Each Coding Tree Block contains one or more square Coding Units (CU).

The Coding Tree Block is split based on a quad-tree structure into several Coding Units. The processing (coding or decoding) order of each Coding Unit in the Coding Tree Block follows the quad-tree structure based on a raster scan order. Figure 5 shows an example of the processing order of Coding Units. In this figure, the number in each Coding Unit gives the processing order of each corresponding Coding Unit of this Coding Tree Block.

In HEVC, several methods are used to code the different syntax element, for example block residuals, information on predictor blocks (motion vectors, INTRA prediction directions, etc.). HEVC uses several types of entropy coding like the Context based Adaptive Binary Arithmetic Coding (CABAC), Golomb-rice Code, or simple binary representation called Fixed Length Coding. Most of the time a binary encoding process is performed to represent the different syntax element. This binary encoding process is also very specific and depends on the different syntax element.

For example, the syntax element called "coeff_abs_level_remaining" contains the absolute value or a part of an absolute of the coefficient residual. The idea of this binary encoding process is to use Golomb-Rice code for the first values and Exponential Golomb for the higher values. More specifically, depending on a given parameter called Golomb Order, this means that for representing the first values, for example values from 0 to 3, a Golomb-Rice code is used, then for higher values, for example values from 4 and above, an Exponential Golomb code is used. The Golomb Order is a parameter used by both the Golomb-Rice code and the exponential Golomb code.

Figure 6 illustrates this principle at the decoding side. The input data of the decoding process are the bitstream 601 and the Order which is known as the Rice Golomb parameter, or the Golomb Order. The output of this process is the decoded symbol 612.

The prefix value is set equal to 1 at step 602 then 1 bit is extracted from the bitstream at step 601 and the variable flag is set equal to the decoded value 603. If this flag is equal to 0 at step 604 the Prefix value is incremented 605 and another bit is extracted from the bitstream 603. When the flag value is equal to 1, the decision module 606 checks if the value Prefix is strictly inferior to 3. If it is true, the N=Order bits are extracted 608 from the bitstream 601 and set to the variable "codeword". This corresponds to the Golomb-Rice representation. The Symbol value 612 is set equal to ((prefix c< Order) + codeword) as depicted in step 609. Where <c' is the left shift operator.

If the Prefix is superior or equal to 3 at step 606, the next step is 610 where N= (prefix -3 + Order) bits are extracted from the bitstream and set to the variable "codeword" 610. The symbol value 611 is set equal to ((1<c(prefix-3fl+2)c< Order) + codeword. This corresponds to the exponential Golomb representation.

In the following, this decoding process, and in a symmetric way the corresponding encoding process, is called Golomb_H with an input parameter corresponding to the Golomb Order. It can be noted in a simple way Golomb_H(Order).

In HEVC, for some syntax elements such as residuals, the Golomb Order is updated in order to adapt the entropy coding to the signal to be encoded. The updating formula tries to reduce the Golomb code size by increasing the Golomb Order when the coefficients have large values. In the HEVC standard, the update is given by the following formula: Order = Min( cLastRiceOrder + (cLastAbsLevel > (3 * (1 << cLastRiceOrder)) ? 1: 0), 4) Where cLastRiceOrder is the last used Order, cLastAbsLevel is the last decoded coeff_abs_level_remaining. Please note that for the first parameter to be encoded or decoded, cLastRiceOrder and cLastAbsLevel are set equal to 0. Morever please note that the parameter Order cannot exceed the value of 4 in this formula. And where the expression (C ? A: B) has the value A if the condition C is true and B if the condition C is false.

The HEVC Range Extension, also commonly called HEVC RExt, is an extension that is currently being drafted of the new video coding standard HEVC.

An aim of this extension is to provide additional tools to code video sequences with additional colour formats and bit-depth, and possibly losslessly. In particular, this extension is designed to support 4:2:2 colour format as well as 4:4:4 video format in addition to 4;2.O video format (see Figure 4). A colour image is generally made of three colour components R, G and B. These components are generally correlated, and it is very common in image and video compression to de-correlate the colour components prior to processing the images. The most common format that de-correlates the colour components is the YUV colour format. YUV signals are typically created from RGB representation of images, by applying a linear transform to the three inputs R, G and B input frames. Y is usually called Luma component, U and V are generally called Chroma components. The term YCbCr' is also commonly used in place of the term YUV'.

Regarding the bit-depth which is the number of bits used to code each colour component of a pixel, if the current HEVC standard is able to deal with 4:2:0 colour format with 8 and 10 bits bit-depth (i.e. 256 to 1,024 possible colours), HEVC RExt is about to be designed to additionally support 4:2:2 and 4:4:4 video format with an extended bit-depth ranging from 8 bits up to 16 bits (i.e. up to 65,536 possible colours). This is particularly useful to have a larger dynamic of colour components.

HEVC RExt is also designed to provide a lossless encoding of the input sequences; this is to have a decoded output 209 strictly identical to the input 101. To achieve this, a number of tools have been modified or added, compared to the conventional HEVC lossy codec. A non-exhaustive list of exemplary modifications or additions to operate losslessly is provided here below: -removal of the quantization step 108 (203 at the decoder); -forced activation of the bypass transform, as normal cosine/sine transforms 107 may introduce errors (204 at the decoder); -removal of tools specifically tailored at compensating quantization noise, such as post filtering 115 (207 at the decoder).

For HEVC RExt, the updating formula of the Golomb Order has been further modified in order to be adapted to deal with higher bit-depth and to take into account very high quality required by application dealing with video compression of extended format (4:2:2 and 4:4:4) including lossless coding. For HEVC RExt, the updating formula has been changed as follows: Order = Min( cLastRiceOrder + (cLastAbsLevel >> (2 + cLastRiceOrder) ), 7) With this formula, the maximum value of Order is 7. Moreover, for the first coding of the coeff_abs_level_remaining for a sub-block of Transform block, the Golomb order is set equal to: Order = Max( 0, cRiceOrder -(transform_skip_flag II cu_transquant_bypass_flag? 1: 2)) where * the variable "transform_skip_flag" is set to 1 if the transform (e.g. OCT 107 or 204) is skipped for the current coding unit and 0 if the transform is used, * the variable "cu_transquant_bypass_flag" is set to 1 if the coding unit is lossless encoded and 0 otherwise, * the variable "cRiceOrder" is set equal to last used Order from another sub-block of the transform block otherwise is set to 0.

Additional tools for HEVC RExt are currently being designed to efficiently encode "screen content" video sequences in addition to natural sequences. The "screen content" video sequences refer to particular video sequences which have a very specific content corresponding to those captured from a personal computer of any other device containing for example text, PowerPoint presentation, Graphical User Interface, tables (e.g. screen shots). These particular video sequences have quite different statistics compared to natural video sequences. In video coding, performance of conventional video coding tools, including HEVC, proves sometimes to be underwhelming when processing such screen content".

The current tools currently discussed on in HEVC RExt to process "screen content" video sequences include the Intra Block Copy mode and the Palette mode.

Prototypes for these modes have shown good coding efficiency compared to the conventional method targeting natural video sequences. Focus is made in this document on the Palette coding mode.

The palette mode of HEVC RExt is a prediction mode. It means that the Palette method is used to build a predictor for the coding of a given coding unit similarly to a prediction performed by motion prediction (Inter case) or by an Intra prediction.

After the generation of the prediction, a residual coding unit is transformed, quantized and coded. In other words, the same processes as described above with reference to Figures 1 and 2 apply.

A palette is generally represented by a table containing a finite set of N-tuple of colors, each color being defined by its components in a given colour space (see for example 803 in Figure 8 based on YUV colour space). For example, in a typical RGB format, the palette is composed of a list of P elements of N-tuple (where N=3 for a RGB). More precisely, each element corresponds to a fixed triplet of colour components in the RGB format. Of course this is not limited to a RGB or YUV colour format. Any other colour format can be represented by a palette and can use a smaller or a higher number of colour components, meaning that N may be different from 3.

At the encoder side, the Palette mode, under consideration in RExt, consists in transforming pixel values of a given input coding unit into indexes called levels identifying the entries in an associated palette. After the transformation, the resulting coding unit or block is composed of levels and is then transmitted to the decoder with the associated palette, generally a table having a finite number of triplets of colours used to represent the coding unit. Since the palette defines a finite number of colours, the transformation into a block of indexes usually approximates the original input coding unit.

To apply the Palette mode at the encoder side, an exemplary way to transform a coding unit of pixels is performed as follows: -find the P triplets describing at best the coding unit of pixels to encode, for example by minimizing overall distortion; -then associate with each pixel of the coding unit the closest colour among the P triplets: the value to encode (or level) is then the index corresponding to the entry of the associated closest colour.

For each coding unit, the palette (i.e. the P triplets found), the block of indexes or levels and the residual representing the difference between the original coding unit and the block of indexes in the colour space (which is the block predictor) are coded in the bitstream 110 and sent to the decoder.

At the decoder, the Palette mode consists in operating the conversion in the reverse way. This means that each decoded index associated with each pixel of the coding unit is replaced by the corresponding colour in the palette decoded from the bitstream, in order to reconstruct the corresponding colour for each pixel of the coding unit. This is the reconstruction of the block of indexes in the colour space (i.e. of the coding unit predictor). Since the Palette mode is a prediction mode, the associated residual is decoded from the bitstream and then added to the reconstructed coding unit predictor to build the final reconstructed coding unit.

Figure 7 further illustrates the principle of Palette mode at the decoder. The prediction mode for the current coding unit is extracted at step 702 from the bitstream 701. Currently, the Palette mode is identified by a flag located before the skip flag in the bitstream (the other coding modes have been described above with reference to Figures 1 and 2). This flag is CABAC coded using a single context. If this mode is the Palette mode 703 then the related syntax of the Palette mode 705, i.e. the information on the palette, the block of levels and the residual, is extracted and decoded 704 from the bitstream 701.

Then, during step 706, two elements are built from the decoded data: the palette 707 and the block of levels 708. From this block of levels and the associated palette, the coding unit predictor in pixel domain 710 is built 709. It means that for each level of the block of levels, a color (RGB or YUV) is associated to each pixel.

Then the coding unit residual is decoded 711 from the bitstream 701. In the current implementation Palette mode, the residual associated with a Palette mode is coded using the common HEVC Inter residual coding method, i.e. using Golomb coding. To obtain the residual of the coding unit, the conventional inverse quantization and inverse transformation are performed. The block predictor 710 is added 713 to this coding unit residual 712 in order to form the reconstructed coding unit 714.

Figure 8 illustrates the principle of the Palette mode at the encoder. The current coding unit 801 is converted into a block 802 of the same size which contains a level for each pixel instead of 3 colour values (Y, U, V) or (R, G, B). The palette 803 associated with this block of levels is built based on coding unit overall distortion minimization and associates at each entry, an entry index or level with corresponding pixel colour values. Please note that for monochrome application, the pixel value can contain only one component.

As mentioned in relation to Figure 7, the palette (as well as the residual) is coded and inserted in the bitstream for each coding unit. In the same way, the block of levels (corresponding to the coding unit predictor) is coded and inserted in the bitstream and an example of the coding is given below with reference to Figure 9. In this example, the block of levels is scanned in a horizontal order.

The block of levels 91 is exactly the same as the one illustrated in Figure 8 under reference 802. The tables 92 and 93 describe the successive syntax elements used to code the block of levels 91. Table 93 should be read as the continuation of table 92. The syntax elements in the table correspond to the encoding of the groups of levels surrounded by bold lines in the block 91.

The block of levels is encoded by group of successive pixels in scan order.

Each group is encoded using a first syntax element giving a prediction direction, a second element giving the repetition, and an optional third element giving the value of the pixel, namely the level. The repetition corresponds to the number of pixels in the group.

These two tables depict the current syntax associated to the Palette mode.

These syntax elements correspond to the encoded information associated in the bitstream for the block of levels 91. In these tables, three main syntax elements are used to fully represent the operations of the Palette mode and are used as follows when successively considering the levels of the block of levels 91.

A first syntax element, called "Pred mode" allows to distinguish between two encoding mode. In a first mode corresponding to "Pred mode" flag equal to "0", a new level is used for the current pixel. The level is immediately signaled after this flag in the bitstreeam. In a second mode corresponding to "Pred mode" flag equal to "1", a "copy up" mode is used. More specifically, this means that the current pixel level corresponds to the pixel level located at the line immediately above starting on the same position for a raster scan order. In that case of "Pred mode" flag equal to "1 there is no need to signal a level immediately after the flag because the value of the level is known by reference to the value of the level of the pixel just above in the block of levels 91.

A second syntax element called "Level" indicates the level value of the palette for the current pixel only in the first mode of "Pred mode".

A third syntax element, called "Run", is used to encode a repetition value in both modes of "Pred mode". Considering that the block of levels 91 is scanned from the top left corner to the bottom right corner, row by row from left to right and top to bottom, the Run syntax element gives the number of successive pixels in block 91 having the same encoding.

This "Run" syntax element has a different meaning which depends on the "pred mode" flag. VThen Pred mode is 0, "Run" element is the number of successive pixels of the predictor block having the same level value. For example, if Run=8 this means that the current "Level" is applied to the current pixel and to the following 8 pixels which corresponds to 9 identical successive samples in raster scan order.

When Pred mode is 1, Run" element is the number of successive pixels of the predictor block having a level value corresponding to the level value of their above pixel in block 91, i.e. where the "copy up" mode is applied. For example, if Run=31 this means that the level of the current pixel is copied from the pixel of the line above as well as the following 31 pixels which corresponds to 32 pixels in total.

Regarding tables 92 and 93, represent the eight steps to represent the block 91 by using the Palette mode. Each step starts with the coding of the "Pred mode" flag which is followed by the "Level" syntax element when "Pred mode" flag equals "0", or by the "Run" syntax element when "Pred mode" flag equals "1". The "Level" syntax element is always followed by a "Run" syntax element.

When the prediction mode decoded for the current block is the palette mode, the decoder first decodes the syntax related to this block and then applied the reconstruction process for the coding unit.

Figure 10 illustrates the decoding process of the syntax elements related to the Palette mode. First, the size of the palette is extracted and decoded 1002 from the bitstream 1001. The exact size of the palette (Palette_size) is obtained by adding ito this size value decoded at step 1002. Indeed, the size is coded by using a unary code for which the value 0 has the smallest number of bits (1 bit) and the size of the palette cannot be equal to 0, otherwise no pixel value can be used to build the block predictor.

Then the process corresponding to the palette values decoding starts. A variable i corresponding to the index of the palette is set equal to 0 at step 1004 next a test is performed at step 1005 to check if i is equal to the palette size (Palette_size). If it is not the case, one palette element is extracted from the bitstream 1001 and decoded 1006 and is then added to the palette with the associated level/index equal to i. Then the variable i is incremented through step 1007. If i is equal to the palette size 1005, the palette has been completely decoded.

Next the process corresponding to the decoding of the block of levels 91 is performed. First, the variable j, corresponding to a pixel counter, is set to 0 as well as the variable syntax_i 1008. Then a check is performed to know if the pixel counter corresponds to the number of pixels contained in the block. If the answer is yes at step 1009 the process ends at step 1017, otherwise the value of the flag "Pred mode" corresponding to one prediction mode is extracted from the bitstieam 1001 and decoded 1010.

The value of "Pred mode" is added to a table at the index syntax_i containing all "Pred mode" value decoded. If the value of this "Pred mode" is equal to 0, step 1011, the syntax element corresponding to "Level" is extracted from the bitstream 1001 and decoded 1012. This variable "Level" is added to a table at the index syntax_i containing all levels decoded. The variable j corresponding to the pixel counter is incremented by one 1013.

Next the "Run" syntax element is decoded at step 1014. If the syntax element "Pred Mode" is equal to 1, step 1011, the "Run" value is also decoded at step 1014. This syntax element "Run" is added to a table at the index syntax_i containing all the runs decoded.

Next at step 1015, the value j is incremented by the value of the run decoded at step 1014. The variable syntax_i is incremented by one to consider the next set of syntax elements. If the counterj is equal to the number of pixels in the block then the syntax to build the block of levels 91 is finished 1017. At the end of this process related to the Palette, the decoder knows the palette, and the tables containing the list of all the "Pred mode", "Level" and "Run" syntax elements associated with the Palette mode of this coding unit. The decoder can then proceed with the reconstruction process of the coding unit as described through Figure 7.

Each palette element, constituted by three values in the above examples, is generally encoded using three binary codes. The length of the binary codes corresponds to the bit-depth of each color component. The palette size is typically encoded using unary code. The "Pred mode" element is encoded using one bit. The "Level" element is encoded using binary code with binary code length equal to b, where 2b is the smallest integer equal or above the palette size. And the "Run" element is encoded using Golomb_H(Order=3) as explained above in relation to Figure 6.

Figure 11 illustrates the reconstruction process to build the block of levels 91 and then the block predictor in the colour space that has to be used as predictor.

The input data of this process are the tables obtained in the process of Figure 10 above, and containing the list of "Fred mode", "Level" and "Run". An additional input data is the size of the coding unit 801 (which is the same as the size of the block of levels 802/91) known from the quadtree (Figure 5) signalled in the bitstream.

In a first step 1101, a variable i, representing a pixel counter, is set equal to O and a variable j, to successively consider each set of syntax elements, is also set equal to 0. At step 1104, the element Pred_mode[j] extracted from the table of "Pred mode" at index j is checked against 0.

If it is equal to 0, a new level is encoded for the current pixel i. As a consequence, the value of the pixel at position i is set equal to the level at the index from the table of levels; Block[i] =Level[j]. This is step 1105. The variable i is incremented by one at step 1106 to consider the next pixel, and the variable k, dedicated to count the pixels already processed in the current Run, is set equal to 0 at step 1107.

A check is performed at step 1108 to determine whether or not k is equal to the "Run" element of the table of runs at the index j: k = Run[j] ?. If not equal, the level of the pixel at position i is set equal to the level value of the pixel at position i-i: Block[i] = Block[i-1]. This is step 1109. The variable i and the variable k are then incremented by one at respectively steps 1110 and 1111. If k = Run[j] at step 1108, the propagation of the left level value is finished and step 1120 is performed (described below).

If Pred_modeD] is different from 0 at step 1104, the "copy up" mode starts with the variable k set equal to Oat step 1112. Next, step 1113 checks whether or not (k-i) is equal to the "Run" element of the table of runs at the index j: k = Run[j]+i? If not equal, the level value of the pixel at position i is set equal to the level value of the pixel at position i of the above line: Block[i] = Block[i-width], where "width" is the width of the block of levels (the same as the coding unit) as deduced from the input size of the coding unit. This is step 1114. Next, the variable i and the variable k are incremented by one at respectively steps 1115 and 1116. If k = Run[j]+1 at step 1113, the prediction mode copy up' is completed and the process goes on at step 1120.

At step 1120, a check is performed to determine whether or not the variable i is equal to the amount of pixels in the block 91/CU 801. If not equal, the variable j is incremented by one at step 1121 to consider the next set of syntax elements and the process loops back to step 1104 described above.

If all the pixels have been processed at step 1120, the final block of levels 91 is obtained at step 1122: this corresponds to table Block[]. Then a final step 1123 consists in converting each level in colour values using the palette 803 decoded using the process of Figure 10. This final step affects pixel values (Y, U, V) or (ft G, B) at each block position according to the level of this position in the block and the corresponding entries in the palette.

Other aspects of the palette mode as introduced in HEVC RExt regard the determination by the encoder of the palette to be used to encode the current coding unit (see Figure 12 below), and the selection of the Pred mode, Level and Run syntax elements at the encoder (see Figure 13 below).

Figure 12 illustrates an exemplary palette determination algorithm at the encoder. The input data of this process are the original coding unit of pixels and its coding unit size. In this example, a YUV palette is built, but other implementations may result in having a RGB palette built in the same way.

At a first step 1201, a variable j representing a pixel counter is set to 0, a variable "Palette_size" to follow the growth of the palette as it is being built is also set to 0, and a variable "TH" representative of a threshold is set to 9. Then at step 1203, the pixel p, i.e. having the index i according to a scanning order, is read at step 1203 from the original coding unit 1204. Then the variable j is set equal to 0 at 1205 and at step 1206 a check is performed to determine whether or not the palette size is equal to the variable j" (meaning that all the palette elements of the palette under construction has been considered).

If the palette size is equal to j, the palette at the index"" is set equal to the pixel value p at step 1209. This means that the current pixel P becomes a new element in the palette, with index j associated with it. More precisely the following assignment is performed: PALy[j] = (VI) PALu[j]= (UI) PAL] = (Vi) where PALyuv are three tables to store the colour values.

The palette size (Palette_size) is incremented by one at step 1210 and an occurrence table Counter is set equal to 1 for the index Palette size' at step 1211.

Then the variable i is incremented by one at step 1213 to consider the next pixel i" of the current coding unit. A check is then performed at step 1214 to determine whether or not all the pixels of the current coding unit have been processed. If they have all been processed, the process is completed by an ordering step 1215 explained later on, otherwise the next pixel is considered at step 1203 described above.

Back to step 1206, if j is different from palette_size, step 1207 is performed where the absolute value for each colour component between p and the palette element at the index j is computed. The formulas are shown in the Figure. If all the absolute differences are strictly less than the predefined threshold TH, the occurrence counter regarding the element "j" in the palette is incremented by one at step 1212.

Step 1207 creates a class for each element of the palette under construction, such a class encompassing colours neighbouring the colour of the element, given the margin TH. Thus step 1212 counts the occurrences of each class. Step 1212 is followed by step 1213 already described.

In the condition of step 1207 is not met, the variable j is incremented by one at step 1208 to consider the next palette element in the palette. This is to compare the other palette colour elements to the current pixel through new occurrence of step 1207.

If no element in the palette meets the criterion of step 1207, a new element is added to the palette as described above with reference to steps 1209, 1210 and 1211.

One may note that the decision module 1207 can compared each color element for a 4:4:4 (YUV or RGB) sequences and can only compare the Luma colour component for 4:2:0 sequences.

At the end of the process of Figure 12, the table "Counter" contains the number of occurrences of the classes defined by the respective palette elements. Then the palette elements are ordered at step 1215 according to their occurrences so that the most frequent element is in the first position (entry with the lowest index or "level") in the palette.

One may also note that the size of the palette can be limited to a maximum size, for example 24 entries. In such a case, if the size of the palette resulting from step 1215 exceeds 24, the palette is reduced by removing the elements (entries) from the 25th position in the ordered palette. It results that a palette has been built.

Turning now to the selection of the Pred mode, Level and Run syntax elements at the encoder, input data of the process of Figure 13 are the original coding unit of pixels, the palette as built through the process of Figure 12 and the coding unit size. In particular, this evaluation is performed when determining which coding mode between INTRA coding, INTER coding and Palette coding has to be used.

At a first step 1301, the variable "i' representing a pixel counter is set to 0.

The process described below seeks to determine the syntax elements for the pixels starting from i. The two modes of prediction are evaluated independently: "Pred mode" = 0 on the right hand part of the Figure, and "Pred mode" =1 on the left hand part of the Figure.

For the copy up' prediction (corresponding to "Fred mode" =1), the variable "i," used to count the number of levels in the current Run is set equal to 0 at step 1303. Then at step 1304 the current level at pixel location i: Block[i + i00], is compared to the level of the pixel located just above in the above line: Block[i + i013, -width], where "width" corresponds to the width of the current coding unit. Note that the level Block[i + of each pixel of the coding unit is determined in parallel at step 1308. This step consists in associating with the pixel at the position i, the closest palette element (in practice its index or level) as already explained above. This step uses the position i, the palette 1306 and the original coding unit 1307.

If Block[i + = Block[i + -width] at step 1304, the variable "i001," is incremented by one at step 1305 to consider the next pixel value of the block of pixels and to indicate that the current pixel level at position i + i can be included in the current "copy up" Run. If Block[i + i] is different from Block[i + i -width] at step 1304 meaning that the current evaluation of a copy up" Run has ended, the variable "i" is transmitted to the decision module 1314. At this stage of the process, the variable "i," corresponds to the number of values copied from the line just above.

For the left value prediction (corresponding to "Pred mode" =0), the loop to determine the Run value (hen) is processed in parallel or sequentially. First the variable Stad used to store the index i of the current pixel is set to "i', and the variable " used to consider successively the pixel levels following index i' is also set equal to i" and the variable "Iefl" used to count the current Run under construction is set equal to 0.

This is step 1309. Next, step 1310 consists to determine whether or not j!= 0 and "Pred_modeD-1]" = 0 and BlockU] = Block[j-1]. Pred_mode[] is a table used by the encoder to store the prediction mode (either 1 or 0 for respectively the "copy up" prediction and the left value prediction). It is filled up progressively at step 1317 described below as the successive pixels are processed, and has been initialized with zero values for example at step 1301: Pred_mode[k]=0 for any k.

If the condition at step 1310 is met, the variable "Ieff" is incremented by one at step 1311 to indicate that the current pixel level at position j can be included in the current "left value" Run, and the variable j is incremented by one at step 1312 to consider the next pixel value of the block of pixels.

If the condition at step 1310 is not met, the variable "j" is compared to "sta" to determine if it is the first pixel value to be examined for the current "left value"Run.

This is step 1313. If "j" is equal to or less than "5tad", meaning that it is the first pixel value to be examined for the current Run, then it starts the current Run and the next pixel value is considered at step 1312 described above. If "j" is strictly higher than Stan, meaning that a first pixel value different from the pixel value of the current "left value"Run has been detected. the variable "heft" which corresponds to the length of the current "left value" Run is transmitted to the decision module 1314. Note that, as the loop for "copy up" prediction, the level Block[i] at the index i is determined in the same loop at step 1308.

After having computed the maximum run for the left value prediction' and the copy up' mode, the variable "heft" and "i0," are compared at step 1314. This is to determine whether or not "i" 0 and "0r," + 2 is higher than left This is an exemplary criterion to select either the copy up mode or the left value prediction mode.

In particular, the parameter "2" may be slightly changed.

The condition at step 1314 means that if "i" is equal to 0 or is smaller than or equal to iier2, the "left value prediction" mode is selected at step 1315. In that case, a "PredMode" variable is set equal to 0 and a Run variable is set equal to "Ieft" at same step 1315. On the other hand, if"i0" is different from 0 and is strictly higher than "ier2" the "copy-up" mode is selected at step 1316. In that case, the "PredMode" variable is set equal to 1 and the Run variable to i001,-1 at step 1316.

Then the tables containing the "Pred_mode" and the "Run" at the encoder are updated with the current value "Fredmode" and "Run", at step 1317. Then, the next position to consider in the block of pixels is computed at step 1318, which corresponds to the current position i incremented by the "run" value +1. Then a check is performed at step 1319 to determine whether the last pixels of the coding unit have been processed. If it is the case, the process ends at step 1320, otherwise the evaluation of the two prediction modes "left prediction" and "copy up" are evaluated starting at steps 1303 and 1309 for the next pixel position to obtain a new set of syntax elements.

At the end of this process, the encoder knows the levels for each sample of the coding unit, and is able to encode the corresponding syntax of the block of levels based on the content of the three tables Pred_modefl, Blockfl and Run[].

As described above, the Palette mode as currently designed in HEVC RExt requires a palette to be transmitted for each coding unit. This represents a large amount of data in the bitstream, and thus a coding cost. The inventors have envisaged an improvement of the Palette mode to improve its coding efficiency According to one aspect of the invention, the current palette for a current coding unit is predicted using a palette predictor. This makes it possible to reduce the amount of information to be transmitted for each palette associated with a coding unit.

Various embodiments are described below.

As the approach of the invention is prediction based, the obtaining of the predictor for the current coding unit are first described with reference to Figures 14 to 18 and 22 and then the syntax elements to actually describe the prediction to the decoder are described with reference to Figures 19 to 22.

General steps of a decoding process implementing the invention is shown in Figure 14 which is based on Figure 7 described above. One skilled in the art would directly understand the corresponding operations at the encoder to appropriately build the bitstream. Blocks l4xx in Figure 14 are similar to blocks 7xx in Figure 7. As shown in the Figure, the main idea of the invention happens at step 1406 where the palette is at least partly predicted from a palette predictor 1416 instead of being entirely decoded from the bitstream. Several ways to obtain 1415 the palette predictor are described below, including obtaining a palette used to predict a previously processed block of pixels or coding unit; a reference palette predictor associated with a coding entity that includes the current coding unit; entries corresponding to values of pixels neighbouring the current coding unit; and at least one entry of the current palette that precedes a current entry to be predicted in the current palette.

According to first embodiments to obtain the palefte predictor, blocks of pixels of the image are processed according to a predefined scanning order as described above with reference to Figure 5 where the CUs of a CTB are coded/decoded according to a scan order. The palette predictor for the current block of pixels is then selected from a set of palettes used to predict previously processed blocks of pixels. Preferably, the palette predictor for the current block of pixels is a palette used for the last processed block of pixels. This may be implemented by storing the last used decoding palette (for the last processed coding unit) in memory. Of course another palette from among the already used palettes may be selected as a palette predictor, in which case an identifier of such selected palette should be provided to the decoder.

According to the scan order used, it is generally observed that the last used palette, and thus the palette predictor, is one of the paleftes used for previously processed coding units that are contiguous to the current coding unit. This ensures high redundancy between the coding units to be taken into account to have an efficient coding. For example, based on Figure 5, the palette predictor can be one of the palettes used to process the left or above coding units 15, 17, 12 and 13.

To avoid poor palette prediction due to a palefte predictor which has content far from the actual content of the current coding unit, specific embodiments provide the reset of the last used palette (or of the set of previously used palettes).

Such reset may occur at each new CTB. i.e. when the current coding unit starts a new coding entity made of blocks of pixels, or at each new line or row of CTBs as introduced above, or even at each new frame.

Such reset also makes it possible for the encoder or the decoder to estimate the palette mode for each CTB (or row of CTBs or frame) in parallel. However, since some correlations often exist between the coding units of 2 CTBs, the reset is preferably performed for each new row of CTBs. This is illustrated using Figure 14a which shows a frame with several CTB5 (each represented by a square). The CTBs on the left in thick lines are the CTBs for which the last used Palette is reset because each of them starts a new line of CTBs.

The reset at the first CTB of a line is a more efficient approach than the reset at Frame level. This is because the CTBs are encoded in a horizontal raster scan order, and thus for the first CTB of a line of CTBs, the last used palette is potentially predicted from a spatially far CTB (the last one of the CTB line just above). Given the spatial distance between the CTBs, the correlation between them is very low and thus a dependency (prediction) between their respective palettes is not worthy.

The reset of the last used palette may mean that no palette is available as a palette predictor. In that case, the palette of the very first coding unit processed in the first CTB of the line cannot be predicted. This reset may be performed by setting the Previous_Palette_size variable (storing the size of the last used palette) to 0.

As it may be worth predicting the palette for the coding unit first processed, a variant includes replacing the last used palette by a by-default palette, so that the by-default palette is used as a palette predictor for the first coding unit of the line of CTBs.

Various ways to generate a by-default palette (both the encoder and the decoder operate in the same way) may be considered. As an example, the by-default palette may include a set of predetermined entries corresponding to colour values equally distributed over a colour space. The equidistribution may concern the three colour components. However, in a preferred embodiment, the colour values may be equally distributed over the Y component. And the U and V component values are fixed, for example at the median value of the U or V components in the colour space. The U and V values may be directly computed from the bit-depth of the components, by assigning the value bit-depth/2 or bit-depth >> 1, where >> is the right shift operator. An example of the distribution along the Y component is the following formula: YLevel =(Level * bit-depth)! Previous_Palette_size, where Level corresponds to the entry index of the by-default table as explained above (and thus is incremented by one at each new value Y), bit-depth is the bit-depth of the Y component. Note that the Previous_Palette_size may be equal to the last used palefte, or to an average Palette_size computed from the last decoded CTB or line of CTBs or frame, or to a predefined number such as 4.

According to other embodiments to obtain the palette predictor, reference palette predictors are associated with respective coding entities of coding units that form the image, such as CTBs, lines of CTBs, slices, slice segment, tile, frame or sequence. And the palette predictor for the current block of pixels is the reference palette predictor associated with the coding entity that includes the current block of pixels. These embodiments may require the reference palefte predictor to be transmitted in the bitstream for each coding entity.

Figure 15 illustrates the decoding process based on reference palette predictors transmitted in the bitstream. This Figure is based on Figure 14 and thus blocks 1501 to 1514 are similar to blocks 1401 to 1414, except that the module 1506 builds the palette using the reference palette predictor 1516 decoded from the bitstream. Of course, a corresponding process is performed at the encoder to include the reference palette predictor in the bitstream.

At the beginning of the CTB decoding (or CTB line, Slice, Frame, etc. decoding), the reference palette predictor 1516 is extracted from the bitstream at step 1515. Note that the use of the reference palette predictor may lead to have no palette provided in the bitstream for any coding unit of the current coding entity CTB. In this case one flag may be provided in the bitstream and thus be extracted therefrom in order to signal not to decode any palette for the coding units. In a variant, this flag may be replaced by using the value 0 for the Palette_size to indicate the decoder that no palette is to be decoded for the current coding unit. This variant requires that the Palette_size be equal to decodedSize instead of decodedSize + 1 in step 1003 above.

To save any bit used for signaling the use of the reference palette predictor, the reference palette predictor can be transmifted at the end of the CTB in case at least one CU of the current CTB is coded using the Palette coding mode.

In any case, the reference palette predictor is extracted and decoded is needed to decode one of the coding units of the current CTB. Module 1502 extracts the prediction mode. If it is not the Palette mode (test 1503), the decoder decodes the CU with the corresponding mode 1517. Otherwise (the palette mode), the palette for the current coding unit is decoded as in Figure 7. However, the building 1506 of the palette 1507 in this embodiment depends on the reference palette predictor 1516. Examples of this dependency and of corresponding syntax elements 1505 are described below.

When the coding unit is decoded (1517, 1520) as described above for Figures 7 and 13, the decoder checks whether or not it was the last coding unit of the CTB at step 1518. Note that this figure doesn't contain the full syntax decoding of a CTB that may involve other syntax elements. In case it was not the last coding unit, the prediction mode for the next CU is extracted at step 1502. In case it was the last coding unit, the remaining processes to entirely decode the CTB are performed (not shown) and the decoder checks whether or not it was the last CTB of the frame at step 1519. If it was not the last CTB, the reference palette predictor for the next CTB is extracted from the bitstream at step 1515 described above. Of course, this figure is not complete and several decoding steps that do not regard the present invention have been omitted.

As described, the reference palette predictor transmitted is used as the predictors of the palette used for each coding unit CU in the current CTB. As described below, the reference palette predictor can be used to predict elements of the palette or, in a variant, the reference palette predictor can be used as the palette for the current coding unit. In that case, the reference palette predictor 1516 is directly transmitted to module 1509 thereby making module 1507 no longer required.

The selection of the reference palette predictor at the encoder may contribute to coding efficiency. Several algorithms can be used to determine the "best" reference palette predictor. For example, the palette used to predict the largest coding unit in the current CTB can be selected as the reference palette predictor for the CTB.

In another example, the palette that minimizes a rate-distortion criterion from the palettes used to predict all the coding units composing the current CTB may be determined and used as the reference palette predictor for the CTB. Of course, other selection criteria may be used.

According to yet other embodiments to obtain the palette predictor, the palette predictor for the current coding unit includes entries corresponding to values of pixels neighbouring the current coding unit. In these embodiments, the palette predictor for the current CU is extracted from the neighboring pixels as exemplary shown in Figure 16.

In this example, the selected pixels are pixels contiguous to the upper and left sides of the current block of pixels, because they belong to the causal area as defined above with reference to Figure 3. These contiguous pixels are shown in grey color in the Figure. In particular, the selected pixels are a fixed number, for example the same three pixels (dark grey pixels 1601 to 1603 in the Figure) as those used for INTRA prediction mode, namely the pixels that are, relative to the contiguous current coding unit, top-left, top-right and bottom-left. Of course, another number of pixels may be considered. Preferably the relevant pixels to select are known by both the encoder and the decoder so that no additional signalling information is needed. However, embodiments may envisage selecting specific pixels at the encoder and then signalling, in the bitstream, the number of selected pixels and those selected pixels.

In one embodiment, a restricted set of neighbouring pixels is considered.

For example, this set of pixels is selected in order that the pixels have the highest spatial distance. This creates diversity and avoids duplicate pixels.

Figure 17 illustrates the generation of the palette predictor for the current coding unit, based on the neighbouring pixels. As noted above, the order of the elements in the palette predictor is important. This requires the definition of classes for the palette elements (here neighbouring pixels). This determination process can be performed at both the encoder and the decoder.

In one embodiment, the neighbouring pixels 1701 are classified 1702. Note that the neighbouring pixels may include pixels that are not directly contiguous to the current coding unit. Each neighboring pixel of the considered set of neighbouring pixels is associated with a class (so to an entry index) depending on its colour distance from already existing entries in the palette predictor, for example using the criteria of step 1207 in Figure 12. In an embodiment, the classes are defined by the three pixels 1601 to 1603 shown in Figure 16. It results a non-ordered palette predictor 1703. In addition, during the classification 1702, the occurrences 1704 of each class are counted.

Based on the non-ordered palette predictor 1703 and the occurrences 1704, the palette predictor 1706 with ordered entries is built at step 1705. Note that the entries having insignificant occurrences (for example below a threshold) may be discarded from the palette predictor.

In an embodiment, two pixels of the same class have exactly the same pixel value (the criterion used for classification thus does not involve absolute value and requires a threshold TH set to zero). Note that in the images targeted by HEVC RExt (and thus by the Palette mode) which include text or screenshots, there are few different values in contiguous coding units. Classifying the pixels based on an identity of pixel values is thus relevant.

Figure 18 illustrates an example of classification. 1801 show the current coding unit with contiguous pixels which have either a first pixel value (represented by class 1'), or a second pixel value (represented by class 2) or a third pixel value (represented by class 3'). The set of neighboring pixels is represented with its related classes in table 1802. Table 1802 gives the occurrences associated with each class.

The palette predictor that is built from table 1802 is shown in table 1803, by ordering the classes according to their occurrences and removing non-significant predictor entries (here entry corresponding to class 1' due to its low number of occurrences, namely 2 in the example). To remove a predictor entry, the algorithm can take into account the number of neighbouring pixels, the number of classes and/or the occurrences of the most probable neighbouring pixels.

According to yet other embodiments to obtain the palette predictor, the current palette has ordered entries, and predicting the current palette using the palette predictor comprises predicting an entry of the current palette from a preceding entry of the same current palette. In other words, the palette predictor when processing a given entry of the palette is made of (includes) the entries that are prior to the given entry in the colour palette currently being built. The current palette is thus intra predicted.

This is illustrated by Figure 22 which shows the decoding process to obtain the predicted palette based on signaling in the bitstream when appropriate. Figure 22 only reproduces the part corresponding to the top left pad of Figure 10, i.e. the generation of the predicted palette. The remainder of Figure 10 thus has to be performed to obtain the syntax elements defining the block predictor 91.

As shown, the Palette_size is decoded and computed at steps 2201 and 2202. Then the first palette element is decoded. As the palette is intra predicted, the first palette element is not predicted and thus is directly decoded from the bitstream.

Then the variable i provided to successively consider each palette entry is set equal to 1 at step 2204. The other palette elements are decoded through the next steps. In particular, for each palette element, a flag, namely Use_Pred, is decoded at step 2206 to determine whether or not (test 2207) the palette element at index i uses intra prediction or not. If it does not use Intra prediction, the palette element is decoded directly from the bitstream at step 2208. Otherwise, index j corresponding to the index of the palette element predictor in the current palette is decoded from the bitstream at step 2210. Note that the encoder may have coded indexj relatively to index i in order to save bits, in which case the decoder operates in the reverse way. Then the residual is decoded at step 2211 and the palette element Pal[i] is set equal to Res[i] + PaID] and added to the palette at step 2212. Then the index i is incremented by one at step 2209 to consider the next palette element. Once all the palette elements have been decoded (test 2205), the process goes on step 1008 of Figure 10.

In one embodiment, the element predictor of the palette element i is the palette element i-i, i.e. the palette entry directly preceding the current palette element in the current palette. In such case, module 2210 can be omitted, and the palette element Pal[i] is set equal to Res[i] + Pal[i-1] when is predicted. In one embodiment, all the palette entries, except the first one, are predicted from the palette element directly preceding them in the current palette. In such case, the Use_pred flag may be omitted since the decoder knows the palette elements to decode using intra prediction. It results that modules 2206 and 2208 can be omitted.

To improve the coding efficiency of the intra prediction of the palette element, the palette element may be ordered according to their values and not to their occurrences at the encoder.

Note that the above various embodiments to obtain the palette predictor may be partly or all combined to provide several basis for predicting all or part of the palette elements.

Turning now to the the syntax elements to actually describe the palette prediction to the decoder, reference is now made to Figures 19 to 21, although Figure 22 described above already introduces a mechanism to define the palette prediction.

The palette predictor is considered to be retrieved and its size (Predictor_of_palette_size) is known.

According to embodiments regarding the syntax elements, predicting the current palette using the palette predictor comprises obtaining a bitmap of flags, each of which defining whether or not a corresponding entry in the palette predictor is selected as an entry of the current palette. It results that in addition to information making it possible for the decoder to retrieve the appropriate palette predictor, only the bitmap needs to be sent to the decoder. This bitmap may be sent in replacement of the palette as defined in HEVC RExt for the current coding unit.

The syntax of the bitmap contains M flags, M being equal to the number of elements in the palette predictor. The ith decoded flag defines whether or not the element i from the palette predictor is used to fill (predict) the current palette for the current coding unit. In a variant, the bitmap may be restricted to a lower number of flags from a flag corresponding to the first element in the palette predictor to the last element that has to be used as an element predictor. The size of the bitmap is specified in the bitstream in a similar fashion as the palette size is specified in HEVC RExt bitstream.

For example, the elements of the palette predictor that are associated with a flag (bit) equal to 1 are copied in the current palette at the first available position, keeping their order.

In another embodiment, additional entries are added at the end of the current palette having the selected entries from the palette predictor. For example, first the bitmap is decoded from the bitstream and the corresponding entries of the palette predictor are copied in the current palette, then additional pixels may be added at the end of the current palette in the same way as the conventional palette transmission.

In one embodiment seeking to provide the predicted palette element as additional palette element, the determining of the Palette_size is adapted to be increased by the number of predicted palette element: to do so, the Palette_size is set equal to the decoded size + the number of flags set equal to 1 in the bitmap (Palette_pred_enable_size). If Palette_pred_enabled_size is equal to 0, the Palette_size is set equal to the decoded size + 1 as described in step 1003.

Figure 19 illustrates the decoding of the palette syntax of these embodiments based on the bitmap of flags. As for Figure 22, Figure 19 is based on Figure 10 but only the part related to the palette decoding is shown.

First, the palette predictor 1902 is obtained at step 1901 according to any of the embodiments described above. In addition, the Predictor_of_palette_size 1903 is also obtained. Module 1905 decodes N flags from the bitstream 1904, where N = Predictor_of_palette_size.

For each flag equal to 1, the corresponding element from the palette predictor is added to the current palette 1907 at the first available index, during step 1906. Palette_pred_enabled_size 1908 representing the number of flags equal to 1 in the bitmap is transmitted to decision module 1910. The size of the remainder of the palette is also decoded from the bitstream 1909. Decision module 1910 determines whether or not Palette_pred_enabled_size is equal to 0. If it is equal to 0 meaning that there is no predicted palette element in the current palette, the Palette_size is set equal to the decoded Size + 1 at step 1911, and the variable i used to successively consider each entry of the current palette is set equal to 0 at step 1912. If Palette_pred_enabled_size is different from 0 meaning that there is at least one predicted palette element in the current palette, the Palette_size is set equal to the decoded Size + Palette_pred_enabled_size at step 1913, and the variable i is set equal to Palette_pied_enabled_size. Then the decoding loop of palette elements is performed through steps 1915, 1916 and 1917 corresponding to steps 1005, 1006 and 1007 of Figure 10. This loop stops when the variable i is equal to the Palette_size. The decoded Palette elements 1916 are added at the end of the current Palette 1907, i.e. after the predicted palette elements. In one embodiment, Palette_size is always set equal to the decoded Size + Palette_pred_enabled_size at step 1913 to simplify the implementation so that modules 1910, 1911 and 1912 can be omitted.

Figure 20 illustrates the process of Figure 19 on an example. The palette predictor obtained using any of the embodiments described above with reference to Figures 14 to 18 is table 2001 which contains five elements associating an index or level with colour values. The decoded flags of the bitmap are represented in table 2002. In this example, two flags are set equal to 1: one for level 0 and one for level 2 of the palette predictor. The corresponding palette elements are thus added to the current palette 2003 with the first level available, respectively level 0 and level 1. Then new palette entries may be decoded from the bitstream as proposed in HEVC RExt and added to the position 2 and 3.

One may note that when the palette predictor is transmitted, only the flags (bitmap) corresponding to the palette predictors is needed. To reduce signaling, the same bitmap may be used for all the coding units belonging to the same CTB, slice, tile, slice segment, frame or sequence for which a single reference palette predictor is transmitted.

According to other embodiments regarding the syntax elements, predicting the current palette using the palette predictor comprises obtaining at least one (possibly two or more) entry residual corresponding to the difference between at least one corresponding entry of the current palette and an entry of the palette predictor. In these embodiments, a residual between the current palette element and the palette predictor is transmitted in the bitstream. The residual Res[i] is equal to Pal[i]-Pal_Pred[j], where Res[i] is the residual for level i, Pal[i] is the current palette element for level i, and Pal_Predu] is the element predictor identified by level j. Note that the palette predictorj usually needs to be transmitted unless it is known by the decoder (for example because j is fixed relatively to i, for instance j=i).

The decoding of the residual for the three colour components is different from the decoding of the palette element. Indeed, as mentioned in the prior art, a palette element is coded with a fix length of N bits, with N = 3* bit-depth. For the residual and in order to save bits, each color component residual may be coded with an adaptive code, such as a Golomb code (in the same way as the coefficients of the block residual).

Figure 21 illustrates a decoding process based on having such residuals between the palette elements and element predictors. Again, this Figure only shows the part related to the palette decoding. In addition, to simplify the Figure, the bitstream has not been represented.

The decoded size of the palette is extracted from the bitstream at step 2101 and the Palette_size is set equal to the decoded Size + 1 at step 2102. The variable i used to successively consider each palette entry is set to 0 at step 2103. Then the loop to decode the palette starts with test 2104 to determine whether or not all the palette entries have been processed. For the palette element i, a flag, Use_pred, is decoded from the bitstream at step 2105 to determine (test 2106) whether or not the palette element i uses prediction or not. If the palette element i does not use prediction, it is decoded at step 2107 using conventional mechanisms and added to the palette at the level equal to i. Then the variable i is incremented by one at step 2108 to consider the next palette element. If the palette element i uses prediction, the predictor index j is decoded from the bitstream at step 2112. Note that for coding efficiency purpose, the length of the code used to encode the predictor index j depends on Predictor_of_Palette_size 2110. Thus, in parallel, the palette predictor 2110 is obtained as described above and Predictor_of_Palette_size 2011 is also obtained.

Once the predictor index j is known, the residual Res[i] of the palette element is also decoded from the bitstream at step 2113. Then, the palette element Pal[i] is computed from the formula Res[i] + Pal_Pred[j] at step 2114 using the palette predictor Pal_Pred 2111. The palette element Pal[i] is then added to the current palette. Next, the variable i is incremented by one at step 2108 to consider the next palette element. At the end of this process, the current palette has been decoded.

In one embodiment, the index j is set equal to i, in which case the predictor index j is no longer required to be transmitted to the decoder. Consequently, module 2112 can be omitted. In addition, a residual may be obtained for every element of the current palette that has a corresponding entry with the same entry index/level in the palette predictor. In this case, if i is superior or equal to the Predictor_of_Palette_size, no residual is decoded. Furthermore, the flag Use_pred is no longer required since the decoder knows which palette elements to predict based on Predictor_of_Palette_size.

Consequently, modules 2105 and 2106 can be omitted. These approaches reduce the number of signaling bits required for the palette prediction, by removing the signaling of the predictors. This is useful when the palette elements are ordered according to their occurrences.

In embodiments, only one or two colour components out of the three (or more) are predicted.

Above have been described several ways to obtain a palette predictor (Figures 14 to 18 and 22) and several ways to define and signal the prediction of the palette from the palette predictor (Figures 19 to 22). These embodiments can be combined, except that when the current palette is intra predicted, only the residual approach can be used (otherwise two entries would be the same in the palette). In a preferred embodiment, the palette predictor is the last decoded CU with a reset at each line of CTBs or at each CTB; and the palette predictor is signaled with a flag bitmap indicating whether or not the elements of the palette predictor have to be copied in the current palette being built.

Figure 23 is a schematic block diagram of a computing device 2300 for implementation of one or more embodiments of the invention. The computing device 2300 may be a device such as a micro-computer, a workstation or a light portable device. The computing device 2300 comprises a communication bus connected to: -a central processing unit 2301, such as a microprocessor, denoted CPU; -a random access memory 2302, denoted RAM, for storing the executable code of the method of embodiments of the invention as well as the registers adapted to record variables and parameters necessary for implementing the method for encoding or decoding at least part of an image according to embodiments of the invention, the memory capacity thereof can be expanded by an optional RAM connected to an expansion port for example; -a read only memory 2303, denoted ROM, for storing computer programs for implementing embodiments of the invention; -a network interface 2304 is typically connected to a communication network over which digital data to be processed are transmitted or received. The network interface 2304 can be a single network interface, or composed of a set of different network interfaces (for instance wired and wireless interfaces, or different kinds of wired or wireless interfaces). Data packets are written to the network interface for transmission or are read from the network interface for reception under the control of the software application running in the CPU 2301; -a user interface 2305 may be used for receiving inputs from a user or to display information to a user; -a hard disk 2306 denoted HD may be provided as a mass storage device; -an I/O module 2307 may be used for receiving/sending data from/to external devices such as a video source or display.

The executable code may be stored either in read only memory 2303, on the hard disk 2306 or on a removable digital medium such as for example a disk.

According to a variant, the executable code of the programs can be received by means of a communication network, via the network interface 2304, in order to be stored in one of the storage means of the communication device 2300, such as the hard disk 2306, before being executed.

The central processing unit 2301 is adapted to control and direct the execution of the instructions or portions of software code of the program or programs according to embodiments of the invention, which instructions are stored in one of the aforementioned storage means. After powering on, the CPU 2301 is capable of executing instructions from main RAM memory 2302 relating to a software application after those instructions have been loaded from the program ROM 2303 or the hard-disc (HD) 2306 for example. Such a software application, when executed by the CPU 2301, causes the steps of the flowcharts shown in Figures 14, 15, 17, 19, 21 and 22 to be performed.

Any step of the algorithms shown in Figures 14,15,17,19,21 and 22 may be implemented in software by execution of a set of instructions or program by a programmable computing machine, such as a PC ("Personal Computer"), a DSP ("Digital Signal Processor") or a microcontroller; or else implemented in hardware by a machine or a dedicated component, such as an FPGA ("Field-Programmable Gate Array") or an ASIC ("Application-Specific Integrated Circuit").

Although the present invention has been described hereinabove with reference to specific embodiments, the present invention is not limited to the specific embodiments, and modifications will be apparent to a skilled person in the art which lie within the scope of the present invention.

Many further modifications and variations will suggest themselves to those versed in the art upon making reference to the foregoing illustrative embodiments, which are given by way of example only and which are not intended to limit the scope of the invention, that being determined solely by the appended claims. In particular the different features from different embodiments may be interchanged, where appropriate.

In the claims, the word comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used.

Claims (33)

1. CLAIMS1. A method for processing a current block of pixels of an image using a palette coding mode, the palette coding mode using a current palette to build a predictor block of indexes to predict the current block of pixels, wherein the current palette comprises a set of entries associating respective entry indexes with corresponding pixel values, the method comprising the step of predicting the current palette using a palette predictor.
2. 2. The method of Claim 1, wherein blocks of pixels of the image are processed according to a predefined scanning order; and the palette predictor for the current block of pixels is selected from a set of palettes used to predict blocks of pixels previously processed according to the palette coding mode.
3. 3. The method of Claim 2, wherein the palette predictor for the current block of pixels is a palette used for the last block of pixels processed using the palette coding mode.
4. 4. The method of Claim 2, wherein the palette predictor for the current block of pixels is selected from the set of palettes used for blocks of pixels previously processed according to the palette coding mode and are contiguous to the current block of pixels.
5. 5. The method of Claim 2, wherein the set of palettes used for previously processed blocks of pixels is reset when the current block of pixels starts a new coding entity made of blocks of pixels or starts a new line of coding entities, each made of blocks of pixels.
6. 6. The method of Claim 5, wherein the set of palettes is reset to a null set.
7. 7. The method of Claim 5, wherein the reset set of palettes comprises a by-default palette.
8. 8. The method of Claim 7, wherein the by-default palette comprises a set of predetermined entries corresponding to pixel values equally distributed over a colour space.
9. 9. The method of Claim 1, wherein reference palette predictors are associated with respective coding entities of blocks of pixels that form the image, and the palette predictor for the current block of pixels is the reference palette predictor associated with the coding entity that includes the current block of pixels.
10. 10. The method of Claim 9, wherein the reference palette predictor associated with a coding entity is used as the palette predictor for each block of pixels composing the coding entity.
11. 11. The method of Claim 9, wherein the reference palette predictors are inserted in a bitstream coding the image.
12. 12. The method of Claim 9, further comprising selecting, as a reference palette predictor for a coding entity, the palette used to predict the largest block of pixels of the coding entity.
13. 13. The method of Claim 9, further comprising selecting, as a reference palette predictor for a coding entity, a palette that minimizes a rate-distortion criterion from the palettes used to predict all the blocks of pixels of the coding entity.
14. 14. The method of Claim 1, wherein the palette predictor for the current block of pixels includes entries corresponding to values of pixels neighbouring the current block of pixels.
15. 15. The method of Claim 14, wherein the pixels neighbouring the current block of pixels are selected from pixels contiguous to the upper and left sides of the current block of pixels.
16. 16. The method of Claim 15, wherein the pixels neighbouring the current block of pixels include the pixels that are, relative to the contiguous current block of pixels, top-left, top-right and bottom-left.
17. 17. The method of Claim 14, wherein a pixel class is associated with each neighbouring pixel, and the neighbouring pixels are ordered according to a number of occurrences of their associated class within a predetermined set of pixels neighbouring the current block of pixels, to give lower entry indexes of the palette predictor to neighbouring pixels having more frequent pixel classes.
18. 18. The method of Claim 14, wherein a predefined number of pixels is selected from a predetermined set of pixels neighbouring the current block of pixels, as entries of the palette predictor, the selected pixels being those having the highest spatial distances between them within the predetermined set of pixels.
19. 19. The method of Claim 1, wherein the current palette has ordered entries, and predicting the current palette using the palette predictor comprises predicting an entry of the current palette from a preceding entry of the same current palefte.
20. 20. The method of Claim 19, wherein a given entry of the current palette is predicted from the entry directly preceding the given entry in the same current palette.
21. 21. The method of Claim 19, wherein all the entries of the current palette, except the first one, are predicted from the entry directly preceding them in the same current palette.
22. 22. The method of two or more of Claims 1 and 9 and 14 and 19, wherein the palette predictor used to predict the current palette combines two or more of: a palette used to predict a previously processed block of pixels; a reference palette predictor associated with a coding entity that includes the current block of pixels; entries corresponding to values of pixels neighbouring the current block of pixels;and at least one entry of the current palette that precedes a current entry to be predicted in the current palette.
23. 23. The method of Claim 1, wherein predicting the current palette using the palette predictor comprises obtaining a bitmap of flags, each of which defining whether or not a corresponding entry in the palette predictor is selected as an entry of the current palette.
24. 24. The method of Claim 23, wherein the bitmap of flags comprises the same number of bits as the number of entries in the palette predictor, and each bit at a position in the bitmap defines whether or not the entry having the corresponding position in the palette predictor is selected as an entry of the current palette.
25. 25. The method of Claim 23, further comprising adding additional entries at the end of the current palette having the selected entries from the palette predictor.
26. 26. The method of Claim 1, wherein predicting the current palette using the palette predictor comprises obtaining at least one entry residual corresponding to the difference between at least one corresponding entry of the current palette and an entry of the palette predictor.
27. 27. The method of Claim 26, wherein the current palette and the palette predictor have respective ordered entries, and each entry residual corresponds to the difference between an entry of the current palette and an entry of the palette predictor that have the same entry index.
28. 28. The method of Claim 27, wherein a residual is obtained for every entry of the current palette that has a corresponding entry with the same entry index in the palette predictor.
29. 29. The method of Claim 1, wherein the pixel values of the entries of the current palette have colour components, and only a subpart of the colour components are predicted using the palette predictor.
30. 30. A device for processing a current block of pixels of an image using a palette coding mode, the palette coding mode using a current palette to build a predictor block of indexes to predict the current block of pixels, wherein the current palette comprises a set of entries associating respective entry indexes with corresponding pixel values, the device comprising a prediction module configured to predicting the current palette using a palette predictor.
31. 31. A device for processing a current block of pixels of an image using a palette coding mode, the palette coding mode using a current palette to build a predictor block of indexes to predict the current block of pixels, wherein the current palette comprises a set of entries associating respective entry indexes with corresponding pixel values, the device being configured to implement the processing method according to any one of Claims ito 29.
32. 32. A non-transitory computer-readable medium storing a program which, when executed by a microprocessor or computer system in a device for processing a current block of pixels of an image using a palette coding mode, the palette coding mode using a current palette to build a predictor block of indexes to predict the current block of pixels, wherein the current palette comprises a set of entries associating respective entry indexes with corresponding pixel values, causes the device to perform the step of predicting the current palette using a palette predictor.
33. 33. A method for processing a current block of pixels of an image using a palette coding mode, the palette coding mode using a current palette to build a predictor block of indexes to predict the current block of pixels, wherein the current palette comprises a set of entries associating respective entry indexes with corresponding pixel values, substantially as herein described with reference to, and as shown in, Figure 13 or 14 or 15 or 17 or 19 or 21 or 22 of the accompanying drawings.