EP2526696A1

EP2526696A1 - Method and device for encoding an image block of an image and corresponding decoding method and device

Info

Publication number: EP2526696A1
Application number: EP10843635A
Authority: EP
Inventors: Xiaozhong Xu; Zhibo Chen
Original assignee: Thomson Licensing SAS
Current assignee: InterDigital VC Holdings Inc
Priority date: 2010-01-22
Filing date: 2010-01-22
Publication date: 2012-11-28
Also published as: WO2011088592A1; EP2526696A4

Abstract

An incoming bit stream (INBS) is residue decoded (RESD). DC coefficient residue (DCCR) is determined and combined with DC prediction value (DCPV) for regaining DC coefficient (COEF). Prediction value (DCPV) is determined using decision operation (DCPD). This decision operation receives mask information (MINB) of neighbouring blocks retrieved from memory (MEM) and mask information (MCB) of the current block as inputs. Mask information (MCB) of the current block results from comparing mask information code word (MICD) separated from bitstream (INBS) with mask code word table MIT wherein mask information code word (MICD) is separated from bit stream (INBS) and mask decoded (MDEC).

Description

METHOD AND DEVICE FOR ENCODING AN IMAGE BLOCK OF AN IMAGE AND CORRESPONDING DECODING METHOD AND DEVICE

TECHNICAL FIELD

The invention is made in the technical field of encoding of an image block comprised in image.

BACKGROUND OF THE INVENTION

Information comprised in raw pixel data of image blocks is highly correlated. Therefore, encoding raw pixel data, as is, would lead to highly redundant code. A way to remove this redundancy is transformation, e.g. using a discrete cosine transform (DCT) . DCT has the properties of separability, symmetry and orthogonality. Separability allows for realizing a two-dimensional DCT by successive application of two one- dimensional DCTs which are also called directional DCT.

By DCT operation, the spatial domain signals are transformed into frequency domain and called coefficients. A first

coefficient of the transformed signal, which is considered to be the average of the spatial domain signals, is called DC coefficient. The remaining coefficients are called AC

coefficients instead. The DC coefficient of a transformed block concentrates the most of the block' s energy and

therefore often has a large non-zero value. This phenomenon brings difficulty to entropy coding, as a large value will generate a long sequence of bits after entropy coding.

Therefore directly encoding the DC coefficient will lead to low coding efficiency.

There are two existing methods to deal with this problem. One is DC prediction as proposed in ITU-T, "Video coding for low bit rate communication Version 2", ITU-T Recommendation H.263 Version 2 (H.263+), 1998. As shown in Fig. la, a current block C, which is yet to-be-encoded, is neighboured by. already encoded blocks UP, block UL and block HL . Block UP is

vertically adjacent to block C from above, block HL is

horizontally adjacent to block C from the left and lower right corner of block UL is adjacent to upper left corner of block H.263+ proposes using DC coefficient of either block UP or block HL as a prediction for the DC coefficient of block C. In this way, a residual of the DC coefficient of block C with respect to the prediction is encoded which commonly shows a reduced absolute value compared to the DC coefficient, as is, and therefore requires a smaller bit amount in encoding.

Another solution is to explore the correlation among a series of DC coefficients, by applying yet another transform on the DC coefficients of the series. For example, in H.264/AVC intra coding, if a 4x4 block is selected, then a 16x16 Macroblock MB is divided into 16 sub-blocks, as shown in Fig.2. After each of these 16 4x4 blocks are transformed, the 16 DC coefficient in a 4x4 form, are transformed again to further reduce their correlation .

A similar strategy is implemented when a partition-based transform scheme is used. A partition-based transform scheme is a transform scheme adapted for transforming image blocks which are partitioned, for instance according texture present in the image block, into two or more partitions of variable size, and variable shape, in particular not necessarily rectangular shape. According to the partition-based transform scheme, the partitions are transformed separately. For example there are three sub-partitions (indicated by number) in one block, as depicted in Fig.3a. The partition-based transform scheme is based on a predominant partition direction,

indicated by the dashed arrows in Fig. 3a. Per partition, spatial domain signal like pixel values are organized in two- dimensional arrays with pixel values located in the partition along the predominant partition direction being organized in a same column. Then, a first directional DCT is performed along the columns. Intermediate coefficients Al' , .. , Jl' resulting from these first DCTs, depicted in Fig. 5a-5c, are then

subjected to a second DCT performed along rows of the

intermediate coefficients. This results in coefficients

depicted in Fig. 6a-6c, wherein the

coefficients |A1 ' |, |E1'|, and JI do carry the DC portion

Therefore, a third DCT is performed across the coefficients l' El' , and kJl' J resulting in final coefficients |Al*|, |E1

and Jl* as depicted in Fig. 7 .

As the partitioning is meant to separate partitions which exhibit significant differences and therefore are preferably not transformed all together in standard block transformation, the third DCT, which is a transform across partitions, is likely to be not very effective. Thus, encoding coefficients is likely to require a bit amount only slightly smaller than the bit amount required for encoding |A1 ' |,

El' , and Jl'

SUMMARY OF THE INVENTION

There is an ongoing effort in the art to specify alternative DC coefficient prediction for the encoding of image blocks in a partition-based transform scheme.

The invention engages in this effort and proposes a method for encoding of an image block comprised in an image according to claim 1 and a corresponding decoding method according to claim 5, as well as a encoding device according to claim 9 and a decoding device according to claim 10. Further, a storage medium carrying an image block according to claim 11 is

proposed .

The features of further advantageous embodiments are specified in the dependent claims.

Said proposed encoding method comprises the steps of analysing texture of the image block for selecting a partitioning mask out of a set of indexed partitioning masks of which at least some are associated with a predominant direction, using the selected partitioning mask for selecting a partition of the image block, determining a DC coefficient and AC coefficients by transforming the selected partition according to a

transform scheme adapted to the selected partitioning mask, using the selected partitioning mask and the selected

partition for determining a further partition of an image block adjacent to the current image block wherein, at least, a further DC coefficient of the further partition is already encoded, using the further DC coefficient of said further partition for determining a prediction, and encoding an index of the selected partitioning mask, the AC coefficients and a residual between the DC coefficient and the determined

prediction .

Using the selected partitioning mask and the selected

partition for determining said further partition allows for using the predominant direction and/or a correspondence in relative positioning in the respective image blocks for selecting the further partition. This allows for selecting the further partition such that it is the one in which the texture of the selected partition continues. Thus, the DC coefficient of the further partition is likely to be the optimal

prediction for the DC coefficient of the selected partition.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the invention are illustrated in the drawings and are explained in more detail in the following description. The exemplary embodiments are explained only for elucidating the invention, but not limiting the invention' s disclosure, scope or spirit defined in the claims.

In the figures: Fig. la exemplarily depicts a current image block, a horizontally adjacent image block to the left of the current image block, a vertically adjacent image block above the current image block and a diagonally adjacent image block to the upper left;

Fig. lb exemplarily depicts a current image block as in Fig.

la with an additional diagonally adjacent image block to the upper right;

Fig. 2 exemplarily depicts a macro image block comprising 16 sub-blocks ;

Fig. 3a-b exemplarily depicts partitioning of an image block into three partitions and predominant directions of the partitions;

Fig. 4a-c exemplarily depicts arrangement of spatial domain

signals of the three partitions depicted in Fig. 3a-b in three separate arrays each comprising spatial domain signals of a single partition;

Fig. 5a-c exemplarily depicts intermediate coefficients

resulting from a first one-dimensional DCT performed, for each array separately, along columns of the spatial domain signals depicted in Fig. 4a-c;

E'ig. 6a-c exemplarily depicts coefficients resulting from a

second one-dimensional DCT performed, for each array separately, along rows of the intermediate coefficients depicted in Fig. 5a-c;

Fig. 7 exemplarily depicts coefficients resulting from a

third one-dimensional DCT performed across the arrays on DC coefficients comprised in the arrays depicted in Fig. 6a-c;

Fig. 8a-d exemplarily depicts different partitioning situations regarding the current image block, the horizontally adjacent image block to the left of the current image block and the vertically adjacent image block above the current image block;

Fig. 9 exemplarily depicts a flowchart of a DC predictor determining method; and

Fig. 10 exemplarily depicts a decoding flowchart.

EXEMPLARY EMBODIMENTS OF THE INVENTION

The invention can be realized on any electronic device

comprising a processing device correspondingly adapted. For instance, the invention can be realized in a television, a mobile phone, a personal computer, a digital still camera, a digital video camera, a video-player, a navigation system or a car video system.

In particular, the invention is exemplarily embodied in a video or still image encoder or in a video or still image decoder.

In such encoder or decoder, a module is comprised which allows for texture based partitioning of a current image block C.

To do so at encoder side, the image block's texture is

analysed and an indexed partitioning mask is selected from an indexed partitioning mask list. The index of the selected partitioning mask is encoded, in the bit stream, together with further code regarding the image block. Partitioning masks of the list are associated with a predominant direction although the associated predominant direction may be "none" for non- directional texture patterns, e.g. circular ones.

At decoder side, the index encoded in the bit stream is decoded and the partitioning mask is selected from the list according to the decoded index. I.e. corresponding lists are present at encoder and decoder.

After partitioning of the current image block C into two or more partitions pixel values of each partition are subjected to discrete cosine transformation at encoder side. Doing so, redundancy in each partition is well removed and only some redundancy remains in the DC coefficient of each partition. At decoder side, respective inverse transformation is performed.

Redundancy in DC coefficients is reduced by residual encoding. Therefore, encoder and decoder, each, comprise a module for predicting DC coefficient. This module uses the indices of the partitioning masks of already encoded, respectively already decoded, image blocks UP, HL, UL, UR as well as the index of the partition mask of the current image block C for

determining a prediction partition. The DC coefficient of the transform of the prediction partition is then used as

prediction for the DC coefficient of the current block' s

transform. At encoder side, residual of the DC coefficient is encoded in the bit stream, and at decoder side, DC coefficient residual is decoded and combined with the prediction.

Based on the indices, the prediction module determines a

further partition of one of the blocks HL, UL, UP, UR which is likely to comprise similar texture as the partition currently to be encoded or decoded. If there is no such further

partition, the prediction is made using an average of DC

coefficients. The partition, from which the averaged DC

coefficients can come, can be comprised in two or more of the already encoded or decoded block UR, UP, UL, HL . Or they are comprised in only one of them.

The relationship of the current image block C and adjacent image blocks HL, UP, UL, and UR is exemplarily depicted in Fig. la and Fig. lb. Exemplarily assumed that encoding of blocks is done from upper left to lower right corner, image blocks

adjacent to the current image block and with already encoded partitions are located in the row atop of a current row

(blocks UL, UP, UR in Fig. lb) comprising the current image block and in said current row (block HL in fig. lb) adjacent to the left of the current image block C. Exemplary constellations of sub-partitions in the current image block C and in adjacent image blocks HL, UP, and UL are depicted in Fig. 8a-d.

For instance, in Fig. 8a, block HL has a horizontal partition and block UP has diagonal right partition wherein current block C has no partition.

DC coefficient of a transform of block C can be predicted using an average of DC coefficients of transforms of the sub^¬ partitions of either block UP or block HL or both blocks.

But, as partitioning is done for separating sub-partitions having sufficiently distinct statistical character due to displaying distinct aspects of texture, DC coefficients of transforms of the sub-partitions are likely to differ

significantly between sub-partitions thus averaging may lead to poor prediction.

Therefore in an exemplary embodiment, DC coefficient of a transform of block C as a whole can be predicted using DC coefficient of transform of a single partition of either block HL or block UP.

An exemplary way for selecting said single partition is based on its "closeness" to or "connectedness" with block C. That is, the single partition selected for prediction is the one having the longest common boundary with current block C. Thus, in the example of Fig. 8a, DC coefficient of partition 1 of block UP is used as prediction. This principle can be also applied to partitions of current block C if current block C is

partitioned also.

For instance, in the exemplary situation depicted in Fig. 8d a predictor for DC coefficient of partition 0 of current image block C is determined using partition 1 of block UP.

Fig. 8d also depicts an exemplary situation where the decision, which partition of adjacent blocks is having the largest

common border or boundary with partition 1 of block C, is ambiguous as both, partition 1 and partition 0 of block HL, do have a common boundary with partition 1 of block C of same length. In this case, determining a prediction by averaging DC coefficients of transforms of partition 1 and partition 0 of block HL is applicable.

In an embodiment, the ways of prediction discussed with respect to Fig. 8d are used if partition pattern is

discontinuous between the current block and the adjacent blocks .

If there is a continuity of partition patterns as exemplarily depicted in Fig. 8b where the partition pattern of block UP is continued in block C, then using corresponding partitions for prediction appears to be appropriable. It is likely that the same texture is present in both blocks thus leading to very similar DC coefficients in corresponding partitions and therefore yielding good predictions.

For determining whether such continuity exists, a predominant direction of the partitioning of the current block is used. Said predominant direction is determined by analysing the texture of the current block C. In an embodiment, there are only a set of indexed partition masks used for partitioning wherein each of the indexed partition mask is associated with a predominant direction although the associated predominant direction can be "none" for an undirected partition mask (e.g. a circular mask) . Then, as soon as a partition mask is

selected based on texture analysis its predominant direction can be determined from a stored index.

If the selected partitioning mask has a predominant direction, texture from a neighbouring block to the current block is analysed as long as the predominant direction is vertical, horizontal, diagonal up or diagonal down. The neighbouring block is selected among already encoded neighbouring blocks such that it is the one located in the image, with respect to current block C, in said predominant direction. If the partition mask of current block C or at least its predominant direction is continuous in the selected

neighbouring block, then DC coefficient of this block can be adopted for prediction. For example, in Fig.8b, block UP and block C are both using vertical partition with 3 subpartitions, then the DC prediction of sub-partition 0 (1,2) in block C should come from the DC values of the sub-partitions 0 (1,2) in block UP, separately. Similarly, if block C has a horizontal pattern, then we should consider first if block HL has a similar horizontal partition. As for a diagonal down partition in block C, the partition mode in block UL can be considered. And for predominantly diagonal left directed partitions, the partitioning mask of block UR can be analysed.

If the above criterion is not met, a more generic DC

prediction scheme is provided.

That is, the DC prediction values of sub-partitions in the current block are independently examined, according to their geometrical location. If a sub-partition of the current block is close to one of the neighboring blocks (or block subpartition), then the prediction candidate from this

neighboring block is used to predict the sub-partition of the current block C. For example in Fig.8c, partition 1 of block C is closer to block HL than to block UP, then the prediction candidate of block HL is used to predict the partition 1 of block C. In this case, the prediction is given by the DC value of partition 1 of block HL .

If it cannot be determined which further partition appears to provide advantages when used for DC coefficient prediction, alone, the DC values of all the sub-partitions in the current block can be predicted the same way. The DC prediction value can come either from block UP, HL, UR or UL. In another exemplary embodiment, intra prediction is used. Below, there is a table of exemplary categorization of different situations according to the mask type of the current block :

The decoder first decodes the mask information for the current block. Then, according to the neighboring blocks' masks information, the DC prediction value for each of the subpartitions in the current block could be acquired through decision operation in Fig.9. Finally, the DC coefficient for one of the current sub-partitions is reconstructed by adding the DC prediction value to the DC coefficient residue (from bitstream) . In Fig. 9, partition direction CPD of current block C is

checked in decision step Dl whether it is predominantly

horizontal and equals partition direction of block HL . If so, prediction PHL is done using corresponding sub-partition of HL.

If not, partition direction CPD of current block C is checked in decision step D2 whether it is predominantly vertical and equals partition direction of block UP. If so, prediction PUP is done using corresponding sub-partition of UP.

If not, partition direction CPD of current block C is checked in decision step D3 whether it is predominantly diagonal down and equals partition direction of block UL . If, so prediction PUL is done using corresponding sub-partition of UL. Similarly, it can be decided whether prediction can be made using block UR if predominant direction is diagonal up.

If not, partition of current block C is checked in decision step D4 whether it is closed to block UP. If so, prediction BUP is done using one or more sub-partitions of block UP.

Either, prediction PUP' is done using a single sub-partition of block UP closed to partition of block C, or, prediction AUP is done using an average of all sub-partitions of block UP. In. decision step D6 it is decided whether there is such single partition based on check of block UP in step CUP.

If not, partition of current block C is checked in decision step D5 whether it is closed to block HL . If so, prediction BHL is done using one or more sub-partitions of block HL.

Either, prediction PHL' is done using a single sub-partition of block UP closed to partition of block C, or, prediction AHL is done using an average of all sub-partitions of block HL . In decision step D7 it is decided whether there is such single partition based on check of block HL in step CHL.

If not, the prediction is made in step ALL using all subpartitions of already encoded neighbouring blocks. Fig. 10 exemplarily depicts, an incoming bit stream INBS which is residue decoded RESD. DC coefficient residue DCCR is determined and combined with DC prediction value DCPV for regaining the DC coefficient COEF.

Prediction value DCPV is determined using the decision

operation DCPD of Fig. 9. This decision operation receives mask information MINB of neighbouring blocks retrieved from memory MEM and mask information MCB of the current block as inputs. Mask information MCB of the current block results from comparing mask information code word MICD separated from bitstream INBS with mask code word table MIT wherein mask information code word MICD is separated from bit stream INBS and mask decoded MDEC.

Claims

Claims :

1. Method for encoding of an image block comprised in an

image, said method comprising

- analysing texture of the image block for selecting a

partitioning mask out of a set of indexed partitioning masks of which at least some are associated with a predominant direction,

- using the selected partitioning mask for selecting a

partition of the image block,

- determining a DC coefficient and AC coefficients by

transforming the selected partition according to a transform scheme adapted to the selected partitioning mask,

- using the selected partitioning mask and the selected

partition for determining a further partition of an image block adjacent to the current image block wherein, at least, a further DC coefficient of the further partition is already encoded,

- using the further DC coefficient of said further partition for determining a prediction, and

- encoding an index of the selected partitioning mask, the AC coefficients and a residual between the DC coefficient and the determined prediction.

2. The method of claim 1, wherein,

- the adjacent image block is determined according to the predominant direction wherein the predominant direction is either horizontal, vertical or diagonal, and

- the further partition is positioned in the adjacent image block at a position corresponding to the position of the selected partition in the image block, wherein the adjacent image block is encoded using a further

prediction mask associated with the same predominant direction, and

- the further DC coefficient is used as said prediction.

3. The method of claim 1, wherein,

- the adjacent block is determined among image blocks

adjacent to the selected partition such that it is the one having the largest common boundary with the selected partition,

- the further partition is determined among partitions of the adjacent image block having a common boundary with the selected partition such that it is the one having the largest common boundary, and

- the further DC coefficient is used as said prediction.

4. The method of claim 1, further comprising,

- averaging the DC coefficient with different further DC coefficients of transforms of different further

partitions comprised in at least one of the adjacent block and different adjacent blocks adjacent to the image block and using the average as said prediction.

5. Method for decoding of an image block, said method

comprising

- decoding a residual, AC coefficients, an indication of a partitioning mask,

- using the partitioning mask indication for selecting a partition mask out of a set of indexed partitioning masks of which at least some are associated with a predominant direction,

- using the partitioning mask for selecting a partition, - using the selected partitioning mask and the selected partition for determining a further partition of an image block adjacent to the current image block wherein, at least, a further DC coefficient of said further partition is already decoded,

- using the further DC coefficient for determining a

prediction,

- using the prediction and the residual for determining a DC coefficient and

- inverse transforming the determined DC coefficient and the decoded AC coefficients for decoding the selected

partition .

6. The method of claim 5, wherein,

- the further partition is positioned in the adjacent image block at a position corresponding to the position of the selected partition in the image block, wherein the adjacent image block is decoded using a further

prediction mask associated with the same predominant direction, and

- the further DC coefficient is used as said prediction.

7. The method of claim 5, wherein,

- the adjacent block is determined among image blocks

- the further partition is determined among partitions of the adjacent image block having a common boundary with the selected partition such that it is the one having the largest common boundary, and - the further DC coefficient is used as said prediction.

8. The method of claim 5, further comprising,

9. Device for encoding of an image block comprised in an

image, said device comprising

- means for analysing texture of the image block for

selecting a partitioning mask out of a set of indexed partitioning masks of which at least some are associated with a predominant direction,

- means for using the selected partitioning mask for

selecting a partition of the image block,

- means for determining a DC coefficient and AC coefficients by transforming the selected partition according to a transform scheme adapted to the selected partitioning mask,

- means for using the selected partitioning mask and the

selected partition for determining a further partition of an image block adjacent to the current image block

wherein, at least, a DC coefficient of the further

partition is already encoded,

- means for using the further DC coefficient of said further partition for determining a prediction, and

- means for encoding an index of the selected partitioning mask, the AC coefficients and a residual between the DC coefficient and the determined prediction.

10. Device for decoding of an image block, said method

comprising - means for decoding a residual, AC coefficients, an indication of a partitioning mask,

- means for using the partitioning mask indication for

selecting a partition mask out of a set of indexed partitioning masks of which at least some are associated with a predominant direction,

- means for using the partitioning mask for selecting a

partition,

- means for using the selected partitioning mask and the selected partition for determining a further partition of an image block adjacent to the current image block wherein, at least, a further DC coefficient of said further partition is already decoded,

- means for using the further DC coefficient for determining a prediction,

- means for using the prediction and the residual for

determining a DC coefficient and

- means for inverse transforming the determined DC

coefficient and the decoded AC coefficients according to a pattern-based transform scheme adapted to the selected partitioning mask for decoding the selected partition.

11. Storage medium carrying a bit stream comprising code of a transform of a partition of a current image block, said storage medium comprising: code of an index of a

partitioning mask, code of AC coefficients and code of a residual between a DC coefficient and a prediction being determined using one or more further partitions of one or more further image blocks which are adjacent to the current image block in the image, wherein the prediction is either a single further DC coefficient from a single further partition or an average of several further DC coefficients from several further partitions comprised in two or more further image blocks and wherein code of the transform of the further partitions used for determining said prediction precedes, in said bit stream, said code of the transform of the partition of current image block