GB2531087A

GB2531087A - Method and device for video coding and decoding

Info

Publication number: GB2531087A
Application number: GB1501515.9A
Authority: GB
Inventors: Laroche Guillaume; Gisquet Christophe; Onno Patrice
Original assignee: Canon Inc
Current assignee: Canon Inc
Priority date: 2014-10-06
Filing date: 2015-01-29
Publication date: 2016-04-13
Also published as: GB201417630D0; GB2533905A; GB201501515D0

Abstract

Encoding a sequence of images comprising performing area-by-area filtering on an image of the sequence, where area-by-area filtering involves selectively applying a filter to different image areas; associating to each image a level corresponding to different image qualities or rate distortion costs; producing a measure of filtering activity in an image based on filtering activity in blocks not used as predictor blocks for an intra block copy mode in which a block is encoded based on an actual block of the current image; and determining other images on which filtering is enabled or disabled in dependence upon the levels and measure. In a second invention the measure is instead based on filtering activity in blocks not encoded using a mode adapted for screen content (such as intra block copy or palette mode encoding). Preferably, the filtering is loop filtering such as sample adaptive offset filtering). The measure may be a ratio of the number of coding elements that require filtering to the number of blocks respectively not used as an IntraBC predictor or not encoded in a screen content mode. If this measure is above a threshold 2210, filtering may be enabled 2211 on all subsequent images having a lower level. Corresponding decoding claims included.

Description

METHOD AND DEVICE FOR VIDEO CODING AND DECODING

The present invention concerns a method and a device for encoding or decoding blocks of pixels in the process of encoding or decoding a video. It concerns more particularly methods to handle parallelization when using INTRA Block Copy mode of HEVC Screen Content extension. It is based on the control of the area available for providing predictor blocks in INTRA Block Copy mode.

It applies more particularly to a mode of coding where a block of pixel is predictively encoded based on a predictor block pertaining to the same image.

This mode of encoding a block of pixel is generally referred to as INTRA Block Copy mode. It is considered as a tool candidate for the Screen content Extension of the High Efficiency Video Coding (HEVC: ISO/IEC 23008-2 MPEG-H Part 2/ ITU-T H.265) international standard and now in the Screen Content extension of the same.

When encoding an image in a video sequence, the image is first divided into coding entities of pixels of equal size referred to as Coding Tree Block (Coding Tree Block). The size of a Coding Tree Block is typically 64 by 64 pixels. Each Coding Tree Block may then be decomposed in a hierarchical tree of smaller blocks which size may vary and which are the actual blocks to encode. These smaller blocks to encode are referred to as Coding Unit (CU).

The encoding of a particular Coding Unit is typically predictive. This means that a predictor block is first determined. Next, the difference between the predictor block and the Coding Unit is calculated. This difference is called the residue. Next, this residue is compressed. The actual encoded information of the Coding Unit is made of some information to indicate the way of determining the predictor block and the compressed residue. Best predictor blocks are blocks as similar as possible to the Coding Unit in order to get a small residue that could be efficiently compressed.

Encoding may be lossy, meaning that information is lost in the encoding process. The decoded block of pixel is not exactly the same as the original Coding Unit. Typically the loss of information comes from a quantization applied to the residue before entropy coding. This quantization allows a higher compression rate at the price of the loss of accuracy. Typically, high frequencies, namely the high level of details, are removed in the block.

Encoding may be lossless, meaning that the residue is not quantized.

This kind of encoding allows retrieving the exact copy of the original samples of the Coding Unit. The lossless encoding is obtained at the expense of compression rate which is much smaller compared to a lossy compression.

The coding mode is defined based on the method used to determine the predictor block for the predictive encoding method of a Coding Unit.

A first coding mode is referred to as INTRJA mode. According to INTRJA mode, the predictor block is built based on the value of pixels immediately surrounding the Coding Unit within the current image. It is worth noting that the predictor block is not a block of the current image but a construction. A direction is used to determine which pixels of the border are actually used to build the predictor block and how they are used. The idea behind INTRA mode is that, due to the general coherence of natural images, the pixels immediately surrounding the Coding Unit are likely to be similar to pixels of the current Coding Unit. Therefore, it is possible to get a good prediction of the value of pixels of the Coding Unit using a predictor block based on these surrounding pixels.

A second coding mode is referred to as INTER mode. According to INTER mode, the predictor block is a block of another image. The idea behind the INTER mode is that successive images in a sequence are generally very similar. The main difference comes typically from a motion between these images due to the scrolling of the camera or due to moving objects in the scene.

The predictor block is determined by a vector giving its location in a reference image relatively to the location of the Coding Unit within the current image. This vector is referred to as a motion vector. According to this mode, the encoding of such Coding Unit using this mode comprises motion information comprising the motion vector and the compressed residue.

We focus in this document on a third coding mode called INTRA Block Copy mode. According to the INTRA Block Copy mode, the block predictor is an actual block of the current image. A block vector is used to locate the predictor block. This block vector gives the location in the current image of the predictor block relatively to the location of the Coding Unit in the same current image. It comes that this block vector shares some similarities with the motion vector of the INTER mode. It is sometime called motion vector by analogy. As there could not be a motion within an image, strictly speaking, and for the sake of clarity, in this document motion vector always refer to the INTER mode while block vector is used for the INTRA Block Copy mode.

The causal principle is the principle that states that all information to decode a particular Coding Unit must be based on already reconstructed Coding Units. At encoding, the whole information may be considered as available. Namely, to encode a given Coding Unit it would be possible to use any information from the entire current images or from all decoded and available other images in the sequence. At decoding, things are different. The decoding of the current images is typically done by decoding sequentially all Coding Unit. The order of decoding follows typically a raster scan order, namely beginning in the upper left of the image, progressing from left to right and from top to bottom. It come that when decoding a given Coding Unit, only the part of the current image located up or left to the current Coding Unit has already been decoded. This is the only available information for the decoding of the current Coding Unit. This has to be taken into account at encoding. For example, a predictor block in INTRA Block Copy mode, should pertain to the part of the image that will be available at decoding.

At decoding, to retrieve a block encoded using INTRA Block Copy mode, first of all, the predictor block is determined using the block vector. Then the residue is decoded and applied to the predictor to obtain a raw reconstructed block.

When the complete image has been reconstructed, some post filtering is applied. Typically a first filter is applied to remove some artefacts in the reconstructed image due to the block encoding. This filter is called the deblocking filter. Typically, while not mandatory, a sample adaptive loop filter (SAO) is then applied to get the final image.

in some decoding architecture, the processing is parallelized in order to speed up the process. In this situation, a particular coding tree block is reconstructed while the previous one is filtered for example. Namely, reconstruction of some coding tree block and filtering of others are made in parallel.

This parallelization has an impact on the INTRA Block Copy mode because predictors used in this mode are raw reconstructed block, meaning that they are non-filtered block. Therefore it may happen that the predictor needed for the decoding of a given bloc belongs to a coding tree block that has already been filtered. It may not be available anymore. Therefore to ensure that the raw reconstructed block remains available for INTRA Block Copy mode, it is necessary to save a version of the raw reconstructed block before applying the filtering in parallelized implementations. This process may prove to be complex and costly.

The present invention has been devised to address one or more of the foregoing concerns.

According to a first aspect of the invention there is provided a method of decoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the method comprising: receiving in the bitstream a delay parameter representing the number of coding elements which may provide a predictor block used to encode a current block, these coding elements being the coding elements preceding the current block in the raster scan order.

In an embodiment, the method further comprises: reconstructing blocks of pixels to obtain raw reconstructed blocks of pixels; post filtering said raw reconstructed blocks of pixels in parallel with the reconstruction; wherein said delay parameter is used to dimension a buffer based on the number of raw reconstructed coding elements to be used in INTRA Block Copy mode.

In an embodiment, the method further comprises reconstructing blocks of pixels to obtain raw reconstructed blocks of pixels; post filtering said raw reconstructed blocks of pixels in parallel with the reconstruction; wherein said delay parameter is used to delay the parallel post filtering until a number of coding elements corresponding to the delay parameter have been reconstructed.

In an embodiment, the coding element is a Coding Tree Block.

In an embodiment, the delay parameter is a power of two.

In an embodiment, the delay parameter is expressed as a number of coding element lines.

According to another aspect of the invention there is provided a method of decoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the method comprising: reconstructing a block of pixels according to INTRA Block Copy mode based on a predictor block belonging to post filtered reconstructed coding elements of the current image.

In an embodiment, all reconstructed area where the Deblocking Filter and Sample Adaptive Offset have been applied is available for INTRA Block Copy mode.

In an embodiment, all reconstructed area, comprising area where the Deblocking Filter and Sample Adaptive Offset have been applied and raw non-filtered reconstructed area, is available for INTRJA Block Copy mode.

In an embodiment, all reconstructed area where the Deblocking Filter and Sample Adaptive Offset have been applied is available for INTRA Block Copy prediction, Deblocking Filter and Sample Adaptive Offset being applied only to the previous fully reconstructed coding elements.

In an embodiment, all reconstructed area where the Deblocking Filter and Sample Adaptive Offset have been applied is available for INTRA Block Copy prediction, Deblocking Filter and Sample Adaptive Offset being applied only to the previous fully reconstructed coding elements except the previous coding element.

In an embodiment, all reconstructed area where the Deblocking Filter and Sample Adaptive Offset have been applied is available for INTRA Block Copy prediction, Deblocking Filter and Sample Adaptive Offset being applied only to the previous fully reconstructed lines of coding elements of maximum size.

According to another aspect of the invention there is provided a method of decoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the method comprising: making a decision for determining for each coding element being reconstructed if post filtering is to be applied to this coding element or not; and wherein only reconstructed coding elements for which post filtering is not to be applied are available for INTRA Block Copy mode.

In an embodiment, it is determined to not apply post filtering to a coding element when at least one block of pixels in this coding element is encoded using INTRA Block Copy mode or Palette mode.

In an embodiment, it is determined to not apply post filtering to a coding element when at least a given proportion of block of pixels in this coding element are encoded using INTRA Block Copy mode or Palette mode.

In an embodiment, it is determined to not apply post filtering to a coding element when at least a given proportion of block of pixels in this coding element are encoded using Palette mode.

In an embodiment, the reconstructed blocks of pixels of the current coding element are further available for INTRA Block Copy mode.

In an embodiment, the reconstructed blocks of pixels of the current and the previous coding elements are further available for INTRA Block Copy mode. a

In an embodiment, the reconstructed blocks of pixels of the current line of coding elements are further available for INTRA Block Copy mode.

In an embodiment, it is determined to not apply post filtering to a coding element when at least one block of pixels in this coding element is encoded using Palette mode or INTRA Block Copy mode with a block vector pointing to another coding element.

In an embodiment, it is determined to not apply post filtering to a coding element when at least one block of pixels in this coding element is encoded using Palette mode or INTRA Block Copy mode with a block vector pointing to another coding element except the previous one.

In an embodiment, it is determined to not apply post filtering to a coding element when at least one block of pixels in this coding element is encoded using Palette mode or INTRA Block Copy mode with a block vector pointing to another coding element line.

In an embodiment, only block of pixels encoded according to Palette mode or INTRA Block Copy mode within reconstructed coding elements for which post filtering is not to be applied are available for INTRA Block Copy mode.

In an embodiment, the decision criterion is applied on the previous coding element to decide if the current coding element needs to be post filtered and if it is available for INTRA Block Copy prediction, the first coding element being not post filtered and being available for INTRA Block Copy mode.

In an embodiment, the decision is made based on a flag in the bitstream.

In an embodiment, a coding element is flagged not to be filtered if it contains a predictor block used for the encoding of a coding unit using INTRA Block Copy mode.

In an embodiment, the reconstructed blocks of pixels of the current and the previous coding elements are further available for INTRA Block Copy mode, the decision is made based on a flag in the bitstream, and a coding element is flagged not to be filtered if it contains a predictor block used for the encoding of a coding unit using INTRA Block Copy mode.

According to another aspect of the invention there is provided a method of decoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the method comprising: determining for each coding element being reconstructed if it is available for INTRA Block Copy prediction or not based on a flag in the bitstream.

In an embodiment, the number of coding element available for INTRA Block Copy prediction is limited to a maximum value.

In an embodiment, the coding elements available for INTRA Block Copy prediction being stored in a dedicated memory, the method further comprises removing the furthest coding element in this dedicated memory when a new coding element flagged as available for INTRA Block Copy prediction is decoded once the maximum value has been reached.

In an embodiment, the maximum value is predetermined.

In an embodiment, the maximum value is received in the bitstream.

According to another aspect of the invention there is provided a method of encoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the method comprising: transmitting in the bitstream a delay parameter representing the number of coding elements which may provide a predictor block used to encode a current block, these coding elements being the coding elements preceding the current block in the raster scan order.

According to another aspect of the invention there is provided a method of encoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRJA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the method comprising: encoding a block of pixels according to INTRA Block Copy mode based on a predictor block belonging to post filtered reconstructed coding elements of the current image.

According to another aspect of the invention there is provided a method of encoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the method comprising: making a decision for determining for each coding element being reconstructed if post filtering is to be applied to this coding element or not; and wherein only reconstructed coding elements for which post filtering is not to be applied are available for INTRA Block Copy mode.

According to another aspect of the invention there is provided a method of encoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the method comprising: determining for each coding element being reconstructed if it is available for INTRA Block Copy prediction; and transmitting this information with a flag in the bitstream.

According to another aspect of the invention there is provided a device for decoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the device comprising: a receiving module for receiving in the bitstream a delay parameter representing the number of coding elements which may provide a predictor block used to encode a current block, these coding elements being the coding elements preceding the current block in the raster scan order.

According to another aspect of the invention there is provided a device for decoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the device comprising: a reconstruction module for reconstructing a block of pixels according to INTRA Block Copy mode based on a predictor block belonging to post filtered reconstructed coding elements of the current image.

According to another aspect of the invention there is provided a device for decoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the device comprising: a decidion module for making a decision for determining for each coding element being reconstructed if post filtering is to be applied to this coding element or not; and wherein only reconstructed coding elements for which post filtering is not to be applied are available for INTRA Block Copy mode.

According to another aspect of the invention there is provided a device for decoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the device comprising: determining for each coding element being reconstructed if it is available for INTRA Block Copy prediction or not based on a flag in the bitstream.

According to another aspect of the invention there is provided a device for encoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the device comprising: a transmission module for transmitting in the bitstream a delay parameter representing the number of coding elements which may provide a predictor block used to encode a current block, these coding elements being the coding elements preceding the current block in the raster scan order.

According to another aspect of the invention there is provided a device for encoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the device comprising: an encoder module for encoding a block of pixels according to INTRA Block Copy mode based on a predictor block belonging to post filtered reconstructed coding elements of the current image.

According to another aspect of the invention there is provided a device for encoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the device comprising: a decision module for making a decision for determining for each coding element being reconstructed if post filtering is to be applied to this coding element or not; and wherein only reconstructed coding elements for which post filtering is not to be applied are available for INTRA Block Copy mode.

According to another aspect of the invention there is provided a device for encoding an image in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being called INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image, the device comprising: a determining module for determining for each coding element being reconstructed if it is available for INTRA Block Copy prediction; and a transmission module for transmitting this information with a flag in the bitstream.

According to another aspect of the invention there is provided a computer program product for a programmable apparatus, the computer program product comprising a sequence of instructions for implementing a method according to the invention, when loaded into and executed by the programmable apparatus.

According to another aspect of the invention there is provided a computer-readable storage medium storing instructions of a computer program for implementing a method according to the invention.

According to another aspect of the invention there is provided a method of encoding an image of a sequence of images in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being an INTRA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image. The method comprises: -associating with the images of the sequence respective levels from a plurality of levels, the levels corresponding to different respective rate-distortion balances or to different image qualities, and for a plurality of coding elements of at least one image of the sequence having a main level N, said level N corresponding to the greater importance of the distortion relative to the importance of cost in the rate-distortion balance or the higher image quality: P1 determining for coding elements of the plurality which are not used as predictor blocks for the INTRA Block Copy mode, whether said coding elements have to be filtered, then for said image having the main level, * determining a ratio between the number of coding elements of the plurality which have to be filtered and the number of coding elements of the plurality which are not used as predictor blocks for the INTRA Block Copy mode, and if said ratio is above a pre-determined threshold then enabling the filtering of a plurality of coding elements of further images of the sequence having a level lower to the main level N. This aspect of the invention offers the possibility to estimate more accurately the SAO statistics for a level of decomposition N in order to take a better decision regarding the SAO filtering of a (the) lower level(s).

In an embodiment, the method is applied to one or more color components of images of the sequence.

In an embodiment, a parameter signals each coding element which is not encoded using the INTRA Block Copy mode.

In an embodiment, the parameter is a flag.

In an embodiment, the provided filtering is the SAO filtering.

In an embodiment, the method further comprising generating in the bitstream a flag related to images, for signaling if the filtering is enabled or not for the images having a level lower than the main level N. In a variant, the method further comprising generating in the bitstream a flag related to slices of images for signaling if the filtering is enabled or not for slices included in images having a level lower than the main level N. According to another aspect of the invention there is provided a method of encoding an image of a sequence of images in a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one or more modes being adapted to coding elements having screen content. The method comprises: -associating with the images of the sequence respective levels from a plurality of levels, the levels corresponding to different respective rate-distortion balances or to different image qualities, and for a plurality of coding elements of at least one image of the sequence having a main level N, said level N corresponding to the greater importance of the distortion relative to the importance of cost in the rate-distortion balance or the higher image quality: determining for the coding elements of the plurality which are not encoded using a mode adapted to coding elements having screen content whether said coding elements have to be filtered, then for said image having the main level determining a ratio between the number of coding elements of the plurality which have to be filtered and the number of coding elements of the plurality which are not encoded using a mode adapted to coding elements having screen content, and if said ratio is above a pre-determined threshold then enabling the filtering of a plurality of coding elements of further images of the sequence having a level lower to the main level.

This aspect of the invention offers the possibility to estimate more accurately the SAO statistics for a level of decomposition N in order to take a better decision regarding the SAO filtering of a (the) lower level(s) when using screen content.

In an embodiment, the method applied to one or more color components of images of the sequence.

In an embodiment, the provided filtering is the SAO filtering.

In an embodiment, the method further comprising generating in the bitstream a flag related to images, for signaling if the filtering is enabled or not for the images having a level lower than the main level N. In an embodiment, the method further comprising generating in the bitstream a flag related to slices of images for signaling if the filtering is enabled or not for slices included in images having a level lower than the main level N. According to another aspect of the invention there is provided a method of encoding a sequence of images, comprising: performing area-by-area loop filtering on at least one image of the sequence and disabling such area-by-area loop filtering on at least one other image of the sequence, such area-by-area loop filtering involving selectively applying loop filtering to different image areas within one image; and associating with the images of the sequence respective levels from a plurality of levels, the levels corresponding to different image qualities or rate-distortion costs; producing a measure of loop filtering activity in one or more images; and determining the images on which such area-by-area loop filtering is disabled in dependence upon the levels and the measure, wherein the measure of loop filtering activity is produced based on loop filtering activity in blocks not using a mode adapted for screen content.

According to another aspect of the invention there is provided a method of encoding a sequence of images, comprising: performing area-by-area loop filtering on at least one image of the sequence and disabling such area-by-area loop filtering on at least one other image of the sequence, such area-by-area loop filtering involving selectively applying loop filtering to different image areas within one image; and associating with the images of the sequence respective levels from a plurality of levels, the levels corresponding to different image qualities or rate-distortion costs; producing a measure of loop filtering activity in one or more images; and determining the images on which such area-by-area loop filtering is disabled in dependence upon the levels and the measure, wherein the measure of loop filtering activity is produced based on loop filtering activity in blocks not used as predictor blocks for a mode in which the block is encoded based on a predictor block being an actual block of the current image.

According to another aspect of the invention there is provided a method of decoding an image of a sequence of images from a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one mode being an INTRJA Block Copy mode in which the block is encoded based on a predictor block being an actual block of the current image. The method comprises: -obtaining for the images of the sequence respective levels from a plurality of levels, the levels corresponding to different respective rate-distortion balances or to different image qualities, and for a plurality of coding elements of at least one image of the sequence having a main level N, said level N corresponding to the greater importance of the distortion relative to the importance of cost in the rate-distortion balance or the higher image quality: determining for coding elements of the plurality which are not used as predictor blocks for the INTRA Block Copy mode, whether said coding elements have to be filtered, then for said image having the main level, determining a ratio between the number of coding elements of the plurality which have to be filtered and the number of coding elements of the plurality which are not used as predictor blocks for the INTRA Block Copy mode, and if said ratio is above a pre-determined threshold then enabling the filtering of a plurality of coding elements of further images of the sequence having a level lower to the main level N. This aspect of the invention is particularly efficient when the corresponding encoding method send flags at slice level which may be very costly. Another way to proceed is to implement a decoding method according to this aspect of the invention.

In an embodiment, the parameter is a flag.

In an embodiment, the provided filtering is the SAO filtering.

According to another aspect of the invention there is provided a method of decoding an image of a sequence of images from a bitstream, the image comprising a plurality of coding elements made of blocks of pixels, each block of pixels being encoded according to a mode out of a plurality of modes, one or more modes being adapted to coding elements having screen content. The method comprises: -obtaining for the images of the sequence respective levels from a plurality of levels, the levels corresponding to different respective rate-distortion balances or to different image qualities, and for a plurality of coding elements of at least one image of the sequence having a main level N, said level N corresponding to the greater importance of the distortion relative to the importance of cost in the rate-distortion balance or the higher image quality: H determining for the coding elements of the plurality which are not encoded using a mode adapted to coding elements having screen content whether said coding elements have to be filtered, then for said image having the main level * determining a ratio between the number of coding elements of the plurality which have to be filtered and the number of coding elements of the plurality which are not encoded using a mode adapted to coding elements having screen content, and if said ratio is above a pre-determined threshold then enabling the filtering of a plurality of coding elements of further images of the sequence having a level lower to the main level.

This aspect of the invention is particularly efficient when the corresponding encoding method send flags at slice level which may be very costly. Another way to proceed is to implement a decoding method according to this aspect of the invention.

In an embodiment, the provided filtering is the SAO filtering.

In an embodiment, the levels of the images of the images of the sequence are obtained from the quantization parameters of the images.

According to another aspect of the invention there is provided a method of decoding a sequence of images, comprising: performing area-by-area loop filtering on at least one image of the sequence and disabling such area-by-area loop filtering on at least one other image of the sequence, such area-by-area loop filtering involving selectively applying loop filtering to different image areas within one image; wherein the images on which area-by-area loop filtering is disabled are determined in dependence upon respective levels associated with the images of the sequence, the levels corresponding to different image qualities, and upon a measure of loop filtering activity in one or more images, and wherein the measure of loop filtering activity is based on loop filtering activity in blocks not using a mode adapted for screen content.

According to another aspect of the invention there is provided a method of decoding a sequence of images, comprising: performing area-by-area loop filtering on at least one image of the sequence and disabling such area-by-area loop filtering on at least one other image of the sequence, such area-by-area loop filtering involving selectively applying loop filtering to different image areas within one image; wherein the images on which area-by-area loop filtering is disabled are determined in dependence upon respective levels associated with the images of the sequence, the levels corresponding to different image qualities, and upon a measure of loop filtering activity in one or more images, and wherein the measure of loop filtering activity is based on loop filtering activity in blocks not used as predictor blocks for a mode in which the block is encoded based on a predictor block being an actual block of the current image According to another aspect of the invention there is provided a device for encoding an image of a sequence of images in a bitstream, the device being configured to implement a method according to one of the previous aspects.

According to another aspect of the invention there is provided a device for decoding a sequence of images from a bitstream, the device being configured to implement a method according to one of the previous aspects.

According to another aspect of the invention there is provided a computer program product for a programmable apparatus, the computer program product comprising a sequence of instructions for implementing a method according to any one of claims 1 to 25, when loaded into and executed by the programmable apparatus.

According to another aspect of the invention there is provided a computer-readable storage medium storing instructions of a computer program for implementing a method according to one of the previous aspects.

According to another aspect of the invention there is provided a bitstream encoded according to the method according to one of the previous aspects.

According to another aspect of the invention there is provided a use, in a decoder, of a bitstream as mentioned above to determine in which images in a sequence of images area-by-area loop filtering is disabled.

At least parts of the methods according to the invention may be computer implemented. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system". Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

Since the present invention can be implemented in software, the present invention can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RF signal.

Embodiments of the invention will now be described, by way of example only, and with reference to the following drawings in which: Figure 1 illustrates the HEVC encoder architecture; Figure 2 illustrates the HEVC decoder architecture; Figure 3 illustrates the Level decomposition of Video frame; Figure 4a and 4b illustrates the loop filtering process of HEVC; Figure 5a and 5b illustrates the sample adaptive Edge offset classification of HEVC; Figure 6 illustrates the sample adaptive band offset classification of HEVC; Figure 7 illustrates the Sample Adaptive Offset parameters representation for the Coding Tree Block mode; Figure 8 illustrates the Sample Adaptive Offset parameters syntax reading; Figure 9 illustrates the Sample Adaptive Offset decoding process; Figure ba and lOb illustrates the concept of decoding process parallelization; Figure 11 illustrates the possible loop filtered pixel and non-loop filtered pixels for one Coding Tree Block; Figure 12 illustrates the concept of the causal area; Figure 13 illustrates the INTRA Block Copy search area; Figure 14 illustrates the available data for INTRA Block Copy in one embodiment of the invention; Figure 15 illustrates the available data for INTRA Block Copy in one embodiment of the invention; Figure 16 illustrates the available data for INTRA Block Copy in one embodiment of the invention; Figure 17 illustrates the selection process of available Coding Tree Block for INTRA Block Copy prediction at encoder and decoder for one embodiment of the invention; Figure 18 illustrates the syntax modification at decoder for one embodiment of the invention; Figure 19 illustrates an encoding selection algorithm for one embodiment of the invention; Figure 20 is a schematic block diagram of a computing device for implementation of one or more embodiments of the invention; Figure 21 illustrates a Group of Pictures (GoP) and its images associated to a level; and Figure 22 illustrates an encoding selection algorithm for one embodiment of the invention.

Figure 1 illustrates the HEVC encoder architecture. In the video encoder, an original sequence 101 is divided into blocks of pixels 102. A coding mode is then affected to each block. There are two families of coding modes typically used in HEVC: the modes based on spatial prediction or INTRA modes 103 and the modes based on temporal prediction or INTER modes based on motion estimation 104 and motion compensation 105. An INTRA Coding Unit is generally predicted from the encoded pixels at its causal boundary by a process called INTRA prediction.

Temporal prediction first consists in finding in a previous or future frame called the reference frame 116 the reference area which is the closest to the Coding Unit in a motion estimation step 104. This reference area constitutes the predictor block. Next this Coding Unit is predicted using the predictor block to compute the residue in a motion compensation step 105.

In both cases, spatial and temporal prediction, a residual is computed by subtracting the Coding Unit from the original predictor block.

In the INTRA prediction, a prediction direction is encoded. In the temporal prediction, at least one motion vector is encoded. However, in order to further reduce the bitrate cost related to motion vector encoding, a motion vector is not directly encoded. Indeed, assuming that motion is homogeneous, it is particularly interesting to encode a motion vector as a difference between this motion vector, and a motion vector in its surrounding. In H.264/AVC coding standard for instance, motion vectors are encoded with respect to a median vector computed between 3 blocks located above and on the left of the current block. Only a difference, also called residual motion vector, computed between the median vector and the current block motion vector is encoded in the bitstream. This is processed in module "Mv prediction and coding" 117. The value of each encoded vector is stored in the motion vector field 118. The neighboring motion vectors, used for the prediction, are extracted from the

motion vector field 118.

Then, the mode optimizing the rate distortion performance is selected in module 106. In order to further reduce the redundancies, a transform, typically a DCI, is applied to the residual block in module 107, and a quantization is applied to the coefficients in module 108. The quantized block of coefficients is then entropy coded in module 109 and the result is inserted in the bitstream 110.

The encoder then performs a decoding of the encoded frame for the future motion estimation in modules 111 to 116. These steps allow the encoder and the decoder to have the same reference frames. To reconstruct the coded frame, the residual is inverse quantized in module 111 and inverse transformed in module 112 in order to provide the "reconstructed" residual in the pixel domain. According to the encoding mode (INTER or INTRA), this residual is added to the INTER predictor 114 or to the INTRA predictor 113.

Then, this first reconstruction is filtered in module 115 by one or several kinds of post filtering. These post filters are integrated in the encoded and decoded loop. It means that they need to be applied on the reconstructed frame at encoder and decoder side in order to use the same reference frame at encoder and decoder side. The aim of this post filtering is to remove compression artifacts.

In Figure 2, have been represented the principle of a decoder. The video stream 201 is first entropy decoded in a module 202. The residual data are then inverse quantized in a module 203 and inverse transformed in a module 204 to obtain pixel values. The mode data are also entropy decoded in function of the mode, an INTRA type decoding or an INTER type decoding is performed. In the case of INTRA mode, an INTRA predictor is determined in function of the INTRA prediction mode specified in the bitstream 205. If the mode is INTER, the motion information is extracted from the bitstream 202. This is composed of the reference frame index and the motion vector residual. The motion vector predictor is added to the motion vector residual to obtain the motion vector 210.

The motion vector is then used to locate the reference area in the reference frame 206. Note that the motion vector field data 211 is updated with the decoded motion vector in order to be used for the prediction of the next decoded motion vectors. This first reconstruction of the decoded frame is then post filtered 207 with exactly the same post filter as used at encoder side. The output of the decoder is the de-com pressed video 209.

This INTRA Block Copy coding mode is particularly well suited for extremely repetitive patterns. In particular, it is known to help coding graphical elements such as glyphs, the graphical representation of a character, or traditional GUI elements, which are very difficult to code using traditional INTRA prediction methods.

It is worth noting that prediction is based on coherence between neighbor Coding Units. This coherence may be geographic when considered within the current frame or temporal when considered across successive frames. This kind of coherence occurs in natural images. As INTRA Block Copy encoding mode is seen as a mode dedicated to text or symbolic images, predication is thought as useless for this kind of image. For instance, there is no reason to have two successive Coding Units in an image representing a text having good predictors close to each other. The first Coding Unit may be the part of letter A", a good predictor block would therefore come from another "A" in the text. While the next Coding Unit would be a "P" letter having a predictor block from another F" in the text. There is no reason, a-priori, to have the two predictor blocks in the same neighborhood. This is why prior art does not contemplate introducing prediction in INTRA Block Copy encoding mode.

The current design of HEVC uses 3 different INTER modes: the Inter mode, the Merge mode and the Merge Skip mode. The main difference between these modes is the data signaling in the bitstream. For the Motion vector coding, the current HEVC standard includes a competitive based scheme for Motion vector prediction compared to its predecessors. It means that several candidates are competing with the rate distortion criterion at encoder side in order to find the best motion vector predictor or the best motion information for respectively the Inter or the Merge mode. An index corresponding to the best predictors or the best candidate of the motion information is inserted in the bitstream. The decoder can derive the same set of predictors or candidates and uses the best one according to the decoded index.

The design of the derivation of predictors and candidate is very important to achieve the best coding efficiency without large impact on complexity. In HEVC two motion vector derivations are used: one for Inter mode (Advanced Motion Vector Prediction (AMVP)) and one for Merge modes (Merge derivation process). The following describes these processes.

In HEVC, it is possible to transmit specific NAL Units, called SEI messages of different types. SEI messages contain information related to the display process, and are therefore optional.

Figure 3 shows the coding structure used in HEVC. According to HEVC and one of its previous predecessors, the original video sequence 301 is a succession of digital images "images i'. As is known per se, a digital image is represented by one or more matrices the coefficients of which represent pixels.

The images 302 are divided into slices 303. A slice is a part of the image or the entire image. In HEVC these slices are divided into non-overlapping Coding Tree Blocks (CTB) 304, generally blocks of size 64 pixels x 64 pixels.

Each Coding Tree Block may in its turn be iteratively divided into smaller variable size Coding Units (CU5) 305 using quadtree decomposition. Coding units are the elementary coding elements and are constituted of two sub units which Prediction Unit (PU) and Transform Units (TU) of maximum size equal to the Coding Unit's size. Prediction Unit corresponds to the partition of the Coding Unit for prediction of pixels values. Each Coding Unit can be further partitioned into a maximum of 4 square Partition Units or 2 rectangular Partition Units 306.

Transform units are used to represent the elementary units that are spatially transform with DCT. A Coding Unit can be partitioned in TU based on a quadtree representation 307.

Each slice is embedded in one NAL unit. In addition, the coding parameters of the video sequence are stored in dedicated NAL units called parameter sets. In HEVC and H.264/AVC two kinds of parameter sets NAL units are employed: first, the Sequence Parameter Set (SPS) NAL unit that gathers all parameters that are unchanged during the whole video sequence.

Typically, it handles the coding profile, the size of the video frames and other parameters. Secondly, Picture Parameter Sets (PPS) codes the different values that may change from one frame to another. HEVC include also Video Parameter Set (VPS) which contains parameters describing the overall structure of the stream.

Figure 4a shows the two cascaded post filtering processes: the deblocking filter and the sample adaptive loop filter (SAO). The input signal of these post filtering is the reconstructed signal Y/Cb/Cr rec obtained after Intra or Inter reconstruction. This signal contains the Luma and Chroma signals and they are independently post filtered. First the deblocking filter is applied. The deblocked pixels Y/Cb/CrDF are then filtered by the Sample Adaptive Offset processing in order to obtain the final reconstructed signal Y/Cb/Cr,j,a, . This filtered frame is the depicted frame. Moreover, this frame is used as the reference frame for the Inter prediction of the following frame.

The aim of the deblocking filter is to visually remove the block artifacts due to quantization and block processing.

In HEVC standard and its extensions, the deblocking filter is applied on 8x8 pixels grid. The length of the filtered border can reach 4 samples. For each Coding Unit the vertical edges filter is applied before the horizontal edges filter.

The deblocking is applied according to several rules. For example if the boundary is a PU or a TU boundary, if the Boundary Strength (BS) is strictly superior to 0, or if the signal variation across border is limited.

Figure 4b shows this process. First the decision on the boundary on TV or Coding Unit is selected in step 401, then the boundary strength is computed in step 402 followed by the deblocking filter parameters in step 403. According to these parameters, it is decided in step 404 to apply or not the deblocking filter and if it is applied, the process selects in step 405 between a strong, step 406, or a weak, step 407, filter.

The HEVC specification defines the deblocking filter on a picture basis, yet the deblocking filter can be implemented on a coding unit basis in order to parallelize coding unit reconstruction and deblocking filter. The neighboring pixels need to be reconstructed to run the deblocking filter, eg. the 4 left pixels rows and the 4 right pixels rows (outside the current block) need to be reconstructed for vertical filtering and the 4 above lines and the 4 bottom lines (outside the current block) need to be reconstructed for horizontal filtering.

Details of the conventional Sample Adaptive Offset filtering are now given with reference to Figures 5a to 9.

The aim of Sample Adaptive Offset (SAO) filtering is to improve the quality of the reconstructed frame by sending additional data in the bitstream in contrast to the deblocking filter where no information is transmitted.

In a conventional Sample Adaptive Offset filter, each pixel is classified into a predetermined class or group and the same offset value is added to every pixel sample of the same class/group. One offset is encoded in the bitstream for each class. Sample Adaptive Offset loop filtering has two Sample Adaptive Offset types for a Coding Tree Block (CTB): the Edge Offset type and the Band Offset type. An example of Edge Offset type is schematically illustrated in Figure 5a and 5b, and an example of Band Offset type is schematically illustrated in Figure 6.

Sample Adaptive Offset filtering is applied Coding Tree Block by Coding Tree Block. In this case the parameters are selected for each Coding Tree Block at the encoder side and parameters are decoded and/or derived for each Coding Tree Block at the decoder side. This offers the possibility of easily encoding and decoding the video sequence by processing each Coding Tree Block at once without introducing delays in the processing of the whole frame.

Moreover, when Sample Adaptive Offset filtering is enabled, only one Sample Adaptive Offset type is used: either the Edge Offset type filter or the Band Offset type filter according to the related parameters transmitted in the bitstream for each classification. These parameters can be copied from the upper and left Coding Tree Block, for example, instead of transmitting all the Sample Adaptive Offset data.

Sample Adaptive Offset filtering may be applied independently for different colour components (e.g. YUV) of the frame. For example, one set of Sample Adaptive Offset parameters may be provided for the luma component Y and another set of Sample Adaptive Offset parameters may be provided for both chroma components Ii and V in common.

A description of the Edge Offset type is now provided with reference to Figure 5a and 5b.

Edge Offset type involves determining an edge index for each pixel by comparing its pixel value to the values of two neighboring pixels. Moreover, these two neighboring pixels depend on a parameter that indicates the direction of these two neighboring pixels with respect to the current pixel. These directions are the 0-degree (horizontal direction), 45-degree (diagonal direction), 90-degree (vertical direction) and 135-degree (second diagonal direction). These four directions are schematically illustrated in Figure 5a.

The table of Figure 5b gives the offset value to be applied to the pixel value of a particular pixel "C" according to the value of the two neighboring pixels Cnl and Cn2 at the decoder side.

When the value of C is less than the two values of neighboring pixels Cnl and Cn2, the offset to be added to the pixel value of the pixel C is "--01".

When the pixel value of C is less than one pixel value of its neighboring pixels (either Cnl or Cn2) and C is equal to one value of its neighbors, the offset to be added to this pixel sample value is "+ 02".

When the pixel value of c is less than one of the pixel values of its neighbors (Cnl or Cn2) and the pixel value of C is equal to one value of its neighbors, the offset to be applied to this pixel sample is 03". When the value of C is greater than the two values of Cnl or Cn2, the offset to be applied to this pixel sample is 04".

When none of the above conditions is met on the current sample and its neighbors, no offset value is added to the current pixel C as depicted by the Edge Index value 2" of the table.

It is important to note that for the particular case of the Edge Offset type, the absolute value of each offset (01, 02, 03, 04) is encoded in the bitstream.

The sign to be applied to each offset depends on the edge index (or the Edge Index in the HEVC specifications) to which the current pixel belongs. According to the table represented in Figure 5b, for Edge Index 0 and for Edge Index 1 (01, 02) a positive offset is applied. For Edge Index 3 and Edge Index 4 (03, 04), a negative offset is applied to the current pixel.

In the HEVC specifications, the direction for the Edge Offset amongst the four directions of Figure 5a is specified in the bitstream by a "sao_eo_class_luma" field for the luma component and a "sao_eo_class_chroma" field for both chroma components U and V. The Sample Adaptive Offset Edge Index corresponding to the index value is obtained by the following formula: Edgelndex = sign (C -Cn2) -sign (Cnl-C) +2 where the definition of the function sign) is given by the following relationships * sign(x) = 1, when x>0 * sign(x) = -1, when xcO * sign(x) = 0, when x0.

In order to simplify the Edge Offset determination for each pixel, the difference between the pixel value of C and the pixel value of both its neighboring pixels Cnl and Cn2 can be shared for current pixel C and its neighbors. Indeed, when Sample Adaptive Offset Edge Offset filtering is applied using a raster scan order of pixels of the current Coding Tree Block or frame, the term sign (Cnl-C) has already computed for the previous pixels (to be precise it was computed as C'-Cn2' at a time when the current pixel C' at that time was the present neighboring pixel Cnl and the neighboring pixel Cn2' was what is now the current pixel C). As a consequence this sign (cr1-c) does not need to be computed again.

A description of the Band Offset type is now provided with reference to Figure 6.

Band Offset type in Sample Adaptive Offset also depends on the pixel value of the sample to be processed. A class in Sample Adaptive Offset Band offset is defined as a range of pixel values. Conventionally, for all pixels within a range, the same offset is added to the pixel value. In the HEVC specifications, the number of offsets for the Band Offset filter is four for each reconstructed block or frame area of pixels (Coding Tree Block), as schematically illustrated in Figure 6.

One implementation of Sample Adaptive Offset Band offset splits the full range of pixel values into 32 ranges of the same size. These 32 ranges are the classes of Sample Adaptive Offset Band offset. The minimum value of the range of pixel values is systematically 0 and the maximum value depends on the bit depth of the pixel values according to the following relationship Max = 2utd1dhhht_1. Classifying the pixels into 32 ranges of the full interval includes 5 bits checking needed to classify the pixels values for fast implementation i.e. only the 5 first bits (5 most significant bits) are checked to classify a pixel into one of the 32 classes! ranges of the full range.

For example, when the bitdepth is 8 bits per pixel, the maximum value of a pixel can be 255. Hence, the range of pixel values is between 0 and 255. For this bitdepth of 8 bits, each class contains 8 pixel values.

In conventional Band Offset type filtering, the start of the band, represented by the grey area (60), that contains four ranges or classes, is signaled in the bitstream to identify the position of the first class of pixels or the first range of pixel values. The syntax element representative of this position is the "sao_band_position" field in the HEVC specifications. This corresponds to the start of class 61 in Figure 6. According to the HEVC specifications, 4 consecutive classes (61, 62, 63 and 64) of pixel values are used and 4 corresponding offsets are signaled in the bitstream.

Figure 7 is a flow chart illustrating steps of a process for decoding Sample Adaptive Offset parameters according to the HEVC specifications. The process of Figure 7 is applied for each Coding Tree Block to generate a set of Sample Adaptive Offset parameters for the Y component and another set of Sample Adaptive Offset parameters common to the U and V components. In order to avoid encoding one set of Sample Adaptive Offset parameters per Coding Tree Block (which is very costly), a predictive scheme is used for the Coding Tree Block mode. This predictive mode involves checking if the Coding Tree Block on the left of the current Coding Tree Block uses the same Sample Adaptive Offset parameters (this is specified in the bitstream through a flag named "sao_nierge_left_flag"). If not, a second check is performed with the Coding Tree Block above the current Coding Tree Block (this is specified in the bitstream through a flag named "sao_merge_up_flag"). This predictive technique enables the amount of data representing the Sample Adaptive Offset parameters for the Coding Tree Block mode to be reduced. Steps of the process are set out below.

In step 701, the process starts by selecting the colour component of the video sequence. In the current version of HEVC, Sample Adaptive Offset parameters are provided for the luma component Y and for both U and V components (together). In the example of a YUV sequence the process starts with the Y component. In step 703, the "sao_merge_left_flag" is read from the bitstream 702 and decoded. If its value is true, then the process proceeds to step 704 where the Sample Adaptive Offset parameters of left Coding Tree Block are copied for the current Coding Tree Block. This enables the type of the Sample Adaptive Offset filter for the current Coding Tree Block to be determined in step 708.

If the outcome is negative in step 703 then the "sao_merge_up_flag" is read from the bitstream and decoded. If its value is true, then the process proceeds to step 705 where the Sample Adaptive Offset parameters of the above Coding Tree Block are copied for the current Coding Tree Block. This enables the type of the Sample Adaptive Offset filter for the current Coding Tree Block to be determined in step 708.

If the outcome is negative in step 705, then the Sample Adaptive Offset parameters for the current Coding Tree Block are read and decoded from the bitstream in step 707. The details of this step are described later with reference to Figure 8. After this step, the parameters are obtained and the type of Sample Adaptive Offset filter is determined in step 708.

In subsequent step 709 a check is performed to determine if the three colour components (Y and U & V) for the current Coding Tree Block have been processed. If the outcome is positive, the determination of the Sample Adaptive Offset parameters for the three components is complete and the next Coding Tree Block can be processed in step 710. Otherwise, (Only Y was processed) U and V are processed together and the process restarts from initial step 701 previously described.

Figure 8 is a flow chart illustrating steps of a process of parsing of Sample Adaptive Offset parameters in the bitstream 801 at the decoder side. In initial step 802, the "sao_type_idx_X" syntax element is read and decoded. The code word representing this syntax element can use a fixed length code or could use any method of arithmetic coding. The syntax element sao_type_idx_X" enables determination of the type of Sample Adaptive Offset applied for the frame area to be processed for the colour component Y or for both Chroma components U & V. For example, for a YUV 4:2:0 sequence, two components are considered: one for Y, and one for U and V. The "sao_type_idx_X" can take 3 values as follows depending on the Sample Adaptive Offset type encoded in the bitstream. 0' corresponds to no Sample Adaptive Offset, 1' corresponds to the Band Offset case illustrated in Figure 6 and 2' corresponds to the Edge Offset type filter illustrated in Figure 5a and 5b.

In the same step 802, a test is performed to determine if the "sao_type_idx_X" is strictly positive. If "saotypeidxX" is equal to CQ signifying that there is no Sample Adaptive Offset for this frame area (Coding Tree Block) for Y if X is set equal to Y and that there is no Sample Adaptive Offset for this frame area for U and V if X is set equal to U and V. The determination of the Sample Adaptive Offset parameters is complete and the process proceeds to step 808. Otherwise if the "sao_type_idx" is strictly positive, this signifies that Sample Adaptive Offset parameters exist for this Coding Tree Block in the bitstream.

Then the process proceeds to step 806 where a loop is performed for four iterations. The four iterations are carried in step 807 where the absolute value of offset j is read and decoded from the bitstream. These four offsets correspond either to the four absolute values of the offsets (01, 02, 03, 04) of the four Edge indexes of Sample Adaptive Offset Edge Offset (see Figure 5b) or to the four absolute values of the offsets related to the four ranges of the Sample Adaptive Offset band Offset (see Figure 6).

Note that for the coding of a Sample Adaptive Offset offset, a first part is transmitted in the bitstream corresponding to the absolute value of the offset.

This absolute value is coded with a unary code. The maximum value for an absolute value is given by the following formula: MAX_abs_SAO_offset_value = (1 cc (M in(bitoepth, 1 0)-5))-1 where << is the left (bit) shift operator.

This formula means that the maximum absolute value of an offset is 7 for a pixel value bitdepth of 8 bits, and 31 for a pixel value bitdepth of 10 bits and beyond.

The current HEVC standard amendment addressing extended bitdepth video sequences provides similar formula for a pixel value having a bitdepth of 12 bits and beyond. The absolute value decoded may be a quantized value which is dequantized before it is applied to pixel values at the decoder for Sample Adaptive Offset filtering. An indication of use or not of this quantification is transmitted in the slice header.

For Edge Offset type, only the absolute value is transmitted because the sign can be inferred as explained previously.

For Band Offset type, the sign is signaled in the bitstream as a second part of the offset if the absolute value of the offset is not equal to 0. The bit of the sign is bypassed when CABAC is used.

After step 807, the process proceeds to step 803 where a test is performed to determine if the type of Sample Adaptive Offset corresponds to the Band Offset type (sao_type_idx_X == 1).

If the outcome is positive, the signs of the offsets for the Band Offset mode are decoded in steps 809 and 810, except for each offset that has a zero value, before the following step 804 is performed in order to read in the bitstream and to decode the position "sao_band_position_X" of the Sample Adaptive Offset band as illustrated in Figure 6.

If the outcome is negative in step 803 ("sao_type_idx_X" is set equal to 2), this signifies that the Edge Offset type is used. Consequently, the Edge Offset class (corresponding to the direction 0, 45, 90 and 135 degrees) is extracted from the bitstream 801 in step 805. If X is equal to Y, the read syntax element is "sao_eo_class_luma" and if X is set equal to U and V, the read syntax element is "sao_eo_class_chroma".

When the four offsets have been decoded, the reading of the Sample Adaptive Offset parameters is complete and the process proceeds to step 808.

Figure 9 is a flow chart illustrating a decoding process of the conventional Sample Adaptive Offset according the HEVC specifications, for example during step 207. This decoding process is also applied in the decoding loop (step 115) at the encoder in order to produce the reference frames used for the motion estimation and compensation of the following frames. This process is related to the Sample Adaptive Offset filtering for one colour component (thus suffix "_X" in the syntax elements has been omitted below).

An initial step 901 comprises determining the Sample Adaptive Offset filtering parameters according to processes depicted in Figures 5a, 5b and 6.

Step 901 gives the sao_type_idx and if it equals 1 the sao_band_position 902 and if it equals 2 the sao_eo_class_luma or sao_eo_class_chroma (according to the colour component processed). It may be noted that if the element sao_type_idx is equal to 0 the Sample Adaptive Offset filtering is not applied.

Step 901 gives also the offsets table of the 4 offsets 903.

A variable i, used to successively consider each pixel Pi of the current block or frame area (Coding Tree Block), is set to 0 in step 904. In step 906, pixel I is extracted from the frame area 905 (the current Coding Tree Block in the HEVC standard) which contains N pixels. This pixel I',? is classified in step 907 according to the Edge offset classification or Band offset classification as described respectively in Figure 5a and 5b and Figure 6 or in accordance with embodiments of the invention to be described later. The decision module 908 tests if 1 is in a class that is to be filtered using the conventional Sample Adaptive Offset filtering.

If J is in a filtered class, the related class number] is identified and the related offset value Of/sc!1 is extracted in step 910 from the offsets table 903. In the case of the conventional Sample Adaptive Offset filtering this Offtce, is then added to the pixel value I in step 911 in order to produce the filtered pixel value I' 912. This filtered pixel] is inserted in step 913 into the filtered frame area 916. In embodiments of the invention, steps 910 and 911 are carried out differently, as will be explained later in the description of those embodiments.

If I is not in a class to be Sample Adaptive Offset filtered then l (909) is inserted in step 913 into the filtered frame area 916 without filtering.

After step 913, the variable i is incremented in step 914 in order to filter the subsequent pixels of the current frame area 905 (if any -test 915). After all the pixels have been processed (i>N) in step 915, the filtered frame area 916 is reconstructed and can be added to the Sample Adaptive Offset reconstructed frame (see frame 208 of Figure 2 or 116 of Figure 1).

For real time or fast implementation, it is often needed to parallelize some encoding and decoding processes. The HEVC standard offers some high level of parallelism as wavefront or tiles or slices for frame parallelism and flexible reference frame management for Inter parallelism but these tools are not mandatory. These tools are more dedicated to fast encoding process. Yet, for decoder (and encoder) implementation it is possible to parallelize some processes.

For a sequential implementation, as depicted in Figure ba, the syntax elements of the block are parsed and then the block is reconstructed. When the reconstruction of this block is ended, the syntax element of the following block are parsed etc...

When the parsing of the syntax is processed in parallel to the block reconstruction (cf. Figure lOb): the decoder can parse the bitstream in order to entropy decode the syntax elements needed for a block reconstruction. These syntax elements are stored for the reconstruction of the corresponding block and the parsing of the following syntax elements for the following blocks can be proceeded. The decoder can reconstruct in parallel the block according to these syntax elements stored. As a consequence, the decoding run time is reduced and the need for buffering minimal. In this example, the delay, (if the parsing take less time that the block reconstruction) is one block.

The loop filtering processes (Deblocking Filter and Sample Adaptive Offset) can be also parallelized. Indeed, as mentioned in the previous sections, even if these processes are defined at picture level in the HEVC specifications, the Deblocking Filter can be proceed at Coding Unit level and the Sample Adaptive Offset at Coding Tree Block level. As a consequence, the loop filtering can be performed in parallel to the block reconstruction with one Coding Tree Block of delay between the block reconstruction and the loop filtering. Yet some data are not available to process all Deblocking Filter and Sample Adaptive Offset on a reconstructed Coding Tree Block. Indeed, as depicted in Figure 11, the 4 pixels on the right row and on the bottom cannot be processed due to the non-availability of their neighboring reconstructed pixels.

The loop filtering can be parallelized to the block reconstruction at higher level. For example, the loop filtering can be performed when 2 lines of Coding Tree Block have been reconstructed. In that case the delay is 2 lines of Coding Tree Blocks. If either Sample Adaptive Offset or Deblocking Filter is deactivated, there is only a need for 2 lines, and thus the delay is 1 line.

The Screen Content Coding extension of HEVC under definition contains additional tools to efficiently code screen coding sequences. The current added tools are the Intra block copy mode, the Palette mode and the residual colour transform. The current invention is dedicated to the Intra block copy mode only, so only this mode is described in the following.

Yet, please note that the Palette mode and the INTRA Block Copy mode are new Intra modes and consequently added to the modules 103 and 205 of respectively Figure 1 and 2.

The Intra Block Copy (IBC) was added as an additional mode for Screen content coding extension of HEVC. This prediction method is particularly well suited for extremely repetitive patterns. In particular, it is known to help coding graphical elements such as glyphs (i.e., the graphical representation of a character) or traditional GUI elements, which are very difficult to code using traditional intra prediction methods.

Figure 12 illustrates how this Intra Block Copy prediction mode works.

At a high-level, an image is divided into Coding Units that are encoded in raster scan order. Thus, when coding block 1201, all the blocks of area 1203 have already been encoded/decoded, and can be considered available to the encoder/decoder. Area 1203 is called the causal area of the Coding Unit 1201.

Once Coding Unit 1201 is encoded/decoded, it will belong to the causal area for the next Coding Unit. This next Coding Unit, as well as all the next ones, belongs to area 1204 illustrated as doted area, and cannot be used for coding the current Coding Unit 1201. It is worth noting that the causal area is constituted by raw reconstructed blocks. The information used to encode a given Coding Unit is not the original blocks of the image for the reason that this information is not available at decoding. The only information available at decoding is the reconstructed version of the blocks of pixels in the causal area, namely the decoded version of these blocks. For this reason, at encoding, previously encoded blocks of the causal area are decoded to provide this reconstructed version of these blocks.

INTRA Block Copy works by signaling a block 1202 in the causal area which should be used to produce a prediction of block 1201. For example, the block 1202 may be found by using a matching algorithm. In the HEVC Screen content Extension, this block is indicated by a block vector 1205, and the residual of this vector according to a predictor is transmitted in the bitstream.

The INTRA Block Copy predictor comes from all the reconstructed causal area of the current frame. As for other Intra modes, the causal area is not loop filtered.

This block vector is the difference in coordinates between a particular point of the Coding Unit 1201 and the equivalent point in the predictor block 1202. Although it would be possible to use subpixel accuracy as for INTER blocks, this displacement is typically in integer units of pixels, therefore not to require costly subpixel interpolation.

In the current INTRA Block Copy design, each INTRA Block Copy Coding Unit can be split into one or 2 PUs as depicted in Figure 3. For the smallest Coding Unit size, SxB, the Coding Unit can be also split into 4 PUs of 4x4 pixels each.

For Inter mode the NxN partition is not available. It means that the 4x4 block size can't be used for Inter mode. The following table summarizes the block size for both modes.

Block sizes IBC mode Inter mode 64x64 (2Nx2N) X X 64x32 (2NxN) X X 32x64(Nx2N) x X 32x32 (2Nx2N) X X 32x16 (2NxN) X X 16x32 (Nx2N) X X 16x16(2Nx2N) X X 16x8(2NxN) X X 8x16(Nx2N) X X 8x8 (2Nx2N) X X 8x4(2NxN) X x 4x8 (Nx2N) X x 4x4(NxN) X In the current implementation of Intra Block Copy prediction mode, the search area at encoder side depends on the blocks sizes. This is represented in

the following table:

Block size Full range search 2 cTBs range search Other (outside the 2CTB area) 64x64 No search 32x32 No search 16x16 classical IBC search classical IBC search 8x8 Hash-based classical IBC search 4x8 classical IBC search 8x4 classical IBC search 4x4 Classical IBC search Please note that the 2NxN and Nx2N PU sizes are tested only for 8x8 Coding Units in the current encoder implementation. These sizes are not depicted in this table. There are 2 types of Intra Block Copy block vector estimation. The first one is the classical INTRA Block Copy search and it corresponds to a dedicated block matching algorithm. The second one is based on the Hash search algorithm. Two search ranges are also defined.

As depicted in Figure 13, for a frame 1301, the two Coding Tree Blocks search range corresponds to the left Coding Tree Block 1303 and to the blocks of the current Coding Tree Block 1302 already encoded/decoded. The blocks of current Coding Tree Block already encoded are depicted in dotted area in Figure 13. The full frame search corresponds to all the Coding Tree Blocks already encoded/decoded 1304.

In the Intra Block Copy mode the "block" vector is the difference in coordinates between a particular point in a block 1201 to encode and the equivalent point in the predictor block 1202 of Figure 12. Although it would be possible to use subpixel accuracy as for INTER blocks, this displacement is in integer units of pixels, therefore it doesn't require costly subpixel interpolation.

This block vector (BV) is itself predicted using a predictor which can be the left, the above BV or the latest decoded block vector of the current Coding Tree Block or the latest of the latest decoded By. These vector predictors come of course from the decoded Intra Block Copy block. With these methods a predictor index is transmitted.

As mentioned in the previous section, INTRA Block Copy is an Intra mode so its predictors come from the raw reconstructed data before any loop filtering. As a consequence for encoder and decoder implementation using parallelization between the block reconstruction process and the loop filtering process, the raw reconstructed block and the final, filtered, signal both need to be stored in memory. So it theoretically doubles the memory needed compared to an implementation without INTRA Block Copy for Intra frame. Another solution to avoid this increase of memory is to apply loop filtering at frame level and consequently to not apply parallelization of the loop filtering with the block reconstruction process.

It is proposed some solutions to solve this issue or inform the decoder of which level of parallelism it may use.

In the first solution, the delay parameter for INTRA Block Copy mode is explicitly signaled in the bitstream. It means that the furthest INTRA Block Copy block predictor in term of coding elements is transmitted in the bitstream. This delay represents the number of coding elements, typically coding tree blocks, which may provide some predictors in INTRJAI Block Copy mode when coding or decoding a given coding unit. These coding elements are the previous ones in the raster scan order used in encoding decoding processes.

At encoder side, this corresponds to the search range used, or a rounding (generally up) of said delay to a multiple of a particular number of coding elements.

In one embodiment, this element is the Coding Tree Block, and the delay is the number of previously decoded (or encoded) Coding Tree Blocks that can be used for INTRA Block Copy prediction, which is transmitted in the bitstream.

In another embodiment, the delay is the number of Coding Tree Block lines. In one embodiment, the value of the number of Coding Tree Blocks can be only a power of 2 (1,2, 4, 8, 16, etc...).

In a specific embodiment, the following coding can be used: IBC Delay Signaling Search range 01 2CTB 1 CIB line 2 CIB lines 1110 4CTBlines 1111 8CTBlines 00 Full frame To offer more flexibility, 2 data can be transmitted. One indicating the Coding Tree Block line has the following: IBC Delay CTB line Number of CTB line Signaling 0 1 1110 2 11110 4 11111 8 0 Full frame And a second one indicating the delay in number of Coding Tree Block per line: IBC Delay CTB Number of CTB line Signaling 1 2 1110 4CTBlines 11110 SCTBlines 11111 l6CTBlines 0 Full line In a further embodiment corresponding to any meaning of the delay, the entropy code used to represent the delay is an unbounded variable-length code.

In a further embodiment, the syntax employed is dependent on the image dimensions: for instance, it may be a fix-length code.

This solution offers to the decoder to know what kind of delay it may introduce between the loop filtering and the reconstructed block. The decoder may use this delay information to introduce a corresponding delay before beginning the parallel filtering of coding elements. It may also be used to determine the size of a memory to store reconstructed coding elements for INTRA Block Copy mode while parallelizing the filtering and the reconstruction of coding elements. This memory may contain the pixels as well as information relevant to post-filtering (quantizer step, partitioning, SAO information, motion or block vectors) for the coding elements indicated by the delay. In a further embodiment, as the filtering steps are sequential, it is possible to infer different sizes for different buffers, e.g. one for pre-filtering pixels, one for information needed in deblocking and one for SAO, in particular as deblocking and SAO may be disabled.

In one embodiment, the proposed signaling is transmitted to the decoder in an SEI message.

In the second solution, INTRA Block Copy mode uses the loop filtered data as predictor and no more the raw reconstructed data. It means that Deblocking Filter and Sample Adaptive Offset are applied on the fly at encoder and decoder side in order to produce loop filtered reference blocks. Namely, it means reconstructing a block of pixels according to INTRA Block Copy mode based on a predictor block belonging to a post filtered reconstructed coding element of the current image.

For a current encoded/decoded Coding Unit, some blocks can't be post filtered at encoder and decoder side. Indeed if all data are not available, it is not possible to apply Deblocking Filter and/or Sample Adaptive Offset.

In one embodiment, all reconstructed area where the Deblocking Filter has been applied and Sample Adaptive Offset have been applied if it has been signaled is available for INTRA Block Copy prediction. Figure 14 shows the loop-filtered area where both Sample Adaptive Offset and Deblocking Filter have been applied. It corresponds to all reconstructed pixels except a border of 4 pixels in the neighboring of the non-reconstructed pixels.

In an additional embodiment, all the reconstructed area is available for INTRA Block Copy prediction, and it contains all possible filtered blocks and the non-possible loop filtered blocks as depicted in Figure 14. It means that in this embodiment some INTRA Block Copy predictor may comprise raw reconstructed pixels as well as filtered reconstructed pixels. This additional embodiment increases the coding efficiency by using additional area. The main advantage of this embodiment is that no additional memory is needed for INTRA Block Copy at decoder side to store reconstructed and non-reconstructed blocks. In a specific embodiment, the Deblocking Filter and Sample Adaptive Offset are considered separately. So some pixels in the loop-filtered area are filtered with Deblocking Filter and Sample Adaptive Offset and others are only Deblocking Filter filtered.

In another embodiment, all the previous fully reconstructed Coding Tree Blocks are loop filtered except the pixels where all data are not available to apply the loop filtering. In this embodiment, pixels of the current Coding Tree Block are considered as non-available for the filtering process. Yet, this loop-filtered area is available for INTRA Block Copy prediction. It corresponds to an implementation where Deblocking Filter and Sample Adaptive Offset are applied Coding Tree Block by Coding Tree Block. So, the filtering process can be parallelized with one Coding Tree Block delay to the block reconstruction process. Figure 15 shows this embodiment. In this embodiment, all filtered data is considered as available for the INTRA Block Copy prediction. In an additional embodiment, the reconstructed blocks of the current Coding Tree Block only are considered as available in addition to the loop filtered data for the prediction of INTRA Block Copy. In an additional embodiment, all other reconstructed data, namely raw data and filtered data, is considered as available for INTRA Block Copy prediction. The advantage of this second embodiment compared to the first one (where all possible loop filtered data is considered as loop filtered for INTRA Block Copy prediction) is that it simplifies the encoder Sample Adaptive Offset selection. Indeed it is complicated to determine Sample Adaptive Offset parameters before the reconstruction of the current Coding Tree Block. So, with this embodiment, the classical Sample Adaptive Offset selection can be used to select Sample Adaptive Offset parameters.

In another embodiment, only the previous fully reconstructed Coding Tree Block, except the previous Coding Tree Block are loop filtered except the pixels where all data are not available to apply the loop filtering. In this embodiment, the pixels of the current Coding Tree Block and the previous one are considered as non-available for the filtering process. This loop-filtered area is available for INTRA Block Copy prediction. In addition the non-filtered reconstructed block of the current Coding Tree Block and the previous one are considered as available for INTRA Block Copy prediction. This embodiment should simplify encoder implementation, because most of the coding efficiency of INTRA Block Copy comes from the use of the current Coding Tree Block and the previous one.

In another embodiment, only the previous fully reconstructed Coding Tree Block lines are loop filtered except the pixels where all data is not available to apply the loop filtering. In this embodiment the pixels of the current Coding Tree Block line are considered as non-available for the filtering process. This loop-filtered area is available for INTRA Block Copy prediction in addition to the reconstructed Coding Tree Block line. Figure 16 shows this loop-filtered area. It corresponds to an implementation where Deblocking Filter and Sample Adaptive Offset are applied Coding Tree Block line by Coding Tree Block line.

So, the filtering process can be parallelized with one Coding Tree Block line delay to the block reconstruction process.

The advantage of solution 2 compared to solution 1 is that it solves the issue for parallelism for all decoder implementations.

The third solution consists in avoiding the Deblocking Filter and Sample Adaptive Offset loop filtering for some Coding Tree Block and in using these Coding Tree Block as INTRA Block Copy prediction area. It means that, based on some criteria of each Coding Tree Block, a decision is made to apply or not the post reconstruction filters to obtain the final image. INTRA Block Copy is then adapted to search for predictors only inside those Coding Tree Blocks that do not have to be filtered. One of the advantages of this solution is that it solves the issue on INTRA Block Copy delay without constraining all decoder implementation to apply Deblocking Filter and Sample Adaptive Offset on the fly as the second solution.

Figure 17 shows the first embodiment. At encoder side, all modes of all Coding Unit of the current Coding Tree Block 1701 are available. For each mode, the encoder and the decoder check if the mode is INTRA Block Copy or Palette 1703. If it is the case the current Coding Tree Block (CTB) is not loop filtered and the current Coding Tree Block is referenced as available for INTRA Block Copy prediction. Please note that only the encoder needs to reference this Coding Tree Block as available for INTRA Block Copy. Indeed, if the encoder has correctly referenced the available Coding Tree Blocks for INTRA Block Copy, it is sure that the INTRA Block Copy predictors come from these Coding Tree Block at decoder side. Please note also that in this embodiment, Sample Adaptive Offset syntax doesn't need to be read. If no Coding Unit are encoded/decoded with Palette or INTRA Block Copy, the Deblocking Filter is applied 1705 on this Coding Tree Block and the Sample Adaptive Offset syntax is extracted from the bitstream (cf. Figure 7 SAO) and Sample Adaptive Offset is applied if needed 1706. The idea behind this embodiment is that if at least one block of the Coding Tree Block is encoded using INTRA Block Copy or Palette mode, it means that the Coding Tree Block is likely to represent screen content. And it has been noticed that Deblocking Filter and Sample Adaptive Offset filtering do not improve significantly screen content blocks. These filters are mainly advantageous on natural content.

In one embodiment, a Coding Tree Block is considered as available for INTRA Block Copy prediction and the loop filtering is not applied according to a proportion of INTRA Block Copy and Palette modes in the Coding Tree Block. It means that the number of pixels encoded with INTRA Block Copy and Palette is taken into account to decide if this Coding Tree Block is loop filtered or not. For example, if 25% of the pixels of a Coding Tree Block have been predicted with Palette or INTRA Block Copy mode, it is considered that this Coding Tree Block is available for INTRA Block Copy prediction and the loop filters process is not applied.

In an additional embodiment, when the number of pixels of a Coding Tree Block which are encoded with INTRA Block Copy and Palette is compared to a threshold to decide if this Coding Tree Block is loop filtered or not, the threshold can depend on the Frame Type: Intra, Inter P or B. In a preferred embodiment, threshold has a higher value for frames having the Intra type than for frames having the Inter type. This Threshold can be equal to 20 per cent of of pixels of a Coding Tree Block which are encoded with INTRA Block Copy and Palette for Intra frames and 10% for Inter P or B frames. In another embodiment, the threshold can depend on the classification (filtered vs. not filtered) of the neighboring CTB (for example the top and/or the left neighboring CTB), or at least one of the neighboring Coding Tree Blocks. If the neighboring Coding Tree Blocks are not filtered, the threshold could be greater than if the (or at least one) neighboring Coding Tree Blocks have not been filtered.

In one embodiment, a Coding Tree Block is considered as available for INTRA Block Copy prediction and the loop filtering is not applied according to the number of Palette Coding Unit or a proportion of Palette pixels.

In one embodiment, in addition to the Coding Tree Block determined as available for INTRA Block Copy prediction (as described in the previous embodiments), the reconstructed blocks of the current Coding Tree Block are available for the INTRA Block Copy prediction.

In one embodiment, in addition to the Coding Tree Block determined as available for INTRA Block Copy prediction, the reconstructed blocks of the current Coding Tree Block and the reconstructed blocks of the previous Coding Tree Block are available for the INTRA Block Copy prediction.

In one embodiment, in addition to the Coding Tree Block or the Coding Unit determined as available for INTRA Block Copy prediction, the reconstructed blocks of the current Coding Tree Block line are available for the INTRA Block Copy prediction.

In one embodiment in addition to the previous decision embodiments, a Coding Unit coded using INTRA Block Copy or its pixel is considered as an INTRA Block Copy Coding Unit in the decision criterion (1703) only if its block vector points to another Coding Tree Block.

In one embodiment in addition to the previous decision embodiments, a Coding Unit coded using INTRA Block Copy or its pixel is considered as an INTRA Block Copy Coding Unit in the decision criterion (1703) only if its block vector points to another Coding Tree Block except the previous one.

In one embodiment in addition to the previous decision embodiments, a Coding Unit coded using INTRA Block Copy or its pixel is considered as an INTRA Block Copy Coding Unit in the decision criterion (1703) only if its block vector points to another Coding Tree Block line.

In one embodiment, when a Coding Tree Block is considered as available for INTRA Block Copy prediction and the loop filtering is not applied, the Sample Adaptive Offset syntax is just set as no Sample Adaptive Offset to simplify the parsing process.

In one embodiment, only the pixels of Palette Coding Unit and INTRA Block Copy Coding Unit are considered as available for INTRA Block Copy prediction. Of course the pixels of the current reconstructed Coding Tree Block should be considered as available for INTRA Block Copy prediction even if all pixels are not encoded with INTRA Block Copy and Palette. In the same way, the previous reconstructed Coding Tree Block or the full current reconstructed Coding Tree Block line should be also considered as available for INTRA Block Copy prediction.

In one embodiment, the decision criterion is applied on the previous Coding Tree Block to decide if the current Coding Tree Block needs to be loop filtered and if it is available for INTRA Block Copy prediction. In this embodiment the first Coding Tree Block is not filtered and is available for INTRA Block Copy prediction.

The proposed solution disables Deblocking Filter and Sample Adaptive Offset for a Coding Tree Block which contains efficient modes for screen content. So the aim of this solution is to classify as screen content a Coding Tree Block for two reasons: the first one is that INTRA Block Copy is very efficient for screen content, so the block predictor of an INTRA Block Copy Coding Unit should come from a block containing screen content. The second reason is that Deblocking Filter and Sample Adaptive Offset are less efficient for screen content than for natural content (especially HEVC Deblocking Filter). So when Deblocking Filter and Sample Adaptive Offset are disabled for a Coding Tree Block containing screen content, the coding efficiency is not necessarily reduced and may even be increased.

Other criterions can be defined to classify a Coding Tree Block as screen content Coding Tree Block. For example, the Intra Pulse Coding Mode (IPCM) can be taken into account in the criterion to decide if the Coding Tree Block is available for INTRA Block Copy prediction. The use of the transform skip can also be taken into account in the criterion.

In the first embodiment, the Coding Tree Block available for INTRA Block Copy prediction are determined on the fly at both encoder and decoder side. In the following embodiments, these Coding Tree Blocks available for the INTRA Block Copy prediction and non-loop filtered are signaled according to a flag (named here Loop_filtering_flag) transmitted in the bitstream. Figure 18 shows this embodiment. This figure is based on Figure 7 of Sample Adaptive Offset Syntax. The loop filtering 1811 is extracted from the bitstream. If this flag is set equal to 1, the Sample Adaptive Offset for each component (Y, U, V) are then decoded and the current Coding Tree Block is loop filtered. If this flag is set equal to 0, no loop filtering is applied on reconstructed Coding Tree Block and this Coding Tree Block can be used as reference for INTRA Block Copy prediction.

In one embodiment, the flag loop_filtering_flag, is coded with the Context-based Adaptive Binary Arithmetic Coding (CABAC). It means that this flag has its own context. As would be known by those skilled in the art, a context model is a probability model for a value to encode, here the loop_filtering_flag. This model may be chosen from a selection of available models depending on the statistics of recently coded data for example.

In an additional embodiment, the context for this flag depends on the decoded value of the previously decoded loop_filtering_flag. For example, the context depends on the value of the loop_filtering_flag of the left and/or right neighbouring CTB.

All other embodiments described previously for solution 3 can be combined with this last embodiment, except embodiments related to the decision criterion. Especially the set of Coding Tree Block available for INTRA Block Copy prediction can contain the current Coding Tree Block or the current Coding Tree Block and the previous one, or the current Coding Tree Block and the full Coding Tree Block line. In the same way, the loop_filtering_flag can be encoded at block level instead of at Coding Tree Block level.

In an additional embodiment, the Sample Adaptive Offset syntax can be non-dependent to the filtering flag to simplify the parsing. At decoder side, this embodiment is quite simple, but at encoder side it depends on the implementation.

In an embodiment, the loop filtering flag for each Coding Tree Block is determined at the end of the encoding of the frame or the slice. Figure 19 gives an example of this encoder algorithm. In this example, we consider that the INTRA Block Copy Block Vector can access the current reconstructed Coding Tree Block (non-loop filtered). In the first step, all loop_filtering_flag are set equal to 1 for all Coding Tree Block of the frame or the slice 1901. For each INTRA Block Copy Coding Unit of the frame or slice, the encoder verifies 1903 if the block vector points to the current Coding Tree Block or not. If not the loop_filtering_flag of the pointed Coding Tree Block or the several Coding Tree Block pointed Coding Tree Block (until 4 Coding Tree Block) by the block vector are set equal to 0. Then the following block vectors are checked. It means that a Coding Tree Block is flagged not to be filtered if it contains a predictor block used for the encoding of a coding unit using INTRA Block Copy mode.

In another encoder embodiment, the loop filtering flag is determined on the fly based on one criterion to determine if the current Coding Tree Block contains screen content as presented for the previous embodiments. Compared to the previous encoder algorithm, this encoder selection offers more flexibility for the implementation but it is less efficient.

The inventors have noticed that specific embodiments of the Sample Adaptive Offset filtering may be affected by the use of the loop filtering flag. An example of theses specific embodiments is described in Figure 20.

Figure 21 shows an example of a Group of Pictures (GOP) comprising images associated with a hierarchy of rate-distortion compromises for the low delay case. As depicted on this figure, the size of image is related to the hierarchy in terms of quality. For example, images with a level equal to "2" have the biggest size and so have a higher quality than images with medium size and the level 1" The images with the smallest size (level equal to "0") have the lower quality.

Another way to evaluate the hierarchy of the images in term of rate-distortion is to consider the Quantization Parameters of the images forming the GoP 2000. As seen in figure 21, the images with the highest quality are associated with Quantization Parameters QP. They have a level equal to 2. The images with the medium quality are associated with Quantization Parameters QP+1. They have a level equal to "1". And, the images with the lowest quality are associated with Quantization Parameters QP÷2. They have a level equal to

Q

In this hierarchy, the quality (absence of distortion) has higher importance (relative to cost) in the rate-distortion compromise for higher levels compared to lower level. The effect is that an image at the top of the hierarchy (level) should have a larger quality than image with lower level. This quality is then propagated, with the help of the Inter coding mode, on the following encoded images which have a lower level in the hierarchy and which have a lower quality in terms of rate distortion compromise than the images with the higher level in the hierarchy. The compromise between rate and distortion can be fixed by the Quantization parameter or by the Langragian parameter (called lambda) in the rate distortion criterion or by both. Images with a lower level have a higher OP value than images with a higher level or the Lagrangian parameter is higher than for images with a higher level (It means that the rate has an higher importance (relative to the distortion) in the rate distortion compromise for lower level).

A classic way of encoding a sequence of images in which area-by-area loop filtering is performed on at least one image of the sequence and is disabled on at one other image of the sequence, Here, area-by-area loop filtering means varying the loop filtering from one image area (block, Largest Coding Unit, etc.) to another within one image. Of course, varying the loop filtering area-by-area in this way incurs certain costs. Firstly, the encoding is more complex because different loop filtering parameters must be generated for each image area requiring a different loop filtering. Secondly, information about the loop filtering parameters usually has to be transmitted from the encoder to the decoder, which in the case of area-by-area loop filtering implies a significant signaling overhead.

By way of background, within a group of images each image may have an associated level within a plurality (hierarchy) of levels. The levels may correspond to different respective rate-distortion balances (compromises) or to different image qualities. The benefit of applying area-by-area loop filtering on an image depends on what level the image has in the hierarchy of levels.

Indeed, the SAO filtering doesn't need to be applied to all the images of the sequence and/or all the images of a group of images, especially when the hierarchy between rate and distortion is used to encode a group of frames. The main advantage of disabling the area-by-area SAO filtering on some images is the improvement of the coding efficiency. Indeed it is better to disable the use of SAO parameters for an entire image than area-by-area in order to save the bitrate dedicated to the signalization which tells not to use the SAO filtering.

For example the SAO parameters (defined in the High Efficiency Video Coding (HEVC: ISO/IEC 23008-2 MPEG-H Part 2/ ITU-T H.265)) include the type of SAO (stored in a variable named codeword sao_type_idx), a value representative of the center of the useful range (stored in the codeword sao_band_position) when the band offset type is used or a value representative the SAO filtering direction when the Edge offset type is used, and the SAO offsets (stored in the codewords sao_offset).

A second advantage is the complexity reduction at encoder side.

Indeed, the area-by-area SAO filtering process doesn't need to be applied on all frames.

A classic way to enable the area-by-area SAO filtering at frame level is to consider the amount of SAO selection for frame with the highest level (frame with level equal to 2 in Figure 21). More precisely the area-by-area SAO filtering is enabled or disabled for the following frames of level 2 and 1 according to the number of CTB filtered with SAO for the previous frame of level 0. The number of CTB filtered by SAO is compared to a threshold which is different for Luma and Chroma components. In other words, the amount of SAO activity in one or more images is considered when determining which images of the group should have area-by-area SAO filtering disabled. In the following description, one measure of SAO activity is the number of SAO selections of an image. This may be the number of times when the SAO filter is selected, or the number of pixels filtered by SAO for all color components or each color component.

The area-by-area SAO filtering is disabled for a target image according to the number of SAO selections in previous encoded images having the same level as the target image in the hierarchy of levels (balances between rate and distortion). This means that the number of SAO selections is determined for each image. And if the number of SAO selections for the level of the target image (current image) is less than a pre-defined threshold, the area-by-area SAO filtering process is disabled for the current image. The number of SAO selections for a level can be the average number of SAO selections for all previously encoded images or a set of previously encoded images with the same level. Or the amount of SAO selections for a level can be the number of SAO selections on the previous image having the same level. In this embodiment, when the area-by-area SAO filtering process is disabled for a given level it is reactivated after the encoding of a pre-determined number of image without SAO. The reactivation of the area-by-area SAO filtering for a level can depend also on the use of Intra frames which refresh the Inter coding dependencies or on content change in the sequence determined by scene change detection algorithm or on QP change for each level of a group of images.

Alternatively, the area-by-area SAO filtering is disabled for a target image having a given level according to the number of SAO selections on images of another level in the hierarchy, ideally, according to the top level of the hierarchy (images with the best quality). In this case, if the number of SAO selections for a higher level in the hierarchy than the level of the current image is less than a pre-defined threshold, the area-by-area SAO filtering is disabled for the current image. The pre-determined threshold can be weighted according to the difference in levels between the reference level (e.g. highest level) and the level of the current image. As the previous embodiment, the number of SAC selections for a level can be the average number of SAC selections for all previously encoded images or for a set of previously encoded images having the same level. In this embodiment, the area-by-area SAO filtering may always be performed on images with the highest level or may be enabled and disabled according to the embodiment described in the immediately-preceding paragraph.

With the previous embodiments comprising the application or not of the loop filtering for each Coding Tree Block (CTB) according to a flag or a classification, the amount of Coding Tree Blocks where SAO is performed can be different from the classical SAC selection (in a variant another block than the CTB may be considered, for example the whole frame).

Indeed, a current CIB can be classified as available for IBC and consequently SAC is disabled even if SAC would have been useful for this CTB. As a consequence, these embodiments have an impact on the SAC statistics used to determine at frame level if SAC needs to be evaluated at encoder side in addition to the quality hierarchy as depicted in Figure 20.

In one embodiment, the SAC statistics are accumulated only for the Coding Tree Blocks where the flag loop_filtering_flag is on as depicted in Figure 22.

More specifically, in a first step 2201, a test is realized to check if the SAO is enabled for a level N of the hierarchy (for instance quality hierarchy) as illustrated in figure 21.

In step 2204 a test is done to determine if the flag Loop_filtering_flag is "on" ("=1") or not. This flag described above, signals if the CTB is filtered implying that it is not available for the Intra Block prediction mode. If the flag Loop_filtering_flag is "off", then a counter NB_available_CTB_PER_FRAME is incremented in a step 2205. This counter indicates the number of Coding Tree Blocks available for the SAO filtering. Otherwise the process loops back to the step 2203.

After incrementing the counter in step 2205, for each color component (step 2202) and for each CTB of a current frame (step 2203) the SAO parameters (whose examples were given above) are evaluated (step 2206).

Then a test 2207 is undertaken to check if the SAO should be selected for the considered CTB according to a quality criteria or the rate-distortion criterion for example, as described in relation to the figure 21.

If the CTB has a quality allowing the SAO filtering then another counter NumSAOCTBNX is incremented in a step 2208. This step allows rectifying the statistics related to the SAO filtering due to the use of IBC mode.

If the SAO filtering should not be selected or after the incrementing step 2208, another test is done for verifying is the considered CIB is the last one (step 2209). If it is not, the process loops back to step 2203.

If the considered CTB is the last one, then in a step 2210 a comparison is made between the ratio of the two counters NB_available_CTB_PER_FRAME and Num_SAO_CTB_N_X and a predetermined threshold Threshold_N-1_X. If the ratio is above the threshold, then the SAO is enabled for the further images of the sequence having a decomposition level N-i or for all other lower levels in a step 2211, at least until it is deactivated If the ratio is below the threshold, then the SAO is disabled for the further images of the sequence having a decomposition level N-i or for all other lower levels in a step 2213, at least until it is reactivated.

In a step 2212, it is checked of the processed color component is the last one. If not, the process loops back to the step 2202. Otherwise it goes to a next step ofafurther coding process, 2214.

More generally, in another embodiment, the SAO statistics are accumulated only for the Coding Tree Blocks which are classified (according to one of the listed criterion defined in reference to figure 17) as non-screen content CTB, implying that the test 2204 would be to determine if the CTB has screencontent. The screen content" video sequences refer to particular video sequences having computer-or machine-generated content, for example text, PowerPoint (RTM) presentations, Graphical User Interfaces, Electronic Program Guides, tables (e.g. screen shots). Such screen content may be characterised by having relatively few colours or by having discrete colours spaced out at relatively wide intervals in the colour spectrum.

In one additional embodiment, when all Coding Tree Blocks of the frame have been classified as used for IBC (which could be signaled with a flag at frame level or slice level) and that no loop filtering is applied on that frame, then the encoder can consider that SAO will not be evaluated for the further frames of a lower level.

The embodiments described in relation to the figure 22 offer the possibility to estimate more accurately the SAO statistics for a level of decomposition N in order to take a better decision regarding the SAO filtering of a lower level. These embodiments allow to coding efficiency.

These embodiments can be applied for all color components or on one or more color components independently. These embodiments can be used to independently apply or not the area-by-area SAO filtering process to one or both SAO types (edge and band).

These embodiments can be used at decoder side to infer the SAO filtering flag at frame level. In this case, the encoder and the decoder apply the same criterion to decide if SAO is applied or not for the current frame. This avoids the need to transmit the SAO filtering flag to the decoder but the complexity of the decoder is increased.

These embodiments can be applied to any kind of loop filtering, and not just to SAO filtering. For example, embodiments can work in the case of Adaptive loop filter (ALF), too. In another embodiment, the statistics determined for SAO (like previous embodiment) are used to decide if ALF is applied or not.

In another embodiment, the statistics determined for ALF (like previous embodiment) are used to decide if SAO is applied or not.

In a fourth solution, the Coding Tree Block available for INTRA Block Copy prediction are signaled as in the previous solution but the post filtering can be applied on it and only raw reconstructed pixels are used for INTRA Block Copy prediction. It means that, by opposition to the third solution, being marked does not mean any more that Coding Tree Block will no longer be filtered to obtain the final image. The main idea is to limit the memory needed to store the whole raw reconstructed frame and to store only some limited numbers of raw reconstructed Coding Tree Block for INTRA Block Copy prediction.

In one embodiment, the amount of Coding Tree Block available for INTRA Block Copy prediction is limited to a certain amount of Coding Tree Block. For each Coding Tree Block a flag is transmitted to indicate that the current reconstructed Coding Tree Block is used as reference for INTRA Block Copy. The flag INTRA Block Copy_reference_Coding Tree Block_flag is set equal to 1 when this Coding Tree Block is used for the INTRA Block Copy prediction and 0 when not.

The maximum number of Coding Tree Block in the memory is set to a value N. At decoder side when the number n of Coding Tree Block reaches this value N, the furthest Coding Tree Block (the lower Coding Tree Block number) is removed from the memory and the current one is added. And no more INTRA Block Copy block predictors can come from the removed Coding Tree Block.

The value N can be fixed or transmitted as in solution 1.

Figure 20 is a schematic block diagram of a computing device 2000 for implementation of one or more embodiments of the invention. The computing device 2000 may be a device such as a micro-computer, a workstation or a light portable device. The computing device 2000 comprises a communication bus connected to: -a central processing unit 2001, such as a microprocessor, denoted CPU; -a random access memory 2002, denoted RAM, for storing the executable code of the method of embodiments of the invention as well as the registers adapted to record variables and parameters necessary for implementing the method for encoding or decoding at least part of an image according to embodiments of the invention, the memory capacity thereof can be expanded by an optional RAM connected to an expansion port for example; -a read only memory 2003, denoted ROM, for storing computer programs for implementing embodiments of the invention; -a network interface 2004 is typically connected to a communication network over which digital data to be processed are transmitted or received.

The network interface 2004 can be a single network interface, or composed of a set of different network interfaces (for instance wired and wireless interfaces, or different kinds of wired or wireless interfaces). Data packets are written to the network interface for transmission or are read from the network interface for reception under the control of the software application running in the CPU 2001; -a user interface 2005 may be used for receiving inputs from a user or to display information to a user; -a hard disk 2006 denoted HD may be provided as a mass storage device; -an I/O module 2007 may be used for receiving/sending data from/to external devices such as a video source or display.

The executable code may be stored either in read only memory 2003, on the hard disk 2006 or on a removable digital medium such as for example a disk. According to a variant, the executable code of the programs can be received by means of a communication network, via the network interface 2004, in order to be stored in one of the storage means of the communication device 2000, such as the hard disk 2006, before being executed.

The central processing unit 2001 is adapted to control and direct the execution of the instructions or portions of software code of the program or programs according to embodiments of the invention, which instructions are stored in one of the aforementioned storage means. After powering on, the CPU 2001 is capable of executing instructions from main RAM memory 2002 relating to a software application after those instructions have been loaded from the program ROM 2003 or the hard-disc (HD) 2006 for example. Such a software application, when executed by the CPU 2001, causes the steps of the flowcharts described herein to be performed.

Any step of the algorithm described herein may be implemented in software by execution of a set of instructions or program by a programmable computing machine, such as a PC ("Personal Computer"), a DSP ("Digital Signal Processor") or a microcontroller; or else implemented in hardware by a machine or a dedicated component, such as an FPGA ("Field-Programmable Gate Array") or an ASIC ("Application-Specific Integrated Circuit").

Although the present invention has been described hereinabove with reference to specific embodiments, the present invention is not limited to the specific embodiments, and modifications will be apparent to a skilled person in the art which lie within the scope of the present invention.

Many further modifications and variations will suggest themselves to those versed in the art upon making reference to the foregoing illustrative embodiments, which are given by way of example only and which are not intended to limit the scope of the invention, that being determined solely by the appended claims. In particular the different features from different embodiments may be interchanged, where appropriate.

In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used.