WO2019002615A1

WO2019002615A1 - Apparatus for block-based predictive decoding and encoding and corresponding methods

Info

Publication number: WO2019002615A1
Application number: PCT/EP2018/067719
Authority: WO
Inventors: Jonathan PFAFF; Martin Winken; Philipp Helle; Thomas Wiegand; Detlev Marpe; Jennifer RASCH; Michael Schäfer; Heiko Schwarz; Mischa Siekmann; Anastasia Henkel
Original assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
Priority date: 2017-06-30
Filing date: 2018-06-29
Publication date: 2019-01-03
Also published as: EP3646607A1; TW201907716A

Abstract

The invention refers to an apparatus (10) for block-based predictive decoding of a picture (200), comprising: an extractor (1 1) extracting from a data stream (100) filter information (20) about applying a diffusion filter (LDF, ADF), an initial prediction signal provider (12) providing an initial prediction signal (21) for a current block (22), a prediction signal modifier (13) providing in case of a positive filter information (20) a modified prediction signal (23) based on the initial prediction signal (21 ) and based on the diffusion filter (LDF, ADF), and a reconstructor (14) decoding a reconstructed version of the current block (22) based on the initial prediction signal (21) in case of a negative filter information (20) and based on the modified prediction signal (23) in case of a positive filter information (20). The invention also refers to an apparatus (10) for block-based predictive decoding of a picture (200) and to corresponding methods.

Description

Apparatus for block-based predictive decoding and encoding and corresponding methods

The invention refers to an apparatus for block-based predictive decoding and to an appa- ratus for block-based predictive encoding of pictures and corresponding methods.

Nowadays many video codecs use block-based predictive coding to compress the data used to represent in combination with a prediction residual the picture content. The different compression techniques perform spatial (intra-picture) and/or temporal (inter-picture) prediction. Blocks in an intra-coded frame or slice are encoded using spatial prediction with respect to neighboring blocks in the same frame, picture or slice. Blocks in an inter- coded frame or slice use either spatial prediction with respect to neighboring macroblocks in the same frame or slice or use temporal prediction with respect to other reference frames.

For the High Efficiency Video Coding (HEVC) standard and its successors, adaptive loop filtering (ALF) was proposed. In general, ALF is an adaptive Wiener filtering technique following the deblocking filter to improve the reference picture used for encoding/decoding of subsequent pictures.

For the High Efficiency Video Coding (HEVC) standard and its successors, an integer motion vector (IMV) mode was proposed. In case of IMV, the motion vectors for inter prediction are integer valued which is advantageous for its signaling. In HEVC, for every block a so called coded block flag (cbf) is sent, which indicates if the corresponding residual is set to zero or not. In case the cbf is zero, no residual is to be sent.

The object of the present invention is to provide an improvement for the encoding and decoding process of a picture.

The object is achieved by an apparatus for block-based predictive decoding of a picture. The apparatus comprises an extractor configured to extract from a data stream filter information about applying a diffusion filter, an initial prediction signal provider configured to provide an initial prediction signal for a current block, a prediction signal modifier configured to provide in case of a positive filter in-formation a modified prediction signal based on the initial prediction signal and based on the diffusion filter, and a reconstructor configured to decode a reconstructed version of the current block based on the initial prediction signal in case of a negative filter in-formation and based on the modified prediction signal in case of a positive filter information.

For the reconstruction of the blocks of the picture - and preferably for each color component of the blocks - a modified prediction signal is used in case of a positive filter information extracted from the data stream. A positive filter information (i.e. the filter information is positive) means that the filter information indicates that a diffusion filter is to be used. Hence, the data stream comprises information about whether a filtered prediction signal (i.e. the modified prediction signal being modified by the application of the diffusion filter) was used during the encoding of the picture. If this is the case, such a filtered prediction signal also has to be used during the reconstruction. In an embodiment, there are different diffusion filters, i.e. kinds of diffusion filters available. In this embodiment, the data stream also carries information about the kind of diffusion filter used during the encoding process and accordingly to be used for the reconstruction.

According to an embodiment, the prediction signal modifier is configured to provide data of an extension of the current block to a larger block comprising the current block and a boundary. The prediction signal modifier is configured to provide an extended prediction signal for the larger block based on the initial prediction signal. The prediction signal modifier is configured to provide a filtered extended prediction signal based on the diffusion filter and based on a filter function fulfilling two boundary conditions referring to the boundary of the larger block. The prediction signal modifier is configured to provide the modified prediction signal based on the filtered extended prediction signal.

In this embodiment, the modified prediction signal for a current block is obtained by the following steps: The current block is enlarged to a larger block and the initial prediction signal for the current block is accordingly fitted to the larger block. Then, the diffusion filter is applied to the extended prediction signal and the so obtained filtered extended prediction signal is restricted to the current block.

In an embodiment, the prediction signal modifier is configured to provide the data of the extension of the current block such that the larger block comprises already reconstructed blocks. In this embodiment, data about already processed blocks are used for generating the larger block.

According to an embodiment, the prediction signal modifier is configured to provide the extended prediction signal based on already reconstructed blocks and/or based on at least one reference picture.

In an embodiment, the two boundary conditions refer to two disjoint subsets belonging to the boundary of the larger block. There is a first boundary condition and a second bounda- ry condition. The two boundary conditions refer to a first subset and to a second subset. The first boundary condition indicates that the filter function for the first subset uses already reconstructed blocks. The second boundary condition indicates that a derivative of the filter function with respect to the outer normal vector field vanishes for the second subset.

According to an embodiment, the prediction signal modifier is configured to provide the filtered extended prediction signal based on a stopping parameter indicating an intensity of the diffusion filter. The stopping parameter refers to a number of iterations to be performed for solving a differential equation.

In an embodiment, the stopping parameter is comprised by the data stream. Alternatively, the stopping parameter is known to the apparatus. Alternatively, the stopping parameter is derived from an already reconstructed block or the prediction signal. The stopping parameter is either known to both, the encoder and the decoder side or the stopping parameter is transmitted e.g. via the data stream or any other medium to the decoder side. Irrespective of the realization, the stopping parameter has to be known to both sides: the encoder side and the decoder/reconstruction side.

According to an embodiment, the prediction signal modifier is configured to provide the filtered extended prediction signal based on the diffusion filter indicating a kind of differential equation to be solved by the filter function. In an embodiment, there is either a linear diffusion filter or an anisotropic diffusion filter to be applied.

In an embodiment, the prediction signal modifier is configured to provide the modified pre- diction signal based on the filtered extended prediction signal by limiting the modified prediction signal to the current block. According to an embodiment, in case of a positive filter information, i.e. the filter information being positive, only integer motion vectors are used for decoding. The following embodiments refers to different cases in which the data stream specifies for the current block a skip modus, a merge modus, the application of an adaptive loop filtering (ALF), a coded block flag setting and an IMV modus. Selection between using the modified prediction signal and the initial prediction signal may, thus, be done depending on settings for the current block pertaining to skip modus, merge modus, the application of the adaptive loop filtering (ALF), the coded block flag (i.e. the residual) with the current block being an intra or inter block being zero or not, or the IMV (integer motion vector) modus.

In an embodiment, the prediction signal modifier is configured to provide the modified pre- diction signal in case of a positive filter information and in case the data stream fails to indicate a skip mode for the current block. In an alternative embodiment, the prediction signal modifier is configured to provide the modified prediction signal in case of a positive filter information and in case the data stream indicates a skip mode for the current block. According to an embodiment, the prediction signal modifier is configured to perform a merging for the modified prediction signal in case of a positive filter information and in case the data stream indicates a merge mode for the current block. In an alternative embodiment, the prediction signal modifier is configured to provide the modified prediction signal in case of a positive filter information and in case the data stream indicates a merge mode for the current block.

In an embodiment, the prediction signal modifier is configured to provide the modified prediction signal in case of a positive filter information and in case the data stream fails to indicate an application of an adaptive loop filter for the current block. Alternatively, the prediction signal modifier is configured to provide the modified prediction signal in case of a positive filter information and in case the data stream indicates an application of an adaptive loop filter for the current block.

In an embodiment, the extractor is configured to extract or parse the filter information only in case if an IMV flag is sent and is positive or in case the IMV flag is not send because of, e.g., a merge modus being signaled in the data stream for the current block, or the current block being part of an intra coded portion/slice so that inter coding mode is not available within that portion/slice. The extractor, in turn, would extract the filter information from the data stream only if the merge modus does not apply for the current block and the current block is not part of an I slice. The prediction signal modifier would then be configured to provide the modified prediction signal only then and only and/or the reconstructor would decode the reconstructed version of the current block based on the modified prediction signal only then and only if the filter information is positive.

In an embodiment, the extractor is configured to extract the filter information only in case if an IMV modus is an allowed coding option for the current block and is also signaled to be applied for the current block in the data stream or in case IMV modus is not an allowed coding option because of, e.g., the current block being part of an intra coded portion/slice, i.e. an I slice. The prediction signal modifier would then be configured to provide the modified prediction signal only then and only and/or the reconstructor would decode the recon- structed version of the current block based on the modified prediction signal only then and only if the filter information is positive. Otherwise (if not present in the data stream), the filter information would be inferred to be negative.

In an embodiment, the extractor is configured to extract the filter information only in case the root cbf of the current block, with the latter being an inter predicted block, or the luma cbf of the current block with the latter being an intra predicted block, is not equal to zero. The root cbf would be a signalization indicating the zeroness of the residual of a tree root block the current block is located in by partitioning the tree root block by recursive partitioning. The luma cbf would be a signalization indicating the zeroness of the residual of the current block itself. Alternatively, the root cbf of an inter block or the luma cbf of an intra block could be signaled in the data stream only (and extracted by the decoder only) if the filter information which would, thus, be extracted from the data stream irrespective of these signalizations, is negative. In the latter case, if the filter information is positive, the root respectively the luma cbf cannot be zero, thus, but would be inferred to be one in- stead, meaning that the prediction residual is present for the current block. The prediction signal modifier is configured to provide the modified prediction signal only then and only and/or the reconstructor would decode the reconstructed version of the current block based on the modified prediction signal only then and only if the filter information is positive.

The object is also achieved by a method for block-based predictive decoding of a picture. The method comprises the following steps:

• extracting from a data stream filter information about applying a diffusion filter,

· providing an initial prediction signal for a current block and an extended prediction signal based on the initial prediction signal,

• providing in case of a positive filter information a modified prediction signal based on the initial prediction signal, the extended prediction signal as its extension and based on the diffusion filter, and

· decoding a reconstructed version of the current block based on the modified prediction signal in case of a positive filter information.

The embodiments of the above mentioned apparatus for block-based predictive decoding of a picture can also be realized by the method, and vice versa.

The object is also achieved by an apparatus for block-based predictive encoding of a picture.

The apparatus comprises: an initial prediction signal provider configured to provide an initial prediction signal for a current block, a prediction signal modifier configured to provide a modified prediction signal based on the initial prediction signal and based on a diffusion filter, a comparator configured to compare an effect of the initial prediction signal with an effect of the modified prediction signal and to generate a comparison result, and an encoder configured to provide a data stream based on the picture and based on the comparison result, wherein the encoder is configured to insert based on the comparison result filter information into the data stream. The filter information is indicating whether a diffusion filter is to be applied for decoding the current block. Alternatively or additionally, the filter information is indicating which diffusion filter is to be applied for decoding the current block.

During the encoding process at least one diffusion filter is applied for the encoding of a current block - and preferably to each color component of the block - to see whether the prediction signal modified with the diffusion filter yields better results than the prediction signal without the diffusion filter. Depending on the outcome, the current block is encoded based on the initial prediction signal without the diffusion filter or based on the modified prediction filter. Further, the information, e.g. a flag, is added to the data stream whether a diffusion filter is to be used or not.

In an embodiment, various diffusion filters are available and tested by the encoder. For this embodiment, an additional information about which diffusion filter is to be used is added to the data stream.

In an embodiment, the prediction signal modifier is configured to provide data of an extension of the current block to a larger block comprising the current block and a boundary. The prediction signal modifier is configured to provide an extended prediction signal for the larger block based on the initial prediction signal. The prediction signal modifier is configured to provide a filtered ex-tended prediction signal based on the diffusion filter and based on a filter function fulfilling two boundary conditions referring to the boundary of the larger block. The prediction signal modifier is configured to provide the modified prediction signal based on the filtered extended prediction signal.

The way to obtain the modified prediction signal is identical to the way used by the apparatus for decoding (or short: decoder) and explained above. Thus, the discussion also holds here.

According to an embodiment, the two boundary conditions refer to two disjoint subsets belonging to the boundary of the larger block. A first boundary condition indicates that the filter function for a first subset of the two subsets describes already encoded blocks. A second boundary condition indicates that a derivative of the filter function with respect to the outer normal vector field vanishes for a second subset of the two subsets.

The boundary conditions are comparable to the conditions used by the decoder as discussed above. The sole difference is that here the first boundary condition refers to encoded blocks whereas the first boundary condition for the decoder refers to already re- constructed blocks. Nevertheless, in both applications the first boundary condition refers to already processed blocks, either being encoded or being reconstructed.

In an embodiment, the prediction signal modifier is configured to provide the filtered extended prediction signal based on a stopping parameter indicating an intensity of the dif- fusion filter. The stopping parameter, i.e. especially its value, is also inserted in an embodiment into the data stream in order to be transmitted to and used by the decoder. Alterna- tively, the stopping parameter is known to the decoder. Alternatively, the stopping parameter is derived from an already reconstructed block or the prediction signal.

According to an embodiment, the prediction signal modifier is configured to provide the filtered extended prediction signal based on the diffusion filter indicating a kind of differential equation to be solved by the filter function.

In an embodiment, the prediction signal modifier is configured to provide the modified prediction signal based on the filtered extended prediction signal by limiting the modified pre- diction signal to the current block.

According to an embodiment, in case of a positive filter information only integer motion vectors are used for encoding. The object is also achieved by a method for block-based predictive encoding of a picture.

The method comprises the following steps:

• providing an initial prediction signal for a current block of the picture and an extended prediction signal based on the initial prediction signal,

• providing a modified prediction signal based on the initial prediction signal, the extended prediction signal and a diffusion filter,

• comparing an effect of the initial prediction signal with an effect of the modified prediction signal and generating a comparison result, and

· providing a data stream based on the picture and based on the comparison result, and

• inserting based on the comparison result filter information into the data stream indicating whether a diffusion filter is to be applied for decoding the current block and/or indicating which diffusion filter is to be applied. The embodiments of the above mentioned an apparatus for block-based predictive encoding of a picture can also be realized by the method, and vice versa.

The invention also refers to a computer program comprising a program code for performing, when running on a computer, a method of any of the foregoing embodiments. The invention also refers to a data stream having a picture encoded thereinto, the data stream being generated by the method for block-based predictive encoding of a picture.

The basis of the invention may be explained again using different words:

A new method is used to generate a prediction signal in video coding obtaining e.g. a hybrid video codec.

For example, in a first step, a video decoder generates a starting prediction signal (also called initial prediction signal) as in the underlying standard used for the encoding of the picture. The starting prediction signal is generated e.g. by motion compensation or intra- picture prediction, in a second step, the decoder modifies the starting/initial prediction signal by applying e.g. a linear diffusion filter or an anisotropic diffusion filter to an extension of the initial prediction signal. The respective diffusion filter is applied in an embodi- ment up to a discrete stopping time (also referred to as stopping parameter) Nstop. The stopping time refers here to the heat kernel and indicates how long the flow is considered with this theory. The stopping time refers to a number of iterations used for solving the differential equation and describes the strength or intensity of the applied filter. The filtered extended prediction signal is then restricted to the current block and is used as a new prediction signal on the current block.

The integration of the diffusion filter into the video codec is explained in the following referring to exemplary embodiments. The steps are especially explained for a decoder but are also valid for the application in an encoder.

Considered is a hybrid video coding standard in which for each color component cmp the content of a video-frame on a block

¾,cmp ^{: =} {( < ) ^e 2²: k_{l cmp}≤ x≤ k_2iCmp: l±_iCmp≤ y≤ _lCmv}, where kl,cmp> ^2,cmp> ,cmp> .cmp ^ ^l.cmp — ^2,anp .cmp — ^2,cmp < is to be generated by a decoder. For every color component cmp, the latter content is given by a function im_cmp: S_(ijCmp→ M.

It is assumed that the hybrid video coding standard operates by predictive coding on the current block Έ_{α αηρ}.

This means that, for every component cmp the decoder constructs a prediction signal pred_cmp-^_diCmp→ R in a way that is determined uniquely by the already decoded bitstream. This prediction signal can, for example, be generated by intra-picture prediction or by motion compensated prediction.

The predictive coding of the standard is extended by the following generation of a new prediction signal pred_cmp using the following steps based on exemplary embodiments:

1. From the bitstream (another expression is data stream), the decoder determines for every component cmp that exactly one of the following options is true: a) No filter is applied.

b) The linear diffusion filter (LDF), whose definition will be given in the following, is to be applied to the prediction signal pred cmp

c) The anisotropic diffusion filter (ADF), whose definition will be given in the following, is to be applied to the prediction signal pred cmp

Hence, in case b) and c), a diffusion filter is to be applied.

2. In different embodiments, at least one of the following options is realized: · in case of skip, no diffusion filter is applied.

• in case of merge, the diffusion filter is also merged.

• in case of skip or merge, a diffusion filter can be applied.

• in case of ALF filtering, the diffusion filter is not applied. • in case of ALF filtering, the diffusion filter can be applied.

• the diffusion can only be applied in case the IMV flag is sent and IMV modus is chosen or in case no IMV flag is sent (e.g. merge or intra) and thus the IMV flag is not sent.

• the diffusion can only be applied in case the IMV modus is possible, can be derived and IMV modus is chosen or in case no IMV is possible (e.g. intra) and thus the IMV flag is not sent.

• the diffusion filter information is only signaled in case the luma cbf of an intra block is not zero.

• the diffusion filter information is only signaled in case the root cbf of an inter block is not zero.

• in case the diffusion filter is applied, no luma cbf of an intra block is signaled since it is not allowed to be not zero. Thus, the luma cbf of an intra block is only signaled if it the diffusion filter information is negative.

in case the diffusion filter is applied, no root cbf of an inter block is signaled since it is not allowed to be not zero. Thus, the root cbf of an inter block is only signaled if it the diffusion filter information is negative.

The following steps are based on a positive filter information obtained from the bitstream indicating the application of a diffusion filter, i.e. it was determined in step 1 that option b) or c) is chosen for a given component cmp.

3. The decoder determines from the bitstream the following data:

a unique integer Nstop_cmp,

a unique x_cmp G E as well as

unique integers k_{l cmp} , k_{2 cmp} , ,cmp '_> ,cmp' with k_liCmp' < k_liCmp, l_liCmp ≤

k_2iCmp ≥ k_{2 crnp}, I .cmp .cm -

Then the decoder extends the current block ~B_{d cmp} to a larger block:

Έα,βχΐ,αηρ ^'-— {(^x> y) £ ^²: k_{l cmp} '≤ x≤ k₂,_cmp : .cmp ≤ ≤ .cmp '}- The stopping parameter Nstop_cmp can be either predefined or signaled separately or deduced from data already available to both encoder and decoder.

For example, the time discretization can be set as x_cmv - 0.25. An example of the block extension is to extend the block by one row to the top and one column to the left, i.e. to put k_{l cmp}' = k_liCmp - 1, k_{2 cmp}' - k_2:Cmp, k.cmp ' = k.cmp ~ 1. / ' — /

l2,cmp ~ ⁱ2,cmp-

4. In an optional embodiment, only integer motion vectors are send.

5. The decoder determines from the bitstream a unique extended prediction signal: pred_{ext cmp} : " a.ext.cmp ^→ ^ such that; pred_ext>cmp(x, y) = pred_cmp(x, y) V(x, y) E Έ_άΜτηρ Π S_d(Cmp.

If, for example, the block is extended by one row to the top and one column to the left, then one lets pred_{ext crnp} be equal to the already reconstructed part of the image on the top row resp. left column.

In case of motion compensated prediction, the extended part on all sides can be set equal to the samples in the reference picture for example.

6. Let

^^d, ext.cmp

= (0_< y) e _diextiCmp : x = k_liCmp' V x = k_2iCmp' V y = l_liCmp' V y = l₂,_cmp'} be the discrete boundary of the larger block ' a_, ext.cmp■

The decoder determines from the bitstream two subsets

and

( ¾,ext.C77ip)2 ^c dBd.ext.cmp with

and

An example of this decomposition is to let (dB_diextiCmp)₁ be the top row and the most left column and to let (d"B_diextiCmp)₂ be the bottom row and the most right column.

In an embodiment, the case that one part of the decomposition is empty while the other one contains everything is explicitly allowed.

7. The decoder constructs a filtered extended prediction signal - which might also be understood as a discrete image - pred_{ext cmp}: _{d ext cmp}→ E based on the kind of diffusion filter indicated by the bitstream as follows. Invoking the parameters Nstop_{cmp i} T_cmp and the sets (d'B_diext:Cmp)₁ and (d _diCXtiCmp)₂ which were determined by the decoder in the previous steps,

If the option 1 b is true, i.e. if a linear diffusion filter is to be used, the decoder defines the discrete image LDF(pred_{ext cmpi} Nstop) as will be explained in the following and if option 1c is true, i.e. if an anisotropic diffusion filter is to be used, the decoder defines the discrete image ADF(pred_{ext CT1ip}, Nstop) as will be explained in the following.

Now the decoder puts pred_{ext cmp} = LDF(pred_extxmp, Nstop), if a linear diffusion filter is to be applied. Otherwise, the decoder puts pred_{ex mp} = ADF(pred_extiCmp, Nstop), if an anisotropic filter is to be applied.

8. The decoder defines a new prediction signal (which also might be understood as a discrete image)

P edcmp- 'Bd.cmp→ K by pred_cmp(x, y) = pred_exttCmp(x, y) for all (x, y) E B_diCmp, where pred_{ext cmp} is as in the previous step.

Then the decoder replaces the prediction signal pred_cmp by pred_cmp. In the following, the two different diffusion filters will be discussed.

The discussion is based on the heat kernel. Thus, in the following the time t refers to the diffusion described by this theory.

Assume that component cmp is fixed, thus in the following the index will be dropped.

Consider B_d: = {x, y e Έ: k_t≤ x < k₂: k≤ y≤ l₂} with k k₂, l_x, i₂ 6 l and k_t < k₂, h < h-

Let im: B_d→ R be a discrete image on B_d.

In the forgoing discussion, im will be chosen as pred_ext and B_d = ^rB _iCXtlCmp ^or a fixed component cmp. If one chooses im to be the reconstructed image, it is clear that the following methods can also be used as post filters for example.

Let B: = [fc_1; fc₂] ^x ih_> hl ^{and tet} im_c: B→ R be a piecewise smooth function which in- terpolates the discrete image im, meaning that one has im_c(x, y) = im{x, y) for all x, y E B_d .

Let dB: = {(x, y) E B: x = V x = k₂ V y = l V y = l₂) be the boundary of B.

Let dB₁ and dB₂ be two piecewise connected subsets of the boundary dB such that dB = dB_t U dB₂ and such that dB₁ n dB₂ = 0.

I. In the following, a definition of the linear diffusion filter operator LDF will be given. Two different methods for the linear filtering will be discussed. 1. Linear Filtering using an iterative method

Let v: dB₂→ M² be the outer normal vector field - being orthogonal with respect to the second subset of the boundary - and let L be the differential operator given by

Let /: [0,∞) χ β→ΐ be the unique smooth function that satisfies the following conditions

Lf(t, ) = 0, V(t, ) E (0, oo) x B, and

f(Q, w) = im_c(w), Vw E B. (1 ) with f{t, ) = im_c(w), V(t, ) G (0,∞) x dB

= 0, V(t,w) G (Ο,οο) xdB_: (2).

The two definitions (2) are referred to as boundary conditions.

The first boundary condition indicates that the function f describes in the first subset of the boundary the already decoded blocks. In case of encoding, the first boundary condition also refers to already processed blocks being already encoded blocks. The second boundary condition indicates a mirror symmetry with respect to the second subset of the boundary.

For a fixed τ G (0,∞):

\elT_d: = {η*τ:ηΕ M} and

define the grid M by : = T_d x B_d and

define the discrete parts of the boundaries as dB₁ : = {(x,y) £ δΒ^.χ,γ G Z],dB_2id:= {(x,y) EdB_z:x,y E %}.

Then, we fix a discretization of the partial differential equation in (1) with corresponding boundary conditions on M. This means that we fix a discretization L_d on M of the operator

L and a discretization D_{v d} of the operator— on dB_2id.

Then we let f_d be the unique solution of this discretized equation. Thus, f_d: M→ E is the unique function which satisfies:

W_d(t,w) = o, v(t,w)e ,

f_d(0,w) = im(w), Vw G B_d. (3) with fd(t, w) = im(w), V(t, w) ET_d x dB_X , D_v,_df(t, w) = 0, V(t, ) G T_d x 5B_2id. (4)

Given a parameter Nstop E N, a "stopping time" and the input image im, we define the new image LDF_Nstop(im) as

LDF_Nstop(im)(x, y) = f_d(Nstop · τ, χ, γ).

Thus, for a fixed τ G (0,∞), a fixed stopping time Nstop and two fixed subsets dB_{l d} c s_d, d/?_{2 d} c 55_d with dfi_{l d} n 5fi₂,d = and dB _d U d5_{2 d} = 3B_d, for the discrete image im we obtain a well-defined discrete image LDF_Nstop(im): B_d→ R.

2. Linear Filtering using a sum of separable filters Consider

0 1 0\ /0 0 0\

V: = l o 0 0 and H: = 1 0 1 .

\0 1 0/ \0 0 0/

Then, L_df_d (τ, w) = 0 from the foregoing method using a finite differences discretization can be expressed as T(V + H) B_d, where + denotes componentwise addition and ° denotes the filter operation. Given a parameter Nstop G M and the input image im, we define the new image LDF_Nstop(im) as

LDF_Nstop (im) = T^Nst°P( + H)^Nst°P ° im (5) Note that in contrary to the method described before the necessary size of B_d now depends on Nstop or its size has to be extended accordingly. In an embodiment, as extension to the left and to the top of the block for example the reconstructed part of the image or in case of motion compensated prediction the corresponding samples of the reference picture can be used. In an additional or alternative embodiment, the right and bottom parts are mirrored symmetrically or taken as the corresponding samples in the reference picture in case of motion compensated prediction. The notion (-)'^Vsi:op is to be understood in the sense of convolution, i.e., for Nstop = 2, it holds that

LDF₂ (im) = T² (V + H)² ° im = T² (V * V + 2 ^■ V * H + H * H) ° im, where * denotes convolution with periodic boundary conditions.

II. A definition of the anisotropic diffusion filter operator ADF will be given as following. The notations of the previous section referring to the linear diffusion filter are kept for the following discussion of the anisotropic diffusion filter.

Now let i = Ci (x, y, t, f{x, y, t), V/(x, y, t)) be a function with values in the symmetric positive semidefinite 2 x 2-matrices.

Let J_p be the locally averaged information of C₁ with respect to some user-chosen parameter p, whereas p = 0 denotes the original matrix _x and is explicitly allowed.

Let C₂ = g Jp), where g is a function with values in the symmetric positive semidefinite 2 x 2-matrices.

Note that it is also possible to use an pre-smoothed version of V/.

Given a piecewise smooth image im: B→ IR, let im.\_Bd : B_d→ l be the discrete image,

For either C = C_t or C = C₂ , the anisotropic diffusion filter is expressed by the following partial differential equation, with the same initial condition as in (1 ) and boundary condi- tions (2): f = div(C - Vf), V(t, w) 6 (0,∞) χ β. (6) The estimate C_x is allowed to vary both temporally and spatially, but is not forced to.

A speed-up alternative of the above process is given by setting C₁{x,y,t,f x,y^'),Vf x,y)) = C_x(x,y), which determines a preferred diffusion direction according to both user-given data and image information. In doing so, the method becomes linear and thus it is possible to replace the iterative process for a fixed stopping parameter Nstop by matrix multiplication. In general, the derivation of J_p depends on the choice of C_t and the parameter p alongside the associated method of locally averaging orientational information.

As in the foregoing, for a fixed τ 6 (0,∞) we let T_d; = {n * τ: n E ] and we define the grid M and the discrete parts of the boundary dB_ld as before.

Then we fix a discretization A_d of the partial differential equation in (6) on M and a discretization D_vd of the operator— on dB_2id,

Then we let f_d: M→ R be the unique solution of the following discretized equation

A ait.w = 0, V(t,w) e M,

fa(0_tw) = im(w), V G B_d. with boundary conditions fa (t, w) = im(w), V(t, w) E T_d x dB_{l dl}

D_v,df(t, w) = 0, V(t, w) G¾ x dB₂ . Given a parameter Nstop £ M, a "stopping time" and the input image im, we define the new image ADF_Nstop(im) as

ADF_Nstop(im)(x,y) = f_d(Nstop^■ τ,χ,γ). One way of defining matrix C is to calculate the tensor product

C₁ = Vf(x, y₎ t)Vf(x, y, tf.

Averaging this can be accomplished by convolving C₁ componentwise with a Gaussian Kp, which gives the structure tensor

Jp- = K_p * (VfVf^T), p > 0.

The symmetric matrix

is positive semidefinite and possesses orthonormal eigenvectors v_t, v₂-

Let g_x (0,∞)→ E be a scalar function, for example

s²

for a fixed parameter μ > 0.

Consider the diagonalization of matrix J_p

where matrix S = (v_t v₂) consists of the eigenvectors of J_p The eigenvectors v_t, v₂ fulfill

+ _Λ/θΊι - ₂2)² + 4Α² ₂

and v_x 1 v₂.

The corresponding eigenvalues λ and λ₂ are given by

where the + sign belongs to λ_χ.

Then, one can define

The function f is in one embodiment - valid for both kinds of differential filter - an infinite series of a convolution of the heat kernel with C, at time t. For a more complete understanding of the present disclosure, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

Fig. 1 shows a schematic embodiment of the apparatus for decoding a picture,

Fig. 2 illustrates the relationship of blocks to be considered during encoding and decoding,

Fig. 3 illustrates the sequence of the different prediction signals and Fig. 4 shows a schematic embodiment of the apparatus for encoding a picture.

Fig. 1 shows an embodiment of an apparatus 10 for a block-based predictive decoding of a picture based on a data stream 100. An extractor 11 extracts from the data stream 100 filter information 20. The filter information 20 indicates whether a diffusion filter is to be used for the reconstruction process or not. The filter information also indicates in case of more than one suitable diffusion filter which kind of diffusion filter is to be used. This is either a linear diffusion filter or an anisotropic diffusion filter. An initial prediction signal provider 12 provides an initial prediction signal 21 for a current block (see Fig. 2). As the decoding is performed block-based, all steps performed by the apparatus 10 are performed for all blocks.

The initial prediction signal 21 is based on the kind of standard used for the encoding of the picture. Hence, the initial prediction signal 21 is used for the reconstruction if the filter information 20 is negative, i.e. if no diffusion filter is to be applied.

A prediction signal modifier 13 receives the initial prediction signal 21 and provides in case of a positive filter information 20 a modified prediction signal 23. The modified pre- diction signal 23 is based on the initial prediction signal 21 and based on a kind of diffusion filter: LDF or ADF indicated by the data stream 100.

Finally, a reconstructor 14 decodes a reconstructed version of the current block 22 based on data from the data stream 100 and in case of a positive filter information 20 based on the modified prediction signal 23. Hence, the reconstructor 14 performs a reconstruction using either the initial prediction signal 21 - in case no diffusion filter is to be applied - or the modified prediction signal 23 - in case a diffusion filter is to be applied - and using the necessary data, e.g. the corresponding residual signal, provided by the data stream 100. The application of the diffusion filter performed by the prediction signal modifier 13 is based on the following steps:

1. The current block is enlarged to a larger block. 2. The initial prediction signal is adapted to the larger block.

3. For the extended prediction signal a differential equation of a filter function f is solved or approximated. The kind of differential equation depends on the kind of diffusion filter to be applied. Further, for two subsets of the boundary of the larger block, the filter function f has to fulfill two conditions: At first, for a first subset, the filter function describes blocks already reconstructed. At second, at a second subset, mirror symmetry is given which is described by the condition that the derivative of the filter function f with respect to the outer normal vec- tor field vanishes for the second subset.

The differential equation is solved in a discrete way with a number of iterations giving an intensity of the application of the diffusion filter. The third step leads to a filtered extended prediction signal.

4. The filtered extended prediction signal is reduced to the current block and is used for the reconstruction of the picture.

Fig. 2 illustrates a current block 22 that is either to be encoded by the encoder or to be reconstructed by the decoder.

For the calculation of the modified prediction signal the current block 22 is enlarged to a larger block 24. The larger block 24 comprises the current block 22 as well as blocks 30 already processed before the current block 22. The processing of the other blocks 30 re- fers either to the already encoded blocks or to the already reconstructed blocks depending on whether an encoding or a decoding is considered.

The larger block 24 comprises a boundary 25 which comprises in the shown embodiment two disjoint subsets: a first subset 26 and a second subset 27. For each subset 26, 27 a boundary condition is defined which has to be fulfilled by a filter function used for the application of the respective diffusion filter.

For the adaptation of the initial prediction signal referring to the current block 22 to the larger block 24, already processed blocks 30 and/or a reference picture 31 is used.

The fig. 3 shows the sequence of the prediction signals.

The initial prediction signal 21 is adapted to describe the larger block. To the obtained extended prediction signal 28 the diffusion filter is applied leading to the filtered extended prediction signal 29 that is afterwards reduced to the current block. The modified prediction signal 23 is finally used for the reconstruction of the current block or is used to con- sider whether the application of the diffusion filter is useful for the encoding of the current block.

Fig. 4 illustrates an apparatus 50 for block-based predictive encoding of a picture 200 and for providing a data stream 100.

An initial prediction signal provider 51 provides an initial prediction signal 21 for each current block 22. The initial prediction signal 21 is given to the prediction signal modifier 52 that provides a modified prediction signal 23 whereas the modification is based on the initial prediction signal 21 and on a diffusion filter.

The initial prediction signal 21 is also submitted to a comparator 53 that compares an effect of the initial prediction signal 21 with an effect of the modified prediction signal 23. The effects refer preferably to the encoding of the picture 200. The comparison is, for ex- ample, performed by calculating the respective rate distortion costs for the initial prediction signal 21 and the modified prediction signal 23. In an embodiment, at least two different diffusion filters - e.g. linear diffusion filter, LDF, and anisotropic diffusion filter, ADF - are used for the modification and are compared accordingly by the comparator 53. Based on the comparison, the comparator 53 provides a comparison result 54 indicating whether a diffusion filter is to be used and/or which diffusion filter is to be used.

The following encoder 55 encodes the current block and uses depending on the comparison result 54 either the initial prediction signal 21 or the modified prediction signal 23.

The encoder 55 inserts into the data stream 100 based on the picture 200 filter information that informs the decoder whether a diffusion filter is to be used and/or which diffusion filter is to be used. Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.

Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software or at least partially in hardware or at least partially in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier. In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non- transitory.

A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods de- scribed herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.

A further embodiment comprises a processing means, for example a computer, or a pro- grammable logic device, configured to or adapted to perform one of the methods described herein.

A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or sys- tern may, for example, comprise a file server for transferring the computer program to the receiver.

In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods de- scribed herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

Claims

1 . Apparatus (10) for block-based predictive decoding of a picture (200), comprising: an extractor (1 1 ) configured to extract from a data stream (100) filter information (20) about applying a diffusion filter (LDF, ADF),

an initial prediction signal provider (12) configured to provide an initial prediction signal (21 ) for a current block (22),

a prediction signal modifier (13) configured to provide in case of a positive filter information (20) a modified prediction signal (23) based on the initial prediction sig- nal (21 ) and based on the diffusion filter (LDF, ADF), and

a reconstructor (14) configured to decode a reconstructed version of the current block (22) based on the initial prediction signal (21 ) in case of a negative filter information (20) and based on the modified prediction signal (23) in case of a positive filter information (20).

2. Apparatus ( 0) according to claim ,

wherein the prediction signal modifier (13) is configured to provide data of an extension of the current block (22) to a larger block (24) comprising the current block (22) and a boundary (25),

wherein the prediction signal modifier ( 3) is configured to provide an extended prediction signal (28) for the larger block (24) based on the initial prediction signal (21 ),

wherein the prediction signal modifier (13) is configured to provide a filtered extended prediction signal (29) based on the diffusion filter (LDF, ADF) and based on a filter function (f) fulfilling two boundary conditions referring to the boundary (25) of the larger block (24), and

wherein the prediction signal modifier (13) is configured to provide the modified prediction signal (23) based on the filtered extended prediction signal (29). 3. Apparatus (10) according to claim 2,

wherein the prediction signal modifier (13) is configured to provide the data of the extension of the current block (22) such that the larger block (24) comprises already reconstructed blocks (30). 4. Apparatus (10) according to claim 2 or 3, wherein the prediction signal modifier (13) is configured to provide the extended prediction signal (28) based on already reconstructed blocks (30) and/or based on at least one reference picture (31 ).

Apparatus (10) according to any of claims 2 to 4,

wherein the two boundary conditions refer to two disjoint subsets (26, 27) belonging to the boundary (25) of the larger block (24),

wherein a first boundary condition indicates that the filter function (f) for a first subset (26) of the two subsets (26, 27) describes already reconstructed blocks (30), and

wherein a second boundary condition indicates that a derivative of the filter function (f) with respect to the outer normal vector field vanishes for a second subset (26) of the two subsets (26, 27).

Apparatus (10) according to any of claims 2 to 5,

wherein the prediction signal modifier (13) is configured to provide the filtered extended prediction signal (29) based on a stopping parameter indicating an intensity of the diffusion filter.

Apparatus (10) according to claim 6,

wherein the stopping parameter is comprised by the data stream (100) or wherein the stopping parameter is known to the apparatus (1 ) or

wherein the stopping parameter is derived from an already reconstructed block or the prediction signal (30).

Apparatus (10) according to any of claims 2 to 7,

wherein the prediction signal modifier (13) is configured to provide the filtered extended prediction signal (29) based on the diffusion filter (LDF, ADF) indicating a kind of differential equation to be solved by the filter function (f).

Apparatus (10) according to any of claims 2 to 9,

wherein the prediction signal modifier (13) is configured to provide the modified prediction signal (23) based on the filtered extended prediction signal (29) by limiting the modified prediction signal (23) to the current block (22).

Apparatus (10) according to any of claims 1 to 9, wherein in case of a positive filter information (20) only integer motion vectors are used for decoding.

Apparatus (10) according to any of claims 1 to 10,

wherein the prediction signal modifier (13) is configured to provide the modified prediction signal (23) in case of a positive filter information (20) and in case the data stream (100) fails to indicate a skip mode for the current block (22),

or

wherein the prediction signal modifier (13) is configured to provide the modified prediction signal (23) in case of a positive filter information (20) and in case the data stream ( 00) indicates a skip mode for the current block (22).

Apparatus (10) according to any of claims 1 to 11 ,

wherein the prediction signal modifier (13) is configured to perform a merging for the modified prediction signal (23) in case of a positive filter information (20) and in case the data stream (100) indicates a merge mode for the current block (22), or

wherein the prediction signal modifier (13) is configured to provide the modified prediction signal (23) in case of a positive filter information (20) and in case the data stream (100) indicates a merge mode for the current block (22).

Apparatus (10) according to any of claims 1 to 12,

wherein the prediction signal modifier (13) is configured to provide the modified prediction signal (23) in case of a positive filter information (20) and in case the data stream (100) fails to indicate an application of an adaptive loop filter for the current block (22),

or

wherein the prediction signal modifier (13) is configured to provide the modified prediction signal (23) in case of a positive filter information (20) and in case the data stream (100) indicates an application of an adaptive loop filter for the current block (22).

Apparatus (10) according to any of claims 1 to 13,

wherein the extractor (11) is configured to parse the filter information from the data stream if an integer motion vector, IMV, flag is present in the data stream for the current block and the IMV flag is positive, wherein the filter information (20) is in- ferred to be negative in case of the IMV flag being present in the data stream for the current block and being negative.

Apparatus (10) according to claim 14,

wherein the extractor is configured to parse the filter information from the data stream also if an IMV modus is not present in the data stream for the current block.

Apparatus (10) according to any of claims 1 to 14,

wherein the extractor is configured to parse the filter information from the data stream in case of the current block being an intra predicted block only if a coded block flag present in the data stream for the current block does not signal a zeron- ess of a prediction residual of the current block.

Apparatus (10) according to any of claims 1 to 14,

wherein the extractor is configured to parse the filter information from the data stream in case of the current block being an inter predicted block only if a coded block flag present in the data stream for a tree root block of the current block does not signal a zeroness of a prediction residual of the tree root block.

Apparatus (10) according to any of claims 1 to 14,

wherein the apparatus is configured to parse, in case of the current block being an intra predicted block, a coded block flag for the current block from the data stream only if the filter information is negative.

Apparatus (10) according to any of claims 1 to 14,

wherein the apparatus is configured to parse, in case of the current block being an inter predicted block, a coded block flag for a tree root block of the current block, from the data stream only if the filter information is negative.

Apparatus (50) for block-based predictive encoding of a picture (200), comprising: an initial prediction signal provider (51) configured to provide an initial prediction signal (21) for a current block (22),

a prediction signal modifier (52) configured to provide a modified prediction signal (23) based on the initial prediction signal (21) and based on a diffusion filter (LDF,

ADF), a comparator (53) configured to compare an effect of the initial prediction signal

(21 ) with an effect of the modified prediction signal (23) and to generate a comparison result (54), and

an encoder (55) configured to provide a data stream (100) based on the picture (200) and based on the comparison result (54),

wherein the encoder (55) is configured to insert based on the comparison result (54) filter information (20) into the data stream (100),

where the filter information (20) is indicating whether a diffusion filter is to be applied for decoding the current block (22), and/or

where the filter information (20) is indicating which diffusion filter (LDF, ADF) is to be applied for decoding the current block (22).

21. Apparatus (50) according to claim 14,

wherein the prediction signal modifier (52) is configured to provide data of an ex- tension of the current block (22) to a larger block (24) comprising the current block

(22) and a boundary (25),

wherein the prediction signal modifier (52) is configured to provide an extended prediction signal (28) for the larger block (24) based on the initial prediction signal

(21 ),

wherein the prediction signal modifier (52) is configured to provide a filtered extended prediction signal (29) based on the diffusion filter (LDF, ADF) and based on a filter function (f) fulfilling two boundary conditions referring to the boundary (25) of the larger block (24), and

wherein the prediction signal modifier (52) is configured to provide the modified prediction signal (23) based on the filtered extended prediction signal (29).

22. Apparatus (50) according to claim 15,

wherein a first boundary condition indicates that the filter function (f) for a first subset (26) of the two subsets (26, 27) describes already encoded blocks (30), and wherein a second boundary condition indicates that a derivative of the filter function (f) with respect to the outer normal vector field vanishes for a second subset (26) of the two subsets (26, 27).

23. Apparatus (50) according to claim 15 or 16, wherein the prediction signal modifier (52) is configured to provide the filtered extended prediction signal (29) based on a stopping parameter indicating an intensity of the diffusion filter.

Apparatus (50) according to any of claims 15 to 17,

wherein the prediction signal modifier (52) is configured to provide the filtered extended prediction signal (29) based on the diffusion filter (LDF, ADF) indicating a kind of differential equation to be solved by the filter function (f).

Apparatus (50) according to any of claims 15 to 18,

wherein the prediction signal modifier (52) is configured to provide the modified prediction signal (23) based on the filtered extended prediction signal (29) by limiting the modified prediction signal (23) to the current block (22).

Apparatus (50) according to any of claims 14 to 19,

wherein in case of a positive filter information (20) only integer motion vectors are used for encoding.

Method for block-based predictive decoding of a picture (200), comprising:

extracting from a data stream (100) filter information (20) about applying a diffusion filter (LDF, ADF),

providing an initial prediction signal (21 ) for a current block (22) and an extended prediction signal (28) based on the initial prediction signal (21),

providing in case of a positive filter information (20) a modified prediction signal (23) based on the initial prediction signal (21), the extended prediction signal (28) and the diffusion filter (LDF, ADF), and

decoding a reconstructed version of the current block (22) based on the modified prediction signal (23) in case of a positive filter information (20).

Method for block-based predictive encoding of a picture (200), comprising:

providing an initial prediction signal (21) for a current block (22) of the picture and an extended prediction signal (28) based on the initial prediction signal (21 ), providing a modified prediction signal (23) based on the initial prediction signal (21 ), the extended prediction signal (28) and a diffusion filter (LDF, ADF), comparing an effect of the initial prediction signal (21 ) with an effect of the modified prediction signal (23) and generating a comparison result (54), and providing a data stream (100) based on the picture (200) and based on the comparison result (54), and

inserting based on the comparison result (54) filter information (20) into the data stream (100) indicating whether a diffusion filter is to be applied for decoding the current block (22) and/or indicating which diffusion filter (LDF, ADF) is to be applied.

29. Computer program comprising a program code for performing, when running on a computer, the method according to claim 27 or 28.

30. Data stream (100) having a picture (200) encoded thereinto, the data stream (100) being generated by a method according to claim 28.