Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
  • H04N19/96—Tree coding, e.g. quad-tree coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

• Embodiments of the present invention refer to a decoder and a method for decoding as well as to an encoder and the corresponding method for encoding pictures.
• the decoding and encoding is performed using partitioning.
• a so-called bi-partitioning is applied to split a block into sub-blocks.
• a partitioning information is used to perform the bi-partitioning for sub- blocks.
• the partitioning of sub-blocks is performed based on syntax elements decoded by context-adaptively entropy coding.
• the partitioning is performed by an apparatus or method supporting at least two modes.
• the partitioning is performed based on a reduced list of split ratios, wherein this fifth aspect especially refers to the reduction process.
• the pictures of a video sequence are coded in a certain order, which is referred to as coding order.
• the coding order of pictures can differ from the capture and display order.
• each video picture is partitioned into blocks.
• a block comprises the samples of a rectangular area of a particular color component.
• the entity of the blocks of all color components that correspond to the same rectangular picture area, together with the associated syntax, is often referred to as unit.
• CTUs coding tree units
• CUs coding units
• PUs prediction units
• TUs transform units
• the associated sample blocks are referred to as coding tree blocks (CTBs), coding blocks (CBs), prediction blocks (PBs), and transform blocks (TBs), respectively.
• a video picture is initially partitioned into fixed sized units (i.e., aligned fixed sized blocks for all color components).
• these units are referred to as coding tree units (CTUs).
• Their size (in luma samples) is transmitted in the sequence parameter set (SPS), which contains high-level syntax elements that are valid for an entire coding video sequence (which consists of an instantaneous decoding refresh (IDR) picture and all following pictures up to (but excluding) the next IDR picture).
• SPS sequence parameter set
• the CTU size can be chosen between 64x64, 32x32, and 16x16 luma samples.
• CTUs coding tree blocks
• the fixed sized CTUs can be adaptively split into smaller units and blocks.
• a picture can be comprised of one or multiple slices.
• a slice represents a collection of CTUs.
• a slice is associated with a slice type (e.g., I, P, or B slice) which determines the set of supported coding modes.
• a slice type e.g., I, P, or B slice
• several types of prediction such as motion vector prediction and intra-picture prediction are only supported between blocks inside a slice.
• the partitioning into coding unit and transform units can be performed in a plurality of ways. For example, within the prior art the following partitioning approaches are described.
• the quadtree splitting is very simple, it has the disadvantage that often an object or motion boundary can only be reasonably approximated by splitting a large block into many smaller blocks. This potentially increases the bit rate for signaling the prediction parameters (such as motion parameters) as well as the bit rate for the transform coding of the prediction error signals.
• the Joint Exploration Test Model [3] specifies a combined quadtree/binary tree (QTBT) approach.
• QTBT quadtree/binary tree
• prediction units PUs
• TUs transform units
• a CTU is first split using the conventional quadtree approach.
• the quadtree leaf nodes can be further partitioned by a binary tree (BT) structure.
• a horizontal split which horizontally splits a NxM block into two Nx(M/2) blocks
• a vertical split which vertically splits a NxN block into two (N/2)xM blocks.
• the binary splitting can be continued until a minimum block width/block height is reached or a maximum number of (binary) splits were executed.
• the resulting blocks are used for coding mode signaling, intra-picture prediction, inter-picture prediction, and transform coding.
• a further extension of the QTBT concept is suggested in [4], where two further splitting modes were introduced for the binary tree part: A horizontal triple split and a vertical triple split.
• an NxM block is split into three blocks: an Nx(M/4), an Nx( /2), and an Nx(M/4) block (from top to bottom).
• the vertical triple split vertically partitions an NxM block into an (N/4)xM, an (N/2)xM, and an (N/4)xM block (from left to right / from first to second).
• picture partitioning into blocks is rendered more efficient in terms of signaling overhead from encoder to decoder, by using multi-level binary split partitioning and signaling this multi-level split partitioning in a manner so that a split direction of a binary split of a certain split level for a certain current block is set in a manner dependent on the split direction used for the parent block of the preceding split level the current block has been derived from by a binary split of the preceding split level.
• the setting may be dependent on both, the partitioning information and the parent block's split direction.
• the partition information may signal the split direction relative to the parent block's split direction which, in turn, might also have been signaled by the partitioning information - relative to the split direction of the grandfather block or absolutely (independent therefrom).
• Embodiments of this aspect provide a block-based decoder 10 for decoding a picture 12 from a data stream 14 in units of coding blocks 16, configured to partition a picture block 18 of the picture 12 into the coding blocks 16 by recursive bi- partitioning with controlling the recursive bi-partitioning according to partition information in the data stream 14, set a first split line direction of a first split line (e.g. line between CUl and CU2 in top left quarter of Fig. 1) along which a predetermined sub-block (top-left quarter in Fig.
• a first split line e.g. line between CUl and CU2 in top left quarter of Fig.
• a predetermined split level (level 2 with level 0 assigned to block 16) of the picture block 16 is split by the recursive bi-partitioning into a pair of child sub-blocks (CU1 and CU2) of a succeeding split level (level 3), depending on a second split line direction of a second split line (line separating upper left and lower left quarter of Fig. 1) along which a grandfather sub-block (left hand half of Fig. 1; note that the term “grandfather sub-biock shall be understood as being broad enough to also cover the picture block 18 itself) of a preceding split level (level 1) of the picture block 16 is split into the predetermined sub-block (upper left quarter) and another sub-block (lower left quarter).
• the block-based decoder may be configured to support a plurality of coding modes (e.g. inter (e.g. temporal) or intra (e.g. spatial) prediction) and derive from the data stream, in units of the coding blocks, a coding mode to be used for decoding the picture from the data stream (cp. mode selector in Fig. 15).
• a coding mode e.g. inter (e.g. temporal) or intra (e.g. spatial) prediction
• the block-based decoder may be configured to support a plurality of coding modes, wherein each coding mode has associated therewith a coding-mode-specific set of one or more coding parameters (e.g. motion vector and, optionally, reference picture index for inter prediction and intra prediction mode, e.g.
• coding parameters e.g. motion vector and, optionally, reference picture index for inter prediction and intra prediction mode, e.g.
• the block- based decoder is configured to derive from the data stream, in units of the coding blocks, the coding mode specific set of one or more coding parameters for decoding the picture from the data stream by deriving from the data stream for each decoding unit the coding-mode-specific set of one or more coding parameters associated with the coding mode signaled to be used for the respective coding block 16 in the data stream (cp. predictors in Fig. 15).
• the block-based decoder may be a predictive decoder and configured to derive from the data stream a prediction residual of the picture in a spectral domain (cp. transform blocks in Fig. 15) in form of spectral transforms representing a piece-wise spectral transform of the prediction residual in units of the coding blocks 16 (e.g. for each coding block 16 the prediction residual would be spectrally decomposed separately at encoder side so as to obtain one transform, e.g. DCT, for one coding unit, and the inverse transform would be used to correct the prediction signal at the area corresponding to the coding block).
• a spectral domain e.g. transform blocks in Fig. 15
• spectral transforms representing a piece-wise spectral transform of the prediction residual in units of the coding blocks 16
• the block-based decoder may be configured to pre-partition the picture 12 into an array of tree root blocks 18 and perform the partitioning of the picture block 18 with respect to each of the array of tree root blocks 18 (as exemplarily depicted in Fig.
• the picture 12 into an array of tree root blocks 18, subject, depending on the partition information, the tree root blocks to a separate partitioning (such as quad-partitioning separately signallable for each tree root block 18, possibly but not exclusively recursively) so as to subdivide the array of the tree root blocks into pre-partition leaf blocks (of possibly different size within the same tree root block 18) and perform the partitioning of the picture block with respect to each of the pre-partition leaf blocks (this alternative is not depicted in Fig. 14; here, each block 18 in Fig.
• the block-based decoder may be configured to set the first split line direction depending on the second split line direction and the partition information (e.g. perpend_split_flag or parallel_split_flag) by modifying (e.g. selecting between adopting the same, or taking the perpendicular direction) the first split line direction according to the partition information to obtain a modified split line direction and setting the second split line direction to be equal to the modified split line direction.
• the partition information e.g. perpend_split_flag or parallel_split_flag
• the block-based decoder may be configured to control the recursive bi- partitioning depending on a binary decision tree (e.g. 20 in Fig. 13; here each node of the tree 20 includes exemplarily more than just a decision of split or no-split, but also indicates split line direction plus split ratio) signaled by the partition information (e.g. perpend_split_flag or parallei_split_flag, and split_ratio_...
• a binary decision tree e.g. 20 in Fig. 13; here each node of the tree 20 includes exemplarily more than just a decision of split or no-split, but also indicates split line direction plus split ratio
• the partition information e.g. perpend_split_flag or parallei_split_flag, and split_ratio_...
• the block-based decoder may be configured to control the recursive bi- partitioning for each sub-block of the picture block belonging to a first predetermined set of split levels (e.g. maybe all split levels, or maybe just for a subset thereof) depending on the partition information (e.g. by split_ratio_%) with respect to a location of a split line along which the respective sub-block is split by the recursive bi-partitioning, transversely to a split line direction of the split line (... in other words, with respect to the split ratio measured transversely to the split line direction).
• a first predetermined set of split levels e.g. maybe all split levels, or maybe just for a subset thereof
• the block-based decoder may be configured to control the recursive bi- partitioning for each sub-block of the picture block belonging to a first predetermined set of split levels (e.g. maybe all split levels, or maybe just for a subset thereof) depending on the partition information (e.g. perpend_split_flag or parallel_split_flag, and split_ratio_... flags) with respect to a split line direction of a split line along which the respective sub-block is split by the recursive bi-partitioning depending on the partition information and a split line direction of a split line along which a parent block of the respective intermediate block is split.
• a first predetermined set of split levels e.g. maybe all split levels, or maybe just for a subset thereof
• the partition information e.g. perpend_split_flag or parallel_split_flag, and split_ratio_... flags
• the second split line direction is horizontal or vertical, and the block-based decoder is configured to set the first split line direction depending on the partition information to be perpendicular or parallel to the second split line direction.
• the block-based decoder may be configured to, in partitioning the picture block, depending on the partition information, decide to split the picture block into a pair of first-split- level sub-blocks, or leave the picture block unsplit, and if the picture block of the picture is split into the pair of first-split-level sub-blocks, depending on the partition information, split a first first-split- level sub-block of the pair of first-split-level sub-blocks into a pair of second-split-level sub-blocks, or leave the first first-split-level sub-block unsplit, and to perform the splitting of the picture block along a third split line having a third split line direction, and setting a fourth split line direction of a fourth split line along which the first first-split-level sub-block is split depending on the third split iine direction.
• the block-based decoder may be configured to, if the picture block of the picture is split into the pair of first-split-level sub-blocks, set the third split direction depending on the partition information (i.e. for the first split the direction is signallable), to derive from the data stream a first split flag obtained by the partition information, and if the first split flag has a first flag state, to split the first first-split-level sub-block into the pair of second-split-level sub-blocks along the second split line with setting the second split direction to be perpendicular to the first split direction, and if the first split flag has a second flag state, leaving the first first-split-level sub-block unsplit (e.g.
• the block-based decoder may, if the picture block of the picture is split into the pair of first-split-level sub-blocks, derive from the data stream a first split flag (e.g. perpend_split_flag) contained by the partition information, and if the first split flag has a first flag state, split the first first- split-level sub-block into the pair of second-split-level sub-blocks along the fourth split line with setting the fourth split direction to be perpendicular to the third split direction, and if the first split flag has a second flag state, derive from the data stream a second split flag (e.g.
• a first split flag e.g. perpend_split_flag
• the block-based decoder may be configured to, in partitioning the picture block, depending on the partition information, decide to split the grandfather sub-block into a pair of sub-blocks including the predetermined sub-block, or to leave the grandfather sub-block unsplit, and if the grandfather sub-block is split into the pair of sub-blocks, depending on the partition information, to split the predetermined sub-block of the pair of sub-blocks into a pair of child sub-blocks, or to leave the predetermined sub-block unsplit, and to perform the splitting of the grandfather sub-block along the second split line having the second split line direction, and setting the first split line direction of the first split line along which the predetermined sub-block is split depending on the second split line direction.
• the block-based decoder may be configured to derive from the data stream a first split flag (e.g. perpend_split_flag) contained by the partition information, and if the first split flag has a first flag state, split the predetermined sub-block (upper left quarter of Fig. 1) into the pair of child sub-blocks (CU1 and CU2) along the first split line with setting the first split line direction to be perpendicular to the second split line direction (separating upper left and lower left quarters in Fig 18), and if the first split flag has a second flag state, to derive from the data stream a second split flag (e.g.
• a first split flag e.g. perpend_split_flag
• the block-based decoder may derive from the data stream a third split flag (e.g. perpend_spiit_flag) contained by the partition information, and if the third split flag has the first flag state, split the other sub-block (lower left quarter of Fig.
• the block-based decoder may be configured to, if the predetermined sub- block is split into the pair of child sub-blocks, derive from the data stream a sixth split flag (e.g. perpend_split_flag) contained by the partition information, and if the sixth split flag has the first flag state, split a first child sub-block (e.g. CUl) of the pair of child sub-blocks into a pair of grandchild sub- blocks (not depicted in Fig. 1) along a seventh split line with setting a seventh split line direction of the seventh split line to be perpendicular to the first split line direction, and if the sixth split flag has the second flag state, derive from the data stream a seventh split flag (e.g.
• the block-based decoder may be configured to, if the predetermined sub- block is split into the pair of child sub-blocks (perpendicularly or in parallel to second split line direction), derive a split ratio (split_ratio_%) from the partition information, indicating at which ratio the predetermined sub-block's width perpendicular to the first split line direction is split.
• the block-based decoder may support a set (e.g.
• the block-based decoder may support a further set of supported split ratios for splitting the predetermined sub-block along the first split line being parallel the second split line direction and be configured to exclude supported split ratios from the further set of supported split ratios according to one or more rules to obtain a further reduced set of supported split ratios, and to infer that the second split flag has the second flag state and skip its derivation if a cardinality of the further reduced set is zero.
• the one of the one or more rules may exclude supported split ratios depending on the width of the predetermined sub-block perpendicular to the first split line.
• the one of the one or more rules may exclude supported split ratios depending on the width of the predetermined sub-block perpendicular to the first split line in a manner so that supported split ratios are excluded which divide the width into two child cub-blocks' widths not being, (measured) in samples, an integer multiple of a predetermined integer (which may be 4, 8 or any other).
• the one of the one or more rules may exclude supported split ratios depending on the width of the predetermined sub-block perpendicular to the first split line in a manner so that supported split ratios are excluded which divide the predetermined sub-block into two child sub-blocks of which at least one is too small to fully enclose (differently speaking not inscribable without crossing the child sub-block's borders, whereas loose inscription would be allowed) at least one of a set of at least one predetermined minimum size blocks (e.g. for example, two such as 8x4 and 4x8).
• a set of at least one predetermined minimum size blocks e.g. for example, two such as 8x4 and 4x8.
• the one of the one or more rules may exclude supported split ratios depending on the width of the predetermined sub-block perpendicular to the first split line in a manner so that supported split ratios are excluded which divide the predetermined sub-block into two child sub-blocks of which at least one is of an aspect ratio (e.g. horizontal width to vertical width, shorter to longer width, vertical to horizontal width) outside an admissible aspect ratio range (e.g. aspect ratio are excluded which themselves or its reciprocal value exceeds a certain threshold).
• an aspect ratio e.g. horizontal width to vertical width, shorter to longer width, vertical to horizontal width
• an admissible aspect ratio range e.g. aspect ratio are excluded which themselves or its reciprocal value exceeds a certain threshold.
• the one of the one or more rules may excludes supported split ratios depending on the width of the predetermined sub-block perpendicular to the first split line in a manner so that a predetermined subset of supported split ratios is excluded which depends on the split level of the recursive bi- partitioning the predetermined sub-block belongs to (in other words, set A may be pre-reduced according to split-level of current sub-block).
• the one of the one or more rules excludes supported split ratios depending on a further split direction and a further split ratio of a further split line of one or more further sub-blocks (e.g. block corresponding to node A in Fig.
• Another embodiment provides a method for decoding a picture 12 from a data stream 14 in units of coding blocks 16, comprising: partitioning a picture block 18 of the picture 12 into the coding blocks 16 by recursive bi- partitioning with controlling the recursive bi-partitioning according to partition information in the data stream 14,
• a first split line direction of a first split line e.g. line between CU1 and CU2 in top left quarter of Fig. 1 along which a predetermined sub-block (top-left quarter in Fig. 1) of a predetermined split level (level 2 with level 0 assigned to block 16) of the picture block 16 is split by the recursive bi-partitioning into a pair of child sub-blocks (CU1 and CU2) of a succeeding split level (level 3), depending on a second split line direction of a second split line (line separating upper left and lower left quarter of Fig. 1) along which a grandfather sub-block (left hand half of Fig.
• Another embodiment provides a block-based encoder for encoding a picture 12 into a data stream 14 in units of coding blocks 16, configured to partition a picture block 18 of the picture 12 into the coding blocks 16 by recursive bi-partitioning with controlling (e.g. selecting the partition information in a manner taking rate/distortion optimization into account) the recursive bi-partitioning according to partition information and inserting the partition information into the data stream 14, to set a first split line direction of a first split line (e.g. line between CU1 and CU2 in top left quarter of Fig. 1) along which a predetermined sub-block (top-left quarter in Fig.
• a first split line direction of a first split line e.g. line between CU1 and CU2 in top left quarter of Fig. 1
• a predetermined sub-block top-left quarter in Fig.
• a predetermined split level (level 2 with level 0 assigned to block 16) of the picture block 16 is split by the recursive bi-partitioning into a pair of child sub-blocks (CU1 and CU2) of a succeeding split level (level 3), depending on a second split line direction of a second split line (line separating upper left and lower left quarter of Fig. 1) along which a grandfather sub-block (left hand half of Fig. 1; note that the term “grandfather sub-block shall be understood as being broad enough to also cover the picture block 18 itself) of a preceding split level (level 1) of the picture block 16 is split into the predetermined sub-block (upper left quarter) and another sub-block (lower left quarter).
• a further embodiment provides the respective method for encoding a picture 12 into a data stream 14 in units of coding blocks 16, comprising: partitioning a picture block 18 of the picture 12 into the coding blocks 16 by recursive bi- partitioning with controlling (e.g. selecting the partition information in a manner taking rate/distortion optimization into account) the recursive bi-partitioning according to partition information and inserting the partition information into the data stream 14,
• a first split line direction of a first split line e.g. line between CU1 and CU2 in top left quarter of Fig. 1 along which a predetermined sub-block (top-left quarter in Fig. 1) of a predetermined split level (level 2 with level 0 assigned to block 16) of the picture block 16 is split by the recursive bi-partitioning into a pair of child sub-blocks (CU1 and CU2) of a succeeding split level (level 3), depending on a second split line direction of a second split line (line separating upper left and lower left quarter of Fig. 1) along which a grandfather sub-block (left hand half of Fig.
• picture partitioning into blocks is rendered more efficient by allowing for an election out of a set of supported split ratios.
• the supported set may be reduced to a reduced set according to one or more rules in order to remove supported split ratios not being allowed or reasonable according to thoughts, thereby reducing the amount of partitioning information necessary.
• Embodiments of this aspect 2 provide a block-based decoder 10 for decoding a picture 12 from a data stream 14 in units of coding blocks 16, which is configured to partition a picture block 18 of the picture 12 into the coding blocks 16 by recursive bi-partitioning with controlling the recursive bi-partitioning according to partition information in the data stream 14, wherein the controlling the recursive bi- partitioning includes controlling the bi-partitioning with respect to a predetermined sub-block (e.g. any of the blocks in Fig. 1 corresponding to any node in Fig. 2, thus including block 18 itself, but also fraction thereof obtained by splits) of the picture block 18 in terms of whether the predetermined sub-block is to be split or to be left unsplit,
• a predetermined sub-block e.g. any of the blocks in Fig. 1 corresponding to any node in Fig. 2, thus including block 18 itself, but also fraction thereof obtained by splits
• split line direction e.g. horizontal or vertical
• the block-based decoder supports a set (e.g. set A) of supported split ratios for splitting the predetermined sub-block and is configured to exclude, for a split of the predetermined sub-block along a split line having a predetermined split line direction, supported split ratios from the set of supported split ratios according to one or more rules to obtain a reduced set of supported split ratios, wherein the block-based decoder is configured to control the bi-partitioning with respect to the predetermined sub-block of the picture block depending on the reduced set of supported split ratios.
• set A e.g. set A
• the decoder may be configured to reduce a fraction of the partition information, used for controlling the recursive bi-partitioning with respect to the predetermined sub- block of the picture biock, depending on a cardinality of the reduced set (e.g. in terms of number of syntax elements contained by, i.e. read from, the partition information for the respective sub-block, or in terms of mean number of bits in bitstream domain such as after entropy coding; see also next paragraph for examples).
• the decoder may be configured to skip, in parsing the fraction of the partition information, a flag (e.g.
• perpend_split_flag or parallel_split_fiag or ver_split_flag or hor_split_flag indicating whether the predetermined sub-block is to be split or to be left unsplit, or a flag (e.g. perpend_split_flag or parailel_split_flag or ver_split_flag or hor_split_flag) indicating the split direction of the split line if the reduced set is empty, and/or to skip, in parsing the fraction of the partition information, an index (e.g.
• split_ratio_ a set of flags such as split_ratio_(8) indicating the split ratio of the split line if the cardinality of the reduced set is one, and/or to change, in entropy decoding the index, a domain of possible values of the index to refer to the reduced set exclusively, or spend shorter codeword lengths for supported split ratios of the reduced set compared to the set of supported split ratios.
• the one of the one or more rules may exclude supported split ratios depending on the width of the predetermined sub-block perpendicular to the predetermined split direction (e.g. 128 in case of block 18 of Fig. 1 with the width being measured vertically or 24 in case of the block corresponding to node F which is split CU 3 and CU 4 with the width being measured horizontally).
• the one of the one or more rules may exclude supported split ratios depending on the width of the predetermined sub-block perpendicular to the predetermined split direction in a manner so that supported split ratios are excluded which divide the width into two child sub-blocks' widths not being, in samples, an integer multiple of a predetermined integer (cp. above).
• the one of the one or more rules may exclude supported split ratios depending on the width of the predetermined sub-block perpendicular to the predetermined split direction in a manner so that supported split ratios are excluded which divide the width into two child sub-blocks' widths being in samples, an integer multiple to each other.
• the one of the one or more rules may exclude supported split ratios depending on the width of the predetermined sub-block perpendicular to the predetermined split direction in a manner so that supported split ratios are excluded which divide the predetermined sub- block into two child sub-blocks of which at least one is too small to fully enclose at least one of a set of at least one predetermined minimum size blocks (cp. above).
• the one of the one or more rules may exclude supported split ratios depending on the width of the predetermined sub-block perpendicular to the predetermined split direction in a manner so that supported split ratios are excluded which divide the predetermined sub- block into two child sub-blocks of which at least one exceeds a predetermined aspect ratio (cp. above).
• the one of the one or more rules may exclude supported split ratios depending on the width of the predetermined sub-block perpendicular to the predetermined split direction in a manner so that a predetermined subset of supported split ratios is excluded which depends on the split level of the recursive bi-partitioning the predetermined sub-block belongs to (cp. above).
• the one of the one or more rules may exclude supported split ratios depending on a further split direction and a further split ratio of a further split line of one or more further sub-blocks of the picture block from which the predetermined sub-block is obtained by the recursive bi-partitioning by splitting the one or more further sub-blocks along the further split line (cp. above).
• the decoder may be configured to control the bi-partitioning with respect to the predetermined sub-block of the picture block depending on the reduced set of supported split ratios by restricting the supported split ratios signallable by the partition information to the reduced set, and one of the one or more rules excludes supported spiit ratios as to reduce a number of redundant settings of the partition information (i.e. the number of setting leading to the same partitioning of the picture block like the ones depicted in Fig. 13 for instance).
• the set of supported split ratios may a set of irreducible fractions
• A AI A'Z X A2' 2 A' AY Qni UN2 ⁇ ill
• the width of the predetermined sub-block both measured perpendicular to the split direction, if the predetermined split direction is vertical, and one of a upper and lower child sub-block resulting from splitting the predetermined sub-block along a split line having the predetermined split direction at the split ratio relative to the width of the predetermined sub-block, both measured perpendicular to the split direction, if the predetermined split direction is horizontal.
• the one of the one or more rules may exclude, if the predetermined split direction is parallel to the split direction of a split line along which the parent sub-block of the predetermined sub-block is split, and if the predetermined sub-block has a width s 0 times the parent sub-block's width, with s 0 being the split ratio of the parent block (e.g. the predetermined sub-block is the one which the nominator relates to), supported split ratios s 1 for which there are a split ratio s of the other block the parent block (e.g.
• the decoder according to this aspect can additionally perform the partitioning according to aspect one, i.e.
• a predetermined split level (level 2 with level 0 assigned to block 16) of the picture block 16 is split by the recursive bi-partitioning into a pair of child sub-blocks (CU1 and CU2) of a succeeding split level (level 3), depending on a second split line direction of a second split line (line separating upper left and lower left quarter of Fig. 1) along which a grandfather sub-block (left hand half of Fig. 1; note that the term “grandfather sub-block shall be understood as being broad enough to also cover the picture block 18 itself) of a preceding split level (level 1) of the picture block 16 is split into the predetermined sub-block (upper left quarter) and another sub- block (lower left quarter).
• An embodiment provides a method for decoding a picture 12 from a data stream 14 in units of coding blocks 16, comprising: partitioning a picture block 18 of the picture 12 into the coding blocks 16 by recursive bi- partitioning with controlling the recursive bi-partitioning according to partition information in the data stream 14, wherein the controlling the recursive bi-partitioning includes controlling the bi-partitioning with respect to a predetermined sub-block (e.g. any of the blocks in Fig. 1 corresponding to any node in Fig. 2, thus including block 18 itself, but also fraction thereof obtained by splits) of the picture block 18 in terms of whether the predetermined sub-block is to be split or to be left unsplit, a split line direction (e.g.
• a split line at which the predetermined sub- block of the picture block is to be split by the recursive bi-partitioning, and/or a split ratio at which a width of the predetermined sub-block of the picture block, perpendicular to the split line, is crossed by the split line; supporting a set (e.g. set A) of supported split ratios for splitting the predetermined sub-block and is configured to exclude, for a split of the predetermined sub-block along a split line having a predetermined split line direction, supported split ratios from the set of supported split ratios according to one or more rules to obtain a reduced set of supported split ratios,
• a set e.g. set A
• this method may be performed using a computer or CPU.
• a computer or CPU may be performed using a computer or CPU.
• an embodiment provides a corresponding computer program.
• An embodiment provides a block-based encoder for encoding a picture 12 into a data stream 14 in units of coding blocks 16, configured to partition a picture block 18 of the picture 12 into the coding blocks 16 by recursive bi-partitioning with controlling the recursive bi-partitioning according to partition information and inserting the partition information into the data stream 14, wherein the controlling the recursive bi-partitioning includes controlling the bi-partitioning with respect to a predetermined sub-block (e.g. any of the blocks in Fig. 1 corresponding to any node in Fig.
• the block-based encoder supports a set (e.g.
• set A) of supported split ratios for splitting the predetermined sub-biock is configured to exclude, for a split of the predetermined sub-block along a split line having a predetermined split line direction, supported split ratios from the set of supported split ratios according to one or more rules to obtain a reduced set of supported split ratios.
• the block- based encoder is configured to control the bi-partitioning with respect to the predetermined sub-block of the picture block depending on the reduced set of supported split ratios.
• This block-based encoder may be configured to control the bi-partitioning with respect to the predetermined sub-block of the picture block depending on the reduced set of supported split ratios in a manner so that the partition information in the data stream depends on the reduced set of supported split ratios.
• Another embodiment provides a method for encoding a picture 12 into a data stream 14 in units of coding blocks 16, configured to partitioning a picture block 18 of the picture 12 into the coding blocks 16 by recursive bi- partitioning with controlling the recursive bi-partitioning according to partition information and inserting the partition information into the data stream 14, wherein the controlling the recursive bi-partitioning includes controlling the bi-partitioning with respect to a predetermined sub-block (e.g. any of the blocks in Fig.
• a set e.g. set A
• supporting a set (e.g. set A) of supported split ratios for splitting the predetermined sub-block is configured to exclude, for a split of the predetermined sub-block along a split line having a predetermined split line direction, supported split ratios from the set of supported split ratios according to one or more rules to obtain a reduced set of supported split ratios
• this method may be performed using a computer or CPU.
• a computer or CPU may be performed using a computer or CPU.
• an embodiment provides a corresponding computer program.
• exclusion rules might be done on the side of the encoder which are not checked or emulated on the side of the decoder as long as the partition information based control of the bi- partitioning remains the same, uses the same set of exclusion rules.
• picture partitioning into blocks - may it be recursive bi-partitioning or any other recursive multi-partitioning - is rendered more efficient by context-adaptively entropy decoding one or more syntax elements controlling a splitting of a predetermined sub-block of a predetermined picture block using a context which depends on a size of one or more neighboring coding blocks into which the picture is partitioned up to the predetermined sub-block in the coding order, in a neighborhood of the predetermined sub-block.
• An embodiments of this aspect 3 provides a block-based decoder 10 for decoding a picture 12 from a data stream 14 in units of coding blocks 16, configured to partition one or more picture blocks 18 of the picture 12 into the coding blocks 16 along a coding order by recursive partitioning with controlling the recursive partitioning according to partition information in the data stream, and to context- adaptively entropy decode from the partition information one or more syntax elements (e.g. split_ratio_%) controlling a splitting of a predetermined sub-block of a predetermined picture block using a context which depends on a size (cp. "area") of one or more neighboring coding blocks (e.g. Al, A2, Bl, and B2) into which the picture is partitioned up to the predetermined sub-block in the coding order, in a neighborhood of the predetermined sub-block.
• a syntax elements e.g. split_ratio_
• the block-based decoder may be configured to pre-partition the picture into an array of picture blocks, and wherein the coding order traverses the picture blocks in a raster scan order.
• the coding order follows a depth-first or breadth first traversal order.
• the one or more syntax elements control the recursive bi- partitioning with respect to a predetermined sub-block in terms of whether the predetermined sub-block is to be split or to be left unsplit, a split direction of a split line at which the predetermined sub-block of the picture block is to be split by the recursive bi-partitioning, and a split ratio at which a width of the predetermined sub-block of the picture block, perpendicular to the width, is crossed by the split line.
• the one of the one or more syntax elements controls the recursive bi- partitioning with respect to a predetermined sub-block in terms of a split ratio at which a width of the predetermined sub-block of the picture block, perpendicular to the width, is crossed by a split line at which the predetermined sub-block of the picture block is to be split by the recursive bi-partitioning and wherein the context for the one syntax element depends on the size of neighboring coding blocks into which the picture is partitioned up to the predetermined sub-block in the coding order, in a neighborhood of the predetermined sub-block.
• the decoder may additionally comprise a decoder according to aspect 1 and/or aspect 2.
• Another embodiments provides a method for decoding a picture 12 from a data stream 14 in units of coding blocks 16, comprising: partitioning one or more picture blocks 18 of the picture 12 into the coding blocks 16 along a coding order by recursive partitioning with controlling the recursive partitioning according to partition information in the data stream,
• context-adaptively entropy decoding from the partition information one or more syntax elements (e.g. split_ratio_%) controlling a splitting of a predetermined sub-block of a predetermined picture block using a context which depends on a size (cp. "area") of one or more neighboring coding blocks (e.g. Al, A2, Bl, and B2) into which the picture is partitioned up to the predetermined sub-block in the coding order, in a neighborhood of the predetermined sub-block.
• syntax elements e.g. split_ratio_
• This method may be performed using a computer program having a program code for performing it.
• Another embodiments provides a block-based encoder for encoding a picture 12 from a data stream 14 in units of coding blocks 16, configured to partition one or more picture blocks 18 of the picture 12 into the coding blocks 16 along a coding order by recursive partitioning with controlling the recursive partitioning according to partition information and inserting the partition information into the data stream, and to context-adaptively entropy encode into the partition information one or more syntax elements (e.g. split_ratio_%) controlling a splitting of a predetermined sub-block of a predetermined picture block using a context which depends on a size (cp. "area") of one or more neighboring coding blocks (e.g.
• Another embodiments provides a corresponding method for encoding a picture 12 from a data stream 14 in units of coding blocks 16, comprising: partitioning one or more picture blocks 18 of the picture 12 into the coding blocks 16 along a coding order by recursive partitioning with controlling the recursive partitioning according to partition information and inserting the partition information into the data stream, and to context-adaptively entropy encoding into the partition information one or more syntax elements (e.g.
• split_ratio_ controlling a splitting of a predetermined sub-block of a predetermined picture block using a context which depends on a size (cp. "area") of one or more neighboring coding blocks (e.g. Al, A2, Bl, and B2) into which the picture is partitioned up to the predetermined sub-block in the coding order, in a neighborhood of the predetermined sub-block.
• a size cp. "area”
• neighboring coding blocks e.g. Al, A2, Bl, and B2
• this method may be performed using a computer or CPU.
• a computer or CPU may be performed using a computer or CPU.
• an embodiment provides a corresponding computer program.
• picture partitioning into blocks - may it be recursive bi-partitioning or any other recursive multi-partitioning - is rendered more efficient by supporting at least two modes between which a switchable is feasible, and which differ in terms of degree of freedom of performing the partitioning.
• Profile and levels may be associated with different modes which, in turn, may be defined by the above-mentioned "adaptation sets", "adaptation sets may be selected in units of whole videos, picture sequences, pictures, slices, or even in units of individual picture blocks.
• the (or some) adaptation sets may be "learned" by collecting big data, and distributed to the decoders and encoders.
• An embodiment according to this aspect 4 provides a block-based decoder for decoding a picture from a data stream in units of coding blocks, configured to partition one or more picture blocks of the picture into the coding blocks along a coding order by recursive partitioning with controlling the recursive partitioning according to partition information in the data stream.
• the block-based decoder supports at least two modes between which the block-based decoder is switchable, and which differ in terms of degree of freedom of performing the partitioning
• the block-based decoder is configured such that the at least two modes differ in one or more of the following: a minimum block size which has to fit into each coding block, a maximum partition tree depth, a maximum partition tree depth following a split of a current sub-block of a current picture block along a split line perpendicular to a split line of a parent block, the current sub-subblock is child of, a maximum block size for sub-blocks of a current picture for which non-symmetric splits are allowed, a maximum aspect ratio for child sub-blocks resulting from a split of a current sub-block of a current picture block, a minimum aspect ratio for child sub-blocks resulting from a split of a current sub-block of a current picture block, preset splits of a current picture block (e.g.
• a set e.g. set A
• supported split ratios for splitting a current sub-block of a current picture block
• one or more rules for excluding, for a split of a current sub-block along a split line having a predetermined split direction, supported split ratios from the set of supported split ratios.
• the block-based decoder may be configured to switch between the at least two modes according to one or more of signaling in the data stream, a history of modes used so far, (e.g. forming a statistics of used modes during decoding) a spatial predictor, (e.g. coding modes, prediction parameters and/or partition information used for neighboring coding/prediction blocks) a degree of extension of a current picture block beyond an outer or internal picture boundary.
• the block-based decoder may be configured to switch between the at least two modes in units of a picture sequence, tree-root blocks of a picture, picture blocks (which might be tree-root blocks).
• the decoder of this aspect 4 may additionally comprise a decoder according to aspect 1,2 and/or 3.
• Another embodiment provides a method for decoding a picture from a data stream in units of coding blocks, comprising: partitioning one or more picture blocks of the picture into the coding blocks along a coding order by recursive partitioning with controlling the recursive partitioning according to partition information in the data stream, and supporting at least two modes or switching between at least two modes (which differ in terms of degree of freedom of performing the partitioning).
• this method may be performed using a computer or CPU.
• an embodiment provides a corresponding computer program.
• Block-based encoder for encoding a picture from a data stream in units of coding blocks, configured to partition one or more picture blocks of the picture into the coding blocks along a coding order by recursive partitioning with controlling the recursive partitioning according to partition information and inserting the partition information into the data stream, wherein the block-based encoder supports at least two modes between which the block-based encoder is switchable, and which differ in terms of degree of freedom of performing the partitioning.
• Another embodiment provides Method for encoding a picture from a data stream in units of coding blocks, comprising: partitioning one or more picture blocks of the picture into the coding blocks along a coding order by recursive partitioning with controlling the recursive partitioning according to partition information and inserting the partition information into the data stream, and supporting at least two modes which differ in terms of degree of freedom of performing the partitioning.
• this method may be performed using a computer or CPU.
• the coding of a split ratio is rendered more effective by using a set of split ratios including, for instance, equal splitting and pairs of asymmetric splits (of mutually reciprocal ratios), reducing this set and using a correspondingly reduced (or pruned) binary tree as binarization scheme for binarizing /and debinarizing) an index into the reduced list.
• An embodiment of aspect 5 provides a decoder (e.g. a module possibly within the extractor of Fig. 15) for decoding an index pointing to a split ratio from a data stream, configured to reduce a base list 30 of split ratios by removing split ratios according to one or more rules so as to obtain a reduced list 32 of split ratios, wherein the base list 30 corresponds to a binary tree 34 leafs 42, 37 of which are associated with respective ones of the split ratios of the base list 30, and the reduced list 32 corresponds to a reduced binary tree 50 resulting from the binary tree 34 by removing, from leafs 42, 37 to main root node 44, each node 40, 38 of the binary tree 34 the child nodes of which do not belong to the reduced list 32, and attaching, in place of each node of the binary tree 34 having merely one non-removed child node, the non-removed child node directly to the node of the binary tree 34 which the respective intermediary node is child of, wherein left (first) branches (
• each split ratio of the reduced list 32 has associated therewith a bin value sequence corresponding to a sequence of bin values associated with the branches which lead, in the reduced binary tree 50, from a main root node 52 of the reduced binary tree 50 to a leaf 54 of the reduced binary tree 50 associated with the respective split ratio, entropy decode a bin string (e.g. formed by split_ratio_... flags) corresponding to a sequence of one or more bins (e.g. formed by the individual split_ratio_...
• the base list 30 is a list of irreducible fractions 1 ⁇ ⁇ 12 ⁇ 21 ⁇ 22 ⁇ 31 32 «m « ⁇ 2
• V fc , a kl + a k2 - b k so that each pair of split ratios kl /b k and k2 /b k , with a k2 b k - a kl and a kl /b k > a k2 /b
• V fc , a kl + a k2 - b k so that each pair of split ratios kl /b k and k2 /b k , with a k2 b k - a kl and a kl /b k > a k2 /b
• the binary tree 34 is composed of a partial tree 36 having 2n leafs 37 all of which are connected to a tree root 38 of the partial tree 36 via n-1 intermediary nodes 40 of the partial tree 36, and correspond, from left to right, to the second to (2n+l)
• the decoder may be configured to perform the entropy decoding context- adaptively for at least a bin subset of the sequence of one or more bins, and to determine a context for the entropy decoding in a manner so that the determination of the context differs among the bins of the bin subset.
• the decoder may be configured to perform the entropy decoding context-adaptively for at least one predetermined bin of the bin string and configured to determine a context for the entropy decoding the predetermined bin depending on a position (e.g. split level) in the binary tree 34 where the node of the reduced binary tree 52 which corresponds to the predetermined bin, originates in the reduction of the binary tree 34 (frankly speaking, e.g. it's occidentalle split level in the binary tree 34 counts, rather than the spit level in the reduced tree 50).
• a position e.g. split level
• the split ratio pointed to by the index is for a recursive bi-partitioning split.
• the decoder is configured to perform the entropy decoding context-adaptively for at least a bin subset of the sequence of one or more bins, and determine, for each bin of the bin subset, a context for the entropy decoding by selecting the context for the respective bin out of a set of contexts depending on coding parameters associated with (e.g. coding modes, prediction parameters and/or partition information used to code/decode block in the...) a spatial neighborhood of a block (18) which the recursive bi-partitioning split relates to.
• the set of contexts out of which the selection takes place differs for at least two bin sub-subsets of the bin subset (e.g. mutually distinct sets of contexts are used for the bins).
• the decoder may be configured to determine, for at least one of the bin subset, the context for the entropy decoding by selecting the context for the at least one bin out of the set of contexts depending on a size (e.g. area) of bi-partitioning leaf blocks in the spatial neighborhood of the block which the recursive bi-partitioning split relates to.
• a size e.g. area
• the decoder may be configured to locate bi-partitioning leaf blocks in the spatial neighborhood of the block which the recursive bi-partitioning split relates to, depending on a split line direction of block which the recursive bi-partitioning split relates to in a manner so that the blocks in the spatial neighborhood neighbor the block which the recursive bi-partitioning split relates to along the split line direction and neighbor each other transversely with respect to the split line direction (e.g. Al and A2 in case of vertical split, and Bl and B2 and case of horizontal split).
• a split line direction of block which the recursive bi-partitioning split relates to in a manner so that the blocks in the spatial neighborhood neighbor the block which the recursive bi-partitioning split relates to along the split line direction and neighbor each other transversely with respect to the split line direction (e.g. Al and A2 in case of vertical split, and Bl and B2 and case of horizontal split).
• the decoder may be is additionally a decoder according to aspect 1, 2, 3 and/or 4.
• An embodiment provides a method for decoding an index pointing to a split ratio from a data stream, comprising:
• reducing a base list 30 of split ratios by removing split ratios according to one or more rules so as to obtain a reduced list 32 of split ratios, wherein the base list 30 corresponds to a binary tree 34 leafs 42,37 of which are associated with respective ones of the split ratios of the base list 30, and the reduced list 32 corresponds to a reduced binary tree 50 resulting from the binary tree 34 by
• each node 40,38 of the binary tree 34 the child nodes of which do not belong to the reduced list 32 , and attaching, in place of each node of the binary tree 34 having merely one non-removed child node, the non-removed child node directly to the node of the binary tree 34 which the respective intermediary node is child of,
• left branches exemplarily assigned “0” in Fig. 5 and Fig. 6
• right branches exemplarily assigned “1” in Fig. 5 and Fig. 6) of the reduced binary tree 50 are associated with a second bin value and each split ratio of the reduced list 32 has associated therewith a bin value sequence corresponding to a sequence of bin values associated with the branches which lead, in the reduced binary tree 50, from a main root node 52 of the reduced binary tree 50 to a leaf 54 of the reduced binary tree 50 associated with the respective split ratio
• entropy decoding a bin string (e.g. formed by split_ratio_... flags) corresponding to a sequence of one or more bins (e.g. formed by the individual split_ratio_... flags) traversing, starting from the main root node 52 of the reduced binary tree 50, the reduced binary tree 50 to one of the reduced binary tree's leafs 54 with each bin of the bin string corresponding to one of the nodes of the reduced binary tree 50 traversed by the sequence of one or more bins;
• a bin string e.g. formed by split_ratio_... flags
• a sequence of one or more bins e.g. formed by the individual split_ratio_... flags
• this method may be performed using a computer or CPU.
• Another embodiments provides a encoder for encoding an index pointing to a split ratio into a data stream, configured to reduce a base list 30 of split ratios by removing split ratios according to one or more rules so as to obtain a reduced list 32 of split ratios, wherein the base list 30 corresponds to a binary tree 34 leafs 42,37 of which are associated with respective ones of the split ratios of the base list 30, and the reduced list 32 corresponds to a reduced binary tree 50 resulting from the binary tree 34 by removing, from leafs 42,37 to main root node 44, each node 40,38 of the binary tree 34 the child nodes of which are both removed, and attaching, in place of each node of the binary tree 34 having merely one non-removed child node, the non-removed child node directly to the node of the binary tree 34 which the respective intermediary node is child of.
• left branches (exemplarily assigned “0” in Fig. 5 and Fig. 6) of the reduced binary tree 50 are associated with a first bin value
• right branches (exemplarily assigned "1” in Fig. 5 and Fig. 6) of the reduced binary tree 50 are associated with a second bin value and each split ratio of the reduced list 32 has associated therewith a bin value sequence corresponding to a sequence of bin values associated with the branches which lead, in the reduced binary tree 50, from a main root node 52 of the reduced binary tree 50 to a leaf 54 of the reduced binary tree 50 associated with the respective split ratio.
• the decoder is configured to binarize the index into a bin string using the reduced binary tree as a binarization scheme, the bin string corresponding to a sequence of one or more bins (e.g. formed by the individual split_ratio_... flags) traversing, starting from the main tree root, the reduced binary tree 50 to one of the reduced binary tree's leafs 54 with each bin of the bin string corresponding to one of the nodes of the reduced binary tree 50 traversed by the sequence of one or more bins; and to entropy encode the bin string (e.g. formed by split_ratio_... flags).
• the bin string corresponding to a sequence of one or more bins (e.g. formed by the individual split_ratio_... flags) traversing, starting from the main tree root, the reduced binary tree 50 to one of the reduced binary tree's leafs 54 with each bin of the bin string corresponding to one of the nodes of the reduced binary tree 50 traversed by the sequence of one or more bins.
• Another embodiments provides a method for encoding an index pointing to a split ratio into a data stream, comprising:
• reducing a base list 30 of split ratios by removing split ratios according to one or more rules so as to obtain a reduced list 32 of split ratios, wherein the base list 30 corresponds to a binary tree 34 leafs 42,37 of which are associated with respective ones of the split ratios of the base list 30, and the reduced list 32 corresponds to a reduced binary tree 50 resulting from the binary tree 34 by
• each split ratio of the reduced list 32 has associated therewith a bin value sequence corresponding to a sequence of bin values associated with the branches which lead, in the reduced binary tree 50, from a main root node 52 of the reduced binary tree 50 to a leaf 54 of the reduced binary tree 50 associated with the respective split ratio, binarizing the index into a bin string using the reduced binary tree as a binarization scheme, the bin string corresponding to a sequence of one or more bins (e.g. formed by the individual split_ratio_...
• this method may be performed using a computer or CPU.
• an embodiment provides a corresponding computer program.
• Fig. 1. shows an exemplary partitioning of 128x128 CTU according to embodiments of the present application.
• Fig. 2 shows the partitioning tree describing the topology in fig. 1 according to embodiments.
• Fig. 3 shows a description of non-leaf nodes of the partitioning tree in fig. 2 (including block sizes and the splits used to subdivide a particular node described by the split direction and the split ratio) according to embodiments.
• Fig. 4 shows Signaling of split direction according to embodiments.
• Fig. 5 shows exemplary signaling of the split ratio for a particular block size according to embodiments.
• Fig. 6 shows reduced signaling from fig. 5 (for cases where not all split ratios are available) according to embodiments.
• Fig. 7 shows three template layouts used to define prohibited partitioning tree patterns according to embodiments.
• Fig. 8 shows exemplary context modeling for a simple binary decision tree for signaling the split ratio according to embodiments.
• Fig. 9 shows exemplary context modeling for another binary decision tree for signaling the split ratio according to embodiments.
• Fig. 10 illustrates location of neighboring blocks according to embodiments.
• Fig. 11 shows an example for redundant parallel splits according to embodiments.
• Fig. 12 shows an example for redundant perpendicular splits according to embodiments.
• Fig. 13 shows an example for higher-order redundancies according to embodiments.
• Fig. 14 shows exemplarily an decoder according to a basic embodiment.
• Fig. 15 shows exemplarily an decoder according to an enhanced embodiment.
• Fig. 16 shows a block diagram of an apparatus for predictively coding a video as an example for a video decoder where a composed prediction concept.
• Fig. 17 shows a block diagram of an apparatus for predictively decoding a video as an example for a video decoder.
• Fig. 18 shows a schematic diagram illustrating an example for a relationship between a prediction residual signal, a prediction signal and a reconstructed signal.
• Fig. 19 shows a schematic illustration an example for limiting the average number of prediction hypotheses over a given area.
• Fig. 20 shows a schematic example for defining the number of prediction hypotheses depending on the block size or geometry.
• a CTU can be split into multiple coding units (CUs).
• a coding unit is the entity for which a coding mode (e.g., intra- or inter-picture coding) is selected.
• the decomposition of a CTU into one or more CUs is specified by a quadtree (QT) and transmitted as part of the bitstream.
• QT quadtree
• a first binary decision (bin) called split_cu_flag is transmitted, which signals whether the entire CTU forms one single CU (i.e., the CTU is not decomposed into multiple CUs) or whether the CTU is partitioned into four equally sized square units.
• a further split_cu_flag is transmitted, which indicates whether the corresponding unit is further partitioned into four equally sized square units. This multi-level splitting is continued until for each of the units it is either signaled that the unit is not further split or a minimum CU size is reached. If a unit has the minimum CU size, no split_.cu_.flag is transmitted, but it is inferred that the unit is not further split.
• the minimum allowed CU size is signaled in the sequence parameter set; it can range from the size of the CTU to 8x8 luma samples (and always presents a square with an integer power of two for both the width and height).
• the resulting units of the described splitting are the coding units (CUs), for which a coding mode and associated parameters are selected and transmitted. Note that the partitioning represents a quadtree (QT) structure.
• Each of the resulting CUs is associated with a coding mode, which is selected at the encoder side and transmitted as part of the bitstream.
• the binary decisions split_cu_flag which specify the partitioning of a CTU into one or multiple CUs are coded using context-based adaptive binary arithmetic coding (CABAC).
• CABAC context-based adaptive binary arithmetic coding
• the CUs of a CTU are processed in the so-called z-scan order. That means, the four blocks that result from a split are processed in raster-scan order; and if any of the blocks is further partitioned, the corresponding four blocks (including the included smaller blocks) are processed before the next block of the higher splitting level is processed.
• a CU is coded in an intra-coding mode, an intra prediction mode for the luma signal and, if the video signal includes chroma components, another intra prediction mode for the chroma signals is transmitted.
• ITU-T H.265 M PEG-H H EVC
• the luma coding block can also be split into four equally sized blocks, in which case, for each of these blocks, a separate luma intra prediction mode is transmitted. This splitting is indicated by a binary syntax element partjnode.
• the actual intra prediction and coding is done on the basis of transform blocks.
• a CU can be partitioned into multiple transform blocks.
• the subdivision into transform blocks is transmitted using a quadtree syntax.
• the root of the quadtree is formed by the coding unit.
• the syntax of the coding unit includes a binary decision called split_transform_flag, which specifies whether the luma component of the CU consists of a single luma transform block or whether it is split into four equally sized square blocks. In the latter case, for each of the resulting luma blocks, another split_transform_flag is transmitted specifying whether the corresponding block is further split.
• This quadtree splitting is continued until for each of the resulting luma blocks, it is signaled that the block is not further split or the block reaches a minimum transform block size (indicated in the sequence parameter set).
• the first split_transform_flag (at the CU level) is not transmitted, but it is inferred that the luma coding block is split into four luma blocks.
• the splitting of the chroma blocks basically follows the splitting of the co-located luma block, i.e., for each luma block the associated chroma blocks cover the same picture area.
• the luma block and the associated chroma blocks that cover the same picture area form a tra nsform unit. There is, however, one exception in the ITU-T H.265 I MPEG-H HEVC standard : If the resulting chroma blocks would be smaller than the minimum transform block size (as specified in the sequence parameter set), only the luma block is split.
• M PEG-H HEVC the binary syntax elements split_transform_flag are coded using CABAC.
• a transform block (TB) is the entity to which a single 2d transform is applied for coding the prediction residual signal. Note that all transform blocks in ITU-T H.265
• the TUs of an intra-coded CU are processed in z-scan order (see above). For each transform block (luma or chroma), first a prediction signal is generated using already reconstructed neighboring samples of the same color component. The algorithm that is used for generating the prediction signal is determined by the intra prediction mode. The resulting prediction error / residual signal is transmitted using tra nsform coding. A two-dimensional transform is applied to the corresponding block of residual samples, the resulting transform coefficients are quantized (typically using uniform reconstruction q ua ntizers), and the resulting quantization indexes, which are a lso referred to as transform coefficient levels, are entropy-coded.
• M PEG-H HEVC uses the concept of context-based adaptive binary arithmetic coding (CABAC).
• CABAC context-based adaptive binary arithmetic coding
• Inter-picture coded CUs can be split into multiple prediction units.
• a prediction unit is the entity of a luma and, for color video, two associated chroma blocks (covering the same picture area), for which a single set of prediction parameters is used.
• a CU can be coded as a single prediction unit, or it can be split into two non-square or four square prediction units. The latter (four square PUs) is only supported if the CU has the minimum CU size specified in the sequence parameter set. The chosen splitting is indicated by the syntax element part_mode.
• MPEG-H HEVC supports in total 8 modes for partitioning a CU into PUs.
• An NxN CU can be pa rtitioned into:
• Each set of motion parameters includes the number of motion hypotheses (one or two in ITU-T H.265
• a merge_flag is transmitted, which indicates whether the motion parameters are completely derived based on already coded motion parameters or whether the motion parameters are transmitted. If the merge_fiag is equal to 1, a list of motion parameter set candidates is derived, which typically includes motion parameters sets of already coded spatially neighboring blocks as well as one or more motion parameter sets that are derived from temporally co- located blocks.
• the used motion parameter set is indicated by a syntax element mergejdx, which specifies an index into the candidate list of motion parameter sets. If merge_flag is equal to 0, the motion parameters are transmitted as part of the bitstream.
• a first syntax element (inter_pred_idc in ITU-T H.265 I MPEG-H HEVC) specifies the number of motion hypotheses. In ITU-T H.265
• the reference picture lists are constructed based on slice header syntax elements.
• Each reference picture list contains one or more already coded pictures (which are stored in a decoded picture buffer) in a specified order.
• the syntax element inter_pred_idc does not only indicate single-hypothesis prediction or bi- prediction, but for single-hypothesis prediction it additionally indicates whether list 0 or list 1 is used.
• the first motion hypothesis refers to list 0 and the second motion hypothesis refers to list 1.
• an associated motion vector is transmitted as part of the bitstream.
• the motion vectors are differentially coded.
• a list of motion vector predictor candidates is derived. It can include (potentially scaled) motion vectors of neighboring PUs as well as scaled motion vectors of temporally preceeding (in terms of coding order), spatially co-located blocks.
• the actually chosen motion vector predictor is signaled via an index into the motion vector predictor iist. Finally the difference between the actual motion vector for the PU and the selected motion vector predictor is transmitted.
• the prediction signal for a PU is formed by a displaced block in the selected reconstructed reference picture, where the displacement relative to the current PU is specified by the motion vector.
• the displacement is typically specified with sub-sample accuracy (in ITU-T H.265
• the prediction signal is generated by interpolating the reconstructed reference picture (typically, using separable FIR filters).
• the final prediction signal of PUs with multi-hypothesis prediction is formed by a weighted sum of the prediction signals for the individual motion hypothesis. Note that luma and chroma blocks of a PU share the same set of motion parameters.
• the prediction signal for an inter-picture coded CU is composed of the prediction signals for the contained PUs.
• the prediction error signal (also called residual signal) for the luma and chroma blocks of a coding unit is transmitted via transform coding.
• the block of luma residual samples of a CU as well as the blocks of chroma residual samples (if present) are partitioned into transform blocks.
• the partitioning into transform blocks is the same as for intra-picture coded CUs described above.
• the partitioning of a CU into transform block is indicated by a quadtree syntax, which is also referred to as residual quadtree (RQT).
• the resulting transform blocks are coded using transform coding: A 2d transform is applied to the block of residual samples, the resulting transform coefficients are quantized, and the resulting transform coefficient levels (quantization indexes) are entropy coded.
• a skip_flag is transmitted. If this flag is equal to 1, it indicates that the corresponding CU consists of a single prediction unit coded in merge mode (i.e., merge_flag is inferred to be equal to 1) and that all transform coefficients are equal to zero (i.e., the reconstruction signal is equal to the prediction signal). In that case, only the mergejdx is transmitted in addition to the skip_flag. If skip_flag is equal to 0, the prediction mode (inter or intra) is signaled, followed by the syntax features described above.
• CABAC context-based adaptive binary arithmetic coding
• Non-binary syntax elements are first binarized, i.e., they are mapped onto a series of binary decisions, which are also referred to as bins. Then, each bin (including binary syntax elements) is coded using a binary arithmetic coder.
• the so-called regular mode one adaptive binary probability model is chosen per bin; after coding the corresponding bin, the binary probability model is updated using the actual value of the bin.
• the entropy coder maintains a certain number of binary probability models.
• the selected probability model depends on the syntax element and the bin number (for non-binary syntax elements). It can additionally depend on already coded syntax elements or bins.
• the selection of binary probability models is also referred to as context modelling.
• MPEG-H HEVC provides a multitude of possibilities for coding a block of samples (such as a coding tree unit).
• the encoder has to select the partitionings (into CUs, PUs, and TUs), the coding modes and associated parameters (such as intra prediction modes and motion parameters), and the transform coefficient levels (quantization indexes).
• the selected coding parameters determine both the reconstruction quality as well as the number of bits required for representing the coding parameter inside the bitstream.
• Modern encoding algorithms are typically based on Lagrangian bit allocation.
• the maximum achievable coding efficiency of a codec is determined by the supported syntax features, such as the set of coding modes, the partitioning options, the supported intra prediction modes, etc.
• partitioning options One of the most important syntax features are the supported partitioning options.
• a flexible partitioning allows an adaptation of the block sizes used for prediction and transform coding to the video signal.
• the partitioning has to be transmitted as part of the bitstream.
• the gain in prediction and transform coding for providing additional partitioning options has to outweigh the additional signaling overhead. Due to that reason, state-of-the-art video coding standards support only very simple partitioning concepts, such as quadtree partitionings.
• a partitioning concept based on generalized binary trees is described, it provides more flexibility than conventional partitioning concepts (such as the quadtree approach) for adapting the blocks used for prediction and transform coding to the video signal to be coded. And since the syntax fortransmitting the selected partitioning is still very simple, the gain in prediction and transform coding typically outweighs the additional signaling overhead. Hence, the generalized partitioning concept is capable of increasing the achievable coding efficiency of a video codec.
• the described partitioning concept can be used for partitioning a block for coding mode signaling, intra-picture prediction, inter-picture prediction, transform coding, or any combination thereof. It can completely replace conventional partitioning concepts, or it can be combined with other partitioning concepts, such as a quadtree partitioning.
• the generalized binary splitting that is presented in this invention represents a superset of all mentioned splitting approaches QT, QTBT, QTBT with additional triple splits, and QTBT with additional asymmetric splits into subblocks of sizes 1/4 and 3/4. It is characterized by a simple syntax and is capable of better adapting the block sizes to object and motion boundaries, which results in an improved coding efficiency.
• a method of video coding using generalized binary trees is presented.
• the presented generalized binary splitting can be applied for all block partitionings required in video coding. In particular, it can be applied for partitioning a block into coding units (for signaling the coding mode), prediction units (for signaling prediction parameters such as motion parameters), or transform units (for transform coding of the residual signal).
• the signaling of the coding mode, the signaling of prediction parameters and the transform coding is applied to the same blocks (i.e., such a block represents a CU, PU, and TU) and the generalized binary splitting is used for specifying the corresponding partitioning of a fixed sized block into coding units (which also represent PUs and TUs). It is, however, also possible to use the generalized binary splitting for specifying the partitioning into CUs, while the resulting CUs can be further partitioned into PUs and/or TUs (either using the generalized binary splitting or any other partitioning concept).
• the generalized binary splitting can be applied individually to color components and auxiliary components, such as alpha channel or depth masks, respectively.
• generalized binary splitting can be applied to groups of those components.
• two different splitting trees can be used: one for partitioning the luma component and other for partitioning the chroma components.
• the generalized binary splitting can be combined with any other splitting.
• the quadtree splitting (or any other splitting concept) can be used on a first level and the resulting blocks can be further partitioned using the generalized binary splitting.
• blocks obtained by the generalized binary splitting can be further partitioned using any other splitting concept.
• the splitting structure can be represented as a binary splitting tree (also referred to as "partitioning tree” in the following).
• Each node represents a binary split with a split direction (horizontal or vertical) and a split ratio (which is the ratio of the width (or height) of the first resulting block and the width (or height) of the original block).
• the split direction is coded relative to the previous split direction (i.e., the corresponding split on the next higher (previous) level) (cp. A6).
• a split is coded as “parallel” or as “perpendicular” split, which always translates to either a “vertical” (splitting the width) or a “horizontal” split (splitting the height). If we assume that the current block was obtained by a horizontal split (at the last (previous) split level), then, if a "parallel split” is signaled, the current block is again horizontally split into two subblocks, and if a "perpendicular split” is signaled, the current block is vertically split into two subblocks (cp. A7) . If the last level split was a vertical split, the "parallel split” specifies a vertical split and the "perpendicular split” specified a horizontal split. Since on the highest level (i.e., for the root level block) no previous split was signaled, it has to be defined by convention whether the last split level is inferred to represent a horizontal or a vertical split.
• a binary split is characterized by the split ratio (cp. A8).
• the split ratio specifies the ratio of the sizes (in number of samples) of the first resulting subblock (from top/left) and the block being split. Note that, since a block is always split into two subblocks, the split ratio also specifies ratio of the block widths (for vertical splits) or the ratio of the block heights (for horizontal blocks).
• the leaf nodes of the partitioning tree represent the actual units (consisting of a luma sample block and, for non-monochrome video, the co-located chroma sample blocks) being coded.
• Coding units which shall be referred to as coding units in the following, are characterized by their positions and sizes, which can be inferred from the partitioning tree.
• Coding cp. A2
• prediction cp. A3
• transform coding cp. A4
• a perpendicular split with a split ratio of 1/2 is the split that can be signaled using the minimum number of bits (or bins) in most configurations and will referred to as "default split" in the following.
• the partitioning is signaled as follows. Without loss of generality, we define that the last split for the root block (which did not result from an actual split) is considered as vertical split. But note that it could also be defined as horizontal split. First a binary decision perpend_split_flag is coded, which indicates whether a perpendicular split applied. If perpend_split_flag is equal to 1 and the last split (i.e., the split from which is current block resulted) was a horizontal split, a vertical split is applied to the current block. If perpend_split_fiag is equal to 1 and the last split was a vertical split (with our above defined convention this includes the case that our current block is the root blocks), this indicates that a horizontal split is applied.
• perpend_split_fiag is equal to 0
• another binary decision parallel_split_flag is transmitted. If parallel_split_flag is equal to 1 and the last split was a horizontal split, the current block is again horizontally split. If parallel_split_flag is equal to 1 and the last split was a vertical split (or was inferred as vertical split by our convention), the current block is vertically split. If parallel_split_flag is equal to 0, the current block is not further split, i.e. it is left unsplit, but represents a leaf block (equivalent to a "no-split" signaling) and becomes a coding block.
• the corresponding binary decisions are not transmitted, but can be inferred at the decoder side. For example, if the last split was a vertical split and the width of the current block is equal to the minimum supported block width, the binary decision parallel_split_flag (which indicates whether the current block is again vertically split) is not transmitted, but inferred to be equal to 0.
• Either or both of the binary decisions perpend_split_flag and parallel_split_flag can be inferred if this is indicated by high level parameters, which may be signaled in the PPS (picture parameter set), SPS (sequence parameter set) or slice header (cp. A19 and A20).
• PPS picture parameter set
• SPS sequence parameter set
• slice header cp. A19 and A20.
• a binary decision tree for signaling the split direction is shown in fig. 4. If a horizontal split is to be applied, the split size is defined as the length of the vertical edge of the block to be split (as the split will introduce a new horizontal edge). For a vertical split, the split size M is the length of the horizontal edge of the block to be split.
• the split pattern consists of alternating horizontal and vertical splits.
• the block is either not further split or it is split using a split direction that is perpendicular to the previous split direction (at the higher level).
• An exception could be made for the root block (where no previous split exists); for the root block it could be explicitly signaled whether a horizontal or a vertical split is applied (or whether the block is not split at all) (cp. A12+A5).
• the split direction is coded using binary decisions hor_spiit_fiag (indicating whether or not the block is horizontally split) and ver_split_flag (indicating whether or not the block is vertically split). These flags could be coded in any order; the second flag is only transmitted if the first flag indicates no splitting in the corresponding direction.
• the split and the split direction are coded using a split_flag and a split_direction_flag, where the split_flag indicates whether the current block is split at all and the split_direction_flag indicates whether the split is applied in horizontal or vertical direction.
• the split_direction_flag indicates whether the split is applied in perpendicular direction to the last split or in the same direction as the last split.
• the split flags are coded using context-based adaptive arithmetic coding (as CABAC in H.265
• a split ratio is coded in addition to the split direction (cp. 18a).
• the set of possible split ratios can be arbitrarily chosen for every block size provided the resulting subblocks can be encoded using the video encoding framework (e.g., an appropriate transformation size has to be available).
• the split ratio may be encoded as an element of a set as well as by binarization of the nominator and the denominator (in the former case, the set has to be predefined).
• the split ratio is coded based on a binary decision tree (with adaptive tree pruning according to the available split ratios) as will be described below in more detail.
• the split ratio can be coded as an index into an ordered list of available split ratios. The index can be mapped onto a series of binary decisions using any binarization scheme.
• the overall set of split ratios is defined as: ⁇ 1/2, 1/4, 3/4, 3/8, 5/8, 1/3, 2/3, 1/5, 2/5, 3/5, 4/5 ⁇ .
• the available set of split ratios represents a subset of this overall set.
• the subset for a particular block and a particular splitdirection can be determined by the size of the current block and/or the series of previous splits. If some of the ratios are not available for a particular split size (e.g., because restrictions on transform or prediction sizes or minimum block sizes prevent these split ratios or because the splits would result in redundant partitionings), the corresponding signaling flags are not transmitted in order to reduce the bit rate.
• all bins of the ratio signaling can be inferred and are not transmitted (e.g., for block sizes larger than the maximum block size for a non-default split). If, for a split direction, none of the split ratios can be applied (e.g., due to prohibitions enforced by high level parameters), the signaling of this split direction can be inferred as false.
• the binary splitting will be applied recursively until no further splitting is inferred or signaled. I.e., if a block is split into two subblocks, for each of the resulting subblocks another split consisting of a split direction and a split ratio is coded (where some or all of the corresponding binary decisions can be inferred based on syntax constraints). This process is continued until it is signaled (or inferred) that no block is further split.
• the generalized binary splitting concept is applied for partitioning fixed sized blocks (such as CTUs) into coding units, which also represent prediction units and transform units. For each of the resulting blocks, a coding mode and associated prediction parameters (such as intra prediction modes or motion parameters) are transmitted and the blocks are also used for transform coding of the residual signal.
• the resulting blocks are used as coding blocks (i.e., for signaling the coding mode), but the coding blocks can be further partitioned into prediction and/or transform blocks (either using the generalized binary splitting concept or any other splitting method).
• the high-level parameters controlling the behavior of generalized binary splitting can include a minimum block size (e.g., specified by a minimum block width and a minimum block height, or specified by a minimum number of samples), a maximum partition tree depth , a maximum partition tree depth following a first non-default split, a maximum block size for a non-default split, maximum and minimum allowed aspect ratios for the resulting blocks as well as constraints regarding the applicable split ratios.
• the high-level parameters may be signaled in a picture parameter set (PPS), sequence parameter set (SPS), slice header or any other high-level syntax structure. To reduce the number of bits required for transmitting the high-level parameters, boundaries and specific domains may be defined for the high-level parameters.
• the maximum aspect ratio may be defined as a power-of-2 bigger than 4, thus significantly reducing the number of possible options.
• the minimum aspect ratio may be defined as the reciprocal of the maximum aspect ratio or vice versa, in which case only the minimum or maximum aspect ratio needs to be transmitted.
• both the split direction and the split ratio are coded using context-based adaptive arithmetic coding (CABAC).
• CABAC context-based adaptive arithmetic coding
• these parameters are coded as a series of binary decisions with adaptive binary probability models (a subset of the bins can also be coded with non-adaptive probability models).
• the used binary probability models for the split decision may be selected based on the size of neighboring blocks.
• the underlying assumption is that small blocks often occur in the neighborhood of other small blocks (i.e., in finely partitioned areas of a video picture). This or a similar assumption may also be used to model the contexts for the split ratio coding. Note that non-symmetric splits results in a smaller and a larger block and smaller blocks may tend to be located in the neighborhood of other small blocks.
• some final partitioning patterns could be generated using more than one partitioning tree.
• some combinations of splits are prohibited by the syntax. This can be achieved by prohibiting some tree patterns from being introduced into the partitioning tree.
• a tree pattern is defined using either the actual "vertical”/"horizontal” or the directional "parallel'Vperpendicular" split direction semantics, possibly mixed.
• a tree pattern also includes split ratios.
• FIG. 1 A specific partial partitioning of a 128x128 block (e.g., a CTU of size 128x128) using the means described in this embodiment is shown in fig. 1.
• the partitioning tree describing the shown topology is shown in fig. 2.
• Fig. 3 explains which splits directions and ratios were used at specific nodes.
• the overall set of split ratios is given by
• split ratios a/b are given as irreducible fractions (numerator and denominator are coprime).
• symmetric split with split ratio 1/2 several non-symmetric split ratios are included in the overall set of split ratios.
• the split ratio (b-a)/b is also included. Both splits yield the same subblock sizes, but for one of the splits, the larger subblock represents the left/top part, and for the other split, the larger subblock represents the right/bottom part of the block to be split.
• the symmetric split is included in the overall set of split ratios, in which case we have
• Criterion 1 Granularity of block_sizes
• the set of available splits can be constraint by a minimum block size. That means, if a split would results in a block size that is smaller than a minimum block size, it is prohibited and removed from the list of available splits.
• the minimum and maximum aspect ratio Another possible constraint is the minimum and maximum aspect ratio. If a split would result in any subblock with an aspect ratio (block width)/(block height) greater than a maximum aspect ratio or less than a minimum aspect ratio, the corresponding split is removed from the set of available splits.
• the minimum aspect ratio could be set equal to the reciprocal of the maximum aspect ratio.
• the maximum aspect ratio could be set equal to 8.
• the split depth represents the depth of the binary split tree structure. Or in other words, for a particular block, the split depth represents the number of splits required to reach the current block from the root block. If the split depth of a current block is already equal to the maximum split depth, no further split is allowed. As an example, the maximum split depth could be set equal to 3.
• Parameters which control the minimum block size, the minimum and maximum aspect ratio, the maximum split depth, and/or values restricting the use of non-symmetric splits are preferably transmitted in a high-level syntax structure, similar to the sequence parameter set, the picture parameter set, or the slice header of H.265
• the first part of Table 1 specifies an example for prohibited parallel splits.
• the parallel splits that are prohibited for the first subblock are labeled by "layout a)" and the parallel splits that are prohibited for the second subblock are labeled by "layout b)".
• the redundancy for consecutive parallel splits could also be removed by only prohibiting splits for the first subblock, or only prohibiting splits for the second subblock, or by using another combination of prohibiting splits for the first and second subblock.
• the coding order of the subblocks is defined in a way that for each split, the first subblock (top or left subblock) or all blocks contained in the first subblock is/are coded before the second subblock (bottom or right subblock) or all block contained in the second subblock.
• the four resulting blocks would be transmitted in the following order: top-left, bottom-left, top-right, bottom-right (see Fig. 12b).
• the described redundant perpendicular splits are typically still very similar, so that prohibiting these splits could actually improve coding efficiency.
• the loss in coding efficiency due to not considering both of the described coding options may be outweighed by the bit rate savings obtained by using shorter codewords for signaling the partitioning.
• a potential gain by considering both ways of splitting is only achievable if the encoder evaluates both options, which would increase the encoding complexity. So, in summary, it depends on the actual coding options and the application whether it is preferable to allow both options in the syntax.
• the described redundant perpendicular splits are prohibited.
• the potential redundancy is supported in the syntax.
• the bitstream contains a syntax element, which indicates whether redundant perpendicular splits are allowed or prohibited by the syntax, in the latter case, the actually used split ratio syntax is depending on the value of a corresponding syntax element.
• the syntax element that indicates whether redundant perpendicular splits are allowed or prohibited could be transmitted as part of any high-level syntax structure, such as the SPS, PPS, or slice header. It is also possible that this syntax element is transmitted as part of the low-level syntax, as for example, the CTU syntax or the CU syntax.
• redundant perpendicular splits are prohibited (either by convention or indicated by a high-level syntax element), it has to be defined which of the two variants is prohibited and which of the variants is supported. This decision could depend on which variant requires less binary decisions for coding or which variant most likely provides better intra-prediction signals or motion vector predictors, etc.
• Table 1 gives an example for specifying prohibited redundant splits. The prohibited perpendicular splits are marked with "layout c)". They have to be interpreted in the following way.
• split ratio A A- atio
• split ratio B B-Ratio
• Figure 7 shows three different partitioning tree patterns that are prohibited if specific split directions and ratios for the nodes A, B, and C are met.
• Table 1 all configurations prohibited in the described embodiment are listed. The local topology produced by those patterns can be exactly replicated using a different partitioning tree pattern.
• the prohibitions are chosen in correspondence to the least expensive splits.
• Layout a) with A-Ratio 3/4 and B-Ratio 2/3 yields the same pattern as Layout b) with A-Ratio 1/2 and B-Ratio 1/2.
• the former is prohibited whereas the latter is allowed due to the fact that 1/2 is less expensive in signaling (when using the signaling scheme described below).
• Method A (1) Vertical split with ratio 1/4 applied to 16x16 block at location (0,0)
• Method B (1) Vertical split with ratio 3/4 applied to 16x16 block at location (0,0)
• both methods yield the following partitioning into 5 subblocks: 4x8 block at location (0,0), 8x8 block at location (4,0), 4x8 block at location (0,8), 8x8 block at location (4,8), and 4x16 block at location (12,0).
• the coding order for both methods is usually different (in Fig. 13, the coding order is indicated by the numbers inside the resulting blocks), so that both ways of splitting actually represent different coding modes. Such higher-order redundancies could be prohibited, or they could be allowed by the syntax.
• the reduction of the set of asymmetric splits has the goal the enable that the areas asymmetrically split before should not be split asymmetrically again.
• This principle may be explained by other words as concept for avoiding one- dimensional / punctual redundancy.
• This split options are prohibited in syntax.
• the result can be that the set of possible splits comprises splits "positioned" in an alternating manner (e.g. left and right) so that the second split is approximated to the mirrored version of the previous split.
• a tree adaption set could define a specific set of leaf nodes or branches, or a set of rules to select leaf nodes or branches added and/or removed from the default tree.
• the aspect of an explicitly signaled tree adaption set might include the signaling of the use of a tree adaption set and/or the signaling of an index, selecting a tree adaption set out of a predefined record of tree adaption sets, using variable or fixed length codes embedded in any high level syntax structure.
• Implicit signaling might range from deriving the use of additional tree adaption sets as well as selecting a tree adaption set out of a record by evaluation of syntax elements coded in any high level syntax structure and/or on CTU-level or below.
• One aspect could link specific profile and level syntax elements to specific tree adaption sets, in order to limit complexity or eliminate unsupported block width, block height or block aspect ratios, which might not be compatible with certain profile- level combinations.
• a tree adaption set might be derived from syntax elements of the temporal or spatial neighborhood, restricting or expanding the default tree, according to assumed local requirements.
• an implicit tree adaption set could be activated by the extraordinary shape of a block at a picture, tile or shape boundary.
• a coding tree unit often has a different block size. For example, assume an HD picture of 1920x1080 luma samples and a CTU size of 128x128 luma samples. Then, the CTUs at the bottom of the picture only have a size of 128x56 luma samples. Here, while the CTUs are initially still considered to consist of 128x128 luma samples, an implicit splitting could be inferred.
• a horizontal split with split ration 1/2 and a following horizontal split with split ratio 7/8 for the first subblock could be inferred, which results in a 128x56 block; the remaining blocks that only include samples outside the picture are not coded.
• the binary decision tree for transmitting the split ratio could also be adapted on a picture or tile boundary. For example, splits that would introduce a block border outside the picture could be eliminated for the blocks at a boundary. This concept could also be used in connection with implicit splits.
• the block is vertically (at right boundary) or horizontally (at bottom boundary) split, but for transmitting the split ratio, all splits that would introduce a block border outside the picture/tile region are eliminated from the corresponding binary decision tree (i.e., the corresponding binary decision in the tree used for signaling the split ratio are inferred to be equal to 0 or 1).
• the high-level syntax includes syntax elements by which the encoder tells the decoder the actual configuration for the binary splitting tree.
• the encoder complexity only allows to evaluate a subset of the splitting options that are supported by the syntax.
• the encoder signals the decoder, which subset of the splitting options are used.
• the high-ievei syntax elements are transmitted in one or more high-level syntax structures, such as the sequence parameter set (SPS: valid for all pictures inside a coded video sequence), the picture parameter set (PPS: valid for one or more pictures), the slice header (valid for a slice), etc.
• the high-level syntax elements that restrict the available splitting option can include one or more of the following parameters:
• a minimum block sizes e.g., specified by a minimum block width and a minimum block height, or specified by a minimum number of samples
• a maximum partition tree depth following a first non-default split i.e., the tree depth after the first split with a split ration not equal to 34 is limited
• a maximum block size for a non-default split i.e., splits with a ratio not equal to 1/2 are only supported if the block size is greater than or equal to a given size
• a minimum block size for a non-default split i.e., splits with a ratio not equal to 1/2 are only supported if the block size is less than or equal to a given size
• the parameters can be included in more than one high-level syntax structures. For example, global parameters specified in the SPS can be overridden by parameters specified in the PPS.
• Fig. 5 depicts signaling diagrams for an exemplary split size of 64 (assuming no further restrictions).
• a special algorithm for pruning the binary decision tree can be employed.
• a particular embodiment of the invention uses a special algorithm to prune the signaling tree (an example for a full decision tree is shown in fig. 5).
• the concept uses basically the same tree of binary decisions for signaling all split ratios, but before any binary decision (bin) is actually coded, the tree is pruned based on the availability of the split ratios. Or in other words, binary decisions that can be inferred to be equal to 0 or 1 based on the availability of the split ratios are not transmitted, but inferred at the decoder side.
• the outputs of the signaling tree (leaf nodes of the tree) are marked according to their availability, where the availability can be derived according to any combination of the rules described above (including, minimum block size, block granularity, minimum and maximum aspect ratio, redundant parallel splits, redundant perpendicular splits, etc.).
• Each node contains an information about the context model that is used to encode the specific bin using the CAB AC engine.
• the reduction (or pruning) algorithm reduces the decision tree as follows. Initially, it is assumed that all nodes of the binary decision tree for signaling the split ratio are available. Then, the leaf nodes of the tree are marked as available or unavailable according to the availability of the corresponding split ratios (which is derived according to any rules, for example, the rules or a subset of the rules specified above). Afterwards, the decision tree is pruned from the bottom to the top according to the following two rules: (1) If a node has no available child node (i.e., both child nodes are marked as unavailable), the node is marked as unavailable (i.e., it is finally removed from the decision tree).
• FIG 6 three examples of reduced signaling decision trees are shown (for the full decision tree of fig. 5).
• fig. 6a an example of a reduced tree without possible outcomes ⁇ 3/8, 5/8 ⁇ is presented.
• figures 6b the outcome ⁇ 1/4 ⁇ is not available.
• figure 6c the ratios ⁇ 1/2, 3/8, 1/4 ⁇ are not available.
• binary decision tree for coding the split ratio for different split sizes are presented.
• the binary decisions are arithmetically coded using adaptive binary probability models.
• the selection of the probability model could depend on any of the following (or a combination thereof):
• the assignment of probability model sets (also referred to as context model sets) to binary decision nodes is not modified for reduced decision trees. That means, for each binary decision, the same context model set (among which a probability model is selected based on additional parameters) is used, regardless of whether the binary decision is coded as part of the full decision tree or any form of a reduced decision tree.
• the depth of a tree following a non-default split is restricted.
• a first non-default split would mean that scarce partitioning is done and the last few available depths will be used to fine-adjust the block edges.
• the probability model is selected based on the current depth of the splitting tree and the sizes of the neighboring blocks, if a subdivided block has small neighboring blocks on one end and large neighboring block on the other, it is assumed that the smaller of the blocks resulting from current subdivision will be placed at the side neighboring with other small blocks.
• the probability model for the binary decision labelled with "ctx6" is selected based on how much do the sizes of neighboring blocks vary (if they vary much, it is assumed that the 1/5 or 4/5 ratio could be more probable as they result in subdivisions with higher difference of areas).
• a specific embodiment could define the context models as follows. A split area is shown in fig. 10.
• the anchors for neighbor access are named Al, A2, Bl, and B2.
• the area is assumed to a special representation of a specific node in a split tree with a specific depth d. On the path to this node, first n ⁇ d splits were default splits, thus simulating a quad-tree of depth q - For coding of decision
• the neighboring blocks have a specific areas areaf/V-J and area(/V 2 ] ,
• the context offset ctx 0 ff is defined as 1 if area(NjJ equals area( 2 ) . If areafNJ is greater than area(/V 2 ) the ctx o; -f is defined as 0. In all other cases it is defined as 2.
• the selection of the binary probability models used for coding the binary decision is based on the assumption that a partition tree of a specific CTU in a picture is similar structured compared to a corresponding partition tree of a so called collocated CTU in an already decoded picture.
• a collocated CTU might be defined as CTU that is obtained when accessing a CTU in a given reference picture using the coordinates derived from the current CTU's position and size, for example the CTU's center position.
• From the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element of the collocated CU might be used to select probability models for signaling split decisions (in fig.4).
• syntax elements from the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element from the obtained collocated CU might be used to select probability models for signaling split ratio decisions (in fig.5-fig.9).
• syntax elements from the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element might be used to infer one or all binary split decisions (in fig.4) without additional signaling.
• syntax elements from the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element from the obtained collocated CU might be used to infer one, any or all split ratio decisions (in fig.5-fig.9).
• split decisions depend on the availability of splits.
• a split can be unavailable for several state dependent reasons. For example, in the particular embodiment defined in (1) for all blocks with widths and heights smaller than 32 a vertical 3/8 split can be available for the sizes
• split_mode xO, yO, cbSizeX, cbSizeY )
• split ratio div4_flag[ xO ][ yO ] 1 ae(v) else
• split decision variables such as SplitRatioDiv4Available, SplitRatioDiv8Available, lsSplitRatioDiv40ffsetCoded, lsSplitRatioDiv80ffsetCoded are subject to the availability constraints exemplified before and in particular the other high level syntax elements such as minimal block sizes and maximal tree depths, given in the description brought forward herein.
• This Lagrangian approach can also be used for deciding between different split modes. For obtaining optimal decisions, it is important to proceed the encoding using a depth-first approach.
• the best rate distortion cost for each of the subblocks has to be considered. This can be ensured by using a depth-first approach.
• the cost for a certain split mode is calculated as follows. First the cost of the first subblock is determined. At this point, the best coding mode (in rate-distortion sense) for the first subblock including all possible split modes is determined. Given the best coding mode for the first subblock, the best coding mode for the second subblock is determined (again, potential splits have to be taken into account).
• the rate- distortion cost / for a split mode is the sum of the rate-distortion costs for the subblocks. This concept can be implemented using a recursive function, similar to the following (simplified) pseudo-code: function rd_cost ( block )
• min _cost distortion ( block ) + lambda * rate ( block ) for ( split_mode ) // loop o er all split modes
• an encoder does not need to evaluate all coding options supported in the syntax. For decreasing the encoder complexity, it is typically beneficial to only test the most likely splitting modes and skip the evaluation of unlikely modes. Most notably, the maximum depth of a tree after the first non-default split is a very important parameter to balance the expected rd-performance and runtime. The maximum block size at which a non-default split is allowed can be used to restrict the testing of partitionings with large blocks, when much smaller blocks are more probable (e.g. for intra slices). Furthermore, the maximum aspect ratio tested in an encoder may be smaller than that supported by the syntax (correspondingly, the minimum aspect ratio tested may be larger than the minimum aspect ratio supported by the syntax).
• heuristic rules that do not change signaling may be incorporated to speed up the split decision process. For instance, during the rd-search a complete set of splits along one direction may be excluded if the corresponding symmetric split gives no improvement compared to the symmetric split in perpendicular direction. This, in particular, implies that a block will not be further divided after the two symmetric splits have been tried and did not result in an rd-cost smaller than that for coding the block without any split. The more asymmetric splits are supported by the syntax, the more such heuristic rules speed-up the encoding process. Another constraint may be set on the aspect ratio of the resulting splitting scheme (if one implements it as encoder only optimization, this would have no impact on signaling).
• the decoder 10 (cf. Fig. 14) is configured to partition the picture 12 of the data stream 14 into blocks 18.
• a recursive bi-partitioning is performed. This partitioning is controlled using partition information.
• a first split line direction of a first split line 16s is set.
• a predetermined sub-block 16a here the top left quarter of the picture block 16 is split by the recursive bi-partitioning into a pair of child sub-blocks CU1 and CU2.
• These sub-blocks CU1 and CU2 form the succeeding split level 3, while the sub-block 16a is formed by split level 1 and 2, assuming that level 0 is assigned to block 16 / 18.
• the setting of the split line 16s is performed in dependency on a second split line direction of a second split line 16s2 along which a grandfather sub-block of the preceding split level 1 of the picture block 16 is split into the predetermined sub-block and in the other sub-block.
• the grandfather sub-block refers to a left half the picture block 16 / 18.
• the grandfather sub-block can also be understood as being the entire picture block 16 or 18 itself, since the relationship to the preceding or succeeding split level (cf. Fig. 2 ) has to be taken into account.
• Another embodiment according to a second aspect provides a decoder for decoding the picture 12 from the data stream 14 which is configured to perform the partitioning of the picture block 18 (cf. Fig. 1) of picture 12 in decoding blocks 16 by recursive bi-partitioning. This is controlled by use of a partition information in the data stream 14, such that a predetermined sub-block, e.g., 16a or another block corresponding to any node in Fig. 2 (i.e., block 18 itself or a fraction) is partitioned taking into account:
• predetermined sub-block e.g., 16a
• split line direction e.g., a horizontal or vertical split line direction
• split line e.g., a horizontal or vertical split line direction
• the decoder 10 performs the picture partitioning by context-adaptively entropy decoding of one or more syntax elements controlling the splitting.
• a context of the data stream 14 is used. It should be noted that the context may, for example, depend on a size of the one or more neighboring coding blocks.
• the decoder 10 may be configured to perform the partitioning in at least two different modes between which the decoder 10 can switch. These two modes differ from each other and may, for example, have the aim:
• maximum partitioning tree depth maximum partitioning tree depth following a split of a current sub-block of a current picture block along a split line perpendicular to a split line of a parent block
• maximum block size for a sub-block of a current picture for which non-symmetrical splits are allowed maximum/minimum aspect ratio of a child sub-block resulting from a split of a current sub- block of a current picture block, etc.
• the decoder 10 may perform the splitting.
• the decoder 10 is configured to reduce a base list (cf. reference numeral 30 of Figs. 5 and 6) of split ratios by removing split ratio according to one or more rules so as to obtain a reduced list 32 of split ratio.
• the base list 30 corresponds to a binary tree 34 leaves 42, 37 of bridge are associated with respect to one of the split ratios of the base list 30.
• the reduced list 32 corresponds to a reduced binary tree 50 resulting from the binary tree 34 by removing, from leaves 42, 37 to main root node 44.
• Each node 40, 38 of the binary tree 34 has child nodes which do belong to the reduced list 32. After that, an attaching, in place of each node of the binary tree 34 having merely one non-removed child node is performed, so that the non-removed child node is directly attached to the node of the binary three 34 which their respective intermediary node is a child of.
• the left, e.g., the first, branches (exemplarily assigned “0" in Figs. 5 and 6) of the reduced binary tree 50 are associated with a first bin value, while the right (second) branches (exemplarily assigned "1" in Figs. 5 and 6) are associated with a second bin value.
• Each split ratio of the reduced list 32 has associated therewith a bin value sequence corresponding to a sequence of a bin vaiue associated with the braches which lead, in reduced binary tree 50, from a main root node 52 of the reduced binary tree 50 to a leaf 54 associated with the respective split ratio. After that, an entropy coding of a bin string corresponding to a sequence of one or more bins is performed.
• the generalized binary splitting framework for partitioning blocks into coding units, prediction units, transform units, etc. may affect other parts of a video codec.
• Another aspect is related to a coding of the last non-zero transform coefficient in a transform block.
• the position of the last non-zero transform coefficient is indicated by transmitting the x-coordinate and the y-coordinate of the last non-zero transform coefficient.
• the corresponding entropy codes were designed for blocks for which both the width and height represent integer powers of two.
• Iast_sig_coeff_ _suffix FL cMax (1 « ( ( last_sig_coeff_y_prefix »
• the binarization for the syntax elements last_sig_coeff_x_prefix, lctst_sig_coeff_y_prefix, last_sig_coeff_x_suffix, last_sig_coeff_y_suffix is modified as follows:
• TR cMaxPrefixX floor( ( log2( TrafoSizeX ) * 2 ) - 0.5 ), last_sig_coeff_x_prefix
• the binarization codes for TrafoSize equal to an integer power of 2 are not changed. Note that version 1 includes redundancies (unnecessary bins for certain codewordes), while the codeword tables for version 2 do not include redundant bins.
• the new partition structure requires various new transforms among them Hadamard transform matrices for prediction residuals.
• the Hadamard transform is used as a very simple approximation of a DCT.
• the sum of absolute Hadamard coefficients is, for example, used as rate-distortion measure in the sub-sample motion search or the pre-selection of intra prediction modes.
• the new luma block dimensions 96, 80, 48, 40, 24, 20, and 12 appear if the maximal CTU size is 128.
• the corresponding Hadamard matrices are not all explicitly constructed. In fact it is sufficient to construct Hadamard matrices of size 20 and 12 using Paley's construction via finite fields. The other sizes are then constructed by Kronecker products since Kronecker products of Hadamard matrices are Hadamard matrices again. The construction of the other matrices via Kronecker products have the advantage that a full large sized Hadamard matrix never has to be constructed entirely.
• the number and topology of neighbors of a particular block may vary. Coding parameters of neighboring block are, for example, used for deriving merge candidates, motion vector prediction candidates, or most probable intra prediction modes.
• an adaption of the state-of-the-art strategy for selecting candidates among the neighboring blocks could led to an improved coding efficiency.
• the state-of-the-art strategies often implicitly assume that blocks have a square shape. But with the generalized splitting concept, the block width can be much larger than the block height or vice versa.
• the long block edge it can be preferable to consider more candidate block along the long block edge than along the short block edge.
• This may include but is not limited to: Scanning for all neighbors along the edges of a block (e.g. to find an extremum or a mean of a measure derived from the neighboring blocks); defining additional neighbor anchor points (e.g. in the case of scanning for merge candidates, adding anchors in the middle of the left and upper edge of a block).
• Fig. 16 shows an apparatus for predictively coding a video 110 composed of a sequence of pictures 120 into a data stream 140. Block-wise predictive coding is used to this end. Further, transform-based residual coding is exemplarily used.
• the apparatus, or encoder, is indicated using reference sign 100.
• Fig. 16 shows a corresponding decoder 200, i.e.
• an apparatus 200 configured to predictively decode the video 110' composed of pictures 120' in picture blocks from the data stream 140, also here exemplarily using transform-based residual decoding, wherein the apostrophe has been used to indicate that the pictures 120' and video 110', respectively, as reconstructed by decoder 200 deviate from pictures 120 originally encoded by apparatus 100 in terms of coding loss introduced by a quantization of the prediction residual signal.
• Fig. 16 and Fig. 17 exemplarily use transform based prediction residual coding, although embodiments of the present application are not restricted to this kind of prediction residual coding. This is true for other details described with respect to Fig. 16 and 17, too, as will be outlined hereinafter.
• the encoder 100 is configured to subject the prediction residual signal to spatial-to-spectral transformation and to encode the prediction residual signal, thus obtained, into the data stream 140.
• the decoder 200 is configured to decode the prediction residual signal from the data stream 14 and subject the prediction residual signal thus obtained to spectral-to-spatial transformation.
• the encoder 100 may comprise a prediction residual signal former 220 which generates a prediction residual 240 so as to measure a deviation of a prediction signal 260 from the original signal, i.e. video 110 or a current picture 120.
• the prediction residual signal former 220 may, for instance, be a subtractor which subtracts the prediction signal from the original signal, i.e. current picture 120.
• the encoder 100 then further comprises a transformer 280 which subjects the prediction residual signal 240 to a spatial-to-spectral transformation to obtain a spectral-domain prediction residual signal 240' which is then subject to quantization by a quantizer 320, also comprised by encoder 100.
• the thus quantized prediction residual signal 240" is coded into bitstream 140.
• encoder 100 may optionally comprise an entropy coder 340 which entropy codes the prediction residual signal as transformed and quantized into data stream 140.
• the prediction residual 260 is generated by a prediction stage 360 of encoder 100 on the basis of the prediction residual signal 240" decoded into, and decodable from, data stream 140.
• the prediction stage 360 may internally, as is shown in Fig. 16, comprise a dequantizer 380 which dequantizes prediction residual signal 240" so as to gain spectral-domain prediction residual signal 240"', which corresponds to signal 240' except for quantization loss, followed by an inverse transformer 400 which subjects the latter prediction residual signal 240"' to an inverse transformation, i.e.
• prediction residual signal 240" which corresponds to the original prediction residual signal 240 except for quantization loss.
• a combiner 420 of the prediction stage 360 then recombines, such as by addition, the prediction signal 260 and the prediction residual signal 240"" so as to obtain a reconstructed signal 460, i.e. a reconstruction of the original signal 120.
• Reconstructed signal 46 may correspond to signal 120'.
• a prediction module 440 of prediction stage 360 then generates the prediction signal 260 on the basis of signal 460 by using, for instance, spatial prediction, i.e. intra prediction, and/or temporal prediction, i.e. inter prediction. Details in this regard are described in the following.
• decoder 200 may be internally composed of components corresponding to, and interconnected in a manner corresponding to, prediction stage 360.
• entropy decoder 500 of decoder 200 may entropy decode the quantized spectral-domain prediction residual signal 240" from the data stream, whereupon dequantizer 520, inverse transformer 540, combiner 560 and prediction module 580, interconnected and cooperating in the manner described above with respect to the modules of prediction stage 360, recover the reconstructed signal on the basis of prediction residual signal 240" so that, as shown in Fig. 17, the output of combiner 560 results in the reconstructed signal, namely the video 110'or a current picture 120' thereof.
• the encoder 100 may set some coding parameters including, for instance, prediction modes, motion parameters and the like, according to some optimization scheme such as, for instance, in a manner optimizing some rate and distortion related criterion, i.e. coding cost, and/or using some rate control.
• encoder 100 and decoder 200 and the corresponding modules 440, 580 respectively, support different prediction modes such as intra-coding modes and inter-coding modes which form a kind of set or pool of primitive prediction modes based on which the predictions of picture blocks are composed in a manner described in more detail below.
• the granularity at which encoder and decoder switch between these prediction compositions may correspond to a subdivision of the pictures 120 and 120', respectively, into blocks. Note that some of these blocks may be blocks being solely intra-coded and some blocks may be blocks solely being inter-coded and, optionally, even further blocks may be blocks obtained using both intra-coding and inter-coding, but details are set-out hereinafter.
• intra-coding mode a prediction signal for a block is obtained on the basis of a spatial, already coded/decoded neighborhood of the respective block.
• intra-coding sub-modes may exist the selection among which, quasi, represents a kind of intra prediction parameter.
• intra-coding sub-modes There may be directional or angular intra-coding sub-modes according to which the prediction signal for the respective block is filled by extrapolating the sample values of the neighborhood along a certain direction which is specific for the respective directional intra-coding sub-mode, into the respective block.
• the intra-coding sub-modes may, for instance, also comprise one or more further sub-modes such as a DC coding mode, according to which the prediction signal for the respective block assigns a DC value to all samples within the respective block, and/or a planar intra-coding mode according to which the prediction signal of the respective block is approximated or determined to be a spatial distribution of sample values described by a two-dimensional linear function over the sample positions of the respective block with deriving tilt and offset of the plane defined by the two-dimensional linear function on the basis of the neighboring samples.
• DC coding mode a DC coding mode
• planar intra-coding mode according to which the prediction signal of the respective block is approximated or determined to be a spatial distribution of sample values described by
• a prediction signal for a block may be obtained, for instance, by temporally predicting the block inner.
• motion vectors may be signaled within the data stream, the motion vectors indicating the spatial displacement of the portion of a previously coded picture of the video 110 at which the previously coded/decoded picture is sampled in order to obtain the prediction signal for the respective block.
• data stream 140 may have encoded thereinto prediction related parameters for assigning to the blocks prediction modes, prediction parameters for the assigned prediction modes, such as motion parameters for inter- prediction modes, and, optionally, further parameters which control a composition of the final prediction signal for the blocks using the assigned prediction modes and prediction parameters as will be outlined in more detail below.
• the data stream may comprise parameters controlling and signaling the subdivision of picture 120 and 120', respectively, into the blocks. The decoder 200 uses these parameters to subdivide the picture in the same manner as the encoder did, to assign the same prediction modes and parameters to the blocks, and to perform the same prediction to result in the same prediction signal.
• Fig. 18 illustrates the relationship between the reconstructed signal, i.e. the reconstructed picture 120', on the one hand, and the combination of the prediction residual signal 240"" as signaled in the data stream, and the prediction signal 260, on the other hand.
• the prediction signal 260 is illustrated in Fig. 18 as a subdivision of the picture area into blocks 800 of varying size, although this is merely an example.
• the subdivision may be any subdivision, such as a regular subdivision of the picture area into rows and columns of blocks, or a multi-tree subdivision of picture 120 into leaf blocks of varying size, such as a quadtree subdivision or the like, wherein a mixture thereof is illustrated in Fig. 18 where the picture area is firstly subdivided into rows and columns of tree-root blocks which are then further subdivided in accordance with a recursive multi-tree subdivisioning to result into blocks 800.
• the prediction residual signal 240"" in Fig. 18 is also illustrated as a subdivision of the picture area into blocks 840. These blocks might be called transform blocks in order to distinguish same from the coding blocks 800.
• Fig. 18 illustrates that encoder 100 and decoder 200 may use two different subdivisions of picture 120 and picture 120', respectively, into blocks, namely one subdivisioning into coding blocks 800 and another subdivision into blocks 840. Both subdivisions might be the same, i.e. each block 80, may concurrently form a transform block 840 and vice versa, but Fig.
• a subdivision into transform blocks 840 forms an extension of the subdivision into blocks 800 so that any border between two blocks 800 overlays a border between two blocks 840, or alternatively speaking each block 800 either coincides with one of the transform blocks 840 or coincides with a cluster of transform blocks 840.
• the subdivisions may also be determined or selected independent from each other so that transform blocks 840 could alternatively cross block borders between blocks 800.
• similar statements are thus true as those brought forward with respect to the subdivision into blocks 800, i.e.
• the blocks 840 may be the result of a regular subdivision of picture area into blocks, arranged in rows and columns, the result of a recursive multi-tree subdivisioning of the picture area, or a combination thereof or any other sort of segmentation.
• blocks 800 and 840 are not restricted to being quadratic, rectangular or any other shape.
• the subdivision of a current picture 120 into blocks 800 at which the prediction signal is formed, and the subdivision of a current picture 120 into blocks 840 at which the prediction residual is coded may not the only subdivision used for coding/decoding. These subdivision from a granularity at which prediction signal determination and residua!
• encoder and decoder may set certain coding parameters which might include some of the aforementioned parameters such as prediction parameters, prediction signal composition control signals and the like.
• Fig. 18 illustrates that the combination of the prediction signal 260 and the prediction residual signal 240"" directly results in the reconstructed signal 120'.
• more than one prediction signal 260 may be combined with the prediction residual signal 240"" to result into picture 120' in accordance with alternative embodiments such as prediction signals obtained from other views or from other coding layers which are coded/decoded in a separate prediction loop with separate DPB, for instance.
• the transform blocks 840 shall have the following significance.
• Transformer 280 and inverse transformer 540 perform their transformations in units of these transform blocks 840.
• many codecs use some sort of DST or DCT for all transform blocks 840.
• Some codecs allow for skipping the transformation so that, for some of the transform blocks 840, the prediction residual signal is coded in in the spatial domain directly.
• encoder 100 and decoder 200 are configured in such a manner that they support several transforms.
• the transforms supported by encoder 100 and decoder 200 could comprise: o DCT-II (or DCT-III), where DCT stands for Discrete Cosine Transform o DST- IV, where DST stands for Discrete Sine Transform o DCT- IV o DST-VII o Identity Transformation (IT)
• transformer 280 would support all of the forward transform versions of these transforms, the decoder 200 or inverse transformer 540 would support the corresponding backward or inverse versions thereof: o Inverse DCT-II (or inverse DCT-III) o Inverse DST- IV o Inverse DCT-IV 0 Inverse DST-VII 0 Identity Transformation (IT)
• the set of supported transforms may comprise merely one transform such as one spectral-to-spatial or spatial-to-spectral transform.
• Figs. 16 - 18 have been presented as an example where the composed- prediction concept described further below may be implemented in order to form specific examples for video encoders and decoders according to the present application.
• the video encoder and decoder of Figs. 16 and 18, respectively represent possible implementations of the video encoders and decoders described herein below.
• the video encoder of Fig. 16 and the video decoder of Fig. when having the subsequently explained embodiments for composed prediction according to the present application built into the video encoder and decoder of Fig. 16 and 17, the video encoder of Fig. 16 and the video decoder of Fig.
• FIG. 17 support, at least as one option, to process a block 800 in the manner outlined in more detail below, or even all blocks a current picture 120 is composed of.
• the embodiments described hereinafter, inter alias refer to a video encoder which equais the encoder 100 of Fig. 16 which treats blocks 800 in the manner outlined in more detail below and the same applies with respect to the decoder of Fig. 18 which, thus, represents an example for a video decoder according to an embodiment where blocks 800 are treated in the manner outlined in more detail below.
• Fig 16 and 17 are, however, only specific examples.
• a video encoder according to embodiments of the present application may, however, perform block-based encoding using the concept outlined in more detail below and being different from the encoder of Fig.
• video decoders may perform decoding from data stream 140 using the composed-prediction coding concept further outlined below, but may differ, for instance, from the decoder 200 of Fig. 17 in that sub-divides picture 120' into blocks in a manner different than described with respect to Fig. 180 and/or in that same does not derive the prediction residual from the data stream 140 in transform domain, but in spatial domain, for instance.
• one subdivision such as the subdivision into transform blocks, may form an extension of the other subdivision as described above, or both subdivisions may form separate extensions of a common primary subdivision such as, for instance, the subdivision of the picture into an array of tree root blocks as described with respect to Fig. 18. And such possibilities also apply for other sub-picture granularities which will be mentioned below such as with respect to the definition of certain prediction parameters, prediction modes, contribution weights or the like. Different subdivisions may be used for different ones of these entities and same may be defined independent from each other, partially independent or as extensions from one another.
• the following description concentrates on predicting blocks 800 at encoder and decoder.
• the aim is to improve the coding efficiency of video using mu!ti-hypotheses with limited memory access bandwidth.
• the average number of prediction hypotheses over a given area is limited.
• the area could be, e.g., a fixed number of (neighboring) blocks, a slice, a picture, a group of pictures, or a whole video sequence.
• this limit would be equal to three and the area would be a picture, it would be possible to have half of the picture using four prediction hypotheses and the other half using two prediction hypotheses. That is, as shown in Fig. 19, for the prediction block 800, i.e., inter predicted block within a predetermined portion of the video, four blocks 840, i.e., four prediction hypotheses are used.
• the number, i.e., the average number, of prediction hypotheses for each inter predicted block may be inserted into the data stream information or on the number of prediction hypotheses for a currently processed inter predicted block dependent on a number of prediction hypotheses of previously processed inter predicted blocks within the predetermined portion and the predetermined criterion.
• the information is coded into the data stream based on the number of prediction hypotheses of the previously processed inter predicted blocks within the predetermined portion and the predetermined criterion, determine an allowed number interval, and, using a binarization of the allowed number interval, code the information into the data stream.
• a maximum of the allowed number interval is the lower, the larger the number of prediction hypotheses of the previously processed inter predicted blocks within the predetermined portion is, and the binarization of the allowed number interval has a mean codeword length which is the lower, the lower the maximum of the allowed number interval is.
• the number of prediction hypotheses is limited depending on the block size or geometry. It is known that the relative prediction memory access overhead of each prediction hypothesis is higher for smaller blocks. Therefore, by allowing a smaller number of hypotheses for smaller blocks, and a larger number of hypotheses for bigger blocks, thus the overall worst-case memory access bandwidth can be kept constant. That is, the Number of hypotheses is determined by comparing an upper limit for the number N of hypotheses depending on the size or geometry of the predetermined inter predicted block. As an example, as shown in Fig.
• the Number N determined by using the above formula indicates a maximum number of prediction hypotheses for the predetermined inter prediction blocks within the predetermined portion, i.e., the number an upper bound with respect to the number of hypotheses signalable for the predetermined inter predicted block.
• the number of prediction hypotheses for each inter predicted block may be inserted into the data stream information, or based on the size or the geometry of the predetermined inter predicted block, determine an allowed number interval, and, using a binarization of the allowed number interval, code the information into the data stream.
• the information regarding the number of the prediction hypotheses may indicate a maximum of the allowed number interval is the lower, the smaller the size of the predetermined inter predicted block is, and the binarization of the allowed number interval has a mean codeword length which is the lower, the lower the maximum of the allowed number interval is.
• the information may indicate that the number of hypotheses is the larger, the larger the predetermined inter predicted block is.
• different interpolation filters are used for different subsets of the prediction hypotheses. For example, for the first and second (uni/bi-predictive) prediction hypotheses, a longer interpolation filter than for the subsequent hypotheses could be used. By doing so, the relative prediction memory access overhead for the subsequent hypotheses would be reduced. In case that a longer interpolation filter is used for a smaller number of the prediction hypotheses, then, a shorter interpolation filter is used for a larger number of the prediction hypotheses. In other words, dependency between the number of the prediction hypotheses and the size of the interpolation filter is monotonic.
• different motion vector accuracies are used for different subsets of the prediction hypotheses.
• the first and second (uni/bi-predictive) prediction hypotheses could use quarter-pel motion vector accuracy, whereas the subsequent hypotheses use full-pel accuracy. Consequently, the relative prediction memory access overhead for the subsequent hypotheses would be equal to one per hypothesis.
• the coarser the motion vector accuracy is used, i.e., the relationship between the number of the prediction hypotheses and the motion vector accuracy is monotonic.
• aspects described in the context of an apparatus it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
• Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.
• the inventive encoded signal can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.
• embodiments of the invention can be implemented in hardware or in software.
• the implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
• Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
• embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer.
• the program code may for example be stored on a machine readable carrier.
• inventions comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
• an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
• a further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.
• the data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary.
• a further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.
• the data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.
• a further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
• a processing means for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
• a further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
• a further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver.
• the receiver may, for example, be a computer, a mobile device, a memory device or the like.
• the apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
• a programmable logic device for example a field programmable gate array
• a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein.
• the methods are preferably performed by any hardware apparatus.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=59631540

Family Applications (1)

Country Status (3)

Families Citing this family (8)

• 2018
  • 2018-07-18 WO PCT/EP2018/069556 patent/WO2019016287A1/en active Search and Examination
  • 2018-07-18 EP EP18739569.4A patent/EP3656127A1/en active Pending
  • 2018-07-19 TW TW110109990A patent/TWI830012B/zh active
  • 2018-07-19 TW TW107125006A patent/TWI725328B/zh active

Also Published As

Similar Documents

Legal Events