RU2786652C2 - Connection between separation limitation elements - Google Patents

Abstract

FIELD: video encoding.

SUBSTANCE: invention relates to means for video encoding. A syntaxis element is obtained from a bit stream. Information on the minimum size, MinQtSizeY, is obtained in brightness samples of a leaf brightness block obtained as a result of separation of a quadrant tree. The maximum size, MaxBtSizeY, is determined in brightness samples of a root brightness block, so that it can be separated using separation of a binary tree, based on MinQtsizeY information and the obtained syntaxis element. Moreover, the syntaxis element is an element of syntaxis of the difference between a logarithm of the base 2 of MinQtSizeY and a logarithm of the base 2 of MaxBtSizeY; or the syntaxis element is an element of syntaxis of the difference between MinQtSizeY and MaxBtSizeY, wherein the difference syntaxis element indicates the mentioned difference in a logarithmic form with the base 2. At the same time, the method additionally contains stages, at which: it is determined whether it is allowed for application of binary separation to an image block based on MaxBtSizeY; and an encoding block of the image block is obtained based on the result of determination.

EFFECT: increase in the efficiency of video encoding due to an increase in the efficiency of image separation.

13 cl, 31 dwg

Description

The field of technology to which the invention belongs

The present invention relates generally to methods for encoding, decoding and dividing an image.

State of the art

The amount of video data required to display even a relatively short video can be significant, which can result in data loss when streamed or otherwise transmitted over a bandwidth-limited communications network. Thus, video data is typically compressed prior to transmission over modern telecommunications networks. The size of the video can also have a significant impact on the amount of data stored in the storage device, as memory resources can be limited. Video compression devices often use source software and/or hardware to encode video data before transmission or storage, thereby reducing the amount of data needed to represent digital video images. The compressed data is then received at the destination by a video decompressor, which decodes the video data. With limited network resources and ever-increasing demands for higher quality video, it is desirable to use improved compression and decompression methods that will increase the compression rate with minimal loss in image quality.

Generally, there are a plurality of separation limiting elements, each element restricting the separation of images (eg, video) with respect to different types of separation methods. These various types of splitting methods include, in particular, binary tree splitting, quadtree splitting, and ternary tree splitting. Partition constraint elements are typically signaled in a bitstream including encoded pictures. The following describes some of the important elements of the split constraint.

A minimum size (MinCbSizeY) of an encoding block may be specified. For example, MinCbSizeY could be 8, which means that an 8x8 parent block cannot be split using any of the splitting methods, since the resulting child block is guaranteed to be smaller than MinCbSizeY (either in width or height ). According to the second example, if MinCbSizeY is 8, the 8x16 parent box cannot be partitioned using, for example, quadtree partitioning, because the resulting four child boxes will be 4x8 (width is 4 and height is 8) and the width of the resulting child blocks will be less than MinCbSizeY. The second example assumed that MinCbSizeY applied to both the width and height of the block, although two different syntax elements could be used independently to constrain the width and height.

The maximum coding tree block size (CtbSizeY) may indicate the size of the maximum coding block in terms of the number of luma samples.

The maximum size (MaxBtSizeY) of a binary tree may be defined as a maximum luminance size (width or height) in terms of the number of coding block samples that can be divided using a binary splitting method. For example, if MaxBtSizeY is 64, a coding block that is larger in size, either in width or in height, cannot be divided using binary separation. This means that a 128x128 block cannot be split using binary splitting, while a 64x64 block can be split using binary splitting.

The minimum size (MinBtSizeY) of a binary tree may be defined as a minimum luminance size (width or height) in terms of the number of coding block samples that can be divided using a binary splitting method. For example, if MinBtSizeY is 16, a coding block that is less than or equal in size, either in width or in height, cannot be divided using binary separation. This means that an 8×8 block cannot be divided using binary partitioning, while a 32×32 block can be divided using binary partitioning.

The minimum size (MinQtSizeY) of the quadtree may be defined as the minimum luminance size of the leaf block resulting from splitting the coding tree unit (CTU) quadtree. Size can specify the width or height of the box in number of samples. It can also specify the width and height together in the case of square boxes. For example, if MinQtSizeY is 16, a coding block having a size less than or equal to 16 cannot be divided into child blocks using a quadtree splitting method. Typically, MinQtSizeY (and its signaled "log2_min_qt_size_intra_slices_minus2" and "log2_min_qt_size_inter_slices_minus2" syntax elements are used to denote the minimum block size of a quadtree. In particular, the size syntax element can be an indirect syntax element, meaning "log2_min_qt_size_intra_slices_minus2" can be a binary logarithm of base 2) (with base 2) the number of luminance samples of the minimum block of the quadtree.

The minimum transform block size (MinTbSizeY) may be defined as the minimum transform block size in terms of the number of coding block samples that can be divided using a ternary splitting method. For example, if MinTbSizeY is 16, a coding block that is less than or equal in size in either width or height cannot be divided using ternary separation. This means that an 8×8 block cannot be divided using ternary partitioning, while a 32×32 block can be divided using ternary partitioning.

The maximum depth (MaxMttDepth) of a multitype tree may be defined as the maximum hierarchy depth for coding units resulting from splitting the multitype tree of a leafy quadtree or CTU. CTU or Coding Tree Block (CTB) describes the maximum block size that is used to divide a picture frame. MaxMttDepth describes an upper limit on the number of consecutive binary or ternary splits that can be applied to produce a child block. As an example, assuming that the CTU size is 128x128 (width is 128 and height is 128) and that MaxMttDepth is 1, each parent block (128x128 in size) can first be split into two 128x64 s child blocks using binary separation. However, the child blocks cannot apply any sequential binary split (to get 128x32 or 64x64 child blocks) because the maximum number of available binary splits has been reached. Note that MaxMttDepth can control the maximum binary split depth or the maximum ternary split depth, or both. If binary and ternary split depths are controlled simultaneously, then one binary split followed by one ternary split can be considered two hierarchical splits. Typically, MaxMttDepth (and its "max_mtt_hierarchy_depth_inter_slices" and "max_mtt_hierarchy_depth_intra_slices" syntax elements) are used to denote the maximum hierarchy depth for coding blocks derived from a multi-type tree.

In addition, "pic_width_in_luma_samples" is a syntax element specifying a picture size element, namely the width of each decoded picture in blocks of luma samples. Pic_width_in_luma_samples should not normally be equal to 0 and should be an integer multiple of MinCbSizeY.

Likewise, "pic_height_in_luma_samples" is a syntax element specifying a picture size element, namely the height of each decoded picture in blocks of luma samples. Pic_height_in_luma_samples should not normally be equal to 0 and should be an integer multiple of MinCbSizeY.

One of the objectives of video encoding is to provide the highest possible quality at the lowest possible speed. One of the factors contributing to the solution of this technical problem is to increase the efficiency of the bit stream structure.

Disclosure of the essence of the invention

In view of the aforementioned technical problem, embodiments of the invention aim to improve the current implementation of image division. In particular, the technical challenge is to increase the availability and flexibility of various image partitioning methods, that is, binary tree, quadtree, and ternary tree partitioning methods. The technical problem is to provide the ability to encode and decode images of large sizes.

Embodiments of the invention are presented in the attached independent claims. Advantageous embodiments of the invention are further defined in the dependent claims.

The above and other technical problems are solved by the subject of independent claims. Additional forms of implementation are obvious from the dependent claims, description and drawings.

Specific embodiments are set forth in the appended independent claims and other embodiments are described in the dependent claims.

In embodiments of the invention, new partitioning rules are established, in particular by establishing relationships between partition constraint elements and transmitting them in the bitstream along with encoded pictures. They can be used to encode video data into a bitstream and decode the bitstream into decoded video data.

According to a first aspect, the invention relates to an apparatus for decoding or processing a bit stream including encoded pictures. The device includes a circuit configured to receive from the bitstream (101) a syntax element; obtain the minimum size information in the luminance samples of the leaf luminance block obtained by splitting the quadtree, MinQtSizeY (105); determine whether the maximum size in luminance samples of the root luminance block can be partitioned using binary tree partitioning, MaxBtSizeY (102), based on the information for MinQtSizeY (105) and the resulting syntax element.

By defining new partitioning rules by establishing relationships between these partition constraint elements, the decoding device is able to increase the availability and flexibility of various image partitioning methods, in particular quadtree and binary tree partitioning.

In an exemplary form of implementing the method according to the first aspect, the circuit is configured to determine MaxBtSizeY given that its lower limit is MinQtSizeY.

In a possible implementation form of the method according to the first aspect or the above implementation, the syntax element is the syntax element (301) of the difference between the base 2 logarithm MinQtSizeY (105) and the base 2 logarithm MaxBtSizeY (102).

Thus, it is possible to easily output the corresponding split constraint elements, for example, at the decoder side, while reducing the information overhead in the bitstream. Difference is an example of a relationship. However, the ratio can also be a proportionality factor, calculation scheme, etc., which makes it possible to obtain MaxBtSizeY from MinQtSizeY.

In a possible implementation form of the method according to the first aspect or the above implementation, the syntax element is a difference syntax element (301) between MinQtSizeY (105) and MaxBtSizeY (102), the difference syntax element may be, i. 2. This difference syntax element provides a compact way to signal a difference.

In an exemplary form of implementation of the method according to the first aspect or the above implementation, the circuit is configured to obtain a syntax element from the bitstream depending on the maximum hierarchy depth for coding blocks obtained by splitting the quadtree leaf block polytype tree (MaxMttDepth) Thus, can be obtained corresponding split constraint elements, for example on the decoder side, while reducing the information overhead in the bitstream.

In addition or alternatively, the schema may be configured to not receive any MaxBtSizeY syntax element from the bitstream if MaxMttDepth is zero.

In addition or alternatively, the schema may be configured to not receive the syntax element from the bitstream (101) if MaxMttDepth (103) is non-zero.

According to a second aspect, the invention relates to an apparatus for generating or processing a bitstream including encoded images, an apparatus (which is configured to: determine a minimum luminance size of a leaf block resulting from splitting a quadtree, MinQtSizeY, determine a maximum luminance size of a coding block, which should be split using binary tree splitting, MaxBtSizeY based on MinQtSizeY, and include information of the specified MinQtSizeY in the bitstream.

This method provides an efficient encoder implementation in which the generated stream has a compact syntax and allows the decoder to efficiently derive the constraint parameters. The above benefits for the decoder side also apply to the encoder side on which the bitstream is generated.

In a possible form of implementation of the method according to the second aspect or the above implementation, the device (its processing circuitry) is configured to determine MaxBtSizeY given that its lower limit is MinQtSizeY.

In an exemplary form of implementing the method according to the second aspect or the above implementation, the device for generating or processing the bitstream may be further configured to include in the bitstream (101) a difference syntax element between the base 2 logarithm MinQtSizeY ( 105) and the base 2 logarithm MaxBtSizeY (102) in the bitstream.

In a possible implementation form of the method according to the second aspect or the above implementation, the syntax element is the syntax element (301) of the difference between MinQtSizeY (105) and MaxBtSizeY (102), the difference syntax element signals said difference in logarithmic scale with base 2.

In a possible form of implementation of the method according to the second aspect or the above implementation, the device is configured to include in the bitstream a syntax element MaxBtSizeY depending on the maximum hierarchy depth for coding blocks resulting from splitting the multitype tree, MaxMttDepth.

In addition or alternatively, the device is configured to not include any MaxBtSizeY syntax element in the bitstream if MaxMttDepth is zero.

In addition or alternatively, the device is configured to include any MaxBtSizeY syntax element in the bitstream if MaxMttDepth is non-zero.

According to a third aspect, the invention relates to a method provided for generating or processing a bitstream (including encoded pictures), including the steps of: determining a minimum luminance size of a leaf block resulting from splitting a quadtree, MinQtSizeY, determining a maximum luminance size of a coding block , which is to be split using binary tree splitting, MaxBtSizeY, based on MinQtSizeY, and including the specified MinQtSizeY information (105) in the bitstream.

According to a fourth aspect, the invention relates to a method provided for decoding or processing a bitstream including encoded pictures, comprising the steps of: deriving a syntax element from the bitstream (101); obtaining the minimum size information in the luminance samples of the leaf luminance block obtained by splitting the quadtree, MinQtSizeY (105); determining whether the maximum size in luminance samples of the root luminance block can be partitioned using binary tree partitioning, MaxBtSizeY (102) based on the information MinQtSizeY (105) and the obtained syntax element.

Wherein the method may further comprise determining whether binary splitting is allowed to be applied to the image block based on MaxBtSizeY; obtaining a coding block of the image block based on the determination result; and obtaining recovered coding block sample values.

Wherein the image block may be a luminance block of encoded images.

Wherein the syntax element may indicate the difference between the base 2 logarithm of MaxBtSizeY (102) and the base 2 logarithm of MinQtSizeY (105); or the syntax element may indicate the difference between MaxBtSizeY and MinQtSizeY.

Where the syntax element may be from a bitstream segment header.

The method according to the third aspect of the invention can be performed by the device according to the first aspect of the invention. Additional features and implementations of the method according to the first aspect of the invention correspond to the features and implementations of the device according to the third aspect of the invention.

The method according to the fourth aspect of the invention can be performed by the device according to the second aspect of the invention. Additional features and implementations of the method according to the second aspect of the invention correspond to the features and implementations of the device according to the fourth aspect of the invention.

The method according to the second aspect can be extended to implementation forms corresponding to the implementation forms of the first device according to the first aspect. Therefore, the implementation form of the method contains the feature(s) of the corresponding implementation form of the first device.

The advantages of the methods according to the fourth aspect are the same as the advantages of the corresponding forms of implementation of the method according to the third aspect.

According to a fifth aspect, the invention relates to a device for decoding a video stream, including a processor and a memory. The memory stores instructions that cause the processor to execute the method according to the third aspect.

According to a sixth aspect, the invention relates to a device for encoding a video stream, including a processor and a memory. The memory stores instructions that cause the processor to execute the method according to the fourth aspect.

According to a seventh aspect, a computer-readable storage medium is provided that stores instructions that, when executed, cause one or more processors to encode video data. The instructions cause one or more processors to execute the method according to the third or fourth aspect, or any possible embodiment of the third or fourth aspect.

According to an eighth aspect, the invention relates to a computer program containing program code for executing the method according to the third or fourth aspect, or any possible embodiment of the third or fourth aspect, when executed on a computer.

The accompanying drawings and the description below set forth the details of one or more embodiments. Other features, objects and advantages will be apparent from the description, drawings and claims.

According to an embodiment, a computer program product is provided that contains program code for controlling a device to perform any of the above methods when the program code is executed by one or more processors of the device.

For the sake of clarity of any of the embodiments disclosed herein, an embodiment may be combined with any one or more other embodiments to provide a new embodiment within the scope of the present invention.

These and other features will be better understood from the following detailed description taken in conjunction with the accompanying drawings and claims.

It should be noted that all devices, elements, blocks and means described in the present invention may be implemented in software or hardware elements, or any combination thereof. All of the steps that are performed by the various entities described herein, as well as the functionality described to be performed by the various entities, are intended to indicate that the respective entity is adapted or capable of performing the respective steps and functions. Even if, in the following description of specific embodiments, a specific functionality or step to be performed by external entities is not reflected in the description of the specific detailed element of that object that performs that particular step or functionality, it should be clear to the skilled artisan that these methods and functionality may be implemented in appropriate software or hardware elements, or in any combination thereof.

Brief description of the drawings

The above aspects and forms of implementation will be explained in the following description of specific implementation options with reference to the accompanying drawings, in which

1 shows a device according to an embodiment of the invention.

2 shows the SPS RBSP syntax for a device according to a second embodiment of the invention.

3 shows the segment header syntax for a device according to a second embodiment of the invention.

4 shows the SPS RBSP syntax for a device according to a third embodiment of the invention.

5 shows the segment header syntax for a device according to a fourth embodiment of the invention.

6 shows the segment header syntax for a device according to a fourth embodiment of the invention.

7 shows a method according to an embodiment of the invention.

Fig.8 shows the usual syntax of SPS RBSP.

9 shows the usual segment header syntax.

10 shows a device according to an embodiment of the invention.

11A is a block diagram illustrating an exemplary coding system that may implement embodiments of the invention.

11B is a block diagram illustrating another example of a coding system that may implement embodiments of the invention.

12 is a block diagram illustrating an example of a video encoder that may implement embodiments of the invention.

13 is a block diagram illustrating an example of a video decoder that may implement embodiments of the invention.

14 is a diagram of a network device according to an embodiment of the invention.

FIG. 15 is a simplified block diagram of a device that can be used as one or both of source device 12 and destination device 14 of FIG. 11A according to an exemplary embodiment.

16A-F show different CU split modes in VVC.

17A shows the forced QT split of the lower boundary of the CTU (128×128) HD (1920×1080).

17B shows forced splitting BT (128x128) of the lower boundary of CTU (128x128) HD (1920x1080) according to an embodiment of the invention.

18 shows an example of defining a boundary.

19A shows an example of a corner case QTBT forced split according to an embodiment of the invention.

19B shows an example of forced QTBT splitting for a corner block according to an embodiment of the invention.

Fig.20 shows a variant of the definition of the border.

21 is an exemplary diagram of an example of blocking using a binary quadtree (QTBT) structure.

FIG. 22 is an exemplary diagram of an example of a tree structure corresponding to block division using the QTBT structure of FIG. 6. FIG.

23 is an illustrative diagram of an example of types of horizontal division of a ternary tree.

Fig. 24 is an exemplary diagram of an example of ternary tree vertical split types.

Fig. 25 is a block diagram showing an example of a video encoder capable of implementing embodiments of the invention;

Fig. 26 is a block diagram showing an exemplary structure of a video decoder capable of implementing embodiments of the invention;

27 is a block diagram showing an exemplary structure of a content delivery system 3100 that implements a content delivery service.

Fig. 28 is a block diagram showing the structure of an example terminal.

Detailed description of embodiments

First of all, it should be noted that although an illustrative implementation of one or more embodiments is provided below, the disclosed systems and/or methods may be implemented using any number of methods known or currently in use. The invention should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the illustrative arrangements and implementations illustrated and described herein, but may be varied within the scope of the appended claims, together with the full scope of their equivalents.

In the following description, reference is made to the accompanying drawings, which form part of the present invention and show, by way of illustration, specific aspects of embodiments of the invention or specific aspects in which embodiments of the present invention may be used. Obviously, embodiments of the invention can be used in other aspects and contain structural or logical changes not shown in the drawings. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

For example, it is obvious that the present invention in connection with the described method can also be true for the corresponding device or system configured to perform the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more blocks, such as functional blocks, to perform the one or more method steps described (for example, one block performing one or more steps, or a plurality of blocks , each of which performs one or more of a plurality of steps) even if such one or more units are not explicitly described and illustrated in the drawings. On the other hand, for example, if a particular device is described in terms of one or more blocks, such as functional blocks, the corresponding method may include one step to perform the functionality of one or more blocks (for example, one step to perform the functionality of one or more blocks). blocks, or a plurality of steps, each of which performs the functionality of one or more of the plurality of blocks) even if such one or more steps is not explicitly described or illustrated in the drawings. In addition, it is understood that features of the various exemplary embodiments and/or aspects described herein may be combined with one another unless specifically noted otherwise.

Video coding generally refers to the processing of a sequence of images that form a video or video sequence. Instead of the term "image", the term "frame" or "image" can be used as synonyms in the field of video coding. The video coding used in the present invention (or the present disclosure) indicates either video coding or video decoding. Video encoding is performed at the source side, typically involving processing (eg, by compressing) the source video images to reduce the amount of data needed to represent the video images (for more efficient storage and/or transmission). Video decoding is performed on the destination side and usually contains inverse processing compared to the encoder for recovering video images. Embodiments relating to "encoding" video images (or images in general, as will be explained later) should be understood to refer to either "encoding" or "decoding" a video sequence. The combination of an encoding part and a decoding part is also called CODEC (Coding and Decoding).

In the case of lossless video coding, the original video images can be reconstructed, that is, the reconstructed video images have the same quality as the original video images (assuming no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, additional compression is performed, for example by quantization, to reduce the amount of data representing video images that cannot be fully reconstructed in the decoder, i.e. the quality of the reconstructed video images is lower or worse compared to the quality of the original video images.

Several video coding standards, starting with H.261, belong to the group of "hybrid lossy video codecs" (ie, combine spatial and temporal sample-domain prediction and 2D transform coding to apply transform-domain quantization). Each picture of a video sequence is usually divided into a set of non-overlapping blocks, and encoding is usually done at the block level. In other words, in the encoder, the video is usually processed, that is, encoded, at the block (video block) level, for example, using spatial (inner picture) prediction and temporal (outer picture) prediction to generate a prediction block, subtracting the prediction block from the current block (block currently being processed/to be processed) to obtain a residual block, transforming the residual block and quantizing the residual block in the transform domain to reduce the amount of data to be transmitted (compressed), while the decoder partially performs inverse processing compared to encoder to the encoded or compressed block to restore the current block for presentation. In addition, the encoder duplicates the processing cycle of the decoder so that both will generate identical predictions (eg, intra and inter prediction) and/or re-restore for processing, ie, encoding, subsequent blocks.

As used herein, the term "block" may be part of an image or frame. For ease of description, embodiments of the invention are described herein with reference to High Efficiency Video Coding (HEVC) or Universal Video Coding (VVC) reference software developed by the Joint Collaborative Group on Video Coding (JCT-VC) ITU-T Video Coding Expert Group (VCEG) and the ISO/IEC Moving Picture Expert Group (MPEG). One of ordinary skill in the art will appreciate that embodiments of the invention are not limited to HEVC or VVC. May refer to CUs (coding units), PUs (prediction units), and TUs (transformation units). In HEVC, the CTU (coding tree unit) is divided into CUs using a quadratic tree structure referred to as a coding tree. The decision as to whether to encode an image area using extrinsic (temporal) or intrinsic (spatial) prediction is made at the CU level. Each CU may be further partitioned into one, two or four PUs according to the partition type of the PU. Within one PU, the same prediction process is applied, and the corresponding information is transmitted to the decoder based on the PU. After obtaining the residual block by applying a prediction process based on the division type of the PU, the CU may be divided into transform units (TUs) according to a different quadtree structure similar to the coding tree for the CU. The latest development of video compression technology uses a quadtree-binary-tree (QTBT) split frame to split a coding block. In the QTBT block structure, the CU may be square or rectangular. For example, a coding tree unit (CTU) is first divided by a quadtree structure. The leaf nodes of the quadtree are further separated by a binary tree structure. The leaf nodes of the binary tree are called coding units (CUs), and this segmentation is used to predict and process the transform without any additional partitioning. This means that CU, PU and TU have the same block size in the QTBT coding block structure. In parallel with the QTBT box structure, it has been proposed to use a plurality of sections, such as a ternary tree (TT) section, the term "device" can also mean "device", "decoder" or "encoder".

In the following embodiments, the encoder 20, the decoder 30, and the encoding system 10 will be described with reference to FIGS. 11-13.

In the following description, reference is made to the accompanying drawings, which form part of the invention and show, by way of illustration, specific aspects of embodiments of the invention or specific aspects in which embodiments of the present invention may be used. Obviously, embodiments of the invention can be used in other aspects and contain structural or logical changes not shown in the drawings. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

For example, it is clear that the invention in connection with the described method may also be true for a corresponding device or system configured to perform the method, and vice versa. For example, if one or more specific method steps are described, the corresponding apparatus may include one or more blocks, such as functional blocks for performing one or more method steps described (for example, one block performing one or more steps, or a plurality of blocks each of which performs one or more of the plurality of steps) even if such one or more units are not explicitly described and illustrated in the drawings. On the other hand, for example, if a particular device is described in terms of one or more blocks, such as functional blocks, the corresponding method may include one step to perform the functionality of one or more blocks (for example, one step to perform the functionality of one or more blocks). blocks, or a plurality of steps, each of which performs the functionality of one or more of the plurality of blocks) even if such one or more steps is not explicitly described or illustrated in the drawings. In addition, it is understood that features of the various exemplary embodiments and/or aspects described herein may be combined with one another unless specifically noted otherwise.

Video coding generally refers to the processing of a sequence of images that form a video or video sequence. Instead of the term "image", the term "frame" or "image" can be used as synonyms in the field of video coding. The video coding used in the present invention (or the present disclosure) indicates either video coding or video decoding. Video encoding is performed at the source side, typically involving processing (eg, by compressing) the source video images to reduce the amount of data needed to represent the video images (for more efficient storage and/or transmission). Video decoding is performed on the destination side and usually contains inverse processing compared to the encoder for recovering video images. Embodiments relating to "encoding" video images (or images in general, as will be explained later) should be understood to refer to either "encoding" or "decoding" a video sequence. The combination of an encoding part and a decoding part is also called CODEC (Coding and Decoding).

In the case of lossless video coding, the original video images can be reconstructed, that is, the reconstructed video images have the same quality as the original video images (assuming no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, additional compression is performed, for example by quantization, to reduce the amount of data representing video images that cannot be fully reconstructed in the decoder, i.e. the quality of the reconstructed video images is lower or worse compared to the quality of the original video images.

Several video coding standards, starting with H.261, belong to the group of "hybrid lossy video codecs" (ie, combine spatial and temporal sample-domain prediction and 2D transform coding to apply transform-domain quantization). Each picture of a video sequence is usually divided into a set of non-overlapping blocks, and encoding is usually done at the block level. In other words, in the encoder, video is usually processed, that is, encoded, at the block (video block) level, for example, using spatial (inner picture) prediction and temporal (outer picture) prediction to generate a prediction block, subtracting the prediction block from the current block ( block currently being processed/to be processed) to obtain a residual block, transforming the residual block, and quantizing the residual block in the transform domain to reduce the amount of data to be transmitted (compressed), while the decoder partially performs inverse processing compared to with the encoder to the encoded or compressed block to reconstruct the current block for presentation. In addition, the encoder duplicates the processing cycle of the decoder so that both will generate identical predictions (eg, intra and inter prediction) and/or re-restore for processing, ie, encoding, subsequent blocks.

As used herein, the term "block" may be part of an image or frame. For ease of description, embodiments of the invention are described herein with reference to High Efficiency Video Coding (HEVC) or Universal Video Coding (VVC) reference software developed by the Joint Collaborative Group on Video Coding (JCT-VC) ITU-T Video Coding Expert Group (VCEG) and the ISO/IEC Moving Picture Expert Group (MPEG). One of ordinary skill in the art will appreciate that embodiments of the invention are not limited to HEVC or VVC. May refer to CUs (coding units), PUs (prediction units), and TUs (transformation units). In HEVC, the CTU (coding tree unit) is divided into CUs using a quadratic tree structure referred to as a coding tree. The decision as to whether to encode an image area using extrinsic (temporal) or intrinsic (spatial) prediction is made at the CU level. Each CU may be further partitioned into one, two or four PUs according to the partition type of the PU. Within one PU, the same prediction process is applied, and the corresponding information is transmitted to the decoder based on the PU. After obtaining the residual block by applying a prediction process based on the division type of the PU, the CU may be divided into transform units (TUs) according to a different quadtree structure similar to the coding tree for the CU. The latest development of video compression technology uses a quadtree-binary-tree (QTBT) split frame to split a coding block. In the QTBT block structure, the CU may be square or rectangular. For example, a coding tree unit (CTU) is first divided by a quadtree structure. The leaf nodes of the quadtree are further separated by a binary tree structure. The leaf nodes of the binary tree are called coding units (CUs), and this segmentation is used to predict and process the transform without any additional partitioning. This means that CU, PU and TU have the same block size in the QTBT coding block structure. In parallel with the QTBT box structure, it has been proposed to use a plurality of sections, such as a ternary tree (TT) section, the term "device" can also mean "device", "decoder" or "encoder".

In the following embodiments, the encoder 20, the decoder 30, and the encoding system 10 will be described with reference to FIGS. 11-13.

11A is a conceptual or schematic block diagram illustrating an example of an encoding system 10, such as a video encoding system 10, that may use the methods of the present invention (the present disclosure). Encoder 20 (eg, video encoder 20) and decoder 30 (eg, video decoder 30) of video coding system 10 represent examples of devices that can be configured to perform methods in accordance with the various examples described in the present invention. As shown in FIG. 11A, the encoding system 10 includes a source device 12 configured to provide encoded data 13, such as an encoded image 13, for example, to a destination device 14 to decode the encoded data 13.

The source device 12 includes an encoder 20 and may additionally, i. e. alternatively, comprise an image source 16, a pre-processing unit 18, such as an image pre-processing unit 18, and a communication interface or communication unit 22.

Image source 16 may comprise or be any type of image capture device, such as for capturing a real image and/or any kind of image or commentary (for screen encoding, some texts on the screen are also considered part of the image or image to be encoded), device generating, such as a computer graphics processor for generating a computer animated image, or any device for producing and/or providing a real image, a computer animated image (e.g., screen content, a virtual reality (VR) image), and/or any combination thereof (e.g., an image augmented reality (AR)). The image source can be any type of memory or storage that stores any of the above images.

A (digital) image is, or can be viewed as, a two-dimensional array or matrix of samples with intensity values. A sample in an array can also be called a pixel (short for picture element) or a pixel. The number of samples in the horizontal and vertical direction (or axis) of the array or image determines the size and/or resolution of the image. Three color components are typically used to represent a color, i.e. an image can be represented or contain three arrays of samples. In RBG format or color space, an image contains a corresponding array of red, green, and blue color samples. However, in video encoding, each pixel is typically represented in a luma/chrominance format or color space, such as YCbCr, which contains a luminance component labeled Y (sometimes L is also used instead) and two chrominance components labeled Cb and Cr. The Y component of luma (or shortness, luminance) represents the brightness or intensity of the gray level (eg, as in a grayscale image), while the two chrominance (or short, chrominance) components Cb and Cr represent chrominance components or color information. Accordingly, an image in YCbCr format contains an array of luma samples of luminance sample values (Y) and two chrominance sample arrays of chrominance values (Cb and Cr). RGB images can be converted or converted to YCbCr format and vice versa, this process is also known as color conversion or conversion. If the image is monochrome, it can only contain an array of brightness samples.

Image source 16 (e.g., video source 16) may be, for example, a camera for capturing an image, a memory, such as an image memory containing or storing a previously captured or generated image, and/or any type of interface (internal or external) for receiving or receiving an image. The camera may be, for example, a local or integrated camera integrated into the source device, the memory may be local or integrated memory, for example, integrated into the source device. The interface may be, for example, an external interface for receiving an image from an external video source, such as an external image capture device such as a camera, external memory, or an external imaging device, such as an external computer GPU, computer, or server. The interface can be anything, such as a wired or wireless interface, an optical interface according to any proprietary or standardized interface protocol. The interface for obtaining image data 17 may be the same interface as the communication interface 22 or part of it.

Unlike the pre-processing unit 18 and the processing performed by the pre-processing unit 18, the image or image data 17 (eg, video data 16) may also be referred to as raw image or original image data 17.

The pre-processing unit 18 is configured to receive (raw) image data 17 and perform pre-processing of the image data 17 to obtain a pre-processed image 19 or pre-processed image data 19. Pre-processing is performed by the pre-processing unit 18, such as cropping, color format conversion (eg from RGB to YCbCr), color correction or denoising. It can be understood that the preprocessing unit 18 may be a possible component.

The encoder 20 (eg, video encoder 20) is configured to receive pre-processed image data 19 and provide encoded image data 21 (more details will be described below, for example, with reference to FIG. 12 or FIG. 14).

Communication interface 22 of source device 12 may be configured to receive encoded image data 21 and transmit encoded image data 21 (or any further processed version thereof) over communication channel 13 to another device, such as destination device 14 or any other device for storage or direct recovery.

The communication interface 22 of the source device 12 may be configured to receive the encoded image data 21 and transmit to another device, for example, to the destination device 14 or any other device for storage or direct retrieval or for processing the encoded image data 21, respectively, before storing the encoded data. 13 and/or transmitting the encoded data 13 to another device, such as a destination device 14 or any other device for decoding or storage.

Destination device 14 includes a decoder 30 (eg video decoder 30) and may additionally, i. e. alternatively, it contains a communication interface or a communication unit 28, a post-processing unit 32 and a display device 34.

The communication interface 28 of the destination device 14 is configured to receive the encoded image data 21 (or any further processed version thereof), for example, directly from the source device 12 or from any other source, for example, a storage device, for example, an encoded image data storage device, and provide encoded picture data 21 to decoder 30.

The communication interface 28 of the destination device 14 is configured to receive encoded image data 21 or encoded data 13, for example, directly from the source device 12 or from any other source, such as a storage device, such as an encoded image data storage device.

Communication interface 22 and communication interface 28 may be configured to transmit or receive encoded image data 21 or encoded data 13 via a direct link between source device 12 and destination device 14, such as a direct wired or wireless connection, or over any network, such as wired or wireless network or any combination thereof, or any private and public network or any combination thereof.

The communication interface 22 may, for example, be configured to pack the encoded image data 21 into an appropriate format, such as packets, and/or process the encoded image data using any kind of transmission coding or processing for transmission over a communication channel or communication network.

The communication interface 28, analogous to the communication interface 22, may, for example, be configured to decompress the encoded data 13 to obtain encoded image data 21.

Communication interface 28 analogous to communication interface 22 may, for example, be configured to receive transmitted data and process transmitted data using any kind of appropriate transmission decoding or processing and/or decompression to obtain encoded image data 21.

Both the communication interface 22 and the communication interface 28 can be configured as unidirectional communication interfaces as shown by the arrow for the encoded image data 13 in FIG. 11A pointing from source device 12 to destination device 14 or as bidirectional communication interfaces, and can be configured, for example, to send and receive messages, for example, to establish a connection, acknowledge and exchange any other information related to the communication channel and/or data transmission, for example, the transmission of encoded image data.

The decoder 30 is configured to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (more details will be described below, for example, based on FIG. 13 or FIG. 15).

The post processor 32 of the destination device 14 is configured to post-process the decoded image data 31 (also referred to as reconstructed image data), such as the decoded image 31, to obtain post-processed image data 33, such as the post-processed image 33. Post-processing performed by block 32 post-processing may include, for example, color format conversion (for example, from YCbCr to RGB), color correction, cropping or resampling, or any other processing, for example, to prepare the decoded image data 31 for display, for example, using the display device 34.

The display device 34 of the destination device 14 is configured to receive post-processed image data 33 for displaying the image to, for example, a user or viewer. The display device 34 may be or comprise any kind of display for presenting the reconstructed image, such as a built-in or external display or monitor. The displays may, for example, consist of liquid crystal displays (LCD), organic light emitting diode (OLED) displays, plasma displays, projectors, micro LED displays, liquid crystals on silicon (LCoS), digital light processor (DLP) or any other display.

Although FIG. 11A depicts source device 12 and destination device 14 as separate devices, device embodiments may also comprise both or both functionality, source device 12 or related functionality, and destination device 14 or related functionality. In such embodiments, source device 12 or related functionality and destination device 14 or related functionality may be implemented using the same hardware and/or software, or using separate hardware and/or software, or any combination thereof.

As will be apparent to those skilled in the art based on the description, the presence and (exact) separation of functionality of various blocks or functions in source device 12 and/or destination device 14 as shown in FIG. 11A may vary depending on the actual device and implementation.

Encoder 20 (eg, video encoder 20) and decoder 30 (eg, video decoder 30) may each be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application programs, application specific integrated circuits ( ASICs), Field Programmable Gate Arrays (FPGAs), Discrete Logic, Hardware, or any combination thereof. If the methods are partially implemented in software, the device may store instructions for the software on a suitable non-volatile computer-readable storage medium and may execute instructions in hardware using one or more processors to perform the methods of the present invention. Any of the above (including hardware, software, a combination of hardware and software, etc.) may be considered one or more processors. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, each of which may be integrated as part of a combined encoder/decoder (CODEC) in a respective device.

Encoder 20 may be implemented via processing circuitry 46 to implement various modules as discussed with respect to encoder 20 in FIG. 12 and/or any other coding system or subsystem described herein. Decoder 30 may be implemented via processing circuitry 46 to implement various modules as discussed with respect to decoder 30 in FIG. 13 and/or any other decoder system or subsystem described herein. The processing circuit may be configured to perform various operations as described below. As shown in FIG. 15, if the methods are partially implemented in software, the device may store instructions for the software on a suitable non-volatile computer-readable storage medium and may execute the instructions in hardware using one or more processors to execute the methods of this invention. Either of video encoder 20 and video decoder 30 may be integrated as part of a combined encoder/decoder (CODEC) in one device, for example, as shown in FIG. 11B.

The source device 12 may be referred to as a video encoding device or a video encoding tool. The destination device 14 may be referred to as a video decoding device or a video decoding tool. Source device 12 and destination device 14 may be examples of video encoding devices or video encoding appliances.

Source device 12 and destination device 14 may comprise any of a wide range of devices, including any kind of portable or stationary devices, such as laptops or laptops, mobile phones, smartphones, tablets or tablet computers, cameras, desktop computers, set-top boxes, televisions, display devices, digital media players, game consoles, video streaming devices (eg, content service servers or content delivery servers), broadcast receiver, broadcast transmitter, and the like. and may or may not use any type of operating system.

In some cases, source device 12 and destination device 14 may be equipped for wireless communication. Thus, source device 12 and destination device 14 may be wireless communication devices.

In some cases, the video coding system 10 illustrated in FIG. 11A is just an example, and the methods of the present invention can be applied to video encoding settings (eg, video encoding or video decoding) that do not necessarily include any communication between encoding and decoding devices. In other examples, data is retrieved from local memory, transferred over a network, and the like. The video encoding device may encode and store data in memory, and/or the video decoding device may retrieve and decode data from memory. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data in memory and/or retrieve and decode data from memory.

For convenience of description, embodiments of the invention are described herein, for example, with reference to High Efficiency Video Coding (HEVC) or Universal Video Coding (VVC) reference software, the next generation video coding standard developed by the Collaborative Video Coding Group (JCT-VC). ) the ITU-T Video Coding Expert Group (VCEG) and the ISO/IEC Moving Picture Expert Group (MPEG). For a person skilled in the art it is obvious that the embodiments of the invention are not limited to HEVC or VVC. It should be understood that for each of the above examples described with reference to video encoder 20, video decoder 30 may be configured to perform a reciprocal process. With respect to signaling syntax elements, video decoder 30 may be configured to receive and parse such syntax element and perform appropriate decoding of the video data. In some examples, video encoder 20 may entropy encode one or more syntax elements into an encoded video bitstream. In such examples, video decoder 30 may parse such a syntax element and decode the associated video data accordingly.

11B is an illustrative diagram of another example of a video coding system 40 including encoder 20 of FIG. 12 and/or decoder 30 of FIG. 13 according to an exemplary embodiment. System 40 may implement methods in accordance with the various examples described in the present invention. In the illustrated implementation, the video coding system 40 may include an imaging device(s) 41, a video encoder 100, a video decoder 30 (and/or a video encoder implemented by processing block(s) 46 logic 47), an antenna 42, one or more processors 43, one or more memory stores 44 and/or a display device 45.

As shown, imaging device(s) 41, antenna 42, processing unit(s) 46, logic circuit 47, video encoder 20, video decoder 30, processor(s) 43, memory storage(s) 44, and/or display device 45 may interact with each other. As discussed, although illustrated with both video encoder 20 and video decoder 30, in various examples, video coding system 40 may include only video encoder 20 or only video decoder 30.

As shown, in some examples, video coding system 40 may include an antenna 42. Antenna 42 may, for example, be configured to transmit or receive an encoded video bitstream. In addition, in some examples, the video encoding system 40 may include a display device 45 . The display device 45 may be configured to present video data. As shown, in some examples, logic 47 may be implemented by processing unit(s) 46 . Processing unit(s) 46 may include application specific integrated circuit (ASIC) logic, graphics processor(s), general purpose processor(s), or the like. Video encoding system 40 may also include additional processor(s) 43, which may likewise include application specific integrated circuit (ASIC) logic, graphics processor(s), general purpose processor(s), and the like. In some examples, logic circuitry 47 may be implemented in hardware, dedicated video coding hardware, and the like, and processor(s) 43 may implement general purpose software, operating systems, and the like. In addition, memory storage(s) 44 may be any type of memory, such as volatile memory (eg, static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (eg, flash memory), memory, etc.), etc. In a non-limiting example, the memory store(s) 44 may be implemented with a cache. In some examples, logic circuitry 47 may access memory store(s) 44 (eg, to implement an image buffer). In other examples, logic circuitry 47 and/or processing unit(s) 46 may include memory stores (eg, cache, etc.) for implementing an image buffer, and the like.

In some examples, video encoder 100 implemented with logic may include an image buffer (eg, via processing unit(s) 46 or memory storage(s) 44) and a graphics processor (eg, processing unit(s) 46). The graphics processor may be communicatively coupled to the image buffer. The graphics processor may include a video encoder 100 implemented by logic 47 to implement various modules, as discussed with reference to FIG. 2 and/or any other coding system or subsystem described herein. The logic circuit may be configured to perform various operations as described herein.

Video decoder 30 may be implemented in a manner similar to that implemented by logic circuitry 47 to implement various modules as discussed with respect to decoder 30 in FIG. 3 and/or any other decoder system or subsystem described herein. In some examples, video decoder 30 may be implemented with logic, may comprise an image buffer (eg, via processing unit(s) 420 or memory storage(s) 44)) and a graphics processor (eg, processing unit(s) 46). The graphics processor may be communicatively coupled to the image buffer. The GPU may include a video decoder 30 implemented by logic 47 to implement various modules, as discussed with reference to FIG. 13 and/or any other decoder system or subsystem described herein.

In some examples, antenna 42 of video coding system 40 may be configured to receive an encoded video bitstream. As discussed, the encoded bitstream may contain data, indicators, index values, mode selection data, and the like associated with encoding a video frame as discussed herein, such as data associated with division coding (e.g., transform coefficients or quantized transform coefficients, possible indicators (as discussed) and/or data defining the coding section). The video coding system 40 may also include a video decoder 30 connected to the antenna 42 and configured to decode the encoded bitstream. The display device 45 is configured to present video frames.

12 shows a schematic/conceptual block diagram of an exemplary video encoder 20 that is configured to implement the methods of the present invention. In the example of FIG. 12, video encoder 20 comprises a residual calculation block 204, a transform processing block 206, a quantization block 208, an inverse quantization block 210 and an inverse transform processing block 212, a reconstruction block 214, a buffer 216, a loop filter block 220, a decoded buffer 230 images (DPB), block 260 processing prediction and block 270 entropy coding. The prediction processing unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a mode select unit 262. Block 244 may include a motion estimation block and a motion compensation block (not shown). The video encoder 20 shown in FIG. 12 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

For example, the residual calculation block 204, the transform processing block 206, the quantization block 208, the prediction processing block 260, and the entropy encoding block 270 form a direct signal path of the encoder 20, while, for example, the inverse quantization block 210, the inverse transform processing block 212, block 214 recovery buffer 216, loop filter 220, decoded picture buffer (DPB) 230, prediction processor 260 form an encoder signal return path in which the encoder return signal path corresponds to the decoder signal path (see decoder 30 in FIG. 13).

An inverse quantization block 210, an inverse transform processing block 212, a reconstruction block 214, a loop filter 220, a decoded picture buffer (DPB) 230, an inter prediction block 244, and an intra prediction block 254 are also related to the formation of an “embedded decoder” of the video encoder 20. The encoder 20 is configured with the ability to receive, for example, at input 202, an image 201 or a block 203 of an image 201, for example, an image of a sequence of images forming a video or video sequence. Image block 203 may also be referred to as the current image block or image block to be encoded, and image 201 as the current image or image to be encoded (particularly in video encoding, to distinguish the current image from other images, such as previously encoded and/or decoded pictures of the same video sequence, that is, a video sequence that also contains the current picture).

A (digital) image is, or can be viewed as, a two-dimensional array or matrix of samples with intensity values. A sample in an array can also be called a pixel (short for picture element) or a pixel. The number of samples in the horizontal and vertical direction (or axis) of the array or image determines the size and/or resolution of the image. Three color components are typically used to represent a color, i.e. an image can be represented or contain three arrays of samples. In RBG format or color space, an image contains a corresponding array of red, green, and blue color samples. However, in video encoding, each pixel is usually represented in a luma and chrominance format or color space, such as YCbCr, which contains a luma component labeled Y (sometimes L is also used instead) and two chrominance components labeled Cb and Cr. The Y component of luminance (or briefly, luminance) represents the brightness or intensity of the gray level (eg, as in a grayscale image), while the two chrominance components (or briefly, chrominance) Cb and Cr represent components of chrominance or color information. Accordingly, an image in YCbCr format contains an array of luma samples of luminance sample values (Y) and two chrominance sample arrays of chrominance values (Cb and Cr). RGB images can be converted to YCbCr format and vice versa, a process also known as color conversion. If the image is monochrome, it can only contain an array of brightness samples. Accordingly, the image may be, for example, an array of luminance samples in monochrome format, or an array of luminance samples and two corresponding arrays of chrominance samples in 4:2:0, 4:2:2 and 4:4:4 color format.

SEPARATION

Embodiments of encoder 20 may include a splitter (not shown in FIG. 12) configured to split image 201 into multiple (generally non-overlapping) image blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264/AVC), or coding tree blocks (CTBs) or coding tree blocks (CTUs) (H.265/HEVC and VVC). The partition block may be configured to use the same block size for all images of the video sequence and the corresponding grid defining the block size, or to change the block size between images, subsets or groups of images and divide each image into appropriate blocks.

In further embodiments, the video encoder may be configured to directly receive block 203 of image 201, such as one, several, or all of the blocks that make up image 201. Image block 203 may also be referred to as the current image block or the image block to be encoded. In one example, prediction processor 260 of video encoder 20 may be configured to perform any combination of the partitioning methods described above.

Like image 201, block 203 is again, or can be viewed as, a two-dimensional array or matrix of samples with intensity values (sample values), albeit smaller than image 201. In other words, block 203 may contain, for example, one array of samples (for example, a luma array in the case of a monochrome image 201) or three arrays of samples (eg, a luminance array and two chrominance arrays in the case of a color image 201) or any other number and/or types of arrays depending on the applied color format. The number of samples in the horizontal and vertical direction (or axis) of block 203 determines the size of block 203. Accordingly, the block may, for example, be an array of MxN samples (M-column per N-row) or an MxN array of transform coefficients.

The encoder 20 shown in FIG. 12 is configured to encode images 201 block by block, for example, encoding and prediction is performed for each block 203.

Embodiments of video encoder 20 as shown in FIG. 12 may be further configured to split and/or encode images using segments (also referred to as video segments), in which an image may be divided into or encoded using one or more segments ( usually without overlap), and each segment may contain one or more blocks (eg CTU) or one or more groups of blocks (eg tiles (H.265/HEVC and VVC) or blocks (VVC)).

Embodiments of video encoder 20 shown in FIG. 12 may be further configured to partition and/or encode pictures using segment/tile groups (also referred to as video tile groups) and/or tiles (also referred to as video tiles) in which the picture may be partitioned or encoded using one or more segment/tile groups (generally non-overlapping), and each segment/tile group may contain, for example, one or more blocks (e.g., CTUs) or one or more tiles, in which each tile , for example, may have a rectangular shape and may contain one or more blocks (eg, CTU), such as integer or fractional blocks.

RESIDUAL CALCULATION

Residual block 204 is configured to calculate residual block 205 based on image block 203 and predictor 265 (more details about predictor 265 are provided later), for example, by subtracting sample values of predictor 265 from sample values of image block 203, sample by sample. (pixel by pixel) to obtain a residual block 205 in the sample area.

TRANSFORMATION

Transform processing unit 206 is configured to apply transforms, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), on the sampled values of the residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207 may also be referred to as residual transform coefficients and represent the residual block 205 in the transform region.

Transform processing unit 206 may be configured to apply DCT/DST integer approximations such as the transforms specified for HEVC/H.265. Compared to the orthogonal DCT, such integer approximations are usually scaled by a certain factor. To preserve the norm of the residual block, which is processed by direct and inverse transformations, additional scaling factors are applied during the transformation. Scaling factors are usually chosen based on certain constraints, such as scaling factors that are a power of two for the shift operation, the bit width of the transform coefficients, the trade-off between accuracy and implementation cost, and so on. Specific scaling factors are, for example, indicated for the inverse transform, for example, by the inverse transform processor 212 in the decoder 30 (and the corresponding inverse transform, for example, by the inverse transform processor 212 in the encoder 20), and appropriate scaling factors for the forward transform can be determined, for example , by the transform processing unit 206 in the encoder 20, respectively.

Embodiments of video encoder 20 (respectively, transform processing unit 206) may be configured to output transform parameters, such as the type of transform, such as directly or encoded or compressed by entropy encoding unit 270, such that, for example, video decoder 30 may receive and use transform parameters for decoding.

QUANTIZATION

Quantizer 208 is configured to quantize transform coefficients 207 to obtain quantized transform coefficients 209, for example by applying scalar quantization or vector quantization. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform factor may be rounded to an m-bit transform factor during quantization, where n is greater than m. The degree of quantization can be changed by adjusting the quantization parameter (QP). For example, for scalar quantization, different scaling may be applied to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, while larger quantization step sizes correspond to coarser quantization. The applicable quantization step size may be indicated by a quantization parameter (QP). The quantization parameter may, for example, be an index to a predetermined set of applicable quantization step sizes. For example, small quantization parameters may correspond to fine quantization (small quantization step sizes), and large quantization parameters may correspond to coarse quantization (large quantization step sizes), or vice versa. Quantization may include dividing by the quantization step size, and corresponding or inverse dequantization, for example by inverse quantizer 210, may include multiplying by the quantization step size. Embodiments according to some standards, for example HEVC may be configured to use a quantization parameter to determine the quantization step size. Typically, the quantization step size can be calculated based on the quantization parameter using a fixed-point approximation of an equation involving division. Additional scaling factors may be introduced for quantization and dequantization to recover the residual block norm, which may be modified due to the scaling used in the fixed-point approximation of the equation for the quantization step size and quantization parameter. In one exemplary implementation, inverse transform scaling and dequantization may be combined. Alternatively, customized quantization tables can be used and signaled from the encoder to the decoder, for example, in a bitstream. Quantization is a lossy operation in which the loss increases with increasing quantization step sizes.

Embodiments of video encoder 20 (respectively, quantizer 208) may be configured to output quantization parameters (QPs), for example, directly or encoded by entropy coding unit 270, such that, for example, video decoder 30 may receive and apply quantization parameters for decoding.

Inverse quantizer 210 is configured to apply the inverse quantization of quantizer 208 to the quantized coefficients to obtain dequantized coefficients 211, for example, by applying the inverse quantization scheme applied by quantizer 208 based on or using the same quantization step size as block 208 quantization. The dequantized coefficients 211 may also be referred to as the dequantized residual coefficients 211 and correspond, although not usually identical to the transform coefficients due to quantization loss, to the transform coefficients 207.

The inverse transform processing unit 212 is configured to apply an inverse transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain an inverse transform block 213 in the sample area. The inverse transform block 213 may also be referred to as the dequantized inverse transform block 213 or the residual inverse transform block 213 .

Reconstructor 214 (eg, adder 214) is configured to add inverse transform block 213 (i.e., reconstructed residual block 213) to prediction block 265 to obtain reconstructed block 215 in the sample area, for example, by adding sample values of reconstructed residual block 213 and values of the samples block 265 prediction.

Alternatively, a buffer block 216 (or "buffer" 216 for short), such as a linear buffer 216, is configured to buffer or store the reconstructed block 215 and corresponding sample values, such as for intra prediction. In further embodiments, the encoder may be configured to use the unfiltered reconstructed blocks and/or corresponding sample values stored in buffer block 216 for any kind of estimation and/or prediction, such as intra prediction.

Embodiments of encoder 20 may be configured where, for example, buffer block 216 is used not only to store reconstructed blocks 215 for intra prediction 254, but also for loop filter block 220 (not shown in FIG. 12) and/or, that, for example, buffer block 216 and decoded picture buffer block 230 form one buffer. Additional embodiments may be configured to use filtered blocks 221 and/or blocks or samples from decoded picture buffer 230 (both not shown in FIG. 12) as input or basis for intra prediction 254.

The loop filter block 220 (or "loop filter" 220 for short) is configured to filter the reconstructed block 215 to obtain a filtered block 221, for example, to smooth out pixel transitions or otherwise improve video quality. The loop filter block 220 is designed to represent one or more loop filters, such as a deblocking filter, an adaptive offset (SAO) filter, or other filters, such as a two-sided filter or an adaptive loop filter (ALF), or sharpening or smoothing filters, or collaborative filters. . Although the loop filter block 220 is shown as a loop filter in FIG. 12, in other configurations, the loop filter block 220 may be implemented as a post loop filter. The filtered block 221 may also be referred to as the filtered reconstructed block 221. The decoded picture buffer 230 may store the reconstructed coding blocks after the loop filter block 220 performs filtering operations on the reconstructed coding blocks.

The loop filter block 220 (or "loop filter" 220 for short) is configured to filter the reconstructed block 215 to obtain a filtered block 221 or, more generally, to filter the reconstructed samples to obtain filtered sample values. The loop filter unit, for example, is configured to smooth out pixel transitions or otherwise improve video quality. The loop filter block 220 may comprise one or more loop filters such as a deblocking filter, an adaptive offset filter (SAO), or one or more other filters such as an adaptive loop filter (ALF), a noise suppression filter (NSF), or any combination thereof. In one example, loop filter block 220 may comprise a deblocking filter, an SAO filter, and an ALF filter. The order of the filtering process can be deblocking filter, SAO and ALF. In another example, a process called luminance chrominance scaling (LMCS) mapping (namely, adaptive in-loop shaper) is added. This process is performed before deblocking. In another example, the deblocking filtering process may also be applied to inner sub-box edges, such as affine sub-box edges, ATMVP sub-box edges, sub-block transform (SBT) edges, and inner sub-partition (ISP) edges. Although the loop filter block 220 is shown as a loop filter in FIG. 12, in other configurations, the loop filter block 220 may be implemented as a post loop filter. The filtered block 221 may also be referred to as the filtered reconstructed block 221.

Embodiments of video encoder 20 (respectively, loop filter block 220) may be configured to output loop filter parameters (such as SAO filter parameters or ALF filter parameters or LMCS parameters), for example, directly or encoded by entropy encoding block 270, such that, for example , decoder 30 may receive and apply the same loop filter parameters or corresponding loop filters for decoding.

Embodiments of encoder 20 (respectively, loop filter block 220) may be configured to output loop filter parameters (such as adaptive sample bias information), for example, encoded directly or by entropy encoding block 270 or any other entropy encoding block, such that, for example, decoder 30 may receive and apply the same loop filter parameters for decoding.

Decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use in encoding video data by video encoder 20. DPB 230 may be formed by any of a variety of memories, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of storage devices. DPB 230 and buffer 216 may be provided by the same storage device or separate storage devices. In some example, the decoded picture buffer (DPB) 230 is configured to store a filtered block 221. The decoded picture buffer 230 can be further configured to store other previously filtered blocks, such as previously restored and filtered blocks 221 of the same current image or different images, such as previously reconstructed images, and may provide complete previously reconstructed, i.e., decoded, images (and corresponding reference blocks and samples) and/or partially reconstructed current image (and corresponding reference blocks and samples), for example, for inter prediction. In some example, if the reconstructed block 215 is reconstructed, but without in-loop filtering, the decoded picture buffer (DPB) 230 is configured to store one or more unfiltered reconstructed blocks 215, or, in general, unfiltered reconstructed samples, for example, if the reconstructed block 215 is not filtered by the block 220 loop filter or any other further processed version of the reconstructed blocks or samples.

The prediction processing unit 260, also referred to as the block prediction processing unit 260, is configured to receive or obtain a block 203 (the current block 203 of the current image 201) and reconstructed image data, such as reference samples of the same (current) image from the buffer 216 and/or reference picture data 231 from one or a plurality of previously decoded pictures from the decoded picture buffer 230, and to process such data for prediction, that is, to provide a prediction block 265, which may be an inter prediction block 245 or an intra prediction block 255.

Mode selector 262 may be configured to select a prediction mode (eg, intra or inter prediction mode) and/or a corresponding predictor 245 or 255 to be used as predictor 265 to calculate the residual block 205 and to reconstruct the reconstructed block 215 .

Embodiments of mode selector 262 may be configured to select a prediction mode (e.g., from those supported by prediction processor 260) that provides the best match, or in other words, minimum residual (minimum residual means better compression for transmission or storage) or minimal signaling overhead (minimum signaling overhead means better compression for transmission or storage), or which takes into account or balances both. The mode selector 262 may be configured to determine a prediction mode based on rate distortion optimization (RDO), i.e., select a prediction mode that provides minimum rate distortion optimization or that the associated rate distortion at least satisfies a prediction mode selection criterion.

Next, a detailed description will be given of the prediction processing process (for example, the prediction processing block 260 and mode selection (for example, the mode selection block 262)) performed by the exemplary encoder 20.

In addition to or alternatively to the above embodiments, in the other embodiments of FIG. 25, the mode selector 260 includes a splitter 262, an inter-predictor 244, and an intra-predictor 254, and is configured to receive or obtain original image data, such as the original block 203 (current block 203 of the current picture 17) and reconstructed picture data, such as filtered and/or unfiltered reconstructed samples or blocks of the same (current) picture and/or from one or more previously decoded pictures, such as from decoded picture buffer 230 or other buffers (eg, linear buffer, not shown). The reconstructed image data is used as reference image data for prediction, such as inter prediction or intra prediction, to obtain prediction block 265 or predictor 265.

The mode selector 260 may be configured to determine or select a split for the current block prediction mode (including no split) and the prediction mode (e.g., intra or inter prediction mode) and generate a corresponding prediction block 265 that is used to calculate the residual block. 205 and to restore the restored block 215.

Embodiments of mode selector 260 may be configured to select a division and prediction mode (e.g., from those supported by or available to mode selector 260) that provide the best match or, in other words, minimum residual (minimum residual means better compression). for transmission or storage), or minimal signaling overhead (minimum signaling overhead means better compression for transmission or storage), or which takes into account or balances both. The mode selector 260 may be configured to determine the division and prediction mode based on rate distortion optimization (RDO), that is, to select a prediction mode that provides minimal rate distortion. Terms such as "best", "minimum", "optimum", etc. in this context does not necessarily refer to an overall "best", "minimum", "optimal", etc., but may also refer to termination performance or selection criteria such as a value above or below a threshold or other constraints , potentially leading to a "non-optimal choice" but reducing complexity and processing time.

In other words, splitter 262 may be configured to split an image from a video sequence into a sequence of coding tree blocks (CTUs), CTU 203 may be further split into smaller block sections or sub-blocks (which again form blocks), e.g., iteratively using splitting. quadtree (QT), binary split (BT) or ternary tree (TT), or any combination thereof, and to perform, for example, prediction for each of the partitions of blocks or subblocks, in which the mode selection contains a choice of the tree structure of the divided block 203, and modes predictions are applied to each of the sections or subblocks of the block.

The following explains in more detail the splitting process (for example, splitter 260) and prediction processing (inter predictor 244 and intra predictor 254) performed by exemplary video encoder 20.

Separation

Splitter 262 may be configured to split an image from a video sequence into a sequence of coding tree blocks (CTUs), and splitter 262 may split (or split) coding tree block (CTU) 203 into smaller sections, such as smaller square or rectangular sized blocks. . For an image that has three sample arrays, the CTU consists of an N×N luma sample block along with two corresponding chrominance sample blocks. The maximum allowable luma block size in CTU is defined as 128x128 in evolving Universal Video Coding (VVC), but a value other than 128x128, such as 256x256, may be specified in the future. Image CTUs can be grouped as segment/tile groups, tiles, or elements. A tile covers a rectangular area of an image, and a tile can be divided into one or more rectangular elements. A rectangular element consists of a number of CTU rows within a tile. A tile that is not divided into several rectangular elements can be called a brick. However, brick is a true subset of tile and is not referred to as tile. VVC supports two tile group modes, namely raster-scanned segment/tile group mode and rectangular segment mode. In raster scan tile group mode, a segment/tile group contains a sequence of tiles with a raster image scan. In Rectangular Segment mode, a segment contains multiple image bricks that together form a rectangular image area. The bricks within a rectangular segment are in the segment's raster scan order. These smaller blocks (which may also be referred to as sub-blocks) can be further subdivided into even smaller sections. This is also called tree splitting or hierarchical tree splitting, in which a root block, e.g. at root level 0 of the tree (hierarchy level 0, depth 0), can be recursively split, e.g. split into two or more blocks of the next lower level of the tree, e.g. , nodes at level 1 of the tree (hierarchy level 1, depth 1) in which these blocks can be split again into two or more next lower level blocks, e.g. tree level 2 (hierarchy level 2, depth 2), etc. . until the split is complete, for example, when a termination criterion is met, such as the maximum tree depth or the minimum block size is reached. Blocks that are not split further are also called leaf blocks or leaf nodes of the tree. A tree using two partitions is called a binary tree (BT), a tree using three partitions is called a ternary tree (TT), and a tree using four partitions is called a quadtree (QT).

For example, a coding tree unit (CTU) may be either a luminance sample CTB, two corresponding chrominance sample CTBs of an image that has three sample arrays, or a monochrome image sample CTB or an image that is encoded using three separate color planes and the syntax structures used for coding samples. Accordingly, a coding tree block (CTB) may be a block of N×N samples for some value of N, such that splitting a component into CTBs is a split. A coding unit (CU) may be or comprise a luminance sample coding unit, two corresponding chrominance sample coding units of an image that has three sample arrays, or a monochrome image sample coding unit or an image that is encoded using three separate color planes and the syntax structures used for coding samples. Accordingly, a coding block (CB) may be a block of M×N samples for some values of M and N, so that the division of the CTB into coding blocks is a division.

In embodiments, for example in accordance with HEVC, a coding tree unit (CTU) may be partitioned into CUs using a square tree structure referred to as a coding tree. The decision as to whether to encode an image area using extrinsic (temporal) or intrinsic (spatial) prediction is made at the leaf-CU level. Each leaf-CU may be further divided into one, two or four PUs according to the partition type of the PU. Within one PU, the same prediction process is applied, and the corresponding information is transmitted to the decoder based on the PU. After obtaining the residual block by applying a prediction process based on the split type of the PU, the leaf CU may be divided into transform units (TUs) according to a different quadtree structure similar to the coding tree for the CU.

In embodiments, for example, in accordance with the latest video coding standard currently being developed, which is called universal video coding (VVC), a merged tree with multiple types of nested square tree using a binary and ternary splitting structure, such as used to split a tree block coding. In a coding tree structure within a coding tree block, a CU may have a square or rectangular shape. For example, a coding tree unit (CTU) is first divided by a quadtree. The leaf nodes of the quadtree can then be further subdivided using a multi-type tree structure. There are four types of splits in the multi-type tree structure: vertical binary split (SPLIT_BT_VER), horizontal binary split (SPLIT_BT_HOR), vertical ternary split (SPLIT_TT_VER), and horizontal ternary split (SPLIT_TT_HOR). The leaf nodes of a multitype tree are called coding units (CUs) and, unless the CU is too large for the maximum transform length, this segmentation is used to predict and process the transform without any additional partitioning. This means that, in most cases, CUs, PUs, and TUs have the same block size in a quadtree with a nested multitype tree coding block structure. An exception occurs when the maximum supported transform length is less than the width or height of the CU's color component. VVC is developing a unique mechanism for signaling section split information in a quadtree with a nested multi-type tree encoding tree structure. In the signaling mechanism, the coding tree unit (CTU) is considered as the root of the quadtree and is first divided by the quadtree structure. Each leaf node of the quadtree (if large enough to be usable) is then further subdivided into a multi-type tree structure. In the multitype tree structure, the first flag (mtt_split_cu_flag) is signaled to indicate whether the node should be further split; when the node is further partitioned, a second flag (mtt_split_cu_vertical_flag) is signaled to indicate the split direction, and then a third flag (mtt_split_cu_binary_flag) is signaled to indicate whether the split is a binary split or a ternary split. Based on the values of mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multitype tree split mode (MttSplitMode) of the CU can be obtained by the decoder based on a given rule or table. It should be noted that for a certain structure, such as 64x64 luma block and 32x32 chrominance pipelining in VVC hardware decoders, TT splitting is prohibited if the width or height of the luma coding block is greater than 64, as shown in FIG. TT splitting is also disabled if the width or height of the chrominance coding unit is greater than 32. The pipeline structure will split the image into virtual pipeline data units (VPDUs), which are defined as non-overlapping blocks in the image. In hardware decoders, sequential blocks (VPDUs) are processed by several stages of the pipeline at the same time. The VPDU size is roughly proportional to the buffer size at most stages of the pipeline, so it is important to keep the VPDU size small. In most hardware decoders, the VPDU size can be set to the maximum transform unit (TB) size. However, in VVC, the separation of the ternary tree (TT) and binary tree (BT) can lead to an increase in the size of VPDUs.

Additionally, it should be noted that when a portion of the tree node box exceeds the bottom or right boundary of the picture, the tree node box is forced to split until all samples of each encoded CU are inside the picture boundaries.

As an example, the intra sub-partition tool (ISP) can divide vertically or horizontally intra-predictive luminance blocks into 2 or 4 sub-partitions depending on the block size.

In one example, mode selector 260 of video encoder 20 may be configured to perform any combination of the partitioning methods described herein. As described above, encoder 20 is configured to determine or select the best or optimal prediction mode from a set of (predefined) prediction modes. The set of prediction modes may include, for example, intra prediction modes and/or inter prediction modes.

The intra prediction mode set may contain 35 different intra prediction modes, such as non-directional modes such as DC (or averaging) mode and planar mode, or directional modes such as defined in H.265, or may contain 67 different intra prediction modes , for example, non-directional modes such as DC mode (or average) and planar mode, or directional modes, such as defined for VVC. For example, several conventional angular intra prediction modes are adaptively replaced by wide-angle intra prediction modes for non-square blocks, such as defined in VVC. As another example, to avoid division operations for DC prediction, only the long side is used to calculate the average for non-square blocks. And the planar mode intra prediction results can be further modified by a position dependent intra prediction combination (PDPC) method.

The intra prediction block 254 is configured to use the reconstructed samples of adjacent blocks of the same current picture to generate the intra prediction block 265 according to the intra prediction mode of the intra prediction mode set.

Block 254 intra prediction (or, in General, block 260 mode selection) is further configured to output parameters of intra prediction (or, in General, information indicating the selected mode of intra prediction for the block) in block 270 entropy encoding in the form of elements 266 syntax for including an encoded image in data 21 so that, for example, video decoder 30 can receive and use prediction parameters for decoding.

The set (or possible) inter prediction modes depends on the available reference pictures (i.e., previous at least partially decoded pictures, e.g. stored in DBP 230) and other inter prediction parameters, e.g. , for example, an area search box around the area of the current block, a reference frame is used to find the best match of the reference block and/or, for example, whether pixel interpolation is applied, such as half/half pixel, quarter pixel and/or 1/16 pixel interpolation or not.

In addition to the aforementioned prediction modes, a skip mode, direct mode, and/or other inter prediction mode may be applied.

For example, extended merge prediction lists merge candidates for such a mode by using the following five types of candidates in order: spatial MVP from spatial neighboring CUs, temporal MVP from co-located CUs, history based MVP from FIFO table, pairwise average MVP, and zero MVs. And, decoder-side motion vector refinement (DMVR) based on two-way matching can be applied to improve the accuracy of the merge mode MVs. Merge mode with MVD (MMVD), which is obtained from the merge mode with motion vector differences. Immediately after sending the skip flag and the merge flag, the MMVD flag is signaled to indicate whether the MMVD mode is used for the CU. And, an adaptive motion vector resolution (AMVR) scheme at the CU level may be applied. AMVR allows you to encode MVD CUs with different precision. Depending on the prediction mode for the current CU, the MVDs of the current CU may be adaptively selected. When a CU is encoded in merge mode, a combined intra/intra prediction (CIIP) mode may be applied to the current CU. To obtain the CIIP prediction, a weighted average of the intra and intra prediction signals is performed. Affine motion compensated predictions, the affine motion field of a block is described by information about the motion of two control points (4 parameters) or three control point motion vectors (6 parameters). Sub-block based temporal motion vector prediction (SbTMVP), which is similar to temporal motion vector prediction (TMVP) in HEVC, but predicts the motion vectors of sub-CUs in the current CU. Bi-Directional Optical Flow (BDOF), formerly called BIO, is a simpler version requiring much less computation, especially in terms of the number of multipliers and the size of the multiplier. Triangular split mode, in this mode, the CU is divided evenly into two triangular partitions using either a diagonal split or an anti-diagonal split. In addition, the bidirectional prediction mode goes beyond simple averaging to provide a weighted average of two prediction signals.

In addition to the aforementioned prediction modes, a skip mode and/or a direct mode may be applied.

Prediction processor 260 may be further configured to partition block 203 into smaller chunks or subblocks, such as iteratively using quadtree partitioning (QT), binary partitioning (BT), or ternary tree partitioning (TT), or any combination thereof, and for performing, for example, prediction for each of the partitions or subblocks of the block, wherein the mode selection comprises selecting the tree structure of the partitioned block 203 and the prediction modes applied to each of the partitions or subblocks of the block.

Inter prediction block 244 may include a motion estimator (ME) (not shown in FIG. 2) and a motion compensation (MC) block (not shown in FIG. 2). The motion estimator is configured to receive or obtain an image block 203 (current image block 203 of the current image 201) and a decoded image 231 or at least one or a plurality of previously reconstructed blocks, e.g., reconstructed blocks of one or a plurality of other/different previously decoded 231 images for motion estimation. For example, the video sequence may comprise the current picture and previously decoded pictures 231 or, in other words, the current picture and previously decoded pictures 231 may be part of or form a sequence of pictures that make up the video sequence.

The encoder 20 may, for example, be configured to select a reference block from a plurality of reference blocks of the same or different pictures from a plurality of other pictures, and provides a reference picture (or reference frame index, ...) and/or an offset (spatial offset) between position ( X, Y) coordinates of the reference block and the position of the current block as related prediction parameters to the motion estimation block (not shown in FIG. 2). This offset is also called motion vector (MV).

The motion compensation block is configured to receive, for example, receive an inter prediction parameter and perform inter prediction based on or using the inter prediction parameter to obtain inter prediction block 265 . The motion compensation performed by the motion compensator may comprise sampling or generating a prediction block based on the motion vector/block determined by the motion estimate, possibly performing interpolation to sub-pixel precision. Interpolation filtering can generate additional pixel samples from known pixel samples, thus potentially increasing the number of prediction block candidates that can be used to encode an image block. After receiving the motion vector for the PU of the current picture block, the motion compensator may find the prediction block pointed to by the motion vector in one of the reference picture lists.

The intra prediction block 254 is configured to receive, for example, an image block 203 (the current image block) and one or a plurality of previously reconstructed blocks, such as reconstructed neighboring blocks of the same image, for intra estimation. Encoder 20 may, for example, be configured to select an intra prediction mode from a plurality of (predetermined) intra prediction modes.

Embodiments of encoder 20 may be configured to select an intra prediction mode based on an optimization criterion, such as minimum residual (eg, intra prediction mode providing prediction block 255 most similar to current picture block 203) or minimum rate distortion.

The intra prediction block 254 is further configured to determine, based on the intra prediction parameter, for example, the selected intra prediction mode, the intra prediction block 255 . In any case, after selecting the intra prediction mode for the block, the intra prediction block 254 is also configured to provide an intra prediction parameter, that is, information indicating the selected intra prediction mode for the block, to the entropy encoding block 270 . In one example, intra predictor 254 may be configured to perform any combination of the intra prediction methods described below.

The entropy encoding unit 270 is configured to apply an entropy encoding algorithm or schemes (e.g., variable length coding (VLC) schemes, context adaptive VLC scheme (CALVC), arithmetic coding scheme, context adaptive binary arithmetic coding (CABAC) based on context adaptive binary arithmetic coding (SBAC), probability interval entropy coding (PIPE), or other entropy coding technologies or methods) on quantized residual coefficients 209, inter prediction parameters, intra prediction parameter, and/or loop filter parameters, individually or jointly (or incompletely) to obtain encoded image data 21 that may be output by output 272, for example in the form of an encoded bitstream 21. The encoded bitstream 21 may be transmitted to video decoder 30 or archived for later transmission or retrieval of the video decode 30. Entropy encoding unit 270 may be further configured to entropy encode other syntax elements for the current video segment being coded.

Other structural variations of video encoder 20 may be used to encode the video stream. For example, non-transform-based encoder 20 may quantize the residual signal directly without transform processor 206 for specific blocks or frames. In another implementation, encoder 20 may have quantizer 208 and inverse quantizer 210 combined into a single block.

FIG. 13 shows an example of a video decoder 30 that is configured to implement the methods of the present invention. Video decoder 30 is configured to receive encoded picture (e.g., encoded bitstream) data 21, such as encoded by encoder 100, to obtain a decoded picture 131. During the decoding process, video decoder 30 receives video data from video encoder 100, such as an encoded video bitstream that represents image blocks of an encoded video segment and associated syntax elements.

In the example of FIG. 13, the decoder 30 includes an entropy decoding block 304, an inverse quantization block 310, an inverse transform processing block 312, a reconstruction block 314 (e.g., adder 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a block 360 prediction processing. The prediction processing block 360 may include an inter prediction block 344 , an intra prediction block 354 , and a mode selection block 362 . Video decoder 30 may, in some examples, perform a decoding process that is generally the reverse of the encoding step described with respect to video encoder 100 in FIG. 12.

As explained with respect to the encoder 20, the inverse quantization block 210, the inverse transform processing block 212, the reconstruction block 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter prediction block 344, and the intra prediction block 354 are also called "inline decoder" generation. of the video encoder 20. Accordingly, the inverse quantizer 310 may be identical in function to the inverse quantizer 110, the inverse transform processor 312 may be identical in function to the inverse transform processor 212, the restorer 314 may be identical in function to the restorer 214, the loop filter 320 may be identical in function to the loop filter 220 and the decoded picture buffer 330 may be identical in function to the decoded picture buffer 230. Therefore, the explanations provided for the respective blocks and functions of the video encoder 20 apply to the respective blocks and functions of the video decoder 30, respectively.

The entropy decoding unit 304 is configured to perform entropy decoding of the encoded image data 21 to obtain, for example, quantized coefficients 309 and/or decoded coding parameters (not shown in FIG. 13), for example, (decoded) any or all of the inter prediction parameters, parameter intra prediction, loop filter parameters, and/or other syntax elements. The entropy decoding block 304 is further configured to forward the inter prediction parameters, the intra prediction parameter, and/or other syntax elements to the prediction processing block 360 . Video decoder 30 may receive syntax elements at the video segment level and/or video block level.

The entropy decoding unit 304 is configured to parse the bitstream 21 (or, in general, the encoded image data 21) and perform, for example, entropy decoding on the encoded image data 21 to obtain, for example, quantized coefficients 309 and/or decoded coding parameters ( not shown in FIG. 13), e.g., any or all of inter prediction parameters (e.g., image reference index and motion vector), intra prediction parameters (e.g., intra prediction mode or index), transform parameters, quantization parameters, loop filter parameters and/or other syntax elements. Entropy decoding unit 304 may be configured to apply decoding algorithms or schemes corresponding to coding schemes as described with respect to entropy encoding unit 270 of encoder 20. Entropy decoding unit 304 may be further configured to provide inter prediction parameters, an intra prediction parameter, and/ or other syntax elements in mode block 360 and other parameters for other blocks of decoder 30. Video decoder 30 may receive syntax elements at the video segment level and/or video block level. In addition or alternatively, segments and corresponding syntax elements, tile groups and/or tiles and corresponding syntax elements can be accepted and/or used.

Inverse quantizer 310 may be identical in function to inverse quantizer 110, inverse transform processor 312 may be identical in function to inverse transform processor 112, restorer 314 may be identical in function to restorer 114, buffer 316 may be identical in function to buffer 116, loop filter 320 may be identical in function to loop filter 120, and decoded image buffer 330 may be identical in function to decoded image buffer 130.

Embodiments of decoder 30 may include a splitter (not shown in FIG. 13). In one example, prediction processing unit 360 of video decoder 30 may be configured to perform any combination of separation methods described above.

Prediction processing block 360 may comprise inter prediction block 344 and intra prediction block 354, in which inter prediction block 344 may resemble inter function prediction block 144 and intra prediction block 354 may resemble intra function prediction block 154. Prediction processing unit 360 is typically configured to perform block predictions and/or obtain prediction block 365 from encoded data 21, and to receive or obtain (explicitly or implicitly) prediction-related parameters and/or selected prediction mode information, for example, from the block 304 entropy decoding.

When a video segment is encoded as an intra (I) segment, the intra prediction block 354 of the prediction processing block 360 is configured to generate a prediction block 365 for a picture block of the current video segment based on the signaled intra prediction mode and data from previously decoded blocks of the current frame, or Images. When a video frame is encoded as an inter-coded (i.e., B or P) segment, the inter prediction block 344 (e.g., motion compensation block) of the prediction processing block 360 is configured to generate prediction blocks 365 for the video block of the current video segment based on the motion vectors and other syntax elements. received from block 304 entropy decoding. For inter prediction, prediction blocks may be obtained from one of the reference pictures in one of the reference picture lists. Video decoder 30 may construct reference frame lists, list 0 and list 1 using default construction methods based on reference pictures stored in DPB 330.

The prediction processing unit 360 is configured to determine prediction information for a video block of the current video segment by analyzing motion vectors and other syntax elements, and use the prediction information to obtain prediction blocks for the current video block being decoded. For example, prediction processing unit 360 uses some of the received syntax elements to determine the prediction mode (eg, intra or inter prediction) used to encode video blocks of a video segment, the type of inter prediction segment (eg, B-segment, P-segment, or GPB-segment ), construction information for one or more reference picture lists for a segment, motion vectors for each inter-coded video block of the segment, inter-prediction status for each inter-coded video block of the segment, and other information for decoding video blocks in the current video segment.

The inverse quantizer 310 is configured to inverse quantize, i. e. dequantizing the quantized transform coefficients provided in the bitstream and decoded by the entropy decoder 304 . The inverse quantization process may include using a quantization parameter calculated by video encoder 100 for each video block in a video segment to determine the amount of quantization, and likewise, the amount of inverse quantization to be applied.

The inverse quantizer 310 may be configured to receive quantization parameters (QPs) (or inverse quantization related information in general) and quantized coefficients from the encoded picture data 21 (e.g., by parsing and/or decoding, e.g., by entropy decoding block 304) and apply, based on the quantization parameters, inverse quantization to the decoded quantized coefficients 309 to obtain dequantized coefficients 311, which may also be referred to as transform coefficients 311.

The inverse transform processing unit 312 is configured to apply an inverse transform, such as an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to obtain residual blocks in the pixel region.

The inverse transform processor 312 may be configured to receive the dequantized coefficients 311, also referred to as transform coefficients 311, and apply transforms to the dequantized coefficients 311 to obtain reconstructed residual blocks 213 in the sample area. The restored residual blocks 213 may also be referred to as transform blocks 313 . The transformation may be an inverse transformation, eg, inverse DCT, inverse DST, inverse integer transformation, or a conceptually similar inverse transformation process. The inverse transform processing unit 312 may be further configured to receive transform parameters or corresponding information from the encoded image data 21 (e.g., by parsing and/or decoding, e.g., by entropy decoding block 304) to determine the transform to be applied to the dequantized coefficients 311.

Reconstructor 314 (eg, adder 314) is configured to add inverse transform block 313 (i.e., reconstructed residual block 313) to prediction block 365 to obtain reconstructed block 315 in the sample area, for example, by adding sample values of reconstructed residual block 313 and values of the samples block 365 prediction.

The loop filter block 320 (either in the encoding loop or after the encoding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321, for example, to smooth pixel transitions or otherwise improve video quality. The loop filter block 320 may comprise one or more loop filters, such as a deblocking filter, an adaptive offset filter (SAO), or one or more other filters, such as an adaptive loop filter (ALF), a noise suppression filter (NSF), or any combination thereof. . In one example, loop filter block 220 may comprise a deblocking filter, an SAO filter, and an ALF filter. The order of the filtering process can be deblocking filter, SAO and ALF. In another example, a process called luminance chrominance scaling (LMCS) mapping (namely, adaptive in-loop shaper) is added. This process is performed prior to deblocking. In another example, the deblocking filter process may also be applied to inner sub-box edges, such as affine sub-box edges, ATMVP sub-box edges, sub-block transform (SBT) edges, and inner sub-partition (ISP) edges. Although in FIG. 13 shows the loop filter block 320 as a loop filter, in other configurations the loop filter block 320 may be implemented as a post loop filter.

Then, the decoded video blocks 321 in that frame or picture are stored in the decoded picture buffer 330, which stores reference pictures used for subsequent motion compensation.

The decoded video blocks 321 of the picture are then stored in the decoded picture buffer 330, which stores the decoded pictures 331 as reference pictures for subsequent motion compensation for other pictures and/or for output, respectively, for display.

The decoder 30 is configured to output the decoded image 331, for example, through an output 332 for presentation or viewing to a user.

Other variants of video decoder 30 may be used to decode the compressed bit stream. For example, decoder 30 may generate an output video stream without loop filter block 320 . For example, the non-transform decoder 30 may perform inverse quantization of the residual signal directly without the inverse transform processing unit 312 for certain blocks or frames. In another implementation, video decoder 30 may have an inverse quantizer 310 and an inverse transform processor 312 combined into one block.

In addition or alternatively to the above embodiments, in other embodiments of FIG. 26, inter prediction block 344 may be identical to inter prediction block 244 (in particular, motion compensation block) and intra prediction block 354 may be identical to inter prediction block 254 in terms of functions and performs split or split decisions and predictions based on splitting and/or prediction parameters or corresponding information obtained from encoded image data 21 (eg, by parsing and/or decoding, eg, by entropy decoding unit 304). Mode application block 360 may be configured to perform prediction (intra or inter prediction) for each block based on the reconstructed images, blocks, or corresponding samples (filtered or unfiltered) to obtain prediction block 365 .

When a video segment is encoded as an intra-coded (I) segment, the intra prediction block 354 of the mode application block 360 is configured to generate a prediction block 365 for a picture block of the current video segment based on the signaled intra prediction mode and data from previously decoded blocks of the current picture. When a video image is encoded as an inter-coded (i.e., B or P) segment, the inter prediction block 344 (e.g., motion compensation block) of the mode application block 360 is configured to generate prediction blocks 365 for the video block of the current video segment based on motion vectors and other elements. syntax received from block 304 entropy decoding. For inter prediction, prediction blocks may be generated from one of the reference pictures in one of the reference picture lists. Video decoder 30 may construct key frame lists, list 0 and list 1 using default construction methods based on reference pictures stored in DPB 330. The same or similar may apply for embodiments using tile groups (e.g. video tile groups) and/or tiles (eg, video tiles) in addition to or as an alternative to segments (eg, video segments), for example, video may be encoded using I, P, or B tile groups and/or tiles.

The mode application block 360 is configured to determine prediction information for a video block of the current video segment by parsing motion vectors or related information and other syntax elements, and uses the prediction information to generate prediction blocks for the current video block being decoded. For example, mode application block 360 uses some of the received syntax elements to determine the prediction mode (eg, intra or inter prediction) used to encode video blocks of a video segment, the type of inter prediction segment (eg, B-segment, P-segment, or GPB-segment ), construction information for one or more reference picture lists for a segment, motion vectors for each inter-coded video block of the segment, inter-prediction status for each inter-coded video block of the segment, and other information for decoding video blocks in the current video segment. The same or similar may apply for or through embodiments using tile groups (e.g., video tile groups) and/or tiles (e.g., video tiles) in addition to or alternatively to segments (e.g., video segments), e.g. video may be encoded using I, P, or B tile groups and/or tiles.

Embodiments of video decoder 30, as shown in FIG. 13, may be configured to split and/or decode images using segments (also referred to as video segments), in which an image may be divided into or decoded using one or more segments (typically without overlap), and each segment may contain one or more blocks (eg CTUs) or one or more groups of blocks (eg tiles (H.265/HEVC and VVC) or blocks (VVC)).

Embodiments of video decoder 30 as shown in FIG. 13 may be configured to partition and/or decode pictures using segment/tile groups (also referred to as video tile groups) and/or tiles (also referred to as video tiles) in which the picture may be partitioned into or decoded using one or more segment/tile groups (generally non-overlapping) and each segment/tile group may contain, for example, one or more blocks (e.g., CTUs) or one or more tiles, in which each tile , for example, may have a rectangular shape and may contain one or more blocks (eg, CTUs), such as full or fractional blocks.

Other variants of the video decoder 30 may be used to decode the encoded picture data 21. For example, decoder 30 may generate an output video stream without loop filter block 320 . For example, the non-transform-based decoder 30 may perform the inverse transform of the residual signal quantization directly without the inverse transform processing unit 312 for certain blocks or frames. In another implementation, video decoder 30 may have an inverse quantizer 310 and an inverse transform processor 312 combined into one block.

It should be understood that in encoder 20 and decoder 30, the processing result of the current step may be further processed and then output to the next step. For example, after the interpolation filtering, motion vector output, or loop filtering, an additional operation such as clipping or shifting may be performed on the result of the interpolation filtering, motion vector output, or loop filtering processing.

14 is a diagram of a video encoding apparatus 400 according to an embodiment of the present invention. The video encoding device 400 is suitable for implementing the disclosed embodiments as described herein. In an embodiment, video encoding device 400 may be a decoder, such as video decoder 30 of FIG. 11A, or an encoder, such as video encoder 20 of FIG. 11A. In an embodiment, video encoding device 400 may be one or more components of video decoder 30 of FIG. 11A or video encoder 20 of FIG. 11A, as described above.

The video encoding device 400 includes input ports 410 and receiver (Rx) units 420 for receiving data; a processor, logical unit, or central processing unit (CPU) 430 for processing data; transmitter (Tx) blocks 440 and data output ports 450; and a memory 460 for storing data. The video encoder 400 may also include optical-electrical (OE) components and electrical-optical (EO) components connected to input ports 410, receiver units 420, transmitter units 440, and output ports 450 for outputting or inputting optical or electrical signals.

Processor 430 is implemented in hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (eg, as a multi-core processor), FPGAs, ASICs, and DSPs. The processor 430 is connected to input ports 410, receiver units 420, transmitter units 440, output ports 450, and memory 460. Processor 430 includes an encoder 470. Encoder 470 implements the disclosed embodiments described above. For example, encoding unit 470 implements, processes, prepares, or provides various encoding operations. Thus, the use of the encoder 470 provides a significant improvement in the functionality of the video encoding device 400 and allows the video encoding device 400 to be converted to another state. Alternatively, encoder 470 is implemented as instructions stored in memory 460 and executed by processor 430.

The memory 460 contains one or more disks, tape drives, and solid state drives and can be used as an overflow storage device, for storing programs when such programs are selected for execution, and for storing instructions and data that are read during program execution. Memory 460 may be volatile and/or non-volatile and may be read only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), and/or static random access memory (SRAM).

FIG. 15 is a simplified block diagram of a device 500 that can be used as one or both of source device 310 and destination device 320 of FIG. 11, according to an exemplary embodiment. Device 500 may implement the methods of the present invention described above. Device 500 may be in the form of a computing system including multiple computing devices, or in the form of a single computing device, such as a mobile phone, tablet computer, laptop computer, laptop computer, desktop computer, and the like.

Processor 502 in device 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device or a plurality of devices capable of manipulating or processing information that currently exists or will be developed in the future. Although the disclosed implementations may be implemented with a single processor, as shown with processor 502, for example, speed and efficiency advantages may be achieved using more than one processor.

In an implementation, memory 504 in device 500 may be read only memory (ROM) or random access memory (RAM). The memory 504 may be any other suitable type of storage device. Memory 504 may include code and data 506 accessed by processor 502 using bus 512. Memory 504 may further include an operating system 508 and application programs 510, application programs 510 include at least one program, which allows the processor 502 to perform the methods described herein. For example, application programs 510 may include applications 1 through N, which further include a video encoding application that performs the methods described herein. Device 500 may also include additional memory in the form of secondary storage 514, which may, for example, be a memory card used with a mobile computing device. Because video communications may contain a significant amount of information, they may be stored in whole or in part in secondary storage 514 and loaded into memory 504 as needed for processing. The device 500 may also include one or more output devices such as a display 518. The display 518 may be, in one example, a touch display that combines a display with a touch element capable of recognizing touch inputs. Display 518 may be connected to processor 502 via bus 512.

The device 500 may also include one or more output devices such as a display 518. The display 518 may be, in one example, a touch display that combines a display with a touch element capable of recognizing touch inputs. Display 518 may be coupled to processor 502 via bus 512. Other output devices that allow a user to program or otherwise use device 500 may be provided in addition to or as an alternative to display 518 or includes a display, the display may be implemented in various ways, including liquid crystal display (LCD), cathode ray tube display (CRT), plasma display, or light emitting diode (LED) display, such as an organic light emitting diode (OLED) display.

The device 500 may also include an imaging device 520, such as a camera, or any other imaging device 520, currently in existence or later developed, that can perceive an image, such as an image of a user operating the device, or interact with the device 500. The imaging device 520 may be positioned to point toward the user operating the device 500. In an example, the position and optical axis of the imaging device 520 may be configured such that the field of view includes an area that immediately adjacent to the display 518 and from which the display 518 is visible.

The device 500 may also include a sound detection device 522, such as a microphone, or any other audio device currently or in the future that can sense sounds in the vicinity of the device 500, or interact with it. The sound detection device 522 may be positioned to face a user operating the device 500 and may be configured to receive sounds, such as speech or other sounds, made by the user when the user operates the device 500.

Although FIG. 15 depicts processor 502 and memory 504 of device 500 as integrated into a single unit, other configurations may be used. The operations of the processor 502 may be distributed among multiple machines (each machine having one or more processors), which may be connected directly or through a local area or other network. Memory 504 may be distributed across multiple machines, such as network memory or memory on multiple machines performing the operations of device 500. Although depicted as a single bus, bus 512 of device 500 may be comprised of multiple buses. In addition, secondary storage 514 may be directly connected to other components of the device 500 or may be accessed via a network and may contain one integrated unit, such as a memory card, or more units, such as multiple memory cards. Thus, device 500 can be implemented in a wide variety of configurations.

EMBODIMENTS RELATED TO THE RELATIONSHIP BETWEEN PARTITION RESTRICTION ELEMENTS

The present invention relates to relationships between partition constraint elements (setting partition rules for different image partition methods), constraint elements that can be signaled via a bitstream that includes a plurality of encoded pictures. Accordingly, the invention provides apparatuses and methods for generating, decoding or processing such a bitstream, in particular including splitting constraint elements in the bitstream, and extracting split constraint elements according to new splitting rules.

It should be noted that the names of the syntax elements described above are used (in this document) as they are commonly used. However, it should be understood that these names can be changed without changing the technical context. Therefore, what should be considered important is the logical meaning of the syntax elements.

Currently, split constraint members (eg, MaxBtSizeY, MaxMttDepth, and MinQtSizeY) are signaled individually in their definition range. В частности, в данный момент, элемент CtbSizeY (т.е. log2_ctu_size_minus2) синтаксиса и элементы MinQtSizeY (т.е. log2_min_qt_size_intra_slices_minus2 и log2_min_qt_size_inter_slices_minus2) синтаксиса и элементы MaxMttDepth (т.е. max_mtt_hierarchy_depth_inter_slices и max_mtt_hierarchy_depth_intra_slices) синтаксиса сигнализируется в наборе параметров последовательности (SPS ). In addition, the difference syntax element between the luma CTB size and MaxBtSizeY (ie log2_diff_ctu_max_bt_size) is signaled in the segment header. The SPS and the segment header may be contained in the bitstream of the picture.

An exemplary SPS syntax, in particular the SPS raw byte sequence (RBSP) payload syntax, is shown in FIG. 8. The semantics of the SPS RBSP of this syntax is as follows.

"Pic_width_in_luma_samples" defines the width of each decoded picture in blocks of luma samples, must not be equal to 0, and must be an integer multiple of MinCbSizeY.

"Pic_height_in_luma_samples" defines the height of each decoded picture in blocks of luma samples, must not be equal to 0, and must be an integer multiple of MinCbSizeY.

"Log2_ctu_size_minus2" plus 2 defines the size of the brightness CTB of each CTU.

The elements CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, PicWidthInCtbsY, PicHeightInCtbsY, PicSizeInCtbsY, PicWidthInMinCbsY, PicHeightInMinCbsY, PicSizeInMinCbsY, PicSixeInSamplesY, PicsInSampleCbsY, PicSizeInMinCbsY, PicSixeInSamplesY, PicsInSampleCbsY, and Pic.InSampleCbsY are usually obtained as below

• CtbLog2SizeY = log2_ctu_size_minus2 + 2

• CtbSizeY = 1 << CtbLog2SizeY

• MinCbLog2SizeY = 2

• MinCbSizeY = 1 << MinCbLog2SizeY

• MinTbSizeY = 4

• MaxTbSizeY = 64

• PicWidthInCtbsY = Ceil(pic_width_in_luma_samples ÷ CtbSizeY)

• PicHeightInCtbsY = Ceil(pic_height_in_luma_samples ÷ CtbSizeY)

• PicSizeInCtbsY = PicWidthInCtbsY * PicHeightInCtbsY

• PicWidthInMinCbsY = pic_width_in_luma_samples / MinCbSizeY

• PicHeightInMinCbsY = pic_height_in_luma_samples / MinCbSizeY

• PicSizeInMinCbsY = PicWidthInMinCbsY * PicHeightInMinCbsY

• PicSizeInSamplesY = pic_width_in_luma_samples * pic_height_in_luma_samples

• PicWidthInSamplesC = pic_width_in_luma_samples / SubWidthC

• PicHeightInSamplesC = pic_height_in_luma_samples / SubHeightC

"Log2_min_qt_size_intra_slices_minus2" plus 2 specifies the minimum size of the leaf block brightness resulting from splitting the CTU quadtree into slices with slice_type equal to 2 (I). The log2_min_qt_size_intra_slices_minus2 value must be between 0 and CtbLog2SizeY - 2 inclusive.

• MinQtLog2SizeIntraY = log2_min_qt_size_intra_slices_minus2 + 2

"Log2_min_qt_size_inter_slices_minus2" plus 2 specifies the minimum size of the leaf block brightness resulting from splitting the CTU quadtree into slices with slice_type equal to 0 (B) or 1 (P). The log2_min_qt_size_inter_slices_minus2 value must be between 0 and CtbLog2SizeY - 2 inclusive.

• MinQtLog2SizeInterY = log2_min_qt_size_inter_slices_minus2 + 2

"Max_mtt_hierarchy_depth_inter_slices" defines the maximum hierarchy depth for coding blocks resulting from splitting a multitype leaf tree quadtree into segments with slice_type equal to 0 (B) or 1 (P). The value of max_mtt_hierarchy_depth_inter_slices must be between 0 and CtbLog2SizeY - MinTbLog2SizeY inclusive.

"Max_mtt_hierarchy_depth_intra_slices" defines the maximum hierarchy depth for coding blocks resulting from splitting a multi-type leaf tree quadtree into segments with slice_type equal to 2 (I). The value of max_mtt_hierarchy_depth_intra_slices must be between 0 and CtbLog2SizeY - MinTbLog2SizeY inclusive.

In addition, FIG. 9 shows an exemplary segment header syntax. The semantics of the fragment header of this syntax are as follows.

"Log2_diff_ctu_max_bt_size" defines the difference between the luminance CTB size and the maximum luminance size (width or height) of a coding block that can be divided using binary division. The log2_diff_ctu_max_bt_size value must be between 0 and CtbLog2SizeY - MinCbLog2SizeY inclusive.

When log2_diff_ctu_max_bt_size is missing, log2_diff_ctu_max_bt_size is assumed to be 2.

The elements MinQtLog2SizeY, MaxBtLog2SizeY, MinBtLog2SizeY, MaxTtLog2SizeY, MinTtLog2SizeY, MaxBtSizeY, MinBtSizeY, MaxTtSizeY, MinTtSizeY, and MaxMttDepth are typically rendered as follows:

• MinQtLog2SizeY = (slice_type == I)? MinQtLog2SizeIntraY: MinQtLog2SizeInterY

• MaxBtLog2SizeY = CtbLog2SizeY - log2_diff_ctu_max_bt_size

• MinBtLog2SizeY = MinCbLog2SizeY

• MaxTtLog2SizeY = (slice_type = = I)? 5:6

• MinTtLog2SizeY = MinCbLog2SizeY

• MinQtSizeY = 1 << MinQtLog2SizeY

• MaxBtSizeY = 1 << MaxBtLog2SizeY

• MinBtSizeY = 1 << MinBtLog2SizeY

• MaxTtSizeY = 1 << MaxTtLog2SizeY

• MinTtSizeY = 1 << MinTtLog2SizeY

• MaxMttDepth = (slice_type = = I)? max_mtt_hierarchy_depth_intra_slices: max_mtt_hierarchy_depth_inter_slices

As shown in FIG. 8, at present, the width and height of a sequence of pictures (eg, pictures of a video sequence) are specified using the "pic_width_in_luma_samples" and "pic_height_in_luma_samples" syntax elements. Typically, these syntax elements must be a multiple of MinCbSizeY to ensure that the image can be divided into minimum size coding blocks. However, although pic_height_in_luma_samples and pic_width_in_luma_samples must be multiples of MinCbSizeY, it is still not guaranteed that the image can be fully subdivided using the available quadtree, binary tree, and ternary tree partitioning methods, respectively. An example of this aspect is explained below:

As an example, it is assumed that:

• MinCbSizeY is 4

• MinQtSizeY is 32

• MaxBtSizeY is 16

In this example, if pic_width_in_luma_samples = 132 and pic_width_in_luma_samples = 132, although the width and height of the image are multiples of 4, it is still not possible to completely separate the image. The reason is that a parent box, for example, size 32x32, cannot be split using binary splitting or quadtree splitting, because if quadtree splitting is applied, the MinQtSizeY constraint will be violated and, if binary splitting is applied, there will be MaxBtSizeY constraint violated. In other words, it is not possible to generate a block with a width or height equal to 4, which, however, is necessary for a complete division of the image.

Thus, as illustrated above, it is currently necessary to solve the technical problem of image separation in full respect of the usual separation rules. In fact, this technical challenge is quite difficult, since it means that some image sizes cannot be encoded and cannot be decoded.

1 shows a device 100 according to a general embodiment of the invention. The device 100 is configured to generate or process, that is, manipulate, a bitstream 101 including encoded images, such as a video bitstream. Device 100 may reside in or may include an encoder configured to encode images and generate bitstream 101. Device 100 may determine relationships between split constraint elements and may include these split constraint elements and/or their relationships in bitstream 101. Thus, device 100 may add separation and/or relation constraint elements to the SPS syntax and/or segment header syntax in bitstream 101.

Similarly, the decoder device analyzes the bitstream and/or obtains from the bitstream and applying the given rules, the restriction parameters related to the separation. The limit parameter then helps to properly decode and recover partitions. The encoder and decoder use (process) the same syntax.

Specifically, device 100 is configured to determine MaxBtSizeY 102 and/or determine MaxMttDepth 103 and MinCbSizeY 104. Device 100 is then configured to determine MinQtSizeY 105 based on MaxBtSizeY 102 and/or based on MaxMttDepth 103 and MinCbSizeY 104. Finally, device 100 is configured to with the ability to include in the bitstream 101 a specific MinQtSizeY 105 either indirectly (that is, by using information from which MinQtSizeY 105 can be obtained) or directly.

In the device 100 according to the first embodiment of the invention, which is based on the device 100 shown in FIG. 1, the range of values of MinQtSizeY 105 may be limited based on the value of MaxBtSizeY 102. For example, the upper bound of MinQtSizeY 105 may be limited to MaxBtSizeY 102. In other words, in the device 100 according to the first embodiment, the minimum value of MinQtSizeY 105 cannot be greater than MaxBtSizeY 102.

Alternatively or in addition to the above, in the device 100 according to the first embodiment, the range of MinQtSizeY 105 values may be limited based on MaxMttDepth 103. In this case, for example:

• If MaxMttDepth 103 is 0, MinQtSizeY 105 may be equal to (or may be derived from) MinCbSizeY 104.

• If MaxMttDepth 103 is greater than 0, the upper bound of MinQtSizeY 105 may be (MinCbSizeY 104 << MaxMttDepth 103). In other words, MinQtSizeY 105 must not be greater than (MinCbSizeY 104 << MaxMttDepth 103).

Notably, in this document, the operation x << y can be mathematically described as x ^y where x is 2 to the power of n, where n is a non-negative integer. In other words, x << y represents shifting x y bits to the left.

Thus, the device 100 according to the first embodiment is configured to establish relationships between MinQtSizeY 105 and MinCbSizeY 104, MaxBtSizeY 102 and/or MaxMttDepth 103, in particular such that the smallest partition block can be obtained by recursively splitting the parent block using a combination of quadtree splitting or binary splitting methods. Since the smallest block (whose size is specified by MinCbSizeY 104) can be generated using the available splitting methods, it is possible to fully split an image that is a multiple of MinCbSizeY 104.

2 and 3 show an RPBS SPS syntax 200 and a segment header syntax 300 for a device according to a second embodiment of the invention. In particular, the traditional RBSP SPS syntax shown in FIG. 8 is changed to the syntax 200 shown in FIG. 2 (new elements in bold, deleted ones in strikethrough). In addition, the traditional segment header syntax shown in FIG. 9 has been changed to the syntax 300 shown in FIG. 3 (new elements in bold, deleted elements in strikethrough). The apparatus according to the second embodiment may be based on the apparatus 100 shown in FIG. 1, or may be a separate embodiment of the invention. In the device according to the second specific variant, the following is implemented:

• MaxBtSizeY 102 is signaled in bitstream 101 relative to MinQtSizeY 105. In other words, as shown in FIG. 3, syntax element 301 of the difference between MaxBtSizeY 102 and MinQtSizeY 105 can be signaled in bitstream 101 (using, for example, a syntax element such as log2_diff_max_bt_size_min_qt_size), and MaxBtSizeY 102 can be obtained (eg, in bitstream decoder 101) based on MinQtSizeY 105 and log2_diff_max_bt_size_min_qt_size. In this case, in the example:

- MaxBtSizeY 102 = MinQtSizeY 105 << log2_diff_max_bt_size_min_qt_size

- It should be noted that in this example, the syntax element 301 of the difference between MaxBtSizeY 102 and MinQtSizeY 105 is signaled on a logarithmic scale (specifically base 2). In this example, log2_diff_max_bt_size_min_qt_size can only have positive integer values or zero.

• MaxMttDepth 103 is signaled in bitstream 101 with respect to MinQtSizeY 105 and MinCbSizeY 104. As shown in FIG. 2, syntax element 201 of the difference between MaxMttDepth 103 and log-2 value MinQtSizeY 105 may be signaled in bitstream 101 (for example, using the syntax element:

diff_max_mtt_hierarchy_depth_log2_min_qt_size). Two such syntax elements 201 are shown in FIG. 2, one for inter_slices, one for intra_slices. In this case, in the example:

- MaxMttDepth 103 = diff_max_mtt_hierarchy_depth_log2_min_qt_size + log2(MinQtSizeY) - log2(MinCbSizeY).

- It should be noted that this example again assumes that diff_max_mtt_hierarchy_depth_log2_min_qt_size is displayed on a logarithmic scale. The log2(x) function corresponds to the base 2 logarithm of x.

3, "log2_diff_max_bt_size_min_qt_size" defines the difference between the maximum luminance size (width or height) of a coding block that can be divided using binary division and the minimum luminance size (width or height) of a coding block that can be divided using tree. quadrants. The log2_diff_ctu_max_bt_size value must be between 0 and CtbLog2SizeY - MinQtLog2SizeY inclusive.

4 shows an RPBS SPS syntax 400 for a device according to a third embodiment of the invention. The device according to the third embodiment may be implemented on the device 100 shown in FIG. 1, or may be a separate embodiment of the invention. As previously mentioned, typically each of the signaled image size elements 405 and 406 (pic_width_in_luma_samples and pic_height_in_luma_samples) is defined as a multiple of the integer MinCbSizeY 104.

In contrast, in the first implementation of the device according to the third embodiment, the image size elements 405 and 406 can be constrained to only have values that are integer multiples of MinQtSizeY 105. splitting the quadtree.

In the second implementation of the device according to the third embodiment, the image width and height may be signaled in bitstream 101 based on MinQtSizeY 105. In particular, the traditional RBSP SPS syntax shown in FIG. elements are in bold, deleted ones are in strikethrough).

In this case, in the SPS syntax 400 shown in FIG. 4, four syntax elements 401-404 may be signaled (e.g., log2_diff_pic_height_min_Qt and log2_diff_pic_width_min_Qt), specifically two syntax elements (height/width) per intra_slice and two syntax elements (height/ width) on inter_slice. Preferably, these syntax elements 401-404 are signaled instead of the actual image size elements 405 and 406, and the image width and height can be determined using the following relationships:

• Image width in luma samples = MinQtSizeY 105 << log2_diff_pic_width_min_Qt

• Image height in luma samples = MinQtSizeY 105 << log2_diff_pic_height_min_Qt

Syntax elements 401-404, in particular differences, may be indicated in SPS syntax 400 based on a logarithmic scale.

5 and 6 show syntaxes 5000 and 600 of two segment headers, respectively, for a device according to a fourth embodiment of the invention. The device according to the fourth embodiment can be obtained on the device 100 shown in FIG. 1, or may be a separate embodiment of the invention.

The device according to the fourth specific embodiment of the invention is concerned with solving the technical problem that if MaxMttDepth 103 is output, signaled or indicated as equal to 0, MaxBtSizeY 102 (or MaxTtSizeY) may still be in bitstream 101, and may have a value greater than MinCbSizeY 104, and MinBtSizeY (or MinTtSizeY) may still have a value equal to MinCbSizeY 104. This condition may create ambiguity in the operation of the encoder and/or decoder, so that complete image frame separation may still not be possible.

In the first implementation of the apparatus according to the fourth embodiment, MaxBtSizeY 102 (or MaxTtSizeY) is transmitted or indicated in bitstream 101 based on the value of MaxMttDepth 103. In particular, the traditional segment header syntax shown in FIG. 9 is changed to syntax 5000 shown in FIG. .5 (new elements in bold). That is, the device is configured to add a MaxBtSizeY 102 (or MaxTtSizeY) syntax element 5001 dependent on MaxMttDepth 103 to the bitstream 101.

In particular, when MaxMttDepth 103 is 0, MaxBtSizeY 102 (or MaxTtSizeY) may not be transmitted in bitstream 101, but may be output (e.g., in a decoder) to be equal to MinCbSizeY 104. Alternatively, MaxBtSizeY 102 (or MaxTtSizeY) is output (e.g., , in the decoder) as equal to a predefined default value, such as 4 or 8 if MaxMttDepth 103 is 0.

"Log2_diff_ctu_max_bt_size" defines again the difference between the luminance CTB size and the maximum luminance size (width or height) of a coding block that can be divided using binary division. The log2_diff_ctu_max_bt_size value must be between 0 and CtbLog2SizeY - MinCbLog2SizeY inclusive.

When log2_diff_ctu_max_bt_size is absent, the following may apply: if slice_type is equal to 2 (I) and max_mtt_hierarchy_depth_intra_slices is equal to 1, log2_diff_ctu_max_bt_size may be output as _f_b_diff_size equal to 2. Otherwise, log2_diff_diff value equal to 2 may be output. equal to CtbLog2SizeY - MinCbLog2SizeY.

In the second implementation of the apparatus according to the fourth embodiment, MaxMttDepth 103 is signaled or indicated in bitstream 101 based on the value of MaxBtSizeY 102 (or MaxTtSizeY). In particular, the traditional segment header syntax shown in FIG. 9 has been changed to the syntax 600 shown in FIG. 6 (new elements in bold). That is, the device is configured to add to the bitstream 101 element 601 of the syntax MaxMttDepth 103 depending on MaxBtSizeY 102 or MaxTtSizeY.

In particular, when MaxBtSizeY 102 (or MaxTtSizeY) is 0, MaxMttDepth 103 may not be transmitted, but may be output, for example, it should be 0 in the decoder. If the value of MaxMttDepth 103 is 0, this means that binary separation is not allowed. In this solution, the syntax element MaxBtSizeY 102 and MaxTtSizeY must be signaled before MaxMttDepth 103 but is not constrained by any parameter set header.

"MaxTtSizeY" is defined as the maximum luminance size (width or height) in terms of the number of coding block samples that can be divided using ternary division. "MinTtSizeY" is defined as the minimum luminance size (width or height) in terms of the number of coding block samples that can be divided using ternary division.

In FIG. 6, "max_mtt_hierarchy_depth" defines the maximum hierarchy depth for coding blocks resulting from splitting a multi-type leaf tree tree of quads in a segment. The value of max_mtt_hierarchy_depth_inter_slices must be between 0 and CtbLog2SizeY - MinTbLog2SizeY inclusive. When max_mtt_hierarchy_depth is absent, the value of max_mtt_hierachiy_depth is set to 0.

7 shows a method 7000 according to an embodiment of the invention. Method 7000 is specifically for managing bit stream 101 and may be performed by device 100 shown in FIG. Method 7000 may also be performed by an encoder encoding images of bitstream 101 into bitstream 101, i. e. generating bitstream 101.

Method 7000 includes step 7001 of determining MaxBtSizeY 102 and/or determining MaxMttDepth 103 and MinCbSizeY 104. In addition, method 7000 includes step 7002 of determining MinQtSizeY 105 based on MaxBtSizeY 102 and/or based on MaxMttDepth 103 and MinCbSizeY 104. Finally, method 7000 includes step 7003, inclusion in bitstream 101 of a certain value MinQtSizeY 105.

10 shows a device 1000 according to a general embodiment of the invention. The device 1000 is configured to generate or process, ie manipulate, a bitstream 101 including encoded images, such as a video bitstream. Apparatus 1000 may be included in or may include an encoder configured to encode pictures and generate bitstream 101. Apparatus 1000 may determine relationships between split constraint elements and may include those split constraint elements and/or their relationships in bitstream 101. Thus, device 1000 may add SPS syntax and/or SPS syntax and/or segment header syntax constraint elements to bitstream 101. Device 1000 is an alternative to device 100 shown in FIG. 1. However, the features of the device 1000 explained below with reference to FIG. 10 may be combined with the features of the devices according to the first, second, third, or fourth specific embodiments described above (if they are not based on the device 100 of FIG. 1).

In particular, device 1000 is configured to determine MinQtSizeY (105). In addition, configured to determine MaxBtSizeY 102 and/or determine MaxMttDepth 103 based on MinQtSizeY 105. information from which the MaxBtSizeY 102 and/or MaxMttDepth 103 values can be derived) or directly.

For example, device 1000 may determine MaxBtSizeY 102 given that its lower limit is MinQtSizeY 105. That is, the range of MaxBtSizeY 102 may be limited to MinQtSizeY 105. For example, the lower bound of MaxBtSizeY 102 may be limited to MinQtSizeY 105. In other words, in the device 1000 the minimum value of MaxBtSizeY 102 cannot be less than MinQtSizeY 105.

Alternatively, or in addition to the above, in the device 1000 MaxMttDepth 103 may be determined assuming that its upper limit is the difference between the value of log2 MinQtSizeY 105 and the value of log2 MinCbSizeY 104. That is, the maximum value of MaxMttDepth 103 cannot be greater than the difference between the log2 MinQtSizeY value of 105 and the log2 MinCbSizeY value of 104.

Thus, the first aspect of the invention provides an apparatus for generating or processing a bitstream including encoded pictures, the apparatus being configured to determine MaxBtSizeY and/or determine MaxMttDepth and MinCbSizeY, determine MinQtSizeY based on MaxBtSizeY and/or based on MaxMttDepth and MinCbSizeY, and include the specified MinQtSizeY in the bitstream.

By determining MinQtSizeY based on MaxBtSizeY and/or based on MaxMttDepth and MinCbSizeY, that is, by defining new splitting rules by establishing relationships between these split constraint elements, the device of the first aspect achieves an increase in the availability and flexibility of various image splitting methods, in particular quadtree splitting and binary tree.

In an implementation form of the first aspect, the device is configured to determine MinQtSizeY given that its upper limit is MaxBtSizeY and/or determine MinQtSizeY given that its upper limit is MinCbSizeY depending on MaxMttDepth.

By establishing such links between the MinQtSizeY, MinCbSizeY, MaxBtSizeY, and MaxMttDepth elements of the split constraint, respectively, it is guaranteed that the smallest split box can be obtained by recursively splitting the parent box using a combination of quadtree splitting methods or binary splitting. Since the smallest block (whose size is specified by MinCbSizeY) can be generated using the available partitioning methods, it is possible to fully partition an image that is a multiple of MinCbSizeY.

In a further implementation form of the first aspect, the device is configured to determine MinQtSizeY as MinCbSizeY if MaxMttDepth is zero, and determine MinQtSizeY as MinCbSizeY depending on MaxMttDepth if MaxMttDepth is greater than zero.

This provides an efficient form of implementation of the prior form of implementation for providing full image partitioning using quadtree and binary tree partitioning, respectively.

A second aspect of the invention provides a device for generating or processing a bitstream including encoded pictures, wherein the device is configured to: determine MinQtSizeY, determine MaxBtSizeY, and/or determine MaxMttDepth based on MinQtSizeY, and

include in the bitstream a specific MaxBtSizeY value and/or a specific MaxMttDepth value.

In an implementation form of the second aspect, the device is configured to determine MaxBtSizeY given that its lower limit is MinQtSizeY and/or determine MaxMttDepth given that its upper limit is the difference between log2 MinQtSizeY and log2 MinCbSizeY.

As in the first aspect, the second aspect defines new partitioning rules by establishing relationships between partition constraint elements. Thus, the device of the second aspect provides increased availability and flexibility of various image partitioning methods, in particular quadtree and binary tree partitioning.

In a further form of implementation of the first or second aspect, the device is configured to include in the bitstream an indication of the relationship, in particular the difference between MinQtSizeY and MaxBtSizeY.

Thus, corresponding split constraint elements can be easily obtained, for example, at the decoder side, while reducing the information overhead in the bitstream. Difference is an example of a relationship. However, the ratio can also be a proportionality factor, calculation scheme, etc., which makes it possible to obtain MaxBtSizeY from MinQtSizeY.

In a further form of implementation of the first or second aspect, the device is configured to include in the bitstream an indication of the relationship, in particular the difference, between the value of log 2 MinQtSizeY and MaxMttDepth.

Thus, it is possible to obtain corresponding split constraint elements, for example, on the decoder side, while reducing the information overhead in the bitstream.

In a further form of implementation of the first or second aspect, the device is configured to determine one or more image size elements that indicate the size, in particular height and width, of bitstream images as an integer multiple of MinQtSizeY, and include one or more image size elements in the bitstream .

As a consequence, a boundary block can always have a quadtree split as an available splitting method.

In a further form of implementation of the first or second aspect, the device is configured to determine one or more image size elements that indicate the size, in particular height and width, of the bitstream images, and include in the bitstream an indication of the relationship between the image size elements and MinQtSizeY.

Thus, it is possible to obtain corresponding split constraint elements, for example, on the decoder side, while reducing the information overhead in the bitstream.

In an additional form of implementation of the first or second aspect, the indication of the relationship between the elements of the image size and MinQtSizeY is based on a logarithmic form.

In a further form of implementation of the first or second aspect, the device is configured to include in the bitstream an indication of MaxBtSizeY or MaxTtSizeY depending on MaxMttDepth.

In this way, the ambiguity normally associated with the operation of an encoder and/or decoder is eliminated, and complete division of an image frame becomes possible.

In a further form of implementation of the first or second aspect, the device is configured to not include any indication of MaxBtSizeY or MaxTtSizeY in the bitstream if MaxMttDepth is zero.

This makes it possible to reduce the information overhead in the bit stream.

In a further form of implementation of the first or second aspect, the device is configured to include in the bitstream a MaxMttDepth indication depending on MaxBtSizeY or MaxTtSizeY.

In this way, the ambiguity normally associated with the operation of an encoder and/or decoder is eliminated, and complete division of an image frame becomes possible.

In a further embodiment of the first or second aspect, the device is configured to not include any indication of MaxMttDepth in the bitstream if MaxBtSizeY or MaxTtSizeY is zero.

This makes it possible to reduce the information overhead in the bit stream.

In a further form of implementation of the first or second aspect, the device comprises or is included in an encoder configured to encode images of the bitstream.

A third aspect of the invention provides a method for generating or processing a bitstream including encoded pictures, wherein the method comprises: determining MaxBtSizeY and/or determining MaxMttDepth and MinCbSizeY, determining MinQtSizeY based on MaxBtSizeY and/or based on MaxMttDepth and MinCbSizeY, and including in bitstream of the specified MinQtSizeY.

In an implementation form of the third aspect, the method comprises determining MinQtSizeY given that its upper limit is MaxBtSizeY and/or determining MinQtSizeY given that its upper limit is MinCbSizeY depending on MaxMttDepth.

In a further implementation form of the third aspect, the method comprises determining MinQtSizeY as MinCbSizeY if MaxMttDepth is zero, and determining MinQtSizeY as MinCbSizeY depending on MaxMttDepth if MaxMttDepth is greater than zero.

In a further implementation form of the third aspect, the method comprises including in the bitstream an indication of a relationship, in particular a difference, between MinQtSizeY and MaxBtSizeY.

In a further form of implementation of the third aspect, the method comprises including in the bitstream an indication of the relationship, in particular the difference between MinQtSizeY and MaxMttDepth.

In a further implementation form of the third aspect, the method comprises determining one or more image size elements that indicate the size, in particular height and width, of bitstream images as an integer multiple of MinQtSizeY, and including one or more image size elements in the bitstream.

In a further implementation form of the third aspect, the method comprises defining one or more image size elements that indicate the size, in particular height and width, of the bitstream images, and including in the bitstream indicating the relationship between the image size elements and MinQtSizeY.

In a further implementation form of the third aspect, the indication of the relationship between image size elements and MinQtSizeY is based on a logarithmic scale.

In a further implementation form of the third aspect, the method comprises including in the bitstream an indication of MaxBtSizeY or MaxTtSizeY depending on MaxMttDepth.

In a further implementation form of the third aspect, the method comprises not including any indication of MaxBtSizeY or MaxTtSizeY in the bitstream if MaxMttDepth is zero.

In a further implementation form of the third aspect, the method comprises including in the bitstream an indication of MaxMttDepth depending on MaxBtSizeY or MaxTtSizeY.

In a further implementation of the third aspect, the method comprises not including any indication of MaxMttDepth in the bitstream if MaxBtSizeY or MaxTtSizeY is zero.

In a further implementation form of the third aspect, the method is performed in an encoder encoding bitstream images.

With the method of the third aspect and its implementation forms, all the advantages and effects described above for the corresponding device of the first aspect and its respective implementation forms can be achieved. A further aspect of the invention is a method for generating or processing a bit stream corresponding to the apparatus of the second aspect.

A fourth aspect of the invention provides a device for generating or processing a bitstream including encoded pictures, in which the device is configured to include in the bitstream an indication of a relationship, in particular a difference, between MinQtSizeY and MaxBtSizeY and/or to include in the bitstream an indication of a relationship, in particular difference between MinQtSizeY and MaxMttDepth.

With reference to the device of the fourth aspect, the respective split constraint elements can be signaled while reducing the overhead in the bitstream. The split constraint elements can be obtained, for example, at the decoder side. The device of the fourth aspect provides an increase in the availability and flexibility of various image partitioning methods, in particular quadtree and binary tree partitioning.

A fifth aspect of the invention provides a device for generating or processing a bitstream including encoded pictures, wherein the device is configured to: determine one or more picture size elements that indicate the size, in particular the height and width, of the bitstream pictures to be an integer multiple of MinQtSizeY and include one or more image size elements in the bitstream; or define one or more image size elements that indicate the size, in particular height and width, of the bitstream images, and include in the bitstream an indication of the relationship between the image size elements and MinQtSizeY.

According to a fifth aspect, as an available partitioning method, a boundary block may always have a quadtree partitioning. In addition, appropriate split constraint elements can be obtained, for example, on the decoder side, while reducing the information overhead in the bitstream. The device according to the fifth aspect provides an increase in the availability and flexibility of various ways of dividing an image, in particular, dividing a quadtree and a binary tree.

The sixth aspect of the invention provides an apparatus for generating or processing a bitstream including encoded images, the apparatus is configured to: include in the bitstream an indication of MaxBtSizeY or MaxTtSizeY depending on MaxMttDepth; or include in the bitstream a MaxMttDepth indication depending on MaxBtSizeY or MaxTtSizeY.

The apparatus of the sixth aspect reduces or eliminates the ambiguity normally associated with the operation of an encoder and/or decoder, and thus maintains complete separation of an image frame. The device according to the sixth aspect provides an increase in the availability and flexibility of various image partitioning methods, in particular quadtree and binary tree partitioning.

A seventh aspect of the invention provides a computer program product containing program code for controlling a device for executing the method of the third aspect, and corresponding implementation forms thereof, when the program code is executed by one or more processors of the device. The device may be a device of the first, second, fourth, fifth or sixth aspect, or any form of implementation thereof.

An eighth aspect of the invention provides an apparatus for decoding a bitstream generated or processed by a device according to the first, second, fourth, fifth, or sixth aspect, or any implementation form thereof.

EMBODIMENTS RELATED TO BOUNDARY SEPARATION

The Next Generation Universal Video Coding (VVC) standard is the latest collaborative video study project of the Video Coding Expert Group (VCEG) of the International Telecommunication Union (ITU-T) and the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Standards Organization Expert Group Moving Picture Exploration Team (MPEG) working together in a partnership known as the Joint Video Exploration Team (JVET). In VVC, the multitype tree (binary/ternary/quadrant) segmentation framework (BT/TT/QT or binary tree/ternary tree/quadtree) replaces or can replace the concept of multitype block partitioning, i.e. eliminates the concept of splitting CUs, PUs, and TUs, except when necessary for CUs that are too large for the maximum transform length, and maintains more flexibility for CU splitting forms. [JVET-J1002].

16A-F illustrate, by way of example, the separation modes currently used in the VTM. Fig. 16A shows a non-split block (no split), Fig. 16B shows a quaternary or quadtree split (QT), Fig. 16C shows a horizontal binary split or binary tree (BT), Fig. 16D shows a vertical binary split or binary tree (BT ), FIG. 16E shows a horizontal ternary partition or ternary tree (TT), FIG. 16F shows a vertical ternary or ternary tree (TT) partition of a block such as a CU or a CTU. Embodiments may be configured to implement split modes as shown in FIGS. 16A-16F.

In embodiments, the following parameters may be defined and indicated by sequence parameter set (SPS) syntax elements for the BT/TT/QT coding tree scheme:

CTU size: The size of the root node of the quadtree.

MinQTSize: The minimum allowable size of a leaf node in a quadtree

MaxBTTSize: The maximum allowable size of the root node of a binary and ternary tree.

MaxBTTDepth: the maximum allowed binary and ternary tree depth, and

MinBTTSize: The minimum allowed size for a binary and ternary tree leaf node.

In other embodiments, the minimum allowed quadtree leaf node size parameter MinQTSize may also be contained in other headers or sets, such as a segment header (SH) or a picture parameter set (PPS).

In the HEVC standard, a coding tree block (CTU) or a coding unit (CU) that is located on segment/picture boundaries will be forced to split using a quadtree (QT) until the bottom right sample of a leaf node is located within the boundary. segment/image. The forced QT split or split does not need to be signaled in the bitstream since both the encoder and decoder, eg video encoder 20 and video decoder 30, are aware of the forced QT being applied. The purpose of forced separation is to provide a CTU/CU boundary by video encoder 20/video decoder 30.

International patent publication number WO 2016/090568 discloses the structure of QTBT (quadtree plus binary tree) and also in VTM 1.0 inherited from HEVC the process of forced separation of CTU/CU at the border. This means that a CTU/CU located at a frame boundary is forced to be split by a quadtree structure (QT) without regard to rate-distortion (RD) optimization until the entire current CU is inside the picture boundary. These forced sections are not signaled in the bitstream.

17A shows an example of a forced split for a high definition (HD) lower bound (128x128) CTU (1920x1080 pixels) split by a forced QT. In Fig. 17, the high-definition image has a resolution of 1920×1080 pixels, and the CTU has a size of 128×128 pixels, or is 128×128 pixels.

In SubCE2 (image edge processing) CE1 (separation) at the San Diego meeting (04.2018) [JVET-J1021], 15 tests were proposed regarding image edge processing using BT, TT or ABT (asymmetric BT). For example, in JVET-K0280 and JVET-K0376, the border is defined as shown in FIG. 18 shows the borders of the image in dashed lines, and the regions of the edge cases are shown in straight lines, namely, the bottom edge case, the corner edge case, and the right edge case. The lower bound can be split by horizontal forced BT or forced QT, the right bound can be split by vertical forced BT or forced QT, and the corner case can only be split by forced QT, in which the decision to use any of the forced BTs or forced QT split is based on an optimization criterion rate distortion and is signaled in the bitstream. Forced split means that the block must be split, for example, forced split is applied to boundary blocks that cannot be encoded using "no split" as shown in FIG. 16A.

If forced border splitting uses QT forced splitting, the MinQTSize split constraint is ignored. For example, in FIG. 13A, if MinQTSize is signaled in the SPS as 32 to match the bound with the forced QT method, it will be necessary to split the QT to a block size of 8x8, which ignores the MinQTSize constraint of 32.

According to embodiments of the invention, if forced QT is used to separate image boundaries, then forced QT separation is performed, eg, not ignoring the separation constraint as specified in, for example, SPS. If additional forced separation is needed, only forced BT is used, which in combination can also be called forced QTBT. In embodiments of the invention, for example, to force a QT split at image boundaries, the MinQTSize split constraint is respected, and no additional signaling is required to force a BT split. Embodiments also allow the separation to be negotiated for normal (non-boundary) blocks and boundary blocks. For example. traditional solutions require two "MinQTSize" parameters, one to separate normal blocks and the other to separate boundary blocks. Embodiments require only one common parameter "MinQTSize" to separate both the normal block and the boundary block, which can be flexibly set between the encoder and decoder, for example, signaling one parameter "MinQTSize". In addition, embodiments require fewer partitions than, for example, forced QT.

Solutions for the Case of Lower Bound and Right Bound

In the case of the bottom and right border, if the block size is larger than MinQTSize, then the division mode for dividing the border image may be selected between forced BT division and forced QT division, for example, based on RDO (rate-distortion optimization). Otherwise (i.e. if the block size is equal to or less than MinQTSize), only the forced separation BT is used to separate the image borders, more specifically, the forced horizontal separation BT is used for the bottom border respectively for the border box located at the bottom border of the image and the vertical forced BT is used for the right border, respectively, for the border box located on the right border of the image.

BT forced splitting may comprise recursively splitting the current block with a horizontal forced border splitting until the current box's sub-partition is at the bottom border of the image, and recursively splitting the sub-partition with a vertically forced border splitting until the leaf the node will not be completely located on the right border of the image. Alternatively, forced partitioning of a BT may comprise recursively splitting the current block with a vertical forced boundary splitting until a sub-partition of the current block is at the bottom boundary, and recursively splitting the sub-partition with a horizontal forced boundary splitting until until the leaf node is completely located on the right border. MinQTSize can also be used to control the division of a non-boundary block.

For example, in the case shown in FIG. 17A, if MinQTSize is equal to or limited to 32, while a rectangular (non-square) block size of 8 samples high or wide is required to fit the image border, forced partitioning will be used to split the 32x32 border block. BT. BT partitions can be further partitioned using the same type of forced BT splitting, for example, in case of forced vertical splitting BT, only additional forced vertical splitting BT is applied, and in case of forced horizontal splitting BT, only additional forced horizontal splitting BT is applied. Forced BT splitting continues until the leaf node is completely within the image.

17B shows an exemplary 128×128 sample lower bound CTU split according to an embodiment of the invention. The lower boundary CTU, which forms the root block or root node of the partition tree, is broken into smaller sections, such as smaller square or rectangular sized blocks. These smaller sections or blocks may be further subdivided into even smaller sections or blocks. In FIG. 17B, the CTU is a first square tree divided into four square blocks 710, 720, 730, and 740, each of which has a size of 64x64 samples. Of these blocks, blocks 710 and 720 are again lower boundary blocks, while blocks 730 and 740 are out of the image (respectively, out of the image) and are not processed.

Block 710 is further divided using a quadtree into four square blocks 750, 760, 770, and 780, each of which has a size of 32x32 samples. Blocks 750 and 760 are located within the image, while blocks 770 and 780 again form the lower boundary blocks. Since the size of block 770 does not exceed MinQTSize, which is equal to 32, for example, a recursive horizontal forced binary split is applied to block 770 until the leaf node is completely within or completely inside the image, for example, until the leaf node block 772 is rectangular a non-square block having 32×16 samples is within the image (after one horizontal binary split), or a leaf node block 774, a rectangular non-square block located at the lower edge of the image and having 32×8 samples, is within the image (after two horizontal binary partitions). The same applies to block 780.

Embodiments of the present invention allow for matching the separation for normal blocks located entirely within the image and the separation of boundary blocks. Boundary boxes are boxes that are not completely inside the image and not completely outside the image. In other words, boundary boxes are boxes that contain a part that is located within the image and a part that is located outside the image. In addition, embodiments of the present invention can reduce signaling since no signaling is required to force a BT split at or below the MinQTSize level.

Corner Case Solutions

In the corner case, some approaches only allow for forced QT splitting, which also ignores the MinQTSize constraint. Embodiments of the present invention provide two solutions for the corner case. A corner case occurs when the block currently being processed is in the corner of the image. This occurs if the current block intersects or is adjacent to two image boundaries (vertical and horizontal).

Solution 1:

The corner case is treated as a lower boundary case or as a right boundary case. Fig.20 shows a variant of the definition of the border. 20 shows the borders of the image in dashed lines and the edge case regions are shown in straight lines. As illustrated, the corner case is defined as the lower boundary case. So the solution is the same as described above for the lower bound case and the right bound case. In other words, first apply the horizontal split (as described for the bottom border case) until the blocks or sections are completely within the image (in the vertical direction) and then apply the vertical split (as described for the right border case) until the leaf the nodes will not be completely within the image (in the horizontal direction).

Solution 2:

The definition of edge cases remains as is. If forced QT is limited by MinQTSize (current block size is equal to or less than MinQTSize), use forced horizontal BT to match lower bound, when lower bound is equal, use forced vertical BT to match right bound.

For example, in FIG. 19A, which shows an implementation of a forced QTBT for a block located in the corner of an image, if MinQTSize is equal to or limited to 32 to force a corner case QT split, an additional BT split after a 32x32 block split will be used until forced separation will not be completed.

19B shows additional details of an exemplary split of an edge CTU at or in a corner of an image according to an embodiment of the invention in which the CTU has a size of 128×128 samples. The CTU is the first quadtree divided into four square blocks, each 64×64 samples in size. Of these boxes, only the upper left box 910 is a boundary box, while the other three are located outside (completely outside) the image and are not further processed. Block 910 is further divided using a quadtree into four square blocks 920, 930, 940, and 950, each of which has a size of 32x32 samples. Block 920 is located inside the image, while blocks 930, 940 and 950 again form boundary blocks. Since the size of these blocks 930, 940 and 950 does not exceed MinQTSize, which is 32, forced binary splitting is applied to blocks 930, 940 and 950.

Block 930 is located on the right border and is split using a recursive vertical forced binary split until the leaf node is within the image, eg block 932 located on the right border of the image (here after two vertical binary splits).

Block 940 is located at the bottom border and is split using a recursive horizontal forced binary split until the leaf node is within the image, such as block 942 located on the right side of the image (here after two horizontal binary splits).

Block 950 is located at the corner boundary and is split using a first recursive horizontal forced binary split until a sub-partition or block, here block 952, is at the bottom border of the image (here after two horizontal binary splits), and then a recursive splitting a sub-partition with a vertically forced boundary split until a leaf node or block, such as block 954, located on the right border of the image (here after two vertical binary splits) or, respectively, until the leaf node is inside the image.

The above approaches can be applied to both decoding and encoding. For decoding MinQTSize can be received via SPS. For encoding, MinQTSize may be transmitted via SPS. Embodiments may use boundary definitions as shown in FIG. 18 or FIG. 20, or other boundary definition methods.

The following are additional embodiments of the present invention. It should be noted that the numbering used in the following section does not have to match the numbering used in the preceding sections.

Option 1 implementation. Separation method comprising:

determining whether the current image block is a boundary block;

if the current box is a boundary box, determining whether the size of the current box exceeds the minimum allowable size of the leaf node of the quadtree;

if the size of the current block does not exceed the minimum allowable size of the leaf node of the quadtree, forced splitting of the binary tree is applied to the current block.

Option 2 implementation. The splitting method of Embodiment 1, wherein the forced binary tree split is a recursive horizontal forced binary split in case the current block is at the bottom border of the image, or is a recursive vertical forced border split in the case that the current block is located at the right border of the image .

Option 3 implementation. The splitting method of embodiment 1 or 2, wherein the forced binary splitting comprises recursively splitting the current block by horizontal forced boundary splitting until a sub-partition of the current block is located directly on the lower border of the image, and recursively splitting the sub-partition with using a vertical forced border split until the entire leaf node is directly on the right border of the image, or vice versa.

Option 4 implementation. The partitioning method as in any one of embodiments 1-3, wherein the minimum allowable quadtree leaf node size is the minimum allowable quadtree leaf node size also used to control non-boundary block splitting.

Option 5 implementation. A decoding method for decoding a block by splitting a block according to the splitting method of any of Embodiments 1 to 4.

Option 6 implementation. The decoding method according to Embodiment 5, wherein the minimum allowable leaf node size of the quadtree is received via the SPS.

Option 7 implementation. An encoding method for encoding a block by splitting a block according to the splitting method of any one of Embodiments 1 to 4.

Option 8 implementation. The coding method of Embodiment 7, wherein the minimum allowable size of the leaf node of the quadtree is transmitted via the SPS.

Option 9 implementation. A device for decoding, containing a logic circuit configured to perform any of the methods of options 5 or 6 implementation.

Option 10 implementation. A device for encoding, containing a logic circuit configured to perform any of the methods of option 7 or 8 implementation.

Option 11 implementation. A persistent storage medium for storing instructions as they are executed by the processor cause the processor to perform any of the methods according to embodiments 1-8.

In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware processor. The computer-readable medium may include a computer-readable medium that corresponds to a tangible medium, such as a data medium, or a communication medium, including any medium that facilitates the transfer of a computer program from one place to another, for example, in accordance with a communication protocol. Thus, computer-readable media can typically correspond to (1) tangible computer-readable storage media that are non-volatile, or (2) communication media such as a signal or carrier wave. Storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the methods described in the present invention. The computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash -memory or any other medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer. Additionally, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are considered as the communication medium. However, it should be understood that computer-readable storage media and storage media do not include connections, carrier waves, signals, or other temporary media, but instead refer to non-volatile, tangible storage media. A disc as used herein includes a compact disc (CD), a laser disc, an optical disc, a digital versatile disc (DVD), a floppy disk, and a Blu-ray disc, where the discs usually reproduce data magnetically and the discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer readable media.

The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Accordingly, the term "processor" as used herein may refer to any of the aforementioned structure, or any other structure suitable for implementing the described methods. Additionally, in some aspects described herein, functionality may be provided within dedicated hardware and/or software modules capable of encoding and decoding or contained in a combined codec. Additionally, the methods may be fully implemented in one or more circuits or logic elements.

The methods of the present invention may be implemented in a wide variety of devices or appliances, including a wireless telephone, an integrated circuit (IC), or a set of ICs (eg, a chipset). The present invention describes various components, modules, or blocks to represent the functional aspects of devices capable of performing the disclosed technologies, but not necessarily requiring implementation in different hardware blocks. Rather, as described above, the various blocks may be combined into a codec hardware block or provided by a set of cooperating hardware blocks including one or more processors, as described above, in combination with suitable software and/or firmware.

The device contains a memory element; and a processor element coupled to the memory element and configured to determine whether the current image block is a boundary block, determine when the current block is a boundary block, determine whether the size of the current block exceeds a minimum allowable size (MinQTSize) of a quadtree (QT), and apply, when the size of the current block does not exceed MinQTSize, force binary tree splitting (BT) to the current block.

Although the present invention presents several embodiments, it should be understood that the disclosed systems and methods can be embodied in many other specific forms without departing from the essence or scope of the present invention. The present examples are to be considered as illustrative and not restrictive, and the present invention is not limited to the details provided herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

Additionally, the methods, systems, subsystems, and technologies described and illustrated in various embodiments as separate or discrete may be combined or integrated with other systems, modules, technologies, or methods without departing from the scope of the present invention. Other elements shown or discussed as related or directly connected or interacting with each other may be indirectly connected or interact through some interface, device or intermediate component, electrically, mechanically or otherwise. Other examples of changes, substitutions, and modifications can be ascertained and performed by a person skilled in the art without deviating from the spirit and scope disclosed herein.

Embodiments of the present invention (or the present disclosure) provide devices and methods for encoding and decoding.

The first aspect relates to a partitioning method, comprising determining whether the current image block is a boundary block and whether the size of the current block exceeds the minimum allowable size of the leaf node of the quadtree; and, if the current box is a boundary box and the size of the current box does not exceed the minimum allowable size (MinQTSize) of the quadtree leaf node, applying forced binary tree splitting (BT) to the current box.

In a first form of implementation of the method according to the first aspect, as such, the forced binary tree split is a recursive horizontal forced binary split in case the current block is located on the bottom border of the image, or a recursive vertical forced border split in case the current block is located on the right border of the image.

In the second form of implementation of the method according to the first aspect as such or any previous form of implementation of the first aspect, forced splitting of the binary tree continues until the leaf node block is within the image.

In a third implementation form of the method according to the first aspect as such or any previous implementation form of the first aspect, forced binary splitting comprises recursively splitting the current block with a horizontal forced boundary splitting until a sub-partition of the current block is at the bottom border of the image; and recursively splitting the sub-partition by vertically forced border splitting until the leaf node is completely on the right border of the image.

In a fourth form of implementation of the method according to the first aspect as such or any previous form of implementation of the first aspect, the forced partition BT comprises recursively partitioning the current block with a vertical forced boundary partition until a sub-partition of the current block is at the bottom boundary; and recursively splitting the sub-partition by means of a horizontal forced boundary split until the leaf node is completely on the right boundary.

In a fifth form of implementation of the method according to the first aspect as such or any previous form of implementation of the first aspect, the method further comprises applying a minimum allowable quadtree leaf node size to control the division of the non-boundary block.

In the sixth form of implementation of the method according to the first aspect as such or any previous form of implementation of the first aspect, the border block is a block that is not completely inside the image and not completely outside the image.

The second aspect relates to a decoding method for decoding a block by dividing the block according to the first aspect as such or any previous form of implementation of the first aspect.

In a first form of implementing the method according to the second aspect, as such, the method further comprises receiving a minimum allowable leaf node size of the quadtree via a sequence parameter set (SPS).

The third aspect relates to an encoding method for encoding a block by dividing the block according to the first aspect as such or any previous form of implementation of the first aspect.

In a first implementation form of the method according to the third aspect, as such, the method further comprises transmitting a minimum allowable quadtree leaf node size via a sequence parameter set (SPS).

The fourth aspect relates to a device for decoding, comprising logic circuits configured to decode the block by dividing the block according to the method of dividing the first aspect as such or any previous form of implementation of the first aspect.

In the first implementation form of the device for decoding according to the fourth aspect, as such, the logic circuit is further configured to receive a minimum allowable size of a leaf node of the quadtree via a sequence parameter set (SPS).

The fifth aspect relates to an encoding device comprising logic circuits configured to encode a block by dividing the block according to the method of dividing the first aspect as such or any previous form of implementation of the first aspect.

In the first implementation form of the decoding apparatus according to the fifth aspect, as such, the logic circuitry is further configured to transmit the minimum allowable size of the leaf node of the quadtree via a sequence parameter set (SPS).

A sixth aspect relates to a persistent storage medium for storing instructions that, when executed by the processor, cause the processor to execute any of the first, second, or third aspects as such, or any preceding form of implementation of the first, second, or third aspect.

A seventh aspect relates to a method comprising determining that the current image block is a boundary block and that the size of the current block is less than or equal to a minimum allowable size (MinQTSize) of a quadtree (QT) leaf node; and applying, in response to the determination, a forced binary tree split (BT) to the current block.

In the first implementation form of the method according to the seventh aspect as such, the current block is located at the bottom border of the image, and in which the forced BT separation is a recursive horizontal forced BT separation.

In the second implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, the current block is located on the right border of the image, and in which the BT forced separation is a recursive vertical BT forced separation.

In a third implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, forced partitioning BT comprises recursively partitioning the current block with a horizontal forced boundary partitioning until a sub-partition of the current block is at a lower boundary; and recursively splitting the sub-partition by vertically forced boundary splitting until the leaf node is completely located on the right boundary.

In a fourth implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, forced partitioning BT comprises recursively partitioning the current block with a vertical forced boundary partitioning until a sub-partition of the current block is at the bottom boundary; and recursively splitting the subdivision by means of a horizontal forced boundary split until the leaf node is completely on the right boundary.

In a fifth form of implementation of the method according to the seventh aspect as such or any previous form of implementation of the seventh aspect, the method further comprises applying MinQTSize to control the division of the non-boundary block.

In a sixth implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, the method further comprises receiving MinQTSize via a sequence parameter set (SPS).

In a seventh implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, the method further comprises transmitting MinQTSize via a Sequence Parameter Set (SPS).

The eighth aspect relates to a device containing a memory; and a processor coupled to the memory and configured to determine whether the current image block is a boundary box, determine when the current block is a boundary box, determine whether the size of the current block is greater than a minimum allowable size (MinQTSize) of a quadtree (QT) leaf node, and apply, when the size of the current block does not exceed MinQTSize, force binary tree splitting (BT) to the current block.

In the first implementation form of the device according to the eighth aspect, the forced BT split as such is a recursive horizontal forced BT split when the current block is located at the bottom border of the image, or a recursive vertical forced BT split when the current block is located at the right border of the image.

In the second implementation form of the device according to the eighth aspect as such or any previous implementation form of the eighth aspect, forced partitioning BT comprises recursively partitioning the current block with a horizontal forced boundary partitioning until a sub-partition of the current block is at a lower boundary; and recursively splitting the sub-partition by vertically forced boundary splitting until the leaf node is completely located on the right boundary.

In the third implementation form of the device according to the eighth aspect as such or any previous implementation form of the eighth aspect, forced partitioning BT comprises recursively partitioning the current block with a vertical forced boundary partitioning until a sub-partition of the current block is at a lower boundary; and recursively splitting the subdivision by means of a horizontal forced boundary split until the leaf node is completely on the right boundary.

In the fourth embodiment of the device according to the eighth aspect as such or any preceding embodiment of the eighth aspect, the processor is further configured to apply MinQTSize to control the division of the non-boundary block.

In a fifth embodiment of the apparatus according to the eighth aspect as such or any preceding embodiment of the eighth aspect, the apparatus further comprises a receiver coupled to the processor and configured to receive a MinQTSize via a sequence parameter set (SPS).

In a sixth embodiment of the apparatus according to the eighth aspect as such or any preceding embodiment of the eighth aspect, the apparatus further comprises a transmitter coupled to the processor and configured to transmit MinQTSize via a sequence parameter set (SPS).

A ninth aspect relates to a computer program product comprising computer-executable instructions stored on a persistent medium that, when executed by a processor, cause a device to determine whether the current image block is a boundary block; determine when the current box is a boundary box, determine whether the size of the current box exceeds the minimum allowable size (MinQTSize) leaf node of the quadtree (QT); and apply, when current block size 0 is not greater than MinQTSize, force binary tree splitting (BT) to the current block.

In the first implementation form of the device according to the eighth aspect, the forced BT split as such is a recursive horizontal forced BT split when the current block is located at the bottom border of the image, or a recursive vertical forced BT split when the current block is located at the right border of the image.

In the second implementation form of the device according to the ninth aspect as such or any previous implementation form of the ninth aspect, the forced partition BT comprises recursively partitioning the current block with a horizontal forced boundary partition until a sub-partition of the current block is at the bottom boundary; and recursively splitting the sub-partition by vertically forced boundary splitting until the leaf node is completely located on the right boundary.

In a third implementation form of the device according to the ninth aspect as such or any previous implementation form of the ninth aspect, forced partitioning BT comprises recursively partitioning the current block with a vertical forced boundary partitioning until a sub-partition of the current block is at a lower boundary; and recursively splitting the subdivision by means of a horizontal forced boundary split until the leaf node is completely on the right boundary.

In the fourth form of implementation of the device according to the ninth aspect as such or any previous form of implementation of the ninth aspect, the instructions further cause the device to apply MinQTSize to control the division of the non-boundary block.

In the fifth form of implementation of the device according to the ninth aspect as such, or any previous form of implementation of the ninth aspect, the instructions further cause the device to receive MinQTSize via sequence parameter set (SPS).

In the sixth form of implementation of the device according to the ninth aspect as such or any previous form of implementation of the ninth aspect, the instructions further cause the device to transmit MinQTSize via sequence parameter set (SPS).

EMBODIMENTS RELATED TO THE RELATIONSHIP BETWEEN SEPARATION CONSTRAINTS FOR NON-BOUNDARY AND BOUNDARY BLOCKS

Next Generation Video Coding (NGVC) does not apply the concept of CU, PU, and TU separation and maintains more flexibility for CU partition forms. The size of the CU corresponds to the size of the coding node and may be square or non-square (eg, rectangular) in shape.

In J. An et al., "Block Partitioning Framework for Next Generation Video Coding", International Telecommunication Union, COM16-C966, September 2015 (hereinafter "VCEG Proposal COM16-C966"), quadtree-binary tree partitioning techniques have been proposed (QTBT) for a future video coding standard beyond HEVC. Modeling has shown that the proposed QTBT structure is more efficient than the quadtree structure in the HEVC used. In HEVC, inter-prediction for small blocks is limited to reduce memory access for motion compensation, and inter-prediction is not supported for 4×4 blocks. In QTBT JEM, these restrictions are removed.

In QTBT, the CU can be square or rectangular. As shown in FIG. 6, a coding tree unit (CTU) is first divided by a quadtree structure. The leaf nodes of the quadtree can be further separated by a binary tree structure. There are two types of partitioning used in a binary tree: symmetrical horizontal and symmetrical vertical partitioning. In each case, the knot is split by dividing the knot in the middle either horizontally or vertically. The leaf nodes of the binary tree are called coding units (CUs) and this segmentation is used to predict and process the transform without any additional partitioning. This means that CU, PU and TU have the same block size in the QTBT coding block structure. A CU sometimes consists of coding blocks (CBs) of different color components, e.g. one CU contains one luma CB and two chrominance CBs in case of P and B segments of 4:2:0 chroma format and sometimes consists of one component CBs, e.g. one CU contains only one luminance CB, or only two chrominance CBs in the case of I-segments.

The following parameters are defined for the QTBT partitioning scheme

- CTU size: the size of the root node of the quadtree, the same concept as in HEVC.

- MinQTSize: minimum allowable size of a leaf node of a quadtree

- MaxBTSize: maximum allowable size of the root node of a binary tree

- MaxBTDepth: maximum allowed binary tree depth

- MinBTSize: minimum allowable size of a binary tree leaf node

In one example of a QTBT partitioning structure, when a quadtree node has a size equal to or less than MinQTSize, the quadtree is not further considered. No additional binary tree partitioning is provided because the size (MinQTSize) is larger than MaxBTSize. Otherwise, the leaf node of the quadtree may be further divided by a binary tree. Therefore, the leaf node of the quadtree is also the root node for the binary tree, and its binary tree depth is 0 (zero). When the depth of the binary tree reaches MaxBTDepth (ie 4), no additional split is considered. If the width of a binary tree node is MinBTSize (ie 4), no additional horizontal split is considered. Similarly, when a binary tree node has a height equal to MinBTSize, no additional vertical split is considered. The leaf nodes of the binary tree are further processed by the prediction and transformation processing without any further separation. In JEM, the maximum CTU size is 256×256 luma samples. The leaf nodes of the binary tree (CUs) can be further processed (eg, by performing a prediction process and a transformation process) without any further splitting.

21 illustrates an example of a block 30 (eg, CTB) split using QTBT splitting techniques. As shown in FIG. 21, using QTBT splitting methods, each of the blocks is split symmetrically at the center of each block. Fig.22 illustrates a tree structure corresponding to the block division in Fig.21. The solid lines in FIG. 22 indicate a quadtree split, and the dotted lines indicate a binary tree split. In one example, at each split (ie, non-leaf) node of the binary tree, a syntax element (eg, flag) is signaled to indicate the type of split performed (eg, horizontal or vertical), where 0 indicates a horizontal split and 1 indicates a vertical split. To split a square tree, you do not need to specify the partition type, because when splitting a square tree, the block is always split horizontally and vertically into 4 sub-blocks of the same size.

As shown in FIG. 22, at node 50, block 30 (corresponding to root 50) is divided into four blocks 31, 32, 33, and 34 shown in FIG. 21 using QT partitioning. Block 34 is not further subdivided and is therefore a leaf node. At node 52, block 31 is further divided into two blocks using BT division. As shown in Fig.22 node 52 is marked with the number 1, which indicates a vertical separation. Thus, a split at node 52 results in a block 37 and a block including both blocks 35 and 36. Blocks 35 and 36 are formed by an additional vertical split at node 54. At node 56, block 32 is further divided into two blocks 38 and 39c. using BT separation.

At node 58, block 33 is divided into 4 blocks of equal size using QT partitioning. Blocks 43 and 44 are formed by QT separation and are not further divided. At node 60, the upper left block is first split using vertical binary tree splitting, resulting in block 40 and a right vertical block. The right vertical block is then split using the horizontal binary tree split into boxes 41 and 42. The bottom right box resulting from the square tree split at node 58 is split at node 62 using the horizontal split tree into boxes 45 and 46. As shown in Fig.22, the node 62 is marked 0, which indicates a horizontal separation.

In addition to QTBT, a block partitioning structure called a multi-type tree (MTT) has been proposed to replace BTs in CU structures based on QTBT, which means that first a CTU can be partitioned using QT partitioning to obtain a CTU and then, secondly, a block can be split using MTT splitting.

The split MTT structure is still a recursive tree structure. MTT uses several different partition structures (for example, two or more). For example, according to MTT technology, two or more different partition structures may be used for each corresponding non-leaf tree structure node at each tree structure depth. The depth of a node in a tree structure may refer to the path length (eg, number of splits) from a node to the root of the tree structure.

MTT uses two types of partitioning: BT partitioning and ternary tree partitioning (TT). The split type can be selected from BT and TT split. The TT split structure differs from the QT or BT split structures in that the TT split structure does not split the block at the center. The central part of the block remains in one subblock. Unlike QT, which forms four blocks, or binary tree, which forms two blocks, splitting according to the TT splitting structure results in three blocks. Examples of partition types according to the TT partition structure include symmetrical partition types (both horizontal and vertical) as well as asymmetric partition types (both horizontal and vertical). In addition, the types of symmetrical partitions according to the TT partition structure may be non-uniform/non-uniform or uniform/uniform. The types of asymmetric partitions according to the CT partition structure are non-uniform/non-uniform. In one example, a TT split structure may include at least one of the following types of splits: horizontal uniform/uniform symmetrical ternary tree, vertical uniform/uniform symmetrical ternary tree, horizontal non-uniform/non-uniform symmetric ternary tree, vertical non-uniform/non-uniform symmetric ternary tree, horizontal non-uniform/non-uniform asymmetric ternary tree, or vertical non-uniform/non-uniform asymmetric ternary tree.

In general, a non-uniform/non-uniform symmetric ternary tree split type is a split type that is symmetrical about the block center line, but where at least one of the three resulting blocks is not the same size as the other two. One preferred example shows that the side blocks are 1/4 of the block size and the center block is 1⁄2 of the block size. A uniform/uniform symmetric ternary tree partition type is a partition type that is symmetrical about the block center line, and all resulting blocks are the same size. Such a split is optional if the height or width of the box, depending on the vertical or horizontal split, is a multiple of 3. A non-uniform/non-uniform asymmetric ternary tree split type is a split type that is not symmetrical about the center line of the box, and where at least , one of the resulting blocks is not the same size as the other two.

23 is a conceptual diagram illustrating possible examples of types of horizontal splitting of a ternary tree. 24 is a conceptual diagram illustrating possible examples of ternary tree vertical split types. In both FIGS. 23 and 24, h represents the block height in luma or chrominance samples and w represents the block width in luma or chrominance samples. Note that the box's corresponding centerline does not represent a box boundary (i.e., ternary tree sections do not split the box across the centerline). Rather, the center line is used to indicate whether a particular section type is symmetrical or asymmetrical about the original block's center line. The center line also runs in the direction of the section.

As shown in FIG. 23, the block 71 is divided by a horizontal uniform/uniform symmetrical partition type. The horizontal uniform/uniform symmetrical partition type forms symmetrical upper and lower halves about the center line of the block 71. The uniform/uniform symmetrical partition type forms three sub-blocks of equal size, each with a height h/3 and a width w. A type of uniform/uniform symmetrical division is possible when the height of the block 71 is divided by 3 without a remainder.

The block 73 is divided by a type of horizontal non-uniform/non-uniform symmetrical partition. The horizontal uneven/non-uniform symmetrical split type produces symmetrical upper and lower halves about the center line of the block 73. The horizontal non-uniform/non-uniform symmetrical split type forms two blocks of equal size (for example, top and bottom blocks with a height of h/4) and a center block of a different size ( for example, the central block height h/2). In one example, according to the type of horizontal non-uniform/non-uniform symmetrical partition, the area of the central block is equal to the combined areas of the upper and lower blocks. In some examples, the type of horizontal non-uniform/non-uniform symmetrical partitioning may be preferred for blocks having a height that is a power of 2 (eg, 2, 4, 8, 16, 32, etc.).

Block 75 is divided by a type of horizontal non-uniform/non-uniform asymmetric separation. The type of horizontal non-uniform/non-uniform asymmetric separation does not form symmetrical upper and lower halves about the center line of the block 75 (ie, the upper and lower halves are asymmetrical). In the example of FIG. 23, the type of horizontal non-uniform/non-uniform asymmetric separation does not form an h/4 height top block, a 3h/8 height center block, and a 3h/8 height bottom block. Of course, other asymmetric arrangements can be used.

As shown in FIG. 24, the block 81 is divided by a type of vertical uniform/uniform symmetrical partition. The vertical even/uniform symmetrical split type forms symmetrical left and right halves about the center line of the block 81. The vertical uniform/uniform symmetrical split type forms three sub-blocks of equal size, each with a width w/3 and a height h. A type of uniform/homogeneous symmetrical division is possible when the width of the block 81 is divided by 3 without a remainder.

The block 83 is divided by a type of vertical non-uniform/non-uniform symmetrical partition. The vertical uneven/non-uniform symmetrical split type generates symmetrical left and right halves about the block center line 83. The vertical uneven/non-uniform symmetrical split type generates symmetrical left and right halves about the center line 83. The vertical uneven/non-uniform symmetrical split type generates two blocks of the same size ( e.g. left and right blocks w/4 wide) and a center block of different sizes (e.g. center block w/2 wide). In one example, according to the type of vertical non-uniform/non-uniform symmetric separation, the area of the center block is equal to the combined areas of the left and right blocks. In some examples, the type of vertical non-uniform/non-uniform symmetrical partitioning may be preferred for blocks having a power of 2 width (eg, 2, 4, 8, 16, 32, etc.).

Block 85 is divided by a type of vertical non-uniform/non-uniform asymmetric separation. The vertical non-uniform/non-uniform asymmetric split type does not form symmetrical left and right halves about the center line of block 85 (ie, left and right halves are asymmetrical). In the example of FIG. 24, the type of vertical non-uniform/non-uniform symmetrical division forms a left block w/4 wide, a center block 3w/8 wide, and a right block 3w/8 wide. Of course, other asymmetric arrangements can be used.

In addition to (or alternatively to) the parameters for QTBT defined above, the following parameters are defined for the MTT split scheme.

- MaxBTSize: maximum allowable size of the root node of a binary tree

- MinBtSize: The minimum allowable size of the root node of the binary tree.

- MaxMttDepth: The maximum depth of a multitype tree.

- MaxMttDepth offset: maximum depth offset of a multitype tree.

- MaxTtSize: the maximum allowable size of the root node of the ternary tree

- MinTtSize: the minimum allowable size of the root node of the ternary tree

- MinCbSize: the minimum size of a coding block.

Embodiments of the present invention may be implemented by a video encoder or video decoder, such as video encoder 20 of FIG. 12 or video decoder 30 of FIG. 13, in accordance with an embodiment of the present invention. One or more building blocks of video encoder 20 or video decoder 30, including a splitter, may be configured to perform the methods of embodiments of the present invention.

In embodiments of the present invention:

In JVET-K1001-v4, log2_ctu_size_minus2, log2_min_qt_size_intra_slices_minus2 and log2_min_qt_size_inter_slices_minus2 are signaled in the SPS (as syntax elements).

The log2_ctu_size_minus2 plus 2 parameter specifies the size of the luminance coding tree block of each CTU. In particular:

CtbLog2SizeY = log2_ctu_size_minus2 + 2 (7-5)

CtbSizeY = 1 << CtbLog2SizeY (7-6)

In other words, CtbLog2SizeY defines a CtbSizeY CTU size log2 value corresponding to the coding tree block (CTB) size for luma (Y).

Additional settings are provided as follows:

MinCbLog2SizeY = 2 (7-7)

MinCbSizeY = 1 << MinCbLog2SizeY (7-8)

MinTbSizeY = 4 (7–9)

MaxTbSizeY = 64 (7-10)

The parameter log2_min_qt_size_intra_slices_minus2 plus 2 defines the minimum size of the leaf block luminance resulting from splitting the CTU quadtree into slices with slice_type equal to 2 (I), that is, inner slices. The log2_min_qt_size_intra_slices_minus2 value must be between 0 and CtbLog2SizeY - 2 inclusive.

MinQtLog2SizeIntraY = log2_min_qt_size_intra_slices_minus2 + 2 (7-22)

The parameter log2_min_qt_size_inter_slices_minus2 plus 2 specifies the minimum size of the leaf block brightness resulting from splitting the CTU quadtree into slices with slice_type equal to 0 (B) or 1 (P), that is, outer slices. The log2_min_qt_size_inter_slices_minus2 value must be between 0 and CtbLog2SizeY - 2 inclusive.

MinQtLog2SizeInterY = log2_min_qt_size_inter_slices_minus2 + 2 (7-23)

MinQtSizeY is defined in (7-30), which means the minimum allowable split size of the quadtree in the luma sample. If the coding block size is less than or equal to MinQtSizeY, splitting the quadtree is not allowed. Additional settings provide the following:

MinQtLog2SizeY=(slice_type==I)? MinQtLog2SizeIntraY: MinQtLog2SizeInterY (7-25)

MaxBtLog2SizeY = CtbLog2SizeY - log2_diff_ctu_max_bt_size (7-26)

MinBtLog2SizeY = MinCbLog2SizeY(7-27)

MaxTtLog2SizeY = (slice_type == I)? 5:6 (7–28)

MinTtLog2SizeY = MinCbLog2SizeY(7-29)

MinQtSizeY = 1 << MinQtLog2SizeY (7-30)

MaxBtSizeY = 1 << MaxBtLog2SizeY (7-31)

MinBtSizeY = 1 << MinBtLog2SizeY (7-32)

MaxTtSizeY = 1 << MaxTtLog2SizeY (7-33)

MinTtSizeY = 1 << MinTtLog2SizeY (7-34)

MaxMttDepth=(slice_type==I)? max_mtt_hierarchy_depth_intra_slices:

max_mtt_hierarchy_depth_inter_slices (7-35)

The max_mtt_hierarchy_depth_intra_slices and max_mtt_hierarchy_depth_inter_slices parameters indicate the maximum hierarchy depth for MTT splitting for inner and outer segments, respectively.

Based on the semantics of log2_min_qt_size_intra_slices_minus2 and log2_min_qt_size_inter_slices_minus2, the ranges of log2_min_qt_size_intra_slices_minus2 and log2_min_qt_size_inter_slices_minus2 are 0 to CtbLog2SizeY-2.

Here, CtbLog2SizeY is defined in the log2_ctu_size_minus2 semantics, which means the log2 value of the luma coding tree block size of each CTU, CtbLog2SizeY in VTM2.0 is 7.

Based on (7-22) and (7-23), MinQtLog2SizeIntraY and MinQtLog2SizeInterY range from 2 to CtbLog2SizeY.

Based on (7-25), the MinQtLog2SizeY range is 2 to CtbLog2SizeY.

Based on (7-30) the MinQtSizeY range in JVET-K1001-v4 is (1 << 2) to (1 << CtbLog2SizeY), in VTM2.0 the range is (1 << 2) to (1 << 7 ), which is between 4 and 128.

In JVET-K1001-v4, log2_diff_ctu_max_bt_size is conditionally signaled in the segment header.

The log2_diff_ctu_max_bt_size parameter defines the difference between the luminance CTB size and the maximum luminance size (width or height) of a coding block that can be divided using binary partitioning. The log2_diff_ctu_max_bt_size value must be between 0 and CtbLog2SizeY - MinCbLog2SizeY inclusive.

When log2_diff_ctu_max_bt_size is missing, log2_diff_ctu_max_bt_size is assumed to be 2.

MinCbLog2SizeY is defined in (7-7), which means the minimum allowed coding block size.

Based on the semantics of log2_diff_ctu_max_bt_size, the range of log2_diff_ctu_max_bt_size is 0 to CtbLog2SizeY - MinCbLog2SizeY.

Based on (7-26), MaxBtLog2SizeY ranges from CtbLog2SizeY to MinCbLog2SizeY.

Based on (7-31), MaxBtSizeY ranges from (1 << CtbLog2SizeY) to (1 << MinCbLog2SizeY).

Based on (7-7), the MaxBtSizeY range in JVET-K1001-v4 is (1 << CtbLog2SizeY) to (1 << 2), because in VTM2.0 CtbLog2SizeY is 7, the MaxBtSizeY range in VTM2.0 is 128 to 4.

Therefore, MinQtSizeY has a range of 4 to (1 << CtbLog2SizeY), in VTM2.0 4 to 128, MaxBtSizeY has a range of (1 << CtbLog2SizeY) to 4, in VTM2.0 128 to 4.

Accordingly, there is a possibility that MinQtSizeY is greater than MaxBtSizeY.

Moreover, based on the current border handling in VVC 2.0, only QT and BT separation is allowed for blocks located on the border (TT not allowed, no separation not allowed).

If the current coding block is located on the boundary and the current size of the coding block cbSizeY satisfies the condition:

MinQtSizeY > cbSizeY > MaxBtSizeY,

neither QT nor BT is possible for the current coding block. Therefore, there is no split mode available for the current block.

IMPLEMENTATION OPTION 1.

The following is a detailed description of the solution (embodiments of the invention) of the above technical problems, including the aspect of the boundary case.

According to an embodiment, in order to solve the above technical problem, the lower bound of MaxBtSizeY should be limited to MinQtSizeY to ensure that MaxBtSizeY is not less than MinQtSizeY. In particular, the lower bound of MaxBtSizeY can be equal to MinQtSizeY, so the range of MaxBtSizeY must be from (1 << CtbLog2SizeY) to (1 << MinQtLog2SizeY), so the range of MaxBtLog2SizeY must be from CtbLog2SizeY to MinQtLog2SizeY, so the range of log2_diff_ctu_max_max_bt_sizebLogSize2 must be from 0 to CtbLog2SizeY- MinQtLog2SizeY.

The corresponding change in the draft (video standard) is in the semantics of log2_diff_ctu_max_bt_size as follows:

log2_diff_ctu_max_bt_size defines the difference between the luminance CTB size and the maximum luma size (width or height) of a coding block that can be divided using binary partitioning. The log2_diff_ctu_max_bt_size value must be between 0 and CtbLog2SizeY - MinQtLog2SizeY inclusive.

The corresponding encoding method implemented by the encoding device (decoder or encoder) may be as follows:

determining whether the current image block is a boundary block;

determining whether the size of the current block exceeds the minimum allowable size of the leaf node of the quadtree;

if the current block is a boundary block and the size of the current block does not exceed the minimum allowable size of the leaf node of the quadtree, applying a binary split to the current block; in which the minimum allowable size of the leaf node of the quadtree does not exceed the maximum allowable size of the root node of the binary tree.

Wherein applying a binary split to the current block may comprise applying a forced binary split to the current block.

In which the current block can be obtained by splitting a picture or a coding tree block (CTU).

In which the method may contain two cases: 1) treeType is equal to SINGLE_TREE or DUAL_TREE_LUMA; 2) treeType is equal to DUAL_TREE_CHROMA. For case 1) the current block is a luminance block and for case 2) the current block is a chrominance block.

Wherein the maximum allowable size of the root node of the binary tree may be the maximum size of the luminance in the luminance samples of the root luminance coding block that can be partitioned using binary tree partitioning.

Wherein the maximum allowable size of the ternary tree root node may be the maximum luminance size in luminance samples of the luminance root coding block that can be partitioned using ternary tree partitioning.

Wherein the minimum allowable quadtree leaf node size may be the minimum luminance size in the luminance leaf block luminance samples resulting from splitting the quadtree.

In this document, coding refers to the coding of a picture, video, or moving picture.

Boundary block means that the image/frame border cuts the block, or in other words, the block is on the image/frame border. In the above embodiment, binary splitting is applied to the current block if the current block is a boundary block (condition 1) and its size does not exceed the minimum allowable leaf node size of the quadtree (condition 2). It should be noted that in some embodiments, ternary or other partitioning may be used instead of binary partitioning. Moreover, in some embodiments, binary splitting may be applied under condition 2 regardless of condition 1. In other words, condition 1 does not need to be evaluated. If the size of the current block is indeed larger than the minimum allowed quadtree leaf node size (i.e. condition 2 is not met), quadtree splitting may be applied.

It should be noted that embodiments may be considered in which binary separation is used only for boundary blocks (condition 1). For non-boundary blocks, the quadtree partition may be the only partition used. The use of binary (or ternary) partitioning at an image/frame boundary has the advantage of alternatively more efficient separation, eg horizontal binary/ternary partitions at a horizontal border and vertical binary/ternary partitions at a vertical border.

Another appropriate encoding method implemented by the encoding device (decoder or encoder) may be as follows: determining if the boundary block size exceeds the minimum allowable size of the leaf node of the quadtree. If the size of the boundary box does not exceed the minimum allowed quadtree leaf node size, the minimum allowed quadtree leaf node size does not exceed the maximum allowed binary tree root node size (e.g., by the standard specification), and binary partitioning is applied to the boundary box.

Alternatively, the border block may not contain a corner block. In other words, a corner box that is clipped by both the vertical and horizontal image/frame boundaries is not considered a boundary box for the purposes of the above condition 1.

IMPLEMENTATION OPTION 2

The following is a description of other embodiments of the present invention (in combination with the above embodiments).

In JVET-K1001-v4 max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices are signaled in SPS. In other words, max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices are syntax elements, which means that their value is contained in a bitstream that also includes the encoded image or video.

In particular, max_mtt_hierarchy_depth_inter_slices defines the maximum hierarchy depth for an encoding block resulting from splitting a multitype tree into leafy quadtrees in segments with slice_type equal to 0 (B) or 1 (P). The value of max_mtt_hierarchy_depth_inter_slices must be between 0 and CtbLog2SizeY - MinTbLog2SizeY inclusive.

max_mtt_hierarchy_depth_intra_slices specifies the maximum hierarchy depth for an coding block resulting from splitting a multitype leaf tree quadtree into segments with slice_type equal to 2 (I). The value of max_mtt_hierarchy_depth_intra_slices must be between 0 and CtbLog2SizeY - MinTbLog2SizeY inclusive.

MinTbSizeY is defined in (7-9), which is fixed at 4, so MinTbLog2SizeY = log2 MinTbSizeY, which is fixed at 2.

MaxMttDepth is defined, which means the maximum allowable splitting depth of a multitype tree. If the current multitype tree split depth is greater than or equal to MaxMttDepth, multitype tree splitting is not allowed (enforced).

Based on the semantics of max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices, the range of max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices is from 0 to CtbLog2SizeY - MinTbLog2SizeY.

Based on (7-35), the MaxMttDepth range is 0 to CtbLog2SizeY - MinTbLog2SizeY. Since CtbLog2SizeY is 7 in VTM2.0, the MaxMttDepth range is 0 to 5.

Therefore, MaxMttDepth has a range of 0 to CtbLog2SizeY - MinTbLog2SizeY, in VTM2.0 0 to 5.

Based on the current border handling in VVC 2.0, only QT and BT splitting is allowed for a block located on a border (not allowed by TT, not allowed without splitting).

If the first technical challenge mentioned above is solved (MaxBtSizeY >= MinQtSizeY), the following condition still holds:

cbSizeY <= MinQtSizeY

MaxMttDepth = 0

For boundary processing, the levels of BT sections (in general, any MTT, including TT) are not enough.

For example, MinQtSizeY is 16, MinTbSizeY is 4, MaxMttDepth is 0.

If a border box with cbSizeY = 16 and the parent section is QT, and that box is still on the boundary, no further split can be performed because the Mttdepth of the current box has reached MaxMttDepth.

The solution (an embodiment of the invention) of this technical problem of the boundary case: in order to solve the mentioned technical problem, the lower limit of MaxMttDepth should be limited to 1 (in other words, it cannot take the value of zero) in order to make sure that after splitting the QT, there are enough levels of different types of splitting of the tree for the boundary case. Or, even more so, the lower bound of MaxMttDepth should be limited (MinQtLog2SizeY-MinTbLog2SizeY) to make sure there are enough levels of the multi-vector tree partition after QT split for both edge and non-edge cases.

The corresponding change in the (standard) project text is in the semantics of max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices as follows:

max_mtt_hierarchy_depth_inter_slices specifies the maximum hierarchy depth for an encoding block resulting from splitting a multitype tree on a leaf quadtree into slices with slice_type equal to 0 (B) or 1 (P). The value of max_mtt_hierarchy_depth_inter_slices must be between 1 and CtbLog2SizeY - MinTbLog2SizeY inclusive.

max_mtt_hierarchy_depth_intra_slices specifies the maximum hierarchy depth for an coding block resulting from splitting a multitype leaf tree quadtree into segments with slice_type equal to 2 (I). The value of max_mtt_hierarchy_depth_intra_slices must be in the range from 1 to CtbLog2SizeY - MinTbLog2SizeY inclusive.

or

max_mtt_hierarchy_depth_inter_slices specifies the maximum hierarchy depth for an coding block resulting from splitting a multitype leaf tree quadtree into slices with slice_type equal to 0 (B) or 1 (P). The value of max_mtt_hierarchy_depth_inter_slices must be between MinQtLog2SizeY-MinTbLog2SizeY and CtbLog2SizeY - MinTbLog2SizeY inclusive.

max_mtt_hierarchy_depth_intra_slices specifies the maximum hierarchy depth for an coding block resulting from splitting a multitype leaf tree quadtree into segments with slice_type equal to 2 (I). The value of max_mtt_hierarchy_depth_intra_slices must be in the range from MinQtLog2SizeY-MinTbLog2SizeY to CtbLog2SizeY - MinTbLog2SizeY inclusive.

The corresponding encoding method implemented by the encoding device (decoder or encoder) may be as follows:

Division of the image into blocks, in which the blocks constitute the boundary block; Applying a binary split to a boundary box with a maximum multitype split boundary depth, in which the maximum multitype split boundary depth is the sum of at least the maximum multitype tree depth and a maximum multitype tree depth offset, in which the maximum multitype tree depth is greater than 0. This option implementation may be combined with option 1 implementation or may be used without option 1 implementation.

Alternatively, the maximum depth of a multitype tree is greater than 0 when binary splitting is applied to the boundary box.

Alternatively, the border block may not contain a corner block.

OPTION 3 IMPLEMENTATION.

In other embodiments of the present invention:

In JVET-K1001-v4 if MinQtSizeY > MaxBtSizeY and MinQtSizeY > MaxTtSizeY.

If cbSize = MinQtsizeY, the partition cannot reach MinCbSizeY (MinTbSizeY and MinCbsizeY are fixed at 4), due to no possible partition mode available.

Solving this technical problem of non-boundary case or edge case: To solve the mentioned technical problem, the lower bound of MaxBtSizeY must be limited to MinQtSizeY to ensure that MaxBtSizeY is not less than MinQtSizeY. Or the lower bound of MaxTtSizeY should be limited to MinQtSizeY to make sure that MaxTtSizeY is not less than MinQtSizeY.

The corresponding change in the text of the project is in the semantics

log2_diff_ctu_max_bt_size defines the difference between the luminance CTB size and the maximum luma size (width or height) of a coding block that can be divided using binary partitioning. The log2_diff_ctu_max_bt_size value must be between 0 and CtbLog2SizeY - MinQtLog2SizeY inclusive.

And/or

log2_min_qt_size_intra_slices_minus2 plus 2 specifies the minimum size of the leaf block luminance resulting from splitting the CTU quadtree into slices with slice_type equal to 2 (I). The log2_min_qt_size_intra_slices_minus2 value must be between 0 and MaxTtLog2SizeY - 2 inclusive.

log2_min_qt_size_inter_slices_minus2 plus 2 defines the minimum size of the leaf block brightness resulting from splitting the CTU quadtree into slices with slice_type equal to 0 (B) or 1 (P). The log2_min_qt_size_inter_slices_minus2 value must be between 0 and MaxTtLog2SizeY - 2 inclusive.

The corresponding encoding method implemented by the encoding device (decoder or encoder) may be as follows:

determining whether the size of the current block exceeds the minimum allowable size of the leaf node of the quadtree;

If the size of the current block is not larger than the minimum allowable size of a quadtree leaf node, a multitype tree split is applied to the current block;

Wherein the minimum allowable size of a leaf node of the quadtree does not exceed the maximum allowable size of a root node of a binary tree, or the minimum allowable size of a leaf node of a quadtree does not exceed the maximum allowable size of a root node of a ternary tree.

Alternatively, the minimum allowable size of a quadtree leaf node does not exceed the maximum allowable size of a binary tree root node, and the minimum allowable size of a quadtree leaf node does not exceed the maximum allowable size of a ternary tree root node.

Alternatively, applying a multiple tree split to the current block, including applying a ternary split to the current block or applying a binary split to the current block.

Alternatively, the border block may not contain a corner block.

OPTION 4 IMPLEMENTATION

In other embodiments of the present invention:

If MaxBtSizeY >= MinQtSizeY, MinQtSizeY > MinTbLog2SizeY and MaxMttDepth <(MinQtLog2SizeY- MinTbLog2SizeY),

If cbSize = MinQtsizeY, the partition cannot reach MinCbSizeY, due to not having enough split levels in the multitype tree.

Solving this technical problem of non-boundary case or edge case: To solve the mentioned technical problem, the lower bound of MaxMttDepth must be limited to (MinQtLog2SizeY- MinTbLog2SizeY) to ensure that there are enough split levels of the multitype tree after QT split for both edge and non-boundary cases.

The corresponding change in the draft text is in the semantics of max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices as follows:

max_mtt_hierarchy_depth_inter_slices specifies the maximum hierarchy depth for an coding block resulting from splitting a multitype leaf tree quadtree into slices with slice_type equal to 0 (B) or 1 (P). The value of max_mtt_hierarchy_depth_inter_slices must be between MinQtLog2SizeY-MinTbLog2SizeY and CtbLog2SizeY - MinTbLog2SizeY inclusive.

max_mtt_hierarchy_depth_intra_slices specifies the maximum hierarchy depth for an coding block resulting from splitting a multitype leaf tree tree into quadtree segments with slice_type equal to 2 (I). The value of max_mtt_hierarchy_depth_intra_slices must be between MinQtLog2SizeY-MinTbLog2SizeY and CtbLog2SizeY - MinTbLog2SizeY inclusive.

The corresponding encoding method implemented by the encoding device (decoder or encoder) may be as follows:

Dividing the image into blocks;

Applying a multitype tree split to a block of blocks with a finite maximum multitype tree depth, in which the final maximum multitype tree depth is the sum of at least the maximum multitype tree depth and a maximum offset depth of the multitype tree, in which the maximum multitype tree depth is greater than or equal to the subtraction the Log2 value of the minimum allowed transform block size from the Log2 value of the minimum allowed quadtree leaf node size, or the maximum depth of the multitype tree is greater than or equal to the subtraction of the Log2 value of the minimum allowed encoding block size from the Log2 value of the minimum allowed quadtree leaf node size.

Alternatively, the block is not a boundary.

Alternatively, the maximum depth offset of a multitype tree is 0.

Alternatively, the block is a boundary block and the multitype tree split is a binary split.

Alternatively, a multitype tree split is (or includes) a ternary split.

Alternatively, the border block may not contain a corner block.

Embodiments 1-4 may be applied at the encoder side to divide the image/frame into coding units and to encode coding units. Embodiments 1-4 may be applied at the decoder side to provide image/frame sections, namely coding blocks, and to decode coding blocks respectively (eg, correctly parse coding blocks from a stream and decode them).

According to some embodiments, a decoder is provided, comprising one or more processors; and a persistent computer-readable storage medium coupled to the processors and storing programs for execution by the processors, wherein the programs, when executed by the processors, configures the decoder to perform any of the methods described above with reference to Embodiments 1-4.

Moreover, an encoder comprising one or more processors is provided; and a persistent computer-readable storage medium coupled to the processors and storing programs for execution by the processors, wherein the programs, when executed by the processors, configures the encoder to perform any of the methods described above with reference to Embodiments 1-4.

Thus, there is provided an encoding method implemented by a decoding apparatus, comprising: determining whether the size of the current block exceeds a minimum allowable size of a leaf node of the quadtree; if the size of the current block is not greater than the minimum allowable size of the leaf node of the quadtree, applying a multi-type tree split to the current block; in which the minimum allowable size of a leaf node of the quadtree does not exceed the maximum allowable size of the root node of a binary tree, or the minimum allowable size of a leaf node of the quadtree does not exceed the maximum allowable size of the root node of a ternary tree.

This approach facilitates efficient separation and signaling of separation parameters for image/video blocks.

Moreover, in some implementations, the method also includes the steps of determining whether the current image block is a boundary block. If the current block is a boundary block and the size of the current block does not exceed the minimum allowable quadtree leaf node size, the method also includes applying a binary split to the current block. It should be noted that in this case the minimum allowable size of the leaf node of the quadtree does not exceed the maximum allowable size of the root node of the binary tree. For example, if the size of the current box does not exceed the minimum allowed size of a leaf node of the quadtree, the above application of a multitype tree split to the current box includes applying a binary split to the current box if the current box is a boundary box and the size of the current box is not greater than the minimum allowed size of a leaf node. quadtree node.

Ensuring binary separation can be especially advantageous for blocks at an image/video frame boundary, such as blocks separated by a boundary. Thus, in some implementations it may be useful to apply the approach to the boundary boxes rather than to the rest of the boxes. However, the present invention is not limited to this, and as mentioned above, the approach of applying binary division for greater division depth for non-boundary blocks can also be applied and signaled efficiently.

In addition to or alternatively to the above embodiments, the minimum allowable quadtree leaf node size does not exceed the maximum allowable binary tree root node size, and the minimum allowable quadtree leaf node size does not exceed the maximum allowable ternary tree root node size.

Applying a multi-type tree split to the current block may include applying a ternary split to the current block, or applying a binary split to the current block. However, the present invention is not limited to this and, in general, a multi-type tree partition may also include additional or other different partition types.

The method may further include determining a maximum allowable size of a root node of the binary tree based on a minimum allowable size of a leaf node of the quadtree. This facilitates efficient signaling/storage of parameters. For example, the maximum allowable size of a root node of a binary tree can be considered equal to the minimum allowable size of a leaf node of a quadtree. However, the present invention is not limited to this, and another relationship can be assumed to obtain the maximum allowable size of the root node of a binary tree.

According to an exemplary embodiment, in addition to or as an alternative to the above embodiments, the method may comprise the steps of dividing an image into blocks, in which the blocks contain the current block. Applying a binary split to the current block comprises applying a binary split to a boundary box with a maximum multitype split boundary depth, wherein the maximum multitype split boundary depth is the sum of at least the maximum multitype tree depth and a maximum multitype tree depth offset, wherein the maximum depth multitype tree is greater than 0. Moreover, in some implementations, the maximum depth of a multitype tree is greater than 0 when applying binary splitting to a boundary box.

According to an embodiment, the method includes dividing the image into blocks (blocks containing the current block). Applying a multitype tree split to the current block includes applying a multitype tree split to the current block of blocks with a finite maximum multitype tree depth, wherein the final maximum multitype tree depth is the sum of at least the maximum multitype tree depth and the maximum offset of the multitype tree depth, in which the maximum depth of the multitype tree is greater than or equal to the subtraction of the Log2 value of the minimum allowed transform block size from the Log2 value of the minimum allowed size of the quadtree leaf node, or the maximum depth of the multitype tree is greater than or equal to the subtraction of the Log2 value of the minimum allowed encoding block size from the Log2 value of the minimum allowed size leaf node of the quadtree. This facilitates additional separation even at large separation depths.

The current block may be a non-boundary block. The maximum depth offset of a multitype tree may be 0. The current block may alternatively or additionally be a boundary box, and the multitype tree split is a binary split. A multitype tree split can be ternary or include a ternary split.

According to an embodiment, an encoding method is provided, comprising the steps of: determining whether the size of the current block exceeds a minimum allowable size of a leaf node of the quadtree; if the size of the current block is not greater than the minimum allowable size of the leaf node of the quadtree, applying a multi-type tree split to the current block; in which the minimum allowable size of a leaf node of the quadtree does not exceed the maximum allowable size of the root node of a binary tree, or the minimum allowable size of a leaf node of the quadtree does not exceed the maximum allowable size of the root node of a ternary tree.

The encoding method may apply any of the above rules and restrictions described in relation to the decoding method. Because the encoder side and the decoder side must share the bitstream. Specifically, the encoding side generates a bitstream after encoding sections resulting from the division described above, while the decoding side parses the bitstream and reconstructs the decoded sections accordingly. The same applies to the embodiments relating to the encoding device (encoder) and the decoding device (decoder) described below.

According to an embodiment, a device for decoding is provided, comprising a circuit configured to: determine whether the size of the current block exceeds a minimum allowable size of a leaf node of the quadtree; if the size of the current block does not exceed the minimum allowable size of the leaf node of the quadtree, apply a multitype tree split to the current block; in which the minimum allowable size of a leaf node of the quadtree does not exceed the maximum allowable size of the root node of a binary tree, or the minimum allowable size of a leaf node of the quadtree does not exceed the maximum allowable size of the root node of a ternary tree. It should be noted that the determination of whether the size of the current block exceeds the minimum allowable size of the leaf node of the quadtree may be performed based on the signaling in the bitstream on the decoding side.

An encoder is also provided, which includes a circuit configured to: determine whether the size of the current block exceeds the minimum allowable size of the leaf node of the quadtree; if the size of the current block does not exceed the minimum allowable size of the leaf node of the quadtree, apply a multitype tree split to the current block; in which the minimum allowable size of a leaf node of the quadtree does not exceed the maximum allowable size of the root node of a binary tree, or the minimum allowable size of a leaf node of the quadtree does not exceed the maximum allowable size of the root node of a ternary tree.

According to an embodiment, a persistent computer-readable storage medium is provided for storing programs to be executed by the processing circuit, in which the programs, when executed by the processing circuit, configures the processing circuit to perform any of the methods mentioned above.

The devices described herein may include processing circuits for performing the various operations and methods described herein. The processing circuit may comprise hardware and software. For example, the processing circuit may include one or more processors and non-volatile memory coupled to one or more processors. The memory may contain program code that, when executed by one or more processors, causes the device to perform specified operations or methods.

The invention has been described using various embodiments, both examples and implementations. However, it will be apparent to those skilled in the art that other variations are possible that may put the present invention into practice based on examination of the drawings, the description of the invention, and the independent claims. In the claims, as well as in the description, the word "comprising" does not exclude other elements or steps, the indefinite article "a" or "an" does not exclude plurality. One element or other block can perform the functions of several objects or objects specified in the claims. The mere fact that certain measures are set forth in mutually different dependent claims does not mean that a combination of these measures cannot be used in an advantageous implementation.

The following is an explanation of the implementations of the encoding method as well as the decoding method as shown in the above embodiments and the system using them.

27 is a block diagram showing a content supply system 3100 for realizing a content distribution service. This content delivery system 3100 includes a capture device 3102, a terminal device 3106, and further includes a display 3126. The capture device 3102 communicates with the terminal device 3106 over a communication line 3104. The communication link may include the communication channel 13 described above. Link 3104 includes, but is not limited to, WIFI, Ethernet, cable, wireless (3G/4G/5G), USB, or any combination thereof, and the like.

The capture device 3102 generates data and may encode the data in an encoding manner as shown in the above embodiments. Alternatively, the capture device 3102 may distribute the data to a streaming server (not shown in the drawings), and the server encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, a camera, smartphone, or tablet, computer or laptop, videoconferencing system, PDA, vehicle-mounted device, or a combination thereof, etc. For example, capture device 3102 may include source device 12 as described above. When the data includes video, the video encoder 20 included in the capture device 3102 may actually perform video encoding processing. When the data includes audio (ie, voice), the audio encoder contained in the capture device 3102 may actually perform audio encoding processing. For some practical scenarios, the capture device 3102 distributes encoded video and audio data by multiplexing them together. For other practical scenarios, such as in a videoconferencing system, encoded audio data and encoded video data are not multiplexed. The capture device 3102 separately distributes the encoded audio data and the encoded video data to the terminal device 3106.

In the content delivery system 3100, the terminal device 310 receives and reproduces the encoded data. Terminal device 3106 may be a device capable of receiving and recovering data, such as a smartphone or tablet 3108, a computer or laptop 3110, a Network Video Recorder (NVR)/Digital Video Recorder (DVR) 3112, a television 3114, a set-top box (STB) 3118 videoconferencing system 3120 video surveillance system 3120 personal digital assistant (PDA) 3122 device 3124 installed on the vehicle, or a combination or the like, configured to decode the above encoded data. For example, terminal 3106 may include destination 14 as described above. When the encoded data includes video, priority is given to the video decoder 30 contained in the terminal to perform video decoding. When the encoded data includes audio, the audio decoder contained in the terminal is given priority to perform audio decoding processing.

For a display terminal such as a 3108 Smartphone or Tablet, 3110 Computer or Laptop, 3112 Network Video Recorder (NVR)/DVR (DVR), 3114 TV, 3122 Personal Digital Assistant (PDA), or 3124 Vehicle Mounted Device, the terminal may transmit the decoded data to its display. For a terminal device not equipped with a display, such as STB 3116, video conferencing system 3118 or video surveillance system 3120, it establishes contact with an external display 3126 to receive and display decoded data.

When each device in this system performs encoding or decoding, an image encoding device or an image decoding device may be used as shown in the above embodiments.

28 is a diagram showing the structure of an example terminal device 3106. After the terminal device 3106 receives a stream from the capturing device 3102, the protocol processing unit 3202 parses the transmission protocol of the stream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Real Time Streaming Protocol (HLS), MPEG-DASH, Real Time Transport Protocol (RTP), Messaging Protocol in real time (RTMP) or any combination thereof or the like.

After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexer 3204 . A demultiplexer 3204 may separate the multiplexed data into encoded audio data and encoded video data. As described above, for some practical scenarios, such as in a videoconferencing system, encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is transmitted to the video decoder 3206 and audio decoder 3208 without using the demultiplexer 3204.

Through the demultiplexing processing, a video elementary stream (ES), audio ES, and optionally subtitles are generated. The video decoder 3206, which includes the video decoder 30 as explained in the above embodiments, decodes the ES video using the decoding method as shown in the above embodiments to generate a video frame, and supplies this data to the synchronization block 3212. An audio decoder 3208 decodes the audio ES to generate an audio frame and provides this data to a sync block 3212. Alternatively, the video frame may be stored in a buffer (not shown in FIG. 18) before being fed to the sync block 3212. Likewise, the audio frame may be stored in a buffer (not shown in FIG. 28) before being fed to the sync block 3212.

The sync block 3212 synchronizes the video frame and the audio frame and supplies video/audio to the video/audio display 3214. For example, the sync block 3212 synchronizes the presentation of video and audio information. The information may be encoded in a syntax using timestamps relating to the presentation of the encoded audio and visual data and timestamps relating to the delivery of the data stream itself.

If there is a subtitle in the stream, the subtitle decoder 3210 decodes the subtitles and synchronizes them with the video frame and audio frame and passes the video/audio/subtitles to the video/audio/subtitle display 3216.

The present invention is not limited to the above system, and in the above embodiments, either the image encoding device or the image decoding device may be included in another system such as an automobile system.

Although embodiments of the invention have been primarily described in terms of video coding, it should be noted that the embodiments of encoding system 10, encoder 20, and decoder 30 (and system 10, respectively) and other embodiments described herein can also be performed. capable of processing or encoding a still image, that is, processing or encoding a single image independently of any previous or subsequent image, as in video encoding. In general, if the image encoding processing is limited to one image 17, only inter prediction blocks 244 (encoder) and 344 (decoder) may be unavailable in the case. All other functions (also referred to as tools or technologies) of video encoder 20 and video decoder 30 can equally be used for still picture processing, e.g. /312, split 262/362, intra prediction 254/354, and/or loop filtering 220, 320 and entropy coding 270 and entropy decoding 304.

Embodiments of, for example, encoder 20 and decoder 30, as well as the functions described herein, for example, in relation to encoder 20 and decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on a computer-readable medium or transmitted over a communication medium as one or more instructions or code and executed by a hardware processor. The computer-readable medium may include a computer-readable medium that corresponds to a tangible medium, such as a data medium, or a communication medium, including any medium that facilitates the transfer of a computer program from one place to another, for example, in accordance with a communication protocol. Thus, computer-readable media can typically correspond to (1) tangible computer-readable storage media that are non-volatile, or (2) communication media such as a signal or carrier wave. Storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the methods described in the present invention. The computer program product may include a computer-readable medium.

The following logical operators or mathematical operators are defined as follows:

The mathematical operators used in the present invention are similar to those used in the C programming language. However, the results of integer division and arithmetic shift operations are more precisely defined, and additional operations such as exponentiation and real-valued division are defined. Numbering and counting rules usually start from 0, for example, "first" is equivalent to 0, "second" is equivalent to 1, and so on.

Arithmetic operators

The following arithmetic operators are defined as follows:

+ Supplement

- Subtraction (as a two-argument operator) or negation (as a unary prefix operator)

* Multiplication including matrix multiplication

x ^y Exponentiation. Specifies x to the power of y. In other contexts, such a notation is used for superscript rather than for interpretation as exponentiation.

/ Integer division with result clipping towards zero. For example, 7/4 and -7/-4 are truncated to 1, -7/4 and 7/-4 are truncated to -1.

÷ Used to indicate division in mathematical equations where no clipping or rounding is expected.

Используется для обозначения деления в математических уравнениях, где не предполагается отсечение или округление.

Used to indicate division in mathematical equations where no clipping or rounding is expected.

Суммирование f (i), где i принимает все целые значения от x до y включительно.

Summation f (i), where i takes all integer values from x to y inclusive.

x % y Module. The remainder of x divided by y is only defined for integers x and y where x>=0 and y>0.

Logical operators

The following logical operators are defined as follows:

x && y Boolean logical operation "and" x and y

x | | y Boolean logical operation "or" x and y

! Boolean logical operation "not"

x? y : z If x is TRUE or not equal to 0, the value of y is evaluated; otherwise, the z value is calculated.

Relational Operators

The following relational operators are defined as follows:

> more than

> = Greater than or equal

< Less than

<= Less than or equal

= = Equal

!= Not equal

When a relational operator is applied to a syntax element or variable that has been assigned the value "na" (not applicable), the value "na" is treated as a separate value for the syntax element or variable. The value "na" is not considered equal to any other value.

Bitwise Operators

The following bitwise operators are defined as follows:

& Bitwise "and". When used with integer arguments, operates on two's complement representation of the integer value. When working with a binary argument that contains fewer bits than another argument, the shorter argument is expanded by adding more significant bits equal to 0.

| Bitwise "or". When used with integer arguments, operates on two's complement representation of the integer value. When working with a binary argument that contains fewer bits than another argument, the shorter argument is expanded by adding more significant bits equal to 0.

^ Bitwise "XOR". When used with integer arguments, operates on two's complement representation of the integer value. When working with a binary argument that contains fewer bits than another argument, the shorter argument is expanded by adding more significant bits equal to 0.

x >> y Arithmetic right shift of the integer representation of x with y's complement of binary digits. This function is only defined for non-negative integer y values. The bits shifted into the most significant bits (MSBs) as a result of a right shift have a value equal to MSB x before the shift operation.

x<<y Arithmetic left shift of the integer representation of x with two's complement of y binary digits. This function is only defined for non-negative integer y values. The bits shifted into the least significant bits (LSBs) as a result of a left shift have a value of 0.

Assignment operators

The following assignment operators are defined as follows:

= Assignment operator

++ Increment, i.e. x + + is equivalent to x = x + 1; when used in an array index, evaluates the value of the variable before the increment operation.

- - Decrement, i.e. x-- is equivalent to x = x - 1; when used in an array index, evaluates the value of the variable before the decrement operation.

+ = Increment by the specified amount, i.e. x + = 3 is equivalent to x = x + 3, and x + = (−3) is equivalent to x = x + (−3).

- = Decrement by the specified amount, i.e. x - = 3 is equivalent to x = x - 3, and x - = (−3) is equivalent to x = x - (−3).

Specifying a Range

The following notation is used to indicate the range of values:

x = y..z x takes integer values from y to z inclusive, where x, y and z are integers and z is greater than y.

Math functions

The following mathematical functions are defined:

abs(x)=

Asin (x) trigonometric inverse sine function operating on argument x, in the range -1.0 to 1.0 inclusive, with an output value in the range −π ÷ 2 to π ÷ 2 inclusive in radians

Atan (x) trigonometric arc tangent function acting on the argument x, with an output value in the range −π ÷ 2 to π ÷ 2 inclusive in radians

atan2(y, x) =

Ceil(x) is the smallest integer greater than or equal to x.

Clip1Y(x) = Clip3(0, (1 << BitDepthY) - 1, x)

Clip1C(x) = Clip3(0, (1 << BitDepthC) - 1, x)

Clip3(x, y, z) =

Cos (x) is a trigonometric cosine function operating on the x argument in radians.

Floor(x) is the largest integer less than or equal to x.

GetCurrMsb (a, b, c, d) =

Ln(x) is the natural logarithm of x (logarithm with base e, where e is a constant with base natural logarithm 2.718 281 828 ...).

Log2(x) base 2 logarithm of x.

Log10(x) is the base 10 logarithm of the number x.

Min(x, y) =

Max(x, y) =

Round( x ) = Sign( x ) * Floor( Abs( x ) + 0.5 )

Sign(x) =

Sin (x) trigonometric sine function, operating on the x argument in radians.

Sqrt( x ) =

Swap( x, y ) = ( y, x )

Tan (x) is a trigonometric tangent function acting on the x argument in radians.

Operation Priority Order

If the order of precedence in an expression is not explicitly specified using parentheses, the following rules apply:

- operations with a higher priority are evaluated before any operation with a lower priority.

- Operations with the same priority are evaluated sequentially from left to right.

The table below lists the priority of operations from highest to lowest; a higher position in the table indicates a higher priority.

For those operators that are also used in the C programming language, the order of precedence used in this specification is the same as in the C programming language.

Table: Priority of operations from highest (top of table) to lowest (bottom of table)

Operations (with operands x, y and z) "x++", "x−−" "!x", "−x" (as unary prefix operator) x ^y

", "x % y""x * y", "x / y", "x ÷ y", "

", "x % y" "x + y", "x − y" (as a two-argument operator), "

" "x<<y", "x>>y" "x < y", "x <= y", "x > y", "x >= y" "x == y", "x != y" "x & y" "x | y" "x && y" "x|||y" "x? y: z" "x..y" "x = y", "x += y", "x -= y"

Text description of logical operations

The text contains a description of logical operations, which is mathematically described in the following form:

if (condition 0)

expression 0

else if (condition 1)

expression 1

...

else / * informative note on the remaining condition * /

expression n

can be described as follows:

... as below / ... the following applies:

- if condition 0, expression 0

- otherwise, if condition 1, expression 1

- ...

- otherwise (informative remark on the remaining condition), the expression n

Each of the "If... Otherwise, if... Otherwise..." in the text starts with "... as specified below" or "... the following applies", immediately followed by "If . ..". The last condition "If... Otherwise, if... Else,..." is always "Otherwise...". Interleaved "If... Otherwise, if... Otherwise..." statements can be identified by matching "... as follows" or "... the following applies" with the ending "Otherwise.. .".

The text contains a statement of logical operations, which is mathematically described in the following form:

If(condition 0a && condition 0b)

expression 0

else if (condition 1a | | condition 1b)

expression 1

...

else

expression n

can be described as follows:

... as below / ... the following applies:

- if all of the following conditions are true, the expression is 0:

- condition 0a

- condition 0b

- Otherwise, if one or more of the following conditions are met, expression 1:

- condition 1a

- condition 1b

- ...

- otherwise the expression n

The text contains a statement of logical operations, which is mathematically described in the following form:

if (condition 0)

expression 0

if (condition 1)

expression 1

can be described as follows:

when condition is 0, expression is 0

when condition 1, expression 1

Abbreviations definitions and glossary

HEVC - High Efficiency Video Coding

VVC - Universal Video Coding

VTM - VVC test model

JEM - Collaborative Exploration Model

CTU - coding tree block

CU - coding unit

BT - binary tree

TT - ternary tree

QT - quadtree or quaternary tree

ABT - asymmetric BT

MTT - multitype tree

AMP - asymmetric section

SH - segment header

SPS - Sequence Parameter Set

PPS - Image Parameter Set

CE - main experiment

SubCE - SubCore experiment (part of the main experiment).

Claims (40)

1. A video bitstream decoding device (101), comprising a circuit configured to: obtaining from the bitstream (101) syntax elements; obtaining information for a minimum size in luminance samples of a luminance leaf block resulting from a splitting of the quadtree, MinQtSizeY (105); determining a maximum size in luminance samples of the root luminance block such that it can be partitioned using binary tree partitioning, MaxBtSizeY (102) based on the information MinQtSizeY (105) and the resulting syntax element; determining whether binary division is allowed to be applied to the image block based on MaxBtSizeY (102); and obtaining a coding block of the image block based on the determination result; and the syntax element is a syntax element (301) of the difference between the base 2 logarithm MinQtSizeY (105) and the base 2 logarithm MaxBtSizeY (102); or the syntax element is a difference syntax element (301) between MinQtSizeY (105) and MaxBtSizeY (102), the difference syntax element signaling said difference in base 2 logarithmic form. 2. The apparatus of claim 1, wherein the circuit is configured to determine MaxBtSizeY (102) given that its lower limit is MinQtSizeY (105). 3. The device according to claim 1 or 2, wherein the circuit is configured to obtain a syntax element from the bitstream depending on the maximum hierarchy depth for coding blocks resulting from splitting the multitype tree, MaxMttDepth. 4. The apparatus of claim 3, wherein the circuit is configured to obtain a syntax element from a bitstream (101) if MaxMttDepth (103), characterizing an upper limit on the number of consecutive binary or ternary divisions applied to obtain a child block, is non-zero. 5. A device (1000) for generating a video bitstream (101), characterized in that it is configured to: determining the minimum brightness size of the leaf block obtained as a result of splitting the quadtree, MinQtSizeY (105), determining a maximum luminance size of a coding block to be partitioned using binary tree partitioning, MaxBtSizeY (102) based on MinQtSizeY (105), and inserting into the bitstream (101) information of a certain MinQtSizeY (105); and inserting a syntax element into the bitstream (101), wherein the syntax element is a syntax element (301) of the difference between the base 2 logarithm MinQtSizeY (105) and the base 2 logarithm MaxBtSizeY (102); or the syntax element is a difference syntax element (301) between MinQtSizeY (105) and MaxBtSizeY (102), the difference syntax element signaling said difference in base 2 logarithmic form. 6. Device (1000) for generating a bit stream (101) including encoded images, according to claim 5, characterized in that it is configured to determine MaxBtSizeY (102), given that its lower limit is MinQtSizeY (105) . 7. Device (1000) for generating a bit stream (101) including encoded images, according to claim 5 or 6, wherein the device is characterized in that it is configured to: inserting, into the bitstream (101), a syntax element (5001) dependent on the maximum hierarchy depth for coding blocks resulting from splitting a multi-type tree, MaxMttDepth (103). 8. Device (1000) for generating a bit stream (101) including encoded images according to claim 7, characterized in that it is configured to inserts, into the bitstream (101), a syntax element if MaxMttDepth (103), characterizing the upper limit of the number of consecutive binary or ternary divisions applied to obtain a child block, is not equal to zero. 9. A method for generating a video bitstream (101), comprising the steps of: determine the minimum brightness size of the leaf block as a result of splitting the quadtree, MinQtSizeY (105), determining a maximum luminance size of a coding block to be partitioned using binary tree partitioning, MaxBtSizeY (102), based on MinQtSizeY (105), and insert, into the bitstream (101), the information of the determined MinQtSizeY (105); and inserting a syntax element into the bitstream (101), wherein the syntax element is a syntax element (301) of the difference between the base 2 logarithm MinQtSizeY (105) and the base 2 logarithm MaxBtSizeY (102); or the syntax element is a difference syntax element (301) between MinQtSizeY (105) and MaxBtSizeY (102), the difference syntax element signaling said difference in base 2 logarithmic form. 10. A method for decoding a video bitstream (101), comprising the steps of: get, from the bitstream (101), a syntax element; getting the minimum size information in the luminance samples of the leaf luminance block obtained by splitting the quadtree, MinQtSizeY (105); determining a maximum size in luminance samples of the root luminance block such that it can be partitioned using binary tree partitioning, MaxBtSizeY (102) based on the MinQtSizeY (105) information and the resulting syntax element; and the syntax element is a syntax element (301) of the difference between the base 2 logarithm MinQtSizeY (105) and the base 2 logarithm MaxBtSizeY (102); or the syntax element is a difference syntax element (301) between MinQtSizeY (105) and MaxBtSizeY (102), the difference syntax element signaling said difference in base 2 logarithmic form; wherein the method further comprises the steps of: determining whether a binary split is allowed to be applied to the image block based on MaxBtSizeY (102); and obtaining an image block coding block based on the determination result. 11. The method of claim 10, wherein the step of determining MaxBtSizeY (102) based on the information for MinQtSizeY (105) and the resulting syntax element comprises a sub-step of determining MaxBtSizeY (102) given that its lower limit is MinQtSizeY (105 ). 12. The method according to claim 10 or 11, wherein the step of obtaining, from the bitstream (101), a syntax element comprises a sub-step of obtaining a syntax element from the bitstream depending on the maximum hierarchy depth for coding blocks obtained from splitting a multi-type tree , MaxMttDepth. 13. The method of claim 12, wherein the step of obtaining the syntax element from the bitstream (101) comprises a sub-step of obtaining the syntax element from the bitstream (101) if MaxMttDepth (103) characterizing the upper limit of the number of consecutive binary or ternary divisions applied to get the child block is non-zero.

Images