TWI707580B - Partitioning of zero unit - Google Patents

Abstract

Zero units (ZU) are used in coding or decoding of video. One example method of video processing includes determining that a block of video data is to be coded as a ZU block, due to the block of video data having dimensions S×T at least one of S and T being a non-power-of two number; partitioning the ZU block into one of two units, three units, or four units; and generating a bitstream by coding the units.

Description

Zero unit division

This patent document generally relates to image and video coding technology.

[Cross references to related applications]

In accordance with applicable patent laws and/or in accordance with the rules of the Paris Convention, this application promptly requests the International Patent Application No. PCT/CN2018/093631 filed on June 29, 2018, and the U.S. Provisional Patent filed on July 2, 2018 The priority and rights of application No. 62/693,415 and International Patent Application No. PCT/CN2018/094767 filed on July 6, 2018. The entire disclosures of the International Patent Application No. PCT/CN2018/093631, U.S. Provisional Application No. 62/693,415 and International Patent Application No. PCT/CN2018/094767 are incorporated by reference as part of the disclosure of this application.

Digital video occupies the largest bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the demand for bandwidth used by digital video will continue to grow.

Described are devices, systems, and methods related to dedicated coding units (CU) and/or coding tree units (CTU) for improving coding efficiency. In particular, the technology of the present disclosure discloses zero units that provide enhancements such as processing sub-blocks (eg, in pictures, slices, slices, etc.) located at the boundaries of video data blocks. The described method can be applied to both existing video coding standards (for example, High Efficiency Video Coding (HEVC)) and future video coding standards or video codecs.

In a representative aspect, the disclosed technology can be used to provide a method of video encoding, which can be implemented in a video encoder. The method includes determining that the video data block will be encoded as a zero unit (ZU) block because the video data block has a size S×T, and at least one of S and T is a power of two; dividing the ZU block into two units, three One unit or one of four units; and the bit stream is generated by the coding unit.

In another example aspect, another method of video processing is disclosed. The method includes receiving a bit stream corresponding to a video data block encoded as a zero unit (ZU) block, the zero unit (ZU) block is divided into two units, three units, or four units, and the video data block has a size S×T; and the video data block is generated by decoding the bit stream.

In another example aspect, another method of video processing is disclosed. The method includes determining that the video data block will be encoded as a zero unit (ZU) block because the block has a height or width that is not a power of two; and dividing the video data block using a partitioning scheme, where the partitioning scheme divides the video data block into One of two units, three units, or four units; a bit stream is generated by encoding video data blocks, where the division scheme uses the same signaling as used for the division of another block that is a non-zero unit block Syntax to signal.

In another example aspect, another method of video processing is disclosed. The method includes receiving a bit stream corresponding to a video data block, the video data block being encoded as a zero unit (ZU) block because the video data block has a size of S×T, at least one of S and T is a power of two, Among them, the video data block is divided using a division scheme of dividing the video data block into two units, three units, or one of four units, and wherein the division scheme is used in the bit stream and used for signaling non The division of zero unit blocks is signaled with the same syntax; based on the signalling, the decoded bit stream is decoded to generate video data blocks.

In another example aspect, another method of video processing is disclosed. The method includes determining that the video data block will be encoded as a zero unit (ZU) block because the video data block has a size of S×T, and at least one of S and T is a power of two; using a selection from the group of ZU block division schemes The partitioning scheme of the ZU block is divided into two units, three units or one of four units; the unit is encoded; and the encoded unit is signaled in the bit stream. Here, the group of ZU block partitioning schemes is a subset of the group of partitioning schemes available for partitioning coding units (CU).

In another example aspect, another method of video processing is disclosed. The method includes receiving a bit stream corresponding to a video data block, the video data block having a size; receiving a signaling notification indicating that since the video data block has a size S×T, at least one of S and T is a power of two The video data block is divided as a zero unit (ZU) block, and the block is divided using a division scheme selected from the group of ZU block division schemes; and based on signaling, the bit stream corresponding to the unit is decoded to reconstruct the video data Piece. Here, the group of ZU block partitioning schemes is a subset of the group of partitioning schemes available for partitioning coding units (CU).

In another example aspect, another method of video processing is disclosed. The method includes determining that the video data block will be encoded as a zero unit (ZU) block because the video data block has a size, and at least one of the sizes is a power of two; when determining that the ZU block is located in an I slice or an intra-coded picture , Divide the ZU block into one of two units, three units, or four units; coding units; and signaling coding units in the bit stream.

In another example aspect, another method of video processing is disclosed. The method includes receiving a bit stream corresponding to a video data block, the video data block having a size; receiving a signaling notification indicating that the video data block includes a unit divided from a zero unit (ZU) block, the zero unit (ZU) A block has at least a height or width that is a non-power of two, and is coded without transform and residual coding. The divided ZU block is located in the I slice or intra-coded picture; and based on the signaling notification, the corresponding decoding The bit stream of the unit is used to reconstruct the video data block.

In another representative aspect, the above method is embodied in the form of processor executable code and stored in a computer-readable program medium.

In another representative aspect, an apparatus configured or operable to perform the above method is disclosed. The device may include a processor that is programmed to implement the method.

In another representative aspect, a video decoder device may implement the method as described herein.

The above and other aspects and features of the technology of the present disclosure are described in more detail in the drawings, specification and claims.

1400, 1500, 1800, 1900: methods

1410~1420, 1510~1530, 1802~1806, 1902~1904: steps

1600: computer system

1605: one or more processors

1610: memory

1615: network interface controller

1625: Interconnect

1700: mobile devices

1701: processor/controller

1702: memory

1703: I/O interface

1704: display

Figure 1 shows an example block diagram of a typical High Efficiency Video Coding (HEVC) video transcoder and decoder.

Figure 2 shows an example of macro block (MB) division in H.264/AVC.

Fig. 3 shows an example of dividing a coding block (CB) into prediction blocks (PB).

4A and 4B respectively show examples of subdividing the coding tree block (CTB) into CB and transform block (TB) and the corresponding quadtree.

Fig. 5 shows an example of the division structure of one frame.

6A and 6B respectively show the subdivision and signaling method of CTB highlighted in the exemplary frame in FIG. 5.

Figures 7A and 7B show examples of subdivision for the largest coding unit (LCU) and the corresponding QTBT (quadtree plus binary tree).

8A to 8E show examples of dividing coding blocks.

Figure 9 shows an example subdivision of QTBT-based CB.

FIGS. 10A to 10I show examples of the division of CBs that support the multi-tree type (MTT), where the multi-tree type (MTT) is a generalization of QTBT.

Fig. 11 shows an example of tree type signaling.

12A to 12C show examples of CTB crossing the picture boundary.

Fig. 13 shows an example of zero cells at the picture boundary.

FIG. 14 shows a flowchart of an example method for video encoding according to the technology of the present disclosure.

FIG. 15 shows a flowchart of another example method for video decoding according to the technology of the present disclosure.

FIG. 16 is a block diagram showing an example of the architecture of a computer system or other control device that can be used to implement various parts of the disclosed technology.

FIG. 17 shows a block diagram of an example embodiment of a mobile device that can be used to implement various parts of the disclosed technology.

Figure 18 is a flowchart of an example method of video processing.

Figure 19 is a flowchart of an example method of video processing.

Due to the increasing demand for higher-resolution video, video coding methods and technologies are ubiquitous in modern technologies. Video transcoders usually include electronic circuits or software that compress or decompress digital video, and video transcoders are continuously improved to provide higher coding efficiency. The video codec converts uncompressed video to a compressed format, or vice versa. Video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of encoding and decoding algorithms, the sensitivity to data loss and error, editing convenience, random access, and end-to-end latency (delay ) Has a complicated relationship. The compression format usually conforms to standard video compression specifications, such as the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), which will be finalized Determined general video coding standards, or other current and/or future video coding standards.

The embodiments of the disclosed technology can be applied to existing video coding standards (for example, HEVC, H.265) and future standards to improve compression performance. In this document, chapter headings are used to improve the readability of the description, rather than limiting the discussion or embodiments (and/or implementation) to each chapter in any way.

The chapter titles are used in this document to facilitate understanding, and the embodiments disclosed in the chapter are not limited to this chapter. In addition, although certain embodiments have been described with reference to general video coding or other specific video transcoders, the disclosed technology can also be applied to other video coding technologies. In addition, although some embodiments describe the video encoding step in detail, it should be understood that the corresponding decoding step of de-encoding will be implemented by the decoder. In addition, the term video processing includes video encoding or compression, video decoding or decompression, and video transcoding, in which video pixels are expressed from one compression format to another compression format or at different compression bit rates.

1. Example embodiment of video encoding

Figure 1 shows an exemplary block diagram of a typical HEVC video encoder and decoder. The coding algorithm for generating a bit stream conforming to HEVC usually proceeds as follows. Each picture is divided into block regions, where the precise block division is transmitted to the decoder. The first picture of the video sequence (and the first picture at each clean random access point of the video sequence) only uses intra prediction (some predictions of the data from area to area are used in the same picture, without Based on other pictures) for encoding. For all remaining pictures of the sequence or pictures between random access points, for most blocks The inter-frame temporal prediction coding mode is often used. The encoding process of inter prediction includes selecting motion data containing selected reference pictures and motion vectors (MV), which will be applied to predict the samples of each block. The encoder and decoder generate the same inter prediction signal by applying motion compensation (MC), which uses MV and mode decision data sent as auxiliary information.

The residual signal of intra-frame prediction or inter-frame prediction is transformed by linear spatial transformation, where the residual signal is the difference between the original block and its prediction. Then the transform coefficients are scaled, quantized, entropy coded and sent with the prediction information.

The encoder replicates the decoder processing loop (see the gray shaded box in Figure 1) so that both will generate the same prediction for subsequent data. Therefore, the quantized transform coefficient is constructed by inverse scaling, and then inversely transformed to reproduce the decoded approximation of the residual signal. The residual is then added to the prediction, and the result of this addition can then be input to one or two loop filters to smooth out artifacts caused by block-by-block processing and quantization. The final picture representation (ie, a copy of the decoder output) is stored in the decoded picture buffer for prediction of subsequent pictures. Generally, the order of encoding or decoding processing of pictures is often different from the order in which they arrive from the source, and it is necessary to distinguish the decoding order (ie, bit stream order) and output order (ie, display order) of the decoder.

It is generally desirable to input video material encoded by HEVC as a progressive image (because the source video originates from this format or because it is produced by deinterlacing before encoding). There is no explicit coding feature in the HEVC design to support the use of interlaced scanning, because interlaced scanning is no longer used in displays and becomes very uncommon in distribution. However, the metadata language has been provided in HEVC The method allows the encoder to indicate that the interlaced video has been transmitted by encoding each field of the interlaced video (that is, the even or odd lines of each video frame) as a separate picture, or that the interlaced video has been transmitted Each interlaced frame is coded as a HEVC coded picture and sent. This provides an effective method of encoding interlaced video without adding to the burden of a special decoding process that the decoder needs to support interlaced video.

1.1. Example of partition tree structure in H.264/AVC

The core of the coding layer in the previous standard is a macro block, which contains a block of 16×16 luma samples and two corresponding 8×8 chroma in the usual case of 4:2:0 color sampling ( chroma) Sample block.

Intra-coded blocks use spatial prediction to take advantage of the spatial correlation between pixels. Two divisions are defined as: 16x16 and 4x4.

The inter coding block uses temporal prediction instead of spatial prediction by estimating the motion between pictures. The 16x16 macro block or any of its sub-macro blocks can be divided into 16x8, 8x16, 8x8, 8x4, 4x8, 4x4 to independently estimate the motion, as shown in FIG. 2. Only one motion vector (MV) is allowed per sub-macro block partition.

1.2. Example of partition tree structure in HEVC

In HEVC, a coding tree unit (CTU) is divided into coding units (CU) by using a quad-tree structure expressed as a coding tree to adapt to various local characteristics. The decision to use inter (temporal) prediction or intra (spatial) prediction to encode picture regions is made at the CU level. According to the prediction unit (PU) division type, each CU can be further divided into one, two or four PUs. In one PU Part, apply the same prediction process, and send relevant information to the decoder based on the PU. After the residual block is obtained by applying a prediction process based on the PU division type, the CU may be divided into transformation units (TU) according to another quad-tree structure similar to the coding tree of the CU. One of the key features of the HEVC structure is that it has multiple partition concepts, including CU, PU, and TU.

Some of the features involved in hybrid video encoding using HEVC include:

(1) Coding tree unit (CTU) and coding tree block (CTB) structure: A similar structure in HEVC is a coding tree unit (CTU), which has a size selected by the encoder and can be larger than a traditional macro block. CTU is composed of luminance CTB and corresponding chrominance CTB and syntax elements. The size of the luminance CTB L×L can be selected as L=16, 32, or 64 samples, and a larger size can usually achieve better compression. Then, HEVC supports the use of tree structure and quadtree-like signaling to divide the CTB into smaller blocks.

(2) Coding Unit (CU) and Coding Block (CB): The quadtree syntax of CTU specifies the size and location of its luminance CB and chrominance CB. The root of the quadtree is associated with the CTU. Therefore, the size of the brightness CTB is the maximum supported size of the brightness CB. The division of CTU into luminance CB and chrominance CB is combined in signaling. One luma CB and usually two chroma CBs and the associated syntax together form a coding unit (CU). The CTB may contain only one CU or may be divided to form multiple CUs, and each CU has an associated division into a tree of prediction units (PU) and transformation units (TU).

(3) Prediction unit and prediction block (PB): whether to use inter picture prediction or The decision of intra-picture prediction to encode picture regions is made at the CU level. The root of the PU partition structure is at the CU level. Depending on the basic prediction type decision, the luminance CB and chrominance CB can then be further divided in size and predicted based on the luminance and chrominance prediction blocks (PB). HEVC supports variable PB sizes from 64×64 to 4×4 samples. Fig. 3 shows an example of PB allowed for M×M CU.

(4) Transform unit (TU) and transform block: use block transform to encode prediction residuals. The root of the TU tree structure is at the CU level. The luminance CB residual may be the same as the luminance transform block (TB), or may be further divided into smaller luminance TB. The same applies to chroma TB. For square TB sizes of 4×4, 8×8, 16×16, and 32×32, an integer basis function similar to a discrete cosine transform (DCT) is defined. For the 4×4 transform of the luma intra picture prediction residuals, an integer transform derived from the form of discrete sine transform (DST) can be specified instead.

1.2.1. Example of dividing the tree structure into TB and TU

For residual coding, CB can be recursively divided into transform blocks (TB). The division is notified by residual quadtree signaling. Only square CB and TB divisions are specified, where blocks can be recursively divided into quadrants, as shown in Figure 4. For a given brightness CB with a size of M×M, a flag indicates whether to divide the CB into four blocks with a size of M/2×M/2. If, as signaled by the maximum depth of the residual quadtree indicated in the SPS, each quadrant can be further divided, a flag is assigned to each quadrant, which indicates whether to divide it into four quadrants. The leaf node block generated by the residual quadtree is a transform block, which is further processed by transform coding. The encoder indicates the maximum and minimum brightness TB size it will use. When the CB size is greater than the maximum In TB size, the division is implicit. When the division will result in the luminance TB size being smaller than the indicated minimum value, no division is implicit. Except when the luminance TB size is 4×4, the chrominance TB size is half of the luminance TB size in each dimension. When the luminance TB size is 4×4, a single 4×4 chrominance TB is used by Area covered by four 4×4 brightness TBs. In the case of a CU for intra picture prediction, the decoded samples of the nearest neighbor TB (within or outside the CB) are used as reference material for intra picture prediction.

Contrary to previous standards, the HEVC design allows TBs to span multiple PBs for inter-predicted CUs, so as to maximize the potential coding efficiency benefits of quad-tree TB partitioning.

1.2.2. Example of picture boundary coding

The boundary of the picture is defined in units of the minimum allowable brightness CB size. Therefore, at the right and bottom borders of the picture, some CTUs may cover part of the area outside the picture boundary. This condition is detected by the decoder, and the CTU quadtree is implicitly partitioned as needed to reduce the CB size to the extent that the entire CB will fit in the picture.

FIG. 5 shows an example of the division structure of a frame, where the resolution is 416×240 pixels, the size is 7CTB×4CTB, and the size of CTB is 64×64. As shown in FIG. 5, the CTB partially located outside the right and bottom borders has an implicit segmentation (dashed line, denoted as 502), and the CUs that are completely outside are skipped (not encoded).

In the example shown in Figure 5, the highlighted CTB (504), the line CTB The index is equal to 2 and the column CTB index is equal to 3, which has 64×48 pixels in the current picture and is not suitable for 64×64 CTB. Therefore, it is forcibly divided into 32×32 without a division flag signal. For the 32×32 in the upper left corner, it is completely covered by the frame. When it chooses to encode in smaller blocks according to the rate-distortion cost (8x8 for 16x16 in the upper left corner, and 16x16 for the rest), it needs to encode several segmentation marks. These division flags (a flag for whether to divide the 32×32 in the upper left corner into four 16×16 blocks, and for signaling whether a 16×16 is further divided, and for the 16×16 in the upper left corner Whether each of the four 8×8 blocks is further divided by the 8×8 flag) must be clearly signaled. A similar situation exists in the 32×32 block in the upper right corner. For the two bottom 32×32 blocks, because they are partially located outside the picture boundary (506), further QT segmentation needs to be applied without signaling. 6A and 6B respectively show the subdivision and signaling method of the CTB (504) highlighted in FIG. 5.

1.2.3. Example of CTB size indication

An example RBSP (Raw Byte Sequence Payload) syntax table for the general sequence parameter set is shown in Table 1.

The corresponding semantics include: log2_min_luma_coding_block_size_minus3 plus 3 specifies the minimum luminance coding block size; and log2_diff_max_min_luma_coding_block_size specifies the difference between the maximum luminance coding block size and the minimum luminance coding block size.

Variables: MinCbLog2SizeY, CtbLog2SizeY, MinCbSizeY, CtbSizeY, PicWidthInMinCbsY, PicWidthInCtbsY, PicHeightInMinCbsY, PicHeightInCtbsY, PicSizeInMinCbsY, PicSizeInCtbsY, PicSizeInSamplesY, PicWidthInSamplesC and PicHeightInSamplesC obtained by: MinCbLog2SizeY = log2_min_luma_coding_block_size_minus3 + 3

CtbLog2SizeY=MinCbLog2SizeY+log2_diff_max_min_luma_coding_block_size

MinCbSizeY=1<<MinCbLog2SizeY

CtbSizeY=1<<CtbLog2SizeY

PicWidthInMinCbsY=pic_width_in_luma_samples/MinCbSizeY

PicWidthInCtbsY=Ceil (pic_width_in_luma_samples÷CtbSizeY)

PicHeightInMinCbsY=pic_height_in_luma_samples/MinCbSizeY

PicHeightInCtbsY=Ceil(pic_height_in_luma_samples÷CtbSizeY)

PicSizeInMinCbsY=PicWidthInMinCbsY*PicHeightInMinCbsY

PicSizeInCtbsY=PicWidthInCtbsY*PicHeightInCtbsY

PicSizeInSamplesY=pic_width_in_luma_samples*pic_height_in_luma_samples

PicWidthInSamplesC=pic_width_in_luma_samples/SubWidthC

PicHeightInSamplesC=pic_height_in_luma_samples/SubHeightC

The variables CtbWidthC and CtbHeightC, which respectively specify the width and height of each chroma CTB array, are obtained by the following: if chroma_format_idc is equal to 0 (monochrome) or separate_colour_plane_flag is equal to 1, then CtbWidthC and CtbHeightC are both equal to 0; otherwise, CtbWidthC and CtbHeightC pass The following gets: CtbWidthC=CtbSizeY/SubWidthC

CtbHeightC=CtbSizeY/SubHeightC

1.3. Example of quadtree plus binary tree block structure with larger CTU in JEM

In some embodiments, reference software called Joint Exploration Model (JEM) is used to explore future video coding technologies. In addition to the binary tree structure, JEM also describes the quad-tree plus binary tree (QTBT) and tri-tree (TT) structures.

1.3.1. Example of QTBT block partition structure

Compared with HEVC, the QTBT structure eliminates the concept of multiple partition types, that is, it eliminates the difference between the concepts of CU, PU, and TU, and provides greater flexibility for CU partition shapes. In the QTBT block structure, the CU may have a square or rectangular shape. As shown in FIG. 7A, the coding tree unit (CTU) is first divided by a quadtree structure. The quad leaf node is further divided by a binary tree structure. There are two types of segmentation in binary tree segmentation, symmetric horizontal segmentation and symmetric vertical segmentation. The binary leaf node is called a coding unit (CU), and the segment is used for prediction and transformation processing without any further division. This means that CU, PU and TU have the same block size in the QTBT coding block structure. In JEM, a CU is sometimes composed of coding blocks (CB) of different color components - for example, a CU contains a luminance CB and two colors in the case of P and B strips in 4:2:0 chroma format. Degree CB; and sometimes composed of a single component CB-for example, a CU contains only one luminance CB or only two chrominance CB in the case of I stripe.

The following parameters are defined for the QTBT division scheme:

-CTU size: the size of the root node of the quadtree, the same concept as in HEVC

-MinQTSize: the minimum allowable quad leaf node size

-MaxBTSize: the maximum allowable size of the root node of the binary tree

-MaxBTDepth: Maximum allowable binary tree depth

-MinBTSize: the minimum allowable size of a binary leaf node

In an example of the QTBT division structure, the CTU size is set to 128×128 luma samples and two corresponding 64×64 chroma sample blocks, MinQTSize is set to 16×16, MaxBTSize is set to 64×64, MinBTSize (for Both width and height) are set to 4×4, and MaxBTDepth is set to 4. The quadtree division is first applied to CTU to generate quad-leaf nodes. The quad leaf node may have a size from 16×16 (ie, MinQTSize) to 128×128 (ie, CTU size). If the quad leaf node is 128×128, it will not be further divided by the binary tree because the size exceeds MaxBTSize (ie 64×64). Otherwise, the quad leaf node can be further divided by the binary tree. Therefore, the quad leaf node is also the root node of the binary tree, and it has a binary tree depth of zero. When the depth of the binary tree reaches MaxBTDepth (ie 4), no further division is considered. When the width of the binary tree node is equal to MinBTSize (ie 4), no further horizontal division is considered. Similarly, when the height of the binary tree node is equal to MinBTSize, no further vertical division is considered. The leaf nodes of the binary tree are further processed through prediction and transformation processing without any further division. In JEM, the maximum CTU size is 256×256 luminance samples.

FIG. 7A shows an example of block division by using QTBT. Figure 7B shows the corresponding tree representation. The solid line indicates the quadtree division, and the dashed line indicates the binary Tree split. In each split (ie, non-leaf) node of the binary tree, a flag is signaled to indicate which split type (ie, horizontal or vertical) is used, where 0 indicates horizontal split, and 1 indicates vertical split. For quadtree partitioning, there is no need to indicate the partition type, because quadtree partitioning always splits the block horizontally and vertically at the same time to generate 4 sub-blocks with the same size.

In addition, the QTBT solution supports the ability to have separate QTBT structures for luminance and chrominance. Currently, for P and B strips, the luminance and chrominance CTB in a CTU share the same QTBT structure. However, for the I slice, the luma CTB is divided into CUs by the QTBT structure, and the chroma CTB is divided into chrominance CUs by another QTBT structure. This means that the CU in the I slice consists of coding blocks of the luma component or two chrominance components, and the CU in the P or B slice consists of coding blocks of all three color components.

In HEVC, inter prediction of small blocks is restricted to reduce memory access for motion compensation, so that bi-directional prediction is not supported for 4×8 and 8×4 blocks, and inter prediction is not supported for 4×4 blocks. In JEM's QTBT, these restrictions are removed.

1.4. Trinomial Tree (TT) for Multifunctional Video Coding (VVC)

FIG. 8A shows an example of quadtree (QT) division, and FIGS. 8B and 8C show examples of vertical and horizontal binary tree (BT) division, respectively. In some embodiments, in addition to quadtrees and binary trees, tri-tree (TT) division is also supported, such as horizontal and vertical center-side tri-trees (as shown in FIGS. 8D and 8E).

In some implementations, two levels of trees are supported: regional trees (quadtrees) and prediction trees (binary trees or ternary trees). First pass the area tree (RT) to the CTU Divide. The RT leaf can be further divided by prediction tree (PT). PT can also be used to further segment the PT leaf until the maximum PT depth is reached. The PT leaf is the basic coding unit. For convenience, it is still called CU. CU cannot be divided further. Both prediction and transformation are applied to CU in the same way as JEM. The whole partition structure is called "multi-type tree".

1.5. Examples of alternative partition structures in video coding technology

In some embodiments, a tree structure called a multi-tree type (MTT) (which is a generalization of QTBT) is supported. In QTBT, as shown in FIG. 9, the coding tree unit (CTU) is first divided by a quadtree structure. The quad leaf node is further divided by a binary tree structure.

The structure of MTT is composed of two types of tree nodes: regional tree (RT) and prediction tree (PT), which supports nine types of divisions, as shown in Figures 10A to 10I. The area tree can recursively divide the CTU into square blocks until it is divided into leaf nodes of 4×4 areas. At each node in the area tree, a prediction tree can be formed from one of three tree types: binary tree, trinomial tree, and asymmetric binary tree. In PT partitioning, it is prohibited to have quadtree partitions in the branches of the prediction tree. As in JEM, the luminance tree and the chrominance tree are separated in the I strip.

Generally, RT signaling is the same as QT signaling in JEM except for context derivation. For PT signaling, up to 4 additional binary bits (bin) are required, as shown in Figure 11. The first binary bit indicates whether the PT is further divided. The context of the binary bit is calculated based on the observation that the possibility of further segmentation is highly correlated with the relative size of the current block and its neighboring blocks. If the PT is further divided, then The second binary bit indicates whether it is divided horizontally or vertically. In some embodiments, the presence of center-side trinomial trees and asymmetric binary trees (ABT) increases the appearance of "high" blocks or "wide" blocks. The third binary bit indicates the type of the divided tree, that is, whether it is a binary tree/trinomial tree or an asymmetrical binary tree. In the case of a binary tree/trinomial tree, the fourth binary bit indicates the type of tree. In the case of an asymmetric binary tree, the fourth binary bit indicates the upward or downward type for horizontally divided trees, and the right or leftward type for vertically divided trees.

1.5.1. Examples of restrictions at the picture boundary

In some embodiments, if the CTB/LCU size is indicated by M×N (usually M is equal to N, as defined in HEVC/JEM), and for CTB located at the boundary of a picture (or slice or slice or other types), K ×L samples are within the picture boundary.

The CU segmentation rules on the bottom and right edges of the picture can be applied to any coding tree configuration QTBT+TT, QTBT+ABT or QTBT+TT+ABT. They include the following two aspects:

(1) If a part of a given coding tree node (CU) is partially located outside the picture, it is always allowed along the relevant boundary direction (horizontal split orientation along the bottom boundary, as shown in Figure 12A, along the vertical Splitting orientation, as shown in Figure 12B) for binary symmetrical splitting of CU. If the lower right corner of the current CU is outside the frame (as shown in FIG. 12C), only the quadtree division of the CU is allowed. In addition, if the current binary tree depth is greater than the maximum binary tree depth and the current CU is on the frame boundary, binary segmentation is enabled to ensure that the frame boundary is reached.

(2) Regarding the trinomial tree segmentation process, in the case where the first boundary or the second boundary between the generated sub-CUs is exactly on the boundary of the picture, the trinomial tree segmentation is allowed. If the partition line (the boundary between the two sub-CUs resulting from the partition) exactly matches the picture boundary, an asymmetric binary tree partition is allowed.

2. Examples of existing implementations

In existing implementations, the width or height of the CTU or CU may not be equal to 2 ^N , where N is a positive integer. These situations are difficult to handle. Specifically, if the number of rows or the number of columns is not in the form of ^2N , it may be difficult to design a transformation with integer operations that does not include division.

In one example, in order to prevent the CTU or CU from having a width or height that is not equal to 2 ^N , the CTU or CU is forced to be divided into smaller ones until both the width and height are in the form of 2 ^N or skip by padding or using transform . If these blocks are processed in a more flexible way, the coding gain can be further improved.

In another example, a transformation is defined for a CU whose width or height is not in the form of ^2N . This transformation is undesirable in actual video coding applications.

3. Example method using zero unit based on the technology of the present disclosure

The embodiments of the technology of the present disclosure overcome the shortcomings of existing implementations, thereby providing higher efficiency for video encoding. Specifically, a zero unit block is proposed as a special CU/CTU, and the block is interpreted as a zero unit if and only if its height and/or width are not in the form of ^2N .

In the following examples described for various implementations, the use of zero units to improve video coding efficiency and enhance existing and future video coding standards is clarified. under The examples of the technology of the present disclosure provided above explain general concepts and are not intended to be construed as limitations. In the examples, unless explicitly indicated to the contrary, the various features described in these examples may be combined. In another example, the various features described in these examples can be applied to a method of picture boundary coding using a backward compatible block size and visual media coding using a partition tree. FIG. 13 shows an example of a zero unit (having the size of pixels or samples) at the picture boundary.

Example A. The zero unit can be further divided into two units (BT or ABT), three units (TT, FTT) or four units (QT, EQT). The division unit divided from the zero unit may be a zero unit, or it may be a normal CU, which has a width or height in the form of ^2N . Assume that the size of the zero unit Z is S×T.

(a) In an example, Z can be divided into two units using BT, both of which have a size of S/2×T.

(b) In an example, Z can be divided into two units using BT, both of which have a size of S×T/2.

2^N+1，Z可以用BT分割為兩個單元，其具有大小為2^N×T和(S-2^N)×T，或(S-2^N)×T和2^N×T。 (c) In an example, assume that 2 ^N <S

2 ^N+1 , Z can be divided into two units by BT, which have sizes of 2 ^N ×T and (S-2 ^N )×T, or (S-2 ^N )×T and 2 ^N ×T.

2^N+1，Z可以用BT分割為兩個單元，其具有大小為S×2^N和S×(T-2^N)，或S×(T-2^N)和S×2^N。 (d) In an example, assume that 2 ^N <T

2 ^N+1 , Z can be divided into two units with BT, which have sizes of S×2 ^N and S×(T-2 ^N ), or S×(T-2 ^N ) and S×2 ^N.

(e) In an example, Z can be divided into three units with TT, which have sizes of S/4×T, S/2×T, and S/4×T.

(f) In an example, Z can be divided into three units with TT, which have sizes of S×T/4, S×T/2, and S×T/4.

(g) In an example, assuming that 2 ^N <S <2 ^N +1, Z can be divided into three units by TT, which have sizes 2 ^N-1 ×T, 2 ^N-1 ×T and (S- 2 ^N )×T, or 2 ^N-1 ×T, (S-2 ^N )×T and 2 ^N-1 ×T, or (S-2 ^N )×T, 2 ^N-1 ×T and 2 ^{N- 1} ×T.

2^N+1，Z可以用TT分割為三個單元，其具有大小為S×2^N-1、S×2^N-1和S×(T-2^N)，或S×2^N-1、S×(T-2^N)和S×2^N-1，或S×(T-2^N)、S×2^N-1和S×2^N-1。 (h) In an example, assume that 2 ^N <T

2 ^N+1 , Z can be divided into three units with TT, which have sizes of S×2 ^N-1 , S×2 ^N-1 and S×(T-2 ^N ), or S×2 ^N-1 , S×(T-2 ^N ) and S×2 ^N-1 , or S×(T-2 ^N ), S×2 ^N-1 and S×2 ^N-1 .

(i) In an example, Z can be divided into four units by QT, each of which has a size of S/2×T/2.

2^N+1，Z可以用QT分割為四個單元，具有大小為2^N×T/2、2^N×T/2、(S-2^N)×T/2和(S-2^N)×T/2，或(S-2^N)×T/2、(S-2^N)×T/2、2^N×T/2和2^N×T/2。 (j) In an example, assume that 2 ^N <S

2 ^N+1 , Z can be divided into four units by QT, with sizes of 2 ^N ×T/2, 2 ^N ×T/2, (S-2 ^N )×T/2 and (S-2 ^N )× T/2, or (S-2 ^N )×T/2, (S-2 ^N )×T/2, 2 ^N ×T/2, and 2 ^N ×T/2.

2^N+1，Z可以用QT分割為四個單元，具有大小為S/2×2^N、S/2×2^N、S/2×(T-2^N)和S/2×(T-2^N)，或S/2×(T-2^N)、S/2×(T-2^N)、S/2×2^N和S/2×2^N。 (k) In an example, assume that 2 ^N <T

2 ^N+1 , Z can be divided into four units by QT, with sizes S/2×2 ^N , S/2×2 ^N , S/2×(T-2 ^N ) and S/2×(T- 2 ^N ), or S/2×(T-2 ^N ), S/2×(T-2 ^N ), S/2×2 ^N and S/2×2 ^N.

2^N+1和2^M<T

2^M+1，Z可以用QT分割為四個單元，具有大小為2^N×2^M、2^N×2^M、(S-2^N)×(T-2^M)和(S-2^N)×(T-2^M)，或(S-2^N)×(T-2^M)、(S-2^N)×(T-2M)、2^N×2^M和2^N×2^M，或2^N×(T-2^M)、2^N×(T-2^M)、(S-2^N)×2M和(S-2^N)×2^M，或(S-2^N)×2^M、(S-2^N)×2^M、2^N×(T-2^M)和2^N×(T-2^M)。 (l) In an example, assume that 2 ^N <S

2 ^N+1 and 2 ^M <T

2 ^M+1 , Z can be divided into four units by QT, with sizes of 2 ^N × 2 ^M , 2 ^N × 2 ^M , (S-2 ^N ) × (T-2 ^M ) and (S-2 ^N ) ×(T-2 ^M ), or (S-2 ^N )×(T-2 ^M ), (S-2 ^N )×(T-2M), 2 ^N ×2 ^M and 2 ^N ×2 ^M , or 2 ^N × (T-2 ^M ), 2 ^N × (T-2 ^M ), (S-2 ^N ) × 2M and (S-2 ^N ) × 2 ^M , or (S-2 ^N ) × 2 ^M , ( S-2 ^N )×2 ^M , 2 ^N ×(T-2 ^M ) and 2 ^N ×(T-2 ^M ).

(m) In an example, the width/height of all division units should be even number. If a division structure results in an odd number of unit width or height, such division structure is automatically prohibited.

Or, in addition, skip the signaling of this division structure.

(n) In an example, Z can be divided into three units with TT.

In an example, assuming 3*2 ^N <S<=3*2 ^N+1 , the sizes of the three units are 2 ^N ×T, 2 ^N+1 ×T, and (S-3*2 ^N )×T .

In an example, assuming 3*2 ^N <T<=3*2 ^N +1, the sizes of the three units are Sx2 ^N , Sx2 ^N+1, and S×(T-3*2 ^N ).

(o) In an example, the width and/or height of all partition units should be in the form of K*M, where M is the minimum width and/or height of the allowed coding unit/prediction unit, such as 4; K is greater than 0 The integer. If a division structure causes the width or height of the unit to be not in this form, the division structure is automatically prohibited.

For example, assuming that the width and height of the division unit in the division structure are W and H, if W<M or H<M or (W&(M-1)!=0) or (H&(M-1)!=0), The division structure is prohibited.

Or, in addition, skip the signaling of this division structure.

Alternatively, the width and/or height of all partitioned non-ZUs should be in the form of K*M, where M is the minimum width and/or height of the allowed coding unit/prediction unit, such as 4. In this case, if the divided zero unit does not follow this restriction but the non-ZU follows this restriction, the structure is still allowed to be divided.

Example B. The split signaling method of ZU is the same as that of normal CU.

a. In one example, different contexts can be used to encode ZU or non-ZU.

b. Or, for ZU, only partial division methods of normal CU are allowed.

i. The subset of normal CU segmentation methods allowed by ZU is determined by ZU size and/or picture/strip/slice boundary position (bottom, right, bottom right, etc.) and/or strip type.

ii. In one example, only QT and BT partition structures are allowed for ZU.

iii. Or, in addition, the ZU division information does not signal whether to use TT and how to use TT (and other types of division structures except QT/BT).

iv. Or, in addition, the split signaling method of ZU remains the same as the split signaling method of normal CU, however, the context of the indication of TT (or other type of partition structure) may further depend on whether the current block is a ZU.

Example C. In one embodiment, ZU must be segmented in I-slice or intra-coded images.

In one example, the width or height of ZU is not in the form of 2N.

In one example, when the following conditions are true, CU is regarded as ZU,

i) W>=T0 and H>=T1. T0 and/or T1 are integers such as 128 or 256.

ii) W>=T0 or H>=T1. T0 and/or T1 are integers such as 128 or 256.

iii) W×H>=T. T is an integer such as 16384 or 65536.

The above examples can be combined in the context of the methods described below-for example, methods 1400 and 1500, which can be implemented at the video decoder and/or video transcoder.

Figure 14 shows a flowchart of an exemplary method for video encoding, which can be implemented in a video transcoder. The method 1400 includes, at step 1410, determining the size of the video data block.

The method 1400 includes, at step 1420, when it is determined that at least one size is a power of two, signaling the video data block as a zero unit (ZU) block, which is not transformable.

In some embodiments, a non-two power is any non-zero number that cannot be expressed in the form of 2N. For example, each integer that does not include a power of two (for example, 1, 3, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 17, 18,...) is non-two The power of.

In some embodiments, non-transformable can be defined in the context of Example 2, so that transform, inverse transform, quantization, and dequantization operations are not invoked on zero units. For example, the non-transformable property of a zero unit is that it is inferred to be encoded with skip mode, and therefore, there is no need to explicitly signal skip mode. In other embodiments, non-transformation may be defined in the context of Example 3, so that although there may be non-zero residuals, the transformation and inverse transformation operations are not defined for the zero unit.

FIG. 15 shows a flowchart of another exemplary method for video encoding, which may be implemented in a video decoder. The flowchart includes some features and/or steps similar to those shown in FIG. 14 and described above. At least some of these features and/or steps may not be separately described in this section.

The method 1500 includes, at step 1510, receiving a bit stream corresponding to a block of video data.

The method 1500 includes, at step 1520, receiving signaling indicating that the video data block is a zero unit (ZU) block, the zero unit (ZU) block is not transformable, and has at least one size that is not a power of two.

The method 1500 includes, at step 1530, decoding the bitstream based on the signaling to reconstruct the video data block.

In some embodiments, the method 1400 and the method 1500, and as described in the context of Example 1, may further include that the size of the video data block is an even number, has a form of 2N, or has a form of 2KN, where K=1, 2, 3, 4.... In other embodiments, the signaling may not include a merge index or skip flag, and/or not include prediction_mode_flag, and/or include the maximum or minimum value of at least one size of the ZU block. In an example, the signaling is in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a coding tree unit (CTU), or a coding unit (CU).

In some embodiments, the motion information of the ZU block is inherited from the motion information of neighboring blocks of size 2N×2M.

In one embodiment, and as described in the context of Example 7, the ZU block is divided into two or more units. In an example, at least one of the two or more cells is a zero cell. In another example, at least one of the two or more units is a coding unit (CU) with a size of 2N×2M.

4. Example implementation of the disclosed technology

16 is a block diagram showing an example of the architecture of a computer system or other control device 1600 that can be used to implement various parts of the technology of the present disclosure. The technology of the present disclosure includes (but is not limited to) the method 1400 and the method 1500. In FIG. 16, a computer system 1600 includes one or more processors 1605 and a memory 1610 connected via an interconnection 1625. Interconnect 1625 may represent any one or more separate physical buses, point-to-point connections, or both, connected by suitable bridges, adapters. Therefore, the interconnection 1625 may include, for example, a system bus, a peripheral component connection (PCI) bus, a two-way transmission bus (HyperTransport) or an industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal Serial bus (USB), IIC (I2C) bus or Institute of Electrical and Electronics Engineers (IEEE) standard 674 bus, sometimes called "Firewire".

The processor(s) 1605 may include a central processing unit (CPU) to control, for example, the overall operation of the host computer. In some embodiments, the processor(s) 1605 implements this by executing software or firmware stored in the memory 1610. The processor(s) 1605 may be or may include one or more programmable general-purpose or special-purpose microprocessors, digital signal processors (DSP), programmable controllers, special-purpose integrated circuits (ASIC), Programmable logic device (PLD), etc., or a combination of these devices.

The memory 1610 may be or include the main memory of the computer system. The memory 1610 represents any suitable form of random access memory (RAM), read-only memory (ROM), flash memory, etc., or a combination thereof. In use, the memory 1610 may contain a machine instruction set, and when the machine instruction set is executed by the processor 1605, The processor 1605 operates to implement the embodiments of the disclosed technology.

Also connected to the processor(s) 1605 via the interconnect 1625 is an (optional) network interface controller 1615. The network interface controller 1615 provides the computer system 1600 with the ability to communicate with remote devices, such as a storage client and/or other storage servers, and the network interface controller 1615 may be, for example, an Ethernet interface Controller or Fibre Channel interface controller.

FIG. 17 shows a block diagram of an example embodiment of a mobile device 1700 that can be used to implement various parts of the disclosed technology, including (but not limited to) method 1400 and method 1500. The mobile device 1700 may be a laptop computer, a smart phone, a tablet computer, a camcorder, or other types of devices capable of processing video. The mobile device 1700 includes a processor or controller 1701 for processing data, and a memory 1702 that communicates with the processor 1701 to store and/or buffer data. For example, the processor 1701 may include a central processing unit (CPU) or a microcontroller unit (MCU). In some implementations, the processor 1701 may include a field programmable gate array (FPGA). In some implementations, the mobile device 1700 includes or communicates with a graphics processing unit (GPU), a video processing unit (VPU) and/or a wireless communication unit for various visual and/or communication data processing functions of a smart phone device . For example, the memory 1702 may include and store processor-executable code, which when executed by the processor 1701 configures the mobile device 1700 to perform various operations, such as receiving information, commands and/or data, processing information and data, and processing The latter information/material is sent or provided to another device such as an actuator or external display.

To support the various functions of the mobile device 1700, the memory 1702 can Store information and data, such as instructions, software, values, images, and other data processed or referenced by the processor 1701. For example, various types of random access memory (RAM) devices, read-only memory (ROM) devices, flash memory devices, and other suitable storage media may have been used to implement the storage function of the memory 1702. In some implementations, the mobile device 1700 includes an input/output (I/O) interface 1703 to interface the processor 1701 and/or the memory 1702 with other modules, units, or devices. For example, the I/O interface 1703 can utilize various types of wireless interfaces compatible with typical data communication standards (for example, between one or more computers in the cloud and the user equipment) to connect the processor 1701 with the memory Body 1702 interface. In some implementations, the mobile device 1700 can interface with other devices via the I/O interface 1703 using a wired connection. The mobile device 1700 can also interface with other external interfaces (such as data memory) and/or a visual or audio display 1704 to retrieve and transmit data and information. The data and information can be processed by a processor, stored in memory or in Display on the display 1704 or the output unit of the external device. For example, the display 1704 may display a video frame including a block (CU, PU, or TU) that applies intra-block copy based on whether a motion compensation algorithm is used and the block is encoded according to the disclosed technology.

In some embodiments, the video decoder device may implement a method using zero units as described herein for video decoding. The various features of this method can be similar to the various methods described above.

In some embodiments, the video decoding method may be implemented using a decoding device, which is implemented on a hardware platform as described in FIG. 16 and FIG. 17.

The following list of solutions can be used to capture some of the practices described in this article Examples.

1. A video encoding method (for example, the method 1800 described in FIG. 18), comprising: determining (step 1802) that a video data block will be encoded into a zero unit (ZU) block, the video data block having a size of S×T; The ZU block is divided (step 1804) into one of two units, three units or four units; and a bit stream is generated (step 1806) through the coding unit.

2. A video decoding method (for example, the method 1900 described in FIG. 19) includes: receiving (step 1902) a bit stream corresponding to a video data block encoded as a zero unit (ZU) block, the zero unit (ZU) The block is divided into two units, three units, or four units, the video data block has a size of S×T; and the video data block is generated (step 1904) by decoding the bit stream.

3. The method according to solution 1 or 2, wherein the ZU block is divided into two units using a binary tree division scheme, and each ZU block has a size of S/2×T.

4. The method according to solution 1 or 2, wherein the ZU block is divided into two units using a binary tree division scheme, and each unit has a size of S×T/2.

2^N+1的ZU塊使用二叉樹劃分方案被劃分為兩個單元的結果，兩個單元的第一單元具有尺寸2^N×T並且兩個單元的第二單元具有尺寸(S-2^N)×T。 5. The method as described in solution 1 or 2, wherein, as 2 ^N <S

2 ^N+1 ZU block is divided into two units using a binary tree division scheme. The first unit of the two units has a size of 2 ^N × T and the second unit of the two units has a size of (S-2 ^N )× T.

2^N+1的ZU塊使用二叉樹劃分方案被劃分為兩個單元的結果，兩個單元的第一單元具有尺寸(S-2^N)×T並且兩個單元的第二單元具有尺 寸2^N×T。 6. The method according to solution 1 or 2, wherein, as 2 ^N <S

2 ^N+1 ZU block is divided into two units using a binary tree division scheme. The first unit of the two units has size (S-2 ^N )×T and the second unit of the two units has size 2 ^N × T.

2^N+1的ZU塊使用二叉樹劃分方案被劃分為兩個單元的結果，兩個單元的第一單元具有尺寸S×2^N並且兩個單元的第二單元具有尺寸S×(T-2^N)。 7. The method according to solution 1 or 2, wherein, as 2 ^N <T

2 ^N+1 ZU block is divided into two units using a binary tree division scheme. The first unit of the two units has a size of S×2 ^N and the second unit of the two units has a size of S×(T-2 ^N ).

2^N+1的ZU塊使用二叉樹劃分方案被劃分為兩個單元的結果，兩個單元的第一單元具有尺寸S×(T-2^N)並且兩個單元的第二單元具有尺寸S×2^N。 8. The method according to solution 1 or 2, wherein as 2 ^N <T

2 ^N+1 ZU block is divided into two units using a binary tree division scheme. The first unit of the two units has size S×(T-2 ^N ) and the second unit of the two units has size S×2 ^N.

9. The method of solution 1 or 2, wherein the ZU block is divided into three units using a three-prong tree division scheme, and each of the two units of the three units has a size of S/4×T and three One of the cells has a size of S/2×T.

10. The method according to solution 1 or 2, wherein the ZU block is divided into three units using a trigeminal tree division scheme, two of the three units have a size of S×T/4 and one of the three units It has a size of S×T/2.

2^N+1的ZU塊使用三叉樹劃分方案被劃分為三個單元的結果，第一單元具有尺寸2^N-1×T，第二單元具有尺寸(S-2^N)×T，並且第三單元具有尺寸2^N-1×T。 11. The method according to solution 1 or 2, wherein as 2 ^N <S

2 ^N+1 ZU block is divided into three units using the trigeminal tree division scheme. The first unit has a size of 2 ^N-1 ×T, the second unit has a size of (S-2 ^N )×T, and the third The cell has a size of 2 ^N-1 ×T.

2^N+1的ZU塊使用三叉樹劃分方案被劃分為三個單元的結果，第一單元具有尺寸S×2^N-1，第二單元具有尺寸S×(T-2^N)，並且第三單元具 有尺寸S×2^N-1。 12. The method according to solution 1 or 2, wherein, as 2 ^N <T

2 ^N+1 ZU block is divided into three units using the tri-tree division scheme. The first unit has size S×2 ^N-1 , the second unit has size S×(T-2 ^N ), and the third The unit has a size of S×2 ^N-1 .

13. The method of solution 1 or 2, wherein the ZU block is divided into four units using a quadtree division scheme, and each unit has a size of S/2×T/2.

2^N+1的ZU塊使用四叉樹劃分方案被劃分為四個單元的結果，第一單元具有尺寸2^N×T/2，第二單元具有尺寸2^N×T/2，並且第三單元具有尺寸S/2×(T-2^N)，並且第四單元具有尺寸S/2×(T-2^N)。 14. The method according to solution 1 or 2, wherein as 2 ^N <S

2 ^N+1 ZU block is divided into four units using a quadtree division scheme. The first unit has a size of 2 ^N × T/2, the second unit has a size of 2 ^N × T/2, and the third unit It has a size of S/2×(T-2 ^N ), and the fourth cell has a size of S/2×(T-2 ^N ).

2^N+1的ZU塊使用四叉樹劃分方案被劃分為四個單元的結果，第一單元具有尺寸S/2×2^N，第二單元具有尺寸S/2×2^N，並且第三單元具有尺寸S/2×(T-2^N)，並且第四單元具有尺寸S/2×(T-2^N)。 15. The method according to solution 1 or 2, wherein, as 2 ^N <T

2 ^N+1 ZU block is divided into four units using the quadtree division scheme. The first unit has size S/2×2 ^N , the second unit has size S/2×2 ^N , and the third unit It has a size of S/2×(T-2 ^N ), and the fourth cell has a size of S/2×(T-2 ^N ).

2^N+1並且2^M<T

2^M+1的ZU塊使用四叉樹劃分方案被劃分為四個單元的結果，第一單元具有尺寸2^N×2^M，第二單元具有尺寸2^N×2^M，並且第三單元具有尺寸(S-2^N)×(T-2^M)，並且第四單元具有尺寸(S-2^N)×(T-2^M)。 16. The method according to solution 1 or 2, wherein as 2 ^N <S

2 ^N+1 and 2 ^M <T

The ZU block of 2 ^M+1 is divided into four units using the quadtree division scheme. The first unit has a size of 2 ^N × 2 ^M , the second unit has a size of 2 ^N × 2 ^M , and the third unit has a size (S-2 ^N )×(T-2 ^M ), and the fourth unit has a size (S-2 ^N )×(T-2 ^M ).

2^N+1並且2^M<T

2^M+1的ZU塊使用四叉樹劃分方案被劃分為四個單元的結果，第一單元具有尺寸2^N×(T-2^M)，第二單元具有尺寸2^N×(T-2^M)，並且第三單元具有尺寸(S-2^N)×2^M，並且第四單元具有尺寸(S-2^N)×2M。 17. The method as described in solution 1 or 2, wherein as 2 ^N <S

2 ^N+1 and 2 ^M <T

The ZU block of 2 ^M+1 is divided into four units using the quadtree division scheme. The first unit has a size of 2 ^N ×(T-2 ^M ), and the second unit has a size of 2 ^N ×(T-2 ^M ), and the third unit has a size (S-2 ^N )×2 ^M , and the fourth unit has a size (S-2 ^N )×2M.

18. The method according to solution 1 or 2, wherein each of the ZU blocks The height and/or width of the cell are even numbers.

19. The method of solution 1 or 2, wherein the ZU block is divided into three units using a tri-tree division scheme.

3*2^N+1的ZU塊被劃分的結果，三個單元中的第一單元具有尺寸2^N×T，三個單元中的第二單元具有尺寸(S-3*2^N)×T，並且三個單元中的第三單元具有尺寸2^N+1×T。 20. The method according to solution 19, wherein, as 3*2 ^N <S

As a result of dividing the 3*2 ^N+1 ZU block, the first of the three units has a size of 2 ^N × T, and the second of the three units has a size of (S-3*2 ^N ) × T, And the third cell of the three cells has a size of ^2N+1 ×T.

3*2^N+1的ZU塊被劃分的結果，三個單元中的第一單元具有尺寸S×2^N，三個單元中的第二單元具有尺寸S×(T-3*2^N)，並且三個單元中的第三單元具有尺寸S×2^N+1。 21. The method according to solution 19, wherein, as 3*2 ^N <T

As a result of dividing the 3*2 ^N+1 ZU block, the first of the three units has a size of S×2 ^N , and the second of the three units has a size of S×(T-3*2 ^N ), And the third cell among the three cells has a size of S×2 ^N+1 .

22. The method of solution 1 or 2, wherein the unit has a size of the form K*M, where M and K are integers.

23. The method of solution 1 or 2, wherein the unit has a size in the form of K*M, M is the minimum height or minimum width of the allowed coding unit and the unit is divided from non-ZU blocks.

24. The method of solution 22, wherein the signaling of the ZU block division scheme that causes units not in the form of K*M is skipped.

25. The method of solution 23, wherein the signaling of a division scheme that causes a unit not in the form of K*M divided from a non-ZU block is skipped.

26. A video encoding method, comprising: determining that a block of video data will be encoded into zero unit (ZU) blocks; using a division scheme to divide the video data block, wherein, The division scheme divides the video data block into one of two units, three units or four units; the bit stream is generated by encoding the video data block, where the division scheme uses the other as a non-zero unit block. The division of the video data block is signaled by the same syntax.

27. A video decoding method, comprising: receiving a bit stream corresponding to a video data block encoded as a zero unit (ZU) block, wherein the video data block is divided into two units, three units, or four units using One of the division schemes divides the video data block, and wherein the division scheme uses the same syntax as used to signal the division of non-zero unit blocks in the bit stream for signaling; based on the signaling, the bit is decoded Stream to generate video data blocks.

28. A video encoding method, comprising: determining that a block of video data will be encoded into zero unit (ZU) blocks, the video data block has a size; and dividing the ZU block into two using a division scheme selected from a group of ZU block division schemes Unit, one of three units, or four units; coding unit; and signaling coding unit in the bit stream. Here, the group of ZU block partitioning schemes is a subset of the group of partitioning schemes available for partitioning coding units (CU).

29. A video decoding method, comprising: receiving a bit stream corresponding to a video data block, the video data block having a size; receiving a signaling notification indicating that the video data block includes units divided from zero unit (ZU) blocks , Using a division scheme selected from the group of ZU block division schemes to divide the divided ZU blocks; and based on signaling, decoding the bit stream corresponding to the unit to reconstruct the video data block. Here, the group of ZU block partitioning schemes can be used to partition a coding unit (CU) partitioning scheme A subset of the group.

30. The method of solution 28 or 29, wherein the group of ZU block division schemes is based on one of the size of the ZU block or the position of the ZU block related to one of picture, slice, slice, or slice types.

31. The method of solution 28 or 29, wherein the group of available ZU block division schemes is limited to the quadtree division scheme and the binary tree division scheme.

32. The method according to solution 28 or 29, wherein the ZU block is divided into three units using a trigeminal tree division scheme, and the signaling of the ZU block does not include information related to the division scheme.

33. The method of solution 28 or 29, wherein the group of ZU block division schemes is the same as the group of division schemes that can be used to divide coding units (CU), and the video data block is selected based on whether the video data block is a ZU block. Let the context of the instruction to divide the ZU block be notified.

34. A video encoding method, comprising: determining that a video data block will be encoded as a zero unit (ZU) block, the video data block has a size; when determining that the ZU block is located in an I slice or an intra-coded picture, dividing the ZU block It is one of two units, three units, or four units; a coding unit; and a unit that signals coding in the bit stream.

35. A video decoding method, comprising: receiving a bit stream corresponding to a video data block, the video data block having a size; receiving a signaling notification indicating that the video data block includes units divided from zero unit (ZU) blocks , The divided ZU block is located in the I slice or intra-coded picture; and based on the signaling notification, the decoding corresponds to The bit stream of the unit to reconstruct the video data block.

36. The method of solution 34 or 35, wherein at least one size of the ZU block is a non-power of two.

37. The method of solution 34 or 35, wherein the video data block is a coding unit (CU) block, and wherein the CU block is treated as a ZU block because at least one size is an integer greater than a threshold.

38. A device in a video system, comprising a processor and a non-transitory memory with instructions thereon, wherein when the instructions are executed by the processor, the processor is made to implement any one of solutions 1 to 37 The method described.

39. A computer program product stored on a non-transitory computer-readable medium, the computer program product including program code for executing the method in any one of solutions 1 to 37.

40. The method, device or system as described in this document.

It can be understood from the foregoing that the specific embodiments of the disclosed technology have been described herein for illustrative purposes, but various modifications can be made without departing from the scope of the present invention. Therefore, the technology of the present disclosure is not limited except for the appended claims.

The subject and functional operation described in this patent document can be realized by various systems, digital electronic circuits, or by computer software, firmware or hardware, including the structure disclosed in the specification and its structural equivalents, or by using them A combination of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, ie, encoded on a tangible and non-transitory computer readable medium One or more computer program instruction modules for the data processing device to execute or control the operation of the data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that affect a machine-readable propagation signal, or a combination of one or more of them. The term "data processing unit" or "data processing device" covers all devices, equipment, and machines used to process data, including, for example, a programmable processor, computer, or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for the computer program in question, for example, constituting processor firmware, protocol stack, database management system, operating system, or one or more of them The code of the combination.

Computer programs (also called programs, software, software applications, scripts or codes) can be written in any form of programming language, including compiled or interpreted languages, and computer programs can be deployed in any form, including stand-alone programs or suitable for computing Modules, components, subroutines, or other units used in the environment. The computer program does not necessarily correspond to the files in the file system. The program can be stored in the part of the document that holds other programs or data (for example, one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordination In a document (for example, a document that stores one or more modules, subprograms, or code parts). Computer programs can be deployed to be executed on one computer or on multiple computers located on one website or distributed on multiple websites and interconnected by a communication network.

The processes and logic flows described in this specification can be executed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and generating output. Process and logic flow can also be A dedicated logic circuit is implemented, and the device can also be implemented as a dedicated logic circuit, such as FPGA (Field Programmable Gate Array) or ASIC (Dedicated Integrated Circuit).

For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, the processor will receive commands and data from read-only memory or random access memory or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, the computer will also include or be operatively coupled to one or more large storage area devices for storing data, such as floppy disks, magneto-optical discs or optical discs, to receive data from the one or more large storage area devices, or Transmit data to the one or more large storage area devices, or both receive and transmit data. However, the computer does not need to have such equipment. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices. The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.

The description and drawings are intended to be considered only exemplary, where exemplary means example. As used herein, the singular forms of "a", "an" and "the" are intended to also include the plural form, unless the context clearly dictates otherwise. In addition, the use of "or" is intended to include "and/or" unless the context clearly dictates otherwise.

Although this patent document contains many details, these details should not be construed as limitations on the scope of any invention or claimable, but as descriptions of features specific to specific embodiments of specific inventions. In this patent document, in a separate implementation Certain features described in the context of an embodiment can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable subcombination. In addition, although the above features can be described as working in certain combinations and even as claimed initially, in some cases, one or more features from the combination can be removed from the claimed combination, and The claimed combination can refer to a sub-combination or a variant of the sub-combination.

Similarly, although operations are depicted in a specific order in the drawings, this should not be understood as requiring that such operations be performed in the specific order shown or in a sequential order, or that all the operations shown are performed to achieve the desired result . In addition, the separation of various system elements in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described, and other implementations, enhancements and modifications can be made based on the content described and shown in this patent document.

1800‧‧‧Method

1802~1806‧‧‧Step

Claims (39)

A video encoding method includes: since a video data block has a size of S×T, and at least one of S and T is a power of two, determining that the video data block will be encoded as a zero unit (ZU) block, wherein the The zero unit cannot be transformed, so that transform, inverse transform, quantization, and dequantization operations are not called on the zero unit; divide the ZU block into one of two units, three units, or four units; and generate by encoding the unit Bit stream. A video decoding method includes: receiving a bit stream corresponding to a video data block encoded as a zero unit (ZU) block, the zero unit (ZU) block is divided into two units, three units, or four units, so The video data block has a size of S×T, wherein the zero unit is not transformable, so that transform, inverse transform, quantization, and dequantization operations are not called on the zero unit; and the video data block is generated by decoding the bit stream. The method described in item 1 or 2 of the scope of patent application, wherein the ZU block is divided into two units using a binary tree division scheme, and each unit has a size of S/2×T. The method described in item 1 or 2 of the scope of patent application, wherein the ZU block is divided into two units using a binary tree division scheme, and each unit has a size of S×T/2. The method described in item 1 or 2 of the scope of patent application, wherein, as 2 ^N <S

2 ^N+1 ZU block is divided into two units using a binary tree division scheme, the first unit of the two units has a size of 2 ^N × T, and the second unit of the two units has a size of (S-2 ^N )×T. The method described in item 1 or 2 of the scope of patent application, wherein, as 2 ^N <S

2 ^N+1 ZU block is divided into two units using a binary tree division scheme, the first unit of the two units has a size (S-2 ^N )×T, and the first unit of the two units The two units have dimensions 2 ^N × T. Such as the method described in item 1 or 2 of the scope of patent application, wherein, as 2 ^N <T

2 ^N+1 ZU block is divided into two units using a binary tree division scheme. The first unit of the two units has a size of S×2 ^N , and the second unit of the two units has a size S×(T-2 ^N ). Such as the method described in item 1 or 2 of the scope of patent application, wherein, as 2 ^N <T

2 ^N+1 ZU block is divided into two units using a binary tree division scheme. The first unit of the two units has a size of S×(T-2 ^N ) and the second unit of the two units The unit has the size S×2 ^N. The method described in item 1 or 2 of the scope of the patent application, wherein the ZU block is divided into three units using a trigeminal tree division scheme, and each of the two units of the three units has a size of S/4 ×T, and one of the three cells has a size of S/2×T. The method according to item 1 or 2 of the scope of patent application, wherein the ZU block is divided into three units using a trigeminal tree division scheme, two of the three units have a size of S×T/4, and One of the three cells has a size of S×T/2. The method described in item 1 or 2 of the scope of patent application, wherein, as 2 ^N <S

2 ^N+1 ZU block is divided into three units using the trigeminal tree division scheme. The first unit has a size of 2 ^N-1 ×T, the second unit has a size of (S-2 ^N )×T, and the third The cell has a size of 2 ^N-1 ×T. Such as the method described in item 1 or 2 of the scope of patent application, wherein, as 2 ^N <T

2 ^N+1 ZU block is divided into three units using the tri-tree division scheme. The first unit has size S×2 ^N-1 , the second unit has size S×(T-2 ^N ), and the third The unit has a size of S×2 ^N-1 . The method described in item 1 or 2 of the scope of patent application, wherein the ZU block is divided into four units using a quadtree division scheme, and each unit has a size of S/2×T/2. The method described in item 1 or 2 of the scope of patent application, wherein, as 2 ^N <S

2 ^N+1 ZU block is divided into four units using a quadtree division scheme. The first unit has a size of 2 ^N × T/2, the second unit has a size of 2 ^N × T/2, and the third unit It has a size of S/2×(T-2 ^N ), and the fourth cell has a size of S/2×(T-2 ^N ). Such as the method described in item 1 or 2 of the scope of patent application, wherein, as 2 ^N <T

2 ^N+1 ZU block is divided into four units using the quadtree division scheme. The first unit has size S/2×2 ^N , the second unit has size S/2×2 ^N , and the third unit It has a size of S/2×(T-2 ^N ), and the fourth cell has a size of S/2×(T-2 ^N ). The method described in item 1 or 2 of the scope of patent application, wherein, as 2 ^N <S

2 ^N+1 and 2 ^M <T

The ZU block of 2 ^M+1 is divided into four units using the quadtree division scheme. The first unit has a size of 2 ^N × 2 ^M , the second unit has a size of 2 ^N × 2 ^M , and the third unit has a size (S-2 ^N )×(T-2 ^M ), and the fourth unit has a size (S-2N)×(T-2 ^M ). The method described in item 1 or 2 of the scope of patent application, wherein, as 2 ^N <S

2 ^N+1 and 2 ^M <T

The ZU block of 2 ^M+1 is divided into four units using the quadtree division scheme. The first unit has a size of 2 ^N ×(T-2 ^M ), and the second unit has a size of 2 ^N ×(T-2 ^M ), the third unit has a size (S-2 ^N )×2 ^M , and the fourth unit has a size (S-2 ^N )×2 ^M. The method according to item 1 or 2 of the scope of patent application, wherein the height and/or width of each unit of the ZU block is an even number. The method according to item 1 or 2 of the scope of the patent application, wherein the ZU block is divided into three units using a tri-tree division scheme. The method described in item 19 of the scope of patent application, wherein, as 3*2 ^N <S

As a result of the 3*2 ^N+1 ZU block being divided, the first unit of the three units has a size of 2 ^N × T, and the second unit of the three units has a size of (S-3*2 ^N )×T, and the third cell of the three cells has a size of ^2N+1 ×T. The method described in item 19 of the scope of patent application, wherein, as 3*2 ^N <T

As a result of dividing a 3*2 ^N+1 ZU block, the first of the three units has a size of S×2 ^N , and the second of the three units has a size of S×(T-3* 2 ^N ), and the third cell of the three cells has a size of S×2 ^N+1 . The method according to item 1 or 2 of the scope of patent application, wherein the unit has a size in the form of K*M, where M and K are integers. The method described in item 1 or 2 of the scope of patent application, wherein the unit has a size in the form of K*M, M is the minimum height or width of the allowed coding unit, and the unit is divided from a non-ZU block owned. The method described in item 22 of the scope of the patent application, wherein the signaling of the ZU block division scheme that causes non-K*M units to be skipped. The method described in item 23 of the scope of patent application, wherein the signaling notification of the division scheme of the non-K*M form unit divided from the non-ZU block is skipped. A video encoding method, including: since the block has a height or width that is not a power of two, determining that a video data block will be encoded as a zero unit (ZU) block, wherein the zero unit is not transformable, so that the zero unit is not called Transformation, inverse transformation, quantization, and dequantization operations; use a partitioning scheme to divide the video data block, wherein the partitioning scheme divides the video data block into one of two units, three units, or four units And generating a bit stream by encoding the video data block, wherein the division scheme uses the same syntax as the signaling used to divide another video data block that is a non-zero unit block to signal . A video decoding method, including: Receive a bit stream corresponding to a video data block, since the video data block has a size of S×T, and at least one of S and T is a power of two, the video data block is encoded as a zero unit (ZU) block , Wherein the video data block is divided using a division scheme of dividing the video data block into two units, three units, or one of four units, and wherein the division scheme is in the bit stream Use the same syntax as that used to signal the division of non-zero unit blocks, where the zero unit is not transformable, so that transform, inverse transform, quantization, and dequantization operations are not called on the zero unit; and The signaling informs that the bit stream is decoded to generate the video data block. A video encoding method includes: since a video data block has a size of S×T, and at least one of S and T is a power of two, determining that the video data block will be encoded as a zero unit (ZU) block, wherein the zero The unit is not transformable, so that the transform, inverse transform, quantization, and dequantization operations are not called on the zero unit; the ZU block is divided into two units, three units, or four using a division scheme selected from the group of ZU block division schemes One of the units; encoding the unit; and signaling the coded unit in the bit stream; wherein the group of the ZU block partitioning scheme is a child of the group of partitioning schemes that can be used to partition the coding unit (CU) set. A video decoding method, including: Receiving a bit stream corresponding to a video data block, the video data block having a size; receiving a signaling notification, the signaling notification indicating that because the video data block has a size of S×T, at least one of S and T is wrong A power of two, the video data block is divided as a zero unit (ZU) block, and the block is divided using a division scheme selected from the group of ZU block division schemes, wherein the zero unit is not transformable, making it non-zero The unit calls transform, inverse transform, quantization, and dequantization operations; and based on the signaling, decodes the bit stream corresponding to the unit to reconstruct the video data block; wherein the ZU block division scheme A group is a subset of groups that can be used to partition a coding unit (CU) division scheme. The method according to item 28 or 29 of the scope of patent application, wherein the group of the ZU block division scheme is based on the size of the ZU block or the ZU related to one of the picture, strip, slice, or strip type One of the location of the block. The method according to item 28 or 29 of the scope of patent application, wherein the group of available ZU block division schemes is limited to the quadtree division scheme and the binary tree division scheme. The method according to item 28 or 29 of the scope of patent application, wherein the ZU block is divided into three units using a trigeminal tree division scheme, and the signaling notification of the ZU block does not include information related to the division scheme information. The method according to item 28 or 29 of the scope of the patent application, wherein the group of the ZU block division scheme and the division that can be used to divide the coding unit (CU) The groups of the schemes are the same, and based on the video data block being the ZU block, the context for signaling the indication of dividing the ZU block is selected. A video encoding method includes: since the video data block has a size, and at least one of the height and the width of the size is a power of two, determining that the video data block will be encoded as a zero unit (ZU) block, wherein The zero unit cannot be transformed, so that the transform, inverse transform, quantization, and dequantization operations are not called for the zero unit; when it is determined that the ZU block is located in an I-slice or intra-coded picture, the ZU block is divided into two One of three units, three units, or four units; encoding the unit; and signaling the encoded unit in the bitstream. A video decoding method, comprising: receiving a bit stream corresponding to a video data block, the video data block having a size; receiving a signaling notification, the signaling notification indicating that the video data block includes a zero unit (ZU) block The divided unit, the zero unit (ZU) block is at least a power of two in height or width, and is coded without transformation and residual coding, and the divided ZU block is located in an I slice or frame In the coded picture, the zero unit is not transformable, so that transform, inverse transform, quantization, and dequantization operations are not called on the zero unit; and based on the signaling notification, the bit stream corresponding to the unit is decoded to reconstruct The video data block. The method according to item 34 or 35 of the scope of patent application, wherein at least one of the height and the width of the size of the ZU block is a non-power of two. The method according to item 34 or 35 of the scope of patent application, wherein the video data block is a coding unit (CU) block, and wherein the CU block is treated as a ZU because at least one size is an integer greater than a threshold Piece. A device in a video system includes a processor and a non-transitory memory on which instructions are stored, wherein, when the instructions are executed by the processor, the processor realizes the first To one or more of the 37 methods. A computer program product stored on a non-transitory computer readable medium. The computer program product includes program code for executing one or more of the methods described in items 1 to 37 in the scope of the patent application.

Images