CN113728631A

CN113728631A - Intra sub-block partitioning and multi-transform selection

Info

Publication number: CN113728631A
Application number: CN202080031501.XA
Authority: CN
Inventors: 张莉; 张凯; 刘鸿彬; 王悦
Original assignee: Beijing ByteDance Network Technology Co Ltd; ByteDance Inc
Current assignee: Beijing ByteDance Network Technology Co Ltd; ByteDance Inc
Priority date: 2019-04-27
Filing date: 2020-04-27
Publication date: 2021-11-30
Anticipated expiration: 2040-04-27
Also published as: WO2020221213A1; CN113728631B

Abstract

Intra subblock partitioning and multi-transform selection are described. In one example aspect, a method of video processing includes, for a transition between a block of video and a bitstream representation of the block, enabling an intra sub-block partitioning (ISP) mode for the block, wherein the block is divided into a plurality of sub-partitions based on the ISP mode; enabling a second mode different from the ISP mode of the block; and performing a conversion based on the ISP mode and the second mode.

Description

Intra sub-block partitioning and multi-transform selection

Cross Reference to Related Applications

This application is proposed to claim the priority and benefit of international patent application No. pct/CN2019/084699 filed 2019 on 4, 27.2019 in due course, according to applicable patent laws and/or regulations of the paris convention. The entire disclosure of international patent application No. pct/CN2019/084699 is incorporated by reference as part of the disclosure of the present application.

Technical Field

This patent document relates to video encoding and decoding techniques, apparatus and systems.

Background

Despite advances in video compression, digital video still accounts for the largest bandwidth used on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the demand for bandwidth for digital video usage is expected to continue to grow.

Disclosure of Invention

This document describes various embodiments and techniques in which a quadratic transform is used during decoding or encoding of a video or image.

In one example aspect, a method of video processing is disclosed. The method includes partitioning a block of video data into sub-blocks using a partitioning mode, performing prediction of a current sub-block of the sub-blocks using at least one line of reference video data that is not adjacent to the current sub-block, and generating a residual signal of the current sub-block based on the prediction.

In yet another example aspect, another method of video processing is disclosed. The method includes dividing a block of video data into sub-blocks using a division mode, performing prediction of the sub-blocks by calculating a matrix vector product based on a reference signal of each sub-block, and generating a residual signal of the sub-blocks based on the prediction.

In yet another example aspect, another method of video processing is disclosed. The method includes receiving a block of video data partitioned into sub-blocks using a partitioning mode, performing prediction of a current sub-block of the sub-blocks using at least one line of reference video data that is not adjacent to a current sub-frame, and reconstructing the current sub-block using the prediction.

In yet another example aspect, another method of video processing is disclosed. The method includes receiving a block of video data partitioned into sub-blocks using a partition mode, performing prediction of the sub-blocks by calculating a matrix vector product based on a reference signal of each sub-block, and reconstructing the sub-blocks using the prediction.

In yet another example aspect, another method of video processing is disclosed. The method includes performing prediction on a block of video data partitioned into sub-blocks using a partition mode, and transforming a residual signal of the sub-blocks based on the prediction. The maximum transform block dimension or the minimum transform block dimension is indicated in the level of a sequence, a picture, a slice group, a slice, or a brick in a bitstream representing a block of video data.

In yet another example aspect, another method of video processing is disclosed. The method includes receiving a bitstream representing a block of video data partitioned into sub-blocks using a partitioning mode, performing an inverse transform on residual signals of the sub-blocks, and reconstructing the sub-blocks using outputs from the inverse transform. The maximum transform block dimension or the minimum transform block dimension is indicated in a sequence, picture, slice group, slice or brick level in the bitstream.

In yet another example aspect, another method of video processing is disclosed. The method includes receiving or transmitting a bitstream representing a block of video data for performing video processing. A block of video data is partitioned into sub-blocks using a partitioning mode, and residual signals of the sub-blocks are quantized in a bitstream, and the sub-blocks share the same quantization information.

In yet another example aspect, another method of video processing is disclosed. The method includes performing prediction on a block of video data partitioned into sub-blocks using a partitioning mode. The reference samples in the first sub-block are modified before being used to perform prediction of the second sub-block. The method also includes encoding or reconstructing a block of video data based on the prediction.

In yet another example aspect, another method of video processing is disclosed. The method includes performing prediction on a block of video data partitioned into sub-blocks using a partitioning mode, and encoding or reconstructing the block of video data based on the prediction. The sub-blocks are partitioned in a plurality of partitioning directions.

In yet another example aspect, another method of video processing is disclosed. The method includes performing prediction of a block of video data to generate a residual signal, performing an explicit transform of the residual signal using one of two transforms, and encoding an output from the implicit transform.

In yet another example aspect, another method of video processing is disclosed. The method includes receiving a block of video data partitioned into one or more sub-blocks, performing an explicit transform of the block of video data using one of two inverse transforms; and reconstructing the block of video data based on the implicit transform.

In yet another example aspect, another method of video processing is disclosed. The method includes enabling an intra sub-block partitioning (ISP) mode for a block of video for a transition between the block and a bitstream representation of the block, wherein the block is divided into a plurality of sub-partitions based on the ISP mode; enabling a second mode different from the ISP mode of the block; and performing a conversion based on the ISP mode and the second mode.

In yet another example aspect, another method of video processing is disclosed. The method includes enabling an intra sub-block partitioning (ISP) mode for a block of video for a transition between the block and a bitstream representation of the block, wherein the block is divided into a plurality of sub-partitions based on the ISP mode, and reference samples located in a first sub-partition to predict a second sub-partition in the block are further modified before being used as a prediction; and performing the conversion based on the ISP mode.

In yet another example aspect, another method of video processing is disclosed. The method includes, for a transition between a block of video and a bitstream representation of the block, enabling an intra sub-block partitioning (ISP) mode for the block, wherein a hybrid partitioning direction is enabled in the ISP mode and the block is partitioned into a plurality of sub-partitions in horizontal and vertical directions; and performing the conversion based on the ISP mode.

In yet another example aspect, another method of video processing is disclosed. The method includes determining, for a transition between a block of video and a bitstream representation of the block, a Multiple Transform Selection (MTS) scheme associated with the block, wherein the MTS scheme is modified to allow partial transforms and corresponding inverse transforms; and performing a conversion based on the determined MTS scheme.

In yet another example aspect, a video encoder is disclosed. The video encoder includes a processor configured to perform one or more of the methods described above.

In yet another example aspect, a video decoder is disclosed. The video decoder comprises a processor configured to perform one or more of the methods described above.

In yet another example aspect, a computer-readable medium is disclosed. The medium includes code stored thereon for performing one or more of the above-described methods.

These and other aspects are described in this document.

Drawings

Fig. 1 shows an example of a block diagram of an encoder.

Fig. 2 shows an example of 67 intra prediction modes.

Fig. 3A-3B illustrate examples of reference samples for wide-angle intra prediction.

FIG. 4 is an example illustration of a discontinuity problem with directions exceeding 45 degrees.

Fig. 5A-5D show example graphical representations of samples used by PDPCs applied to diagonal and adjacent corner intra modes.

Fig. 6 depicts an example of four reference rows.

Fig. 7 is an example of the division of 4 × 8 and 8 × 4 blocks.

Fig. 8 is an example of division of all blocks except for 4 × 8, 8 × 4, and 4 × 4.

Fig. 9 is an example of Affine Linear Weighted Intra Prediction (ALWIP) for a 4x4 block.

FIG. 10 is an example of ALWIP for an 8x8 block.

FIG. 11 is an example of ALWIP for an 8x4 block.

FIG. 12 is an example of ALWIP for a 16x16 block.

Fig. 13 shows an example of quadratic transformation in JEM.

Fig. 14 shows an example of the proposed reduced quadratic transform (RST).

FIG. 15 is a diagram of sub-block transform modes SBT-V and SBT-H.

FIG. 16 is a block diagram of an example hardware platform for performing the techniques described in this document.

Fig. 17 shows an example of missed partitioning.

Fig. 18 is a flow diagram of a video processing method in accordance with one or more examples of the present technology.

Fig. 19 is a flow diagram of a video processing method in accordance with one or more examples of the present technology.

Fig. 20 is a flow diagram of a video processing method in accordance with one or more examples of the present technology.

Fig. 21 is a flow diagram of a video processing method in accordance with one or more examples of the present technology.

Fig. 22 is a flow diagram of a video processing method in accordance with one or more examples of the present technology.

Fig. 23 is a flow diagram of a video processing method in accordance with one or more examples of the present technology.

Fig. 24 is a flow diagram of a video processing method in accordance with one or more examples of the present technology.

Detailed Description

Section headings are used in this document to facilitate ease of understanding and do not limit the embodiments disclosed in the sections to only that section. Furthermore, although certain embodiments are described with reference to multi-function video codecs or other specific video codecs, the disclosed techniques are also applicable to other video codecs techniques. Furthermore, although some embodiments describe video encoding steps in detail, it will be understood that the corresponding steps of de-encoding will be performed by the decoder. Furthermore, the term video processing covers video encoding or compression, video decoding or decompression and video transcoding, where video pixels are represented from one compression format to another compression format or at different compression bit rates.

1. Overview

This patent document relates to video encoding and decoding techniques. In particular, it relates to related transforms in video codecs. It can be applied to existing video codec standards, such as HEVC, or to-be-completed standards (multifunctional video codec). It may also be applicable to future video codec standards or video codecs.

2. Preliminary discussion

Video codec standards were developed primarily by developing the well-known ITU-T and ISO/IEC standards. ITU-T makes H.261 and H.263, ISO/IEC makes MPEG-1 and MPEG-4Visual, and the two organizations together make the H.262/MPEG-2 video and the H.264/MPEG-4 Advanced Video Codec (AVC) and H.265/HEVC standards. Starting from h.262, the video codec standard is based on a hybrid video codec structure, in which temporal prediction and transform coding is utilized. In order to explore future video codec technologies other than HEVC, VCEG and MPEG united in 2015 to form a joint video exploration team (jfet). Thereafter, JFET adopted many new approaches and introduced them into reference software known as the "Joint exploration model" (JEM). In month 4 of 2018, the joint video experts group (jfet) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11(MPEG) holds true in the VVC standard, which has a 50% reduction in bit rate compared to HEVC.

2.1 codec flow for typical video codecs

Fig. 1 shows an example encoder block diagram for a VVC, which contains three loop filter blocks: deblocking Filter (DF), Sample Adaptive Offset (SAO), and ALF. Unlike DF using predefined filters, SAO and ALF reduce the mean square error between original and reconstructed samples by adding offsets and applying Finite Impulse Response (FIR) filters, respectively, with the original samples of the current picture, with the side information of the coding signaling the offsets and filter coefficients. ALF is located at the final processing stage of each picture and can be viewed as a tool that attempts to capture and fix artifacts produced by previous stages.

2.2 Intra mode coding with 67 Intra prediction modes

To capture any edge direction present in natural video, the number of directional intra modes extends from 33 to 65 used in HEVC. Additional directional modes are indicated by the red dashed arrows in fig. 6, with the plane and DC modes remaining unchanged. These denser directional intra prediction modes are applicable to all block sizes as well as luma and chroma intra prediction.

As shown in fig. 2, the conventional intra-angular prediction direction is defined from 45-135 degrees in the clockwise direction. In VTM2, several conventional intra prediction modes are adaptively replaced by wide-angle intra prediction modes for non-square blocks. The alternative mode is signaled using the original method and remapped to the index of the wide mode after parsing. The total number of intra prediction modes is unchanged, i.e. 67, and the intra mode codec is unchanged.

In HEVC, each intra coded block has a square shape and the length of each side thereof is a power of 2. Therefore, the intra predictor is generated using the DC mode without division. In VVV2, chunks may have a rectangular shape, which typically requires the use of a division operation for each chunk. To avoid division operations for DC prediction, only the longer side is used to calculate the average of non-square blocks.

2.3 Wide-Angle Intra prediction of non-Square blocks

The conventional intra-angular prediction direction is defined from 45 degrees to-135 degrees in the clockwise direction. In VTM2, several conventional intra prediction modes are adaptively replaced by wide-angle intra prediction modes for non-square blocks. The alternative mode is signaled using the original method and remapped to the index of the wide mode after parsing. The total number of intra prediction modes for a certain block is unchanged, i.e. 67, and the intra mode codec is unchanged.

To support these prediction directions, a top reference of length 2W +1 and a left reference of length 2H +1 are defined as shown in FIGS. 3A-3B.

The number of modes of the replacement mode in the wide-angle direction mode depends on the aspect ratio of the block. Alternative intra prediction modes are shown in table 1.

Table 1-intra prediction mode replaced by wide-angle mode

Condition	Alternative intra prediction modes
		W/H＝＝2	Modes 2,3,4,5,6,7
W/H>2	Modes 2,3,4,5,6,7,8,9,10,11
		W/H＝＝1	Is free of
H/W＝＝1/2	Modes 61,62,63,64,65,66
		H/W<1/2	Modes 57,58,59,60,61,62,63,64,65,66

As shown in fig. 4, in the case of wide-angle intra prediction, two vertically adjacent prediction samples may use two non-adjacent reference samples. Thus, a low-pass reference sampling filter and side smoothing are applied to the wide-angle prediction to reduce the increased gap Δ p_αThe negative effects of (c).

2.4 location-dependent Intra prediction combining

In VTM2, the result of intra prediction for planar mode is further modified by a position dependent intra prediction combining (PDPC) method. PDPC is an intra prediction method that involves unfiltered boundary reference samples and a combination of HEVC intra prediction and filtered boundary reference samples. PDPC applies to the following intra modes without signaling: plane, DC, horizontal, vertical, lower left corner pattern and its eight adjacent corner patterns, and upper right corner pattern and its eight adjacent corner patterns.

The prediction samples pred (x, y) are predicted using a linear combination of intra prediction mode (DC, plane, angle) and reference samples according to the following equation:

wherein R is_x,-1，R_-1,yRespectively, represent reference samples located at the top and left of the current sample (x, y), and R_-1,-1Representing the reference sample located in the upper left corner of the current block.

If PDPC is applied to DC, planar, horizontal and vertical intra modes, no additional boundary filter is needed, which is needed in case of HEVC DC mode boundary filter or horizontal/vertical mode edge filter.

5A-5D illustrate reference samples (R) for PDPC applied over various prediction modes_x,-1，R_-1,yAnd R_-1,-1) The definition of (1). The prediction samples pred (x ', y') are located at (x ', y') within the prediction block. Reference sample R_x,-1Is given by: x ═ x '+ y' +1, and reference sample point R_-1,yIs similarly given by: y ═ x '+ y' + 1.

Fig. 5A to 5D provide definitions of samples used by PDPCs applied to diagonal and adjacent corner intra modes.

The PDPC weights depend on the prediction mode and are shown in table 2.

TABLE 2 example of PDPC weights according to prediction modes

Prediction mode	wT	wL	wTL
				Diagonal upper right	16>>((y’<<1)>>shift)	16>>((x’<<1)>>shift)	0
Diagonal left lower	16>>((y’<<1)>>shift)	16>>((x’<<1)>>shift)	0
				Adjacent diagonal right upper	32>>((y’<<1)>>shift)	0	0
Adjacent diagonal left lower	0	32>>((x’<<1)>>shift)	0

2.5. Multiple reference rows

Multiple Reference Line (MRL) intra prediction uses more reference lines for intra prediction. In fig. 6, an example of 4 reference rows is depicted, where the samples of segments a and F are not taken from the reconstructed neighboring samples, but are filled with the closest samples from segments B and E, respectively. HEVC intra picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, 2 additional lines (reference line 1 and reference line 3) are used.

The index (mrl _ idx) of the selected reference row is signaled and used to generate the intra predictor. For reference row indices greater than 0, only additional reference row modes in the MPM list are included and only the MPM index is signaled without the remaining modes. The reference row index is signaled before the intra prediction mode, and the plane and DC mode are excluded from the intra prediction mode if a non-zero reference row index is signaled.

MRL is disabled for the first row blocks inside the CTU to prevent the use of extended reference samples outside the current CTU row. Also, PDPC is disabled when additional lines are used.

2.6 Intra-frame sub-block partitioning (ISP)

ISPs are proposed that divide luma intra prediction blocks vertically or horizontally into 2 or 4 sub-partitions according to the block size dimension, as shown in the table. Fig. 7 and 8 show examples of two possibilities. Fig. 7 shows examples of the division of 4 × 8 and 8 × 4 blocks. Fig. 8 shows an example of division of all blocks except for 4 × 8, 8 × 4, and 4 × 4. All sub-partitions satisfy the condition of at least 16 samples. For block sizes, if 4 × N or N × 4(N >8) is allowed, there may be 1 × N or N × 1 sub-partitions.

Table 3: number of sub-partitions depending on block size

For each of these sub-partitions, a residual signal is generated by entropy-decoding the coefficients transmitted by the encoder, and then inverse-quantizing and inverse-transforming them. Then, intra prediction is performed on the sub-partition, and finally a corresponding reconstructed sample is obtained by adding the residual signal to the prediction signal. Thus, the reconstructed value for each sub-partition will be available to generate a prediction for the next partition, which will repeat the process, and so on. All sub-partitions share the same intra mode.

Table 4 shows example transform types based on the intra prediction mode(s).

Table 4: specification of trTypeHor and trTypeVer depending on predModeIntra

2.6.1 example syntax and semantics

Table 5 shows an example codec unit syntax.

Table 5: coding/decoding unit syntax

Table 6 shows an example transform unit syntax. Some example variables include:

an intra _ subportions _ mode _ flag x0 y0 equal to 1 indicates that the current intra codec unit is partitioned into numintrasubportions x0 y0 rectangular transform block sub-partitions. Intra _ sub partitions _ mode _ flag x0 y0 equal to 0 means that the current intra codec unit is not partitioned into rectangular transform block sub-partitions.

When intra _ sub _ modes _ flag [ x0] [ y0] is not present, it is inferred to be equal to 0.

intra _ sub _ partitions _ split _ flag x0 y0 specifies whether the intra sub-partition type is horizontal or vertical. When intra _ sub _ splits _ flag [ x0] [ y0] is not present, it is inferred as follows:

if cbHeight is greater than MaxTbSizeY, then intra _ sub _ splits _ flag [ x0] [ y0] is inferred to be equal to 0.

Otherwise (cbWidth is greater than MaxTbSizeY), intra _ sub _ split _ flag [ x0] [ y0] is inferred to be equal to 1.

The variable intrasubportionssplit type specifies the partition type for the current luma codec block, as shown in table 79. Intrasubportionssplit type is derived as follows:

if intra _ subportions _ mode _ flag [ x0] [ y0] is equal to 0, IntraSubPartitionsSplitType is set equal to 0.

Otherwise, IntraSubPartitionsSplitType is set equal to 1+ intra _ sub _ partitionsSplit flag [ x0] [ y0 ].

TABLE 6 conversion Unit syntax

Table 7 illustrates example name associations with IntraSubPartitionsSplitType

Table 7 associates the name of IntraSubPartitionsSplitType

IntraSubPartitionsSplitType	Name of IntraSubPartitionsSplitType
		0	ISP_NO_SPLIT
1	ISP_HOR_SPLIT
		2	ISP_VER_SPLIT

The variable NumIntraSubPartitions specifies the number of transform block sub-partitions into which an intra luma coded block is partitioned. NumIntraSubPartations is derived as follows:

if IntraSubPartitions SplitType is equal to ISP _ NO _ SPLIT, then NumIntraSubPartitions is set equal to 1.

Otherwise, if one of the following conditions is met, NumIntraSubPartitions is set equal to 2: cbWidth equals 4 and cbHeight equals 8, cbWidth equals 8 and cbHeight equals 4.

Otherwise, NumIntraSubPartitions is set equal to 4.

2.7 affine Linear weighted Intra prediction (ALWIP, also known as matrix-based Intra prediction)

Affine linear weighted intra prediction (ALWIP, also known as matrix-based intra prediction (MIP)) is proposed.

2.7.1 Generation of reduced prediction signals by matrix vector multiplication

Adjacent reference samples are first downsampled by averaging to generate a reduced reference signal bdry_red. The reduced prediction signal pred is then calculated by calculating the matrix vector product and adding an offset_red。

pred_red＝A·bdry_red+b。

Here, A is a matrix having W_red·H_redRow, if W ═ H ═ 4, then it has 4 columns, in all other cases it has 8 columns. b is the dimension W_red·H_redThe vector of (2).

2.7.2. Graphical representation of the entire ALWIP Process

The whole process of averaging, matrix vector multiplication and linear interpolation is shown for different shapes in fig. 9 to 12. Note that the remaining shapes are processed as in one of the cases shown.

Given a 4x4 block, as shown in fig. 9, the ALWIP takes two averages along each axis of the boundary. The resulting four input samples enter a matrix vector multiplication. The matrix is taken from the set S _ 0. After adding the offset, this yields 16 final predicted samples. Linear interpolation is not necessary for generating the prediction signal. Therefore, a total of (4 · 16)/(4 · 4) ═ 4 multiplications are performed per sample.

Given an 8x8 block, as shown in fig. 10, the ALWIP takes four averages along each axis of the boundary. The resulting eight input samples enter matrix vector multiplication. The matrix is taken from the set S _ 1. This produces 16 samples at odd positions of the prediction block. Therefore, a total of (8 · 16)/(8 · 8) ═ 2 multiplications are performed per sample. After adding the offset, these samples are interpolated vertically by using the reduced top boundary. Horizontal interpolation is then performed by using the original left boundary.

Given an 8x4 block, as shown in fig. 11, the ALWIP takes four averages along the horizontal axis of the boundary and takes four original boundary values on the left boundary. The resulting eight input samples enter matrix vector multiplication. The matrix is taken from the set S _ 1. This produces 16 samples at each of the odd horizontal and vertical positions of the prediction block. Therefore, a total of (8 · 16)/(8 · 4) ═ 4 multiplications are performed per sample. After adding the offset, these samples are interpolated horizontally by using the original left boundary. The transpose case is handled accordingly.

Given a 16x16 block, as shown in fig. 12, the ALWIP takes four averages along each axis of the boundary. The resulting eight input samples enter matrix vector multiplication. The matrix is taken from the set S _ 2. This results in 64 samples at odd positions of the prediction block. Therefore, a total of (8 · 64)/(16 · 16) ═ 2 multiplications are performed per sample. After adding the offset, these samples are interpolated vertically by using the eight averages of the top boundary. Horizontal interpolation is then performed by using the original left boundary. In this case, the interpolation process does not add any multiplication. Thus, in general, two multiplications are required per sample to compute the ALWIP prediction.

For larger shapes, the process is essentially the same and it is easy to verify that the number of multiplications per sample is less than four.

For a W × 8 block with W >8, only horizontal interpolation is needed, since samples are given at odd horizontal and every vertical position.

Finally, for a W × 4 block with W >8, let a _ kbe be a matrix, which is generated by truncating each row corresponding to an odd entry along the horizontal axis of the downsample block. Therefore, the output size is 32, and again, only horizontal interpolation needs to be performed.

The transpose case is handled accordingly.

2.7.1 syntax and semantic examples

Table 8 shows an example codec Unit syntax

TABLE 8 coding/decoding Unit syntax

2.8 Multiple Transform Sets (MTS) in VVC

2.8.1 explicit Multiple Transform Sets (MTS)

In VTM4, large block size transforms up to 64 × 64 in size are enabled, which are mainly useful for higher resolution video, such as 1080p and 4K sequences. For a transform block with a size (width or height, or both) equal to 64, the high frequency transform coefficients are zeroed out so that only the low frequency coefficients remain. For example, for an mxn transform block, where M is the block width and N is the block height, when M equals 64, only the left 32 columns of transform coefficients are retained. Similarly, when N equals 64, only the top 32 rows of transform coefficients are retained. When the transition skip mode is used for large blocks, the entire block will be used without zeroing any values.

In addition to DCT-II, which has been adopted in HEVC, inter and intra coded blocks are residual coded using a Multiple Transform Selection (MTS) scheme. It uses a plurality of selected transforms from the DCT8/DST 7. The newly introduced transformation matrices are DST-VII and DCT-VIII. The table below shows the basis functions of the selected DST/DCT.

To preserve the orthogonality of the transform matrices, the transform matrices are quantized more accurately than the transform matrices in HEVC. In order to keep the median of the transform coefficients in the 16-bit range, all coefficients should have 10 bits after the horizontal and vertical transforms.

To control the MTS scheme, separate enable flags are specified at the SPS level for intra-frame and inter-frame, respectively. When MTS is enabled at SPS, CU level flag is signaled to indicate whether MTS is applied. Here, MTS is applied only to luminance. The MTS CU level flag is signaled when the following conditions are met.

-width and height both less than or equal to 32

-CBF flag equal to one

If the MTS CU flag is equal to zero, the DCT2 is applied in both directions. However, if the MTS CU flag is equal to one, the other two flags are additionally signaled to indicate the transform type in the horizontal and vertical directions, respectively. The transformation and signaling mapping tables are shown in tables 3-10. When transform matrix precision is involved, an 8-bit one-pass transform kernel is used. Therefore, all transform kernels used in HEVC remain unchanged, including 4-point DCT-2 and DST-7, 8-point, 16-point and 32-point DCTs-2. Furthermore, other transform kernels, including 64-point DCT-2, 4-point DCT-8, 8-point, 16-point, 32-point DST-7, and DCT-8, all use 8-bit primary transform kernels.

To reduce the complexity of large sizes of DST-7 and DCT-8, the high frequency transform coefficients of DST-7 and DCT-8 with a size (width or height, or both) equal to 32 are zeroed out. Only the coefficients in the 16x16 low frequency region are retained.

As in HEVC, the residual of a block may be coded with a transform skip mode. To avoid redundancy of syntax coding, the transform skip flag is not signaled when the CU level MTS _ CU _ flag is not equal to 0. The block size restriction for transform skipping is the same as MTS in JEM4, indicating that transform skipping applies to a CU when both the block width and height are equal to or less than 32.

2.8.1.1 example syntax and semantics

The MTS index may be signaled in the bitstream and this design is referred to as explicit MTS. In addition, an alternative method of directly deriving the matrix from the transform block size, i.e. implicit MTS, is also supported.

For explicit MTS, it supports all codec modes. Whereas for implicit MTS, only intra mode is supported. Table 9 shows an example picture parameter set syntax.

Table 9 picture parameter set RBSP syntax.

Table 10 shows an example transform unit syntax.

TABLE 10 transform Unit syntax

Some of the example variables include:

transform _ skip _ flag x0 y0 specifies whether or not a transform is applied to the luma transform block. The array indices x0, y0 specify the position of the top-left luma sample of the transform block under consideration relative to the top-left luma sample of the picture (x0, y 0). transform _ skip _ flag x0 y0 equal to 1 means that no transform is applied to the luma transform block. transform _ skip _ flag x0 y0 equal to 0 indicates that the decision whether to apply a transform to the luma transform block depends on other syntax elements. When transform _ skip _ flag [ x0] [ y0] is not present, it is inferred to be equal to 0.

tu _ mts _ idx [ x0] [ y0] specifies which transform kernels are applied to the residual samples along the horizontal and vertical directions of the associated luma transform block. The array indices x0, y0 specify the position of the top-left luma sample of the transform block under consideration relative to the top-left luma sample of the picture (x0, y 0).

When tu _ mts _ idx [ x0] [ y0] is not present, it is inferred to be equal to 0.

In the CABAC decoding process, one context is used to decode transform _ skip _ flag, and the truncated unary code is used to binarize tu _ mts _ idx. Each bin (bin) of tu _ mts _ idx is context coded, for the first bin, using a quadtree depth (i.e., cqtDepth) to select a context; for the remaining bins, one context is used.

Table 11 shows an example assignment of ctxInc to syntax elements.

Table 11 assigns ctxInc to syntax elements with context codec bin

2.8.2 implicit Multi-transform set (MTS)

It should be noted that ISPs, SBTs and MTSs enabled but with implicit signaling are all considered implicit MTSs.

implicitMtsEnabled is used to define whether implicit MTS is enabled. The variable implicitimtsenenabled is derived as follows:

if sps _ mts _ enabled _ flag is equal to 1 and one of the following conditions is true, then imitmitsuntsenabled is set equal to 1:

IntraSubPartitionsSplitType not equal to ISP _ NO _ SPLIT

-cu _ sbt _ flag equal to 1 and Max (nTbW, nTbH) less than or equal to 32

-sps _ explicit _ mts _ INTRA _ enabled _ flag and sps _ explicit _ mts _ inter _ enabled _ flag are both equal to 0 and CuPredMode [ xTbY ] [ yTbY ] is equal to MODE _ INTRA

Otherwise, implicitMtsEnabled is set equal to 0.

The variable trTypeHor specifying the horizontal transform kernel and the variable trTypeVer specifying the vertical transform kernel are derived as follows:

if cIdx is greater than 0, then trTypeHor and trTypeVer are set equal to 0.

Otherwise, if implicitMtsEnabled is equal to 1, then the following applies:

if IntraSubPartitionsSplitType is not equal to ISP _ NO _ SPLIT, then trTypeHor and trTypeVer are specified in table 815 according to intraPredMode.

Otherwise, if cu _ sbt _ flag is equal to 1, trTypeHor and trTypeVer are specified in table 814 according to cu _ sbt _ horizontal _ flag and cu _ sbt _ pos _ flag.

Otherwise (sps _ explicit _ mts _ intra _ enabled _ flag and sps _ explicit _ mts _ inter _ enabled _ flag equal to 0), trTypeHor and trTypeVer are derived as follows:

trTypeHor＝(nTbW>＝4&&nTbW<＝16&&nTbW<＝nTbH)？1:0(8 1030)

trTypeVer＝(nTbH>＝4&&nTbH<＝16&&nTbH<＝nTbW)？1:0(8 1031)

otherwise, trTypeHor and trTypeVer are specified in Table 12 according to tu _ mts _ idx [ xTbY ] [ yTbY ].

Table 12 depends on the specification of trTypeHor and trTypeVer for tu _ mts _ idx [ x ] [ y ]

tu_mts_idx[x0][y0]	0	1	2	3	4
						trTypeHor	0	1	2	1	2
trTypeVer	0	1	1	2	2

Table 13 shows example specifications of trTypeHor and trTypeVer depending on cu _ sbt _ horizontal _ flag and cu _ sbt _ pos _ flag.

Table 13 depends on the specification of trTypeHor and trTypeVer for cu _ sbt _ horizontal _ flag and cu _ sbt _ pos _ flag

cu_sbt_horizontal_flag	cu_sbt_pos_flag	trTypeHor	trTypeVer
				0	0	2	1
0	1	1	1
				1	0	1	2
1	1	1	1

2.9 reduced quadratic transform (RST)

Indivisible quadratic transformation (NSST) in JEM 2.9.1

In JEM, a quadratic transform is applied between the forward primary transform and the quantization (at the encoder) and between the dequantization and the inverse primary transform (at the decoder side). As shown in fig. 10, performing a 4x4 (or 8x8) quadratic transform depends on the block size. For example, a 4x4 quadratic transform is applied for small blocks (i.e., min (width, height) <8), and an 8x8 quadratic transform is applied every 8x8 blocks for larger blocks (i.e., min (width, height) > 4).

Fig. 13 shows an example of quadratic transformation in JEM.

The application of the indivisible transformation is described below using the input as an example. To apply the indivisible transform, 4X4 input block X

First expressed as a vector

The indivisible transformation is calculated as

Wherein

A transform coefficient vector is indicated, and T is a 16x16 transform matrix. The 16x1 coefficient vector is then scanned using the order of the block (horizontal, vertical, or diagonal)

Reorganized into 4x4 chunks. The coefficients with smaller indices will be placed in the 4x4 coefficient block with smaller scan indices. There are a total of 35 transform sets and 3 indivisible transform matrices (kernels) are used per transform set. The mapping from intra prediction mode to transform set is predefined. For each transform set, the selected undivided quadratic transform candidate is further specified by an explicitly signaled quadratic transform index. Each intra CU signals an index in the bitstream after the transform coefficients.

2.9.2 reduced quadratic transform (RST)

RST is introduced and 4 transform set (instead of 35 transform set) mappings are introduced. For 8x8 and 4x4 blocks, 16x64 (which may be further reduced to 16x48) and 16x16 matrices, respectively, are employed. For convenience, the 16x64 (possibly further reduced to 16x48) transform is denoted RST8x8 and the 16x16 transform is denoted RST4x 4. Fig. 11 shows an example of RST.

Fig. 14 shows an example of the proposed reduced quadratic transform (RST).

2.10 sub-block transforms

For inter-predicted CUs with CU cbf equal to 1, CU sbt flag may be signaled to indicate whether the entire residual block or a sub-portion of the residual block is decoded. In the former case, the inter-frame MTS information is further parsed to determine the transform type of the CU. In the latter case, one part of the residual block is coded by the inferred adaptive transform, while another part of the residual block is zeroed. SBT does not apply to the combined inter-intra mode.

In the sub-block transform, a position-dependent transform is applied to a luminance transform block in SBT-V and SBT-H (always using the chroma TB of DCT-2). The two positions of SBT-H and SBT-V are associated with different core transformations. More specifically, the horizontal and vertical translation of each SBT position is specified in fig. 15. For example, the horizontal and vertical transforms for SBT-V position 0 are DCT-8 and DST-7, respectively. When one side of the residual TU is greater than 32, the corresponding transform is set to DCT-2. Thus, the sub-block transforms collectively specify TU bricks, cbf, and the horizontal and vertical transforms of the residual block, which may be considered syntax shortcuts for the case where the main residual of the block is on one side of the block.

2.10.1 example syntax and semantics

Table 14 shows an example codec unit syntax.

TABLE 14 coding/decoding Unit syntax

Table 15 shows an example residual codec syntax.

Table 15 residual codec syntax.

3. Examples of problems addressed by embodiments

The current design has the following problems:

1. the ISP cannot be enabled when multiple reference rows (MRLs) are enabled.

2. Transform Skip (TS) cannot be enabled when using ISPs. However, enabling the ISP and TS simultaneously may achieve similar functionality as BDPCM without adding additional modules for processing the BDPCM.

3. Each sub-partition signals Delta QP, which results in it being signaled multiple times for the ISP codec block.

4. The ISP mode (e.g., intra _ sub _ modes _ flag) is signaled to be enabled when the width or height of a block is not greater than maxtsizey, and the partition direction (i.e., partition type, horizontal/vertical direction) is signaled when neither the width nor the height is greater than maxtsizey. If one of them is larger than MaxTbSizeY, the following applies:

a. if the height is greater than MaxTbSizeY, then horizontal partitioning is used (i.e., intra _ sub _ partitions _ split _ flag [ x0] [ y0] is inferred to be equal to 0).

b. Otherwise (width greater than MaxTbSizeY), vertical partitioning will be used (i.e., intra _ sub _ partitions _ split _ flag [ x0] [ y0] is inferred to be equal to 1).

Such a design is based on the following assumptions: only the width or height may be twice as large as MaxTbSizeY, which limits flexibility. When the maximum transform size is set to, for example, 32x32, and the CU size is, for example, 128x128, it will be divided into 4 128x32 sub-partitions according to the rule. However, when the maximum transform size is 32x32, one 128x32 sub-partition is not allowed to be coded in VVC. How to handle the situation is unknown.

ISP and sub-block transformations are both considered as implicit MTS, since no signalling is required to inform the transformation matrix. The sub-block transform may support block sizes up to 64x64 when MaxSbtSize. However, the setting of the implicit MTS only checks whether Max (width, height) is less than or equal to 32. In addition, when cu _ sbt _ flag is equal to 1, the implicit MTS is automatically set to 1, so there is no need to check the transform size.

TS is part of MTS. However, signaling enabling/disabling of TS and signaling of the maximum TS size is signaled in the PPS. The MTS enable/disable flag is signaled in the SPS.

7. Redundancy checks for block sizes are determined in current VVC designs for signaling transform _ skip _ flag and tu _ mts _ idx.

4. Example embodiments and techniques

The following list of embodiments should be considered as examples to explain the general concept. These examples should not be construed in a narrow sense. Furthermore, these embodiments may be combined in any manner.

In the following description, a block size is represented by W × H, where W is the block width and H is the block height. The maximum transform block size is represented by MaxTbW × MaxTbH, where MaxTbW and MaxTbH are the width and height, respectively, of the maximum transform block. The size of the smallest transform block is denoted by MinTbW MinTbH, where MinTbW and MinTbH are the width and height, respectively, of the smallest transform block. It should be noted that MRL may represent those techniques that use non-adjacent reference lines in the current picture to predict the current block, while ALWIP may represent those techniques that use a matrix-based intra prediction method. They are not limited to those mentioned in the prior art.

With respect to an ISP:

1. it is proposed that both intra sub-block partitioning (ISP) mode and multi-reference line (MRL) mode (e.g., the reference line may not be the closest one) may be enabled to encode a block.

a. In one example, all sub-partitions use the same reference row index for intra prediction.

b. Alternatively, only the first K (e.g., K ═ 1) sub-partition follows the reference row index (e.g., signaled in the bitstream). The remaining sub-partitions still use the closest reference row for intra prediction.

c. In one example, whether MRL is applied to the remaining sub-partitions (e.g., the sub-partitions other than the first sub-partition) may depend on the partition direction in the ISP or/and the intra prediction mode or/and the dimensions of the block.

i. For example, if a block is divided in the horizontal direction in the ISP, MRL may be applied to the remaining sub-partitions only if the lower left or/and upper neighboring reference (reconstructed) samples of the block are used in intra prediction of the first sub-partition. For example, the prediction mode is less than or equal to 50 in fig. 2.

For example, if a block is divided horizontally in the ISP, MRL may not be applied to the remaining sub-partitions when the upper right neighboring reference (reconstructed) samples of the block are used in intra prediction of the first sub-partition. For example, the prediction mode is greater than 50 in fig. 2.

For example, if a block is partitioned in the vertical direction in the ISP, MRL may be applied to the remaining sub-partitions only if the left or/and upper right neighboring reference (reconstructed) samples of the block are used in intra prediction of the first sub-partition. For example, the prediction mode is greater than or equal to 18 in fig. 2.

For example, if a block is partitioned in the vertical direction in the ISP, MRL may not be applied to the remaining sub-partitions when the lower left neighboring reference (reconstructed) samples of the block are used in intra prediction of the first sub-partition. For example, in fig. 2 the prediction mode is less than 18.

2. An indication of ISP mode information (e.g., on/off, split direction) may be signaled before the MRL related information is signaled.

a. In one example, when ISP mode is enabled for a block, signaling of MRL related information, such as reference to a row index, may be skipped.

i. Alternatively, further, the reference row index is considered to be 0.

3. One block may enable both the ALWIP and ISP simultaneously.

a. Alternatively, in addition, the selection of a matrix for a sub-partition may depend on the intra mode and/or the dimensions of the sub-partition.

b. Alternatively, in addition, an indication of the ALWIP mode (e.g., intra _ lwip _ flag and related intra modes) and an indication of the ISP mode (e.g., intra _ sub _ modes _ flag and intra _ sub _ split _ flag)

4. One block may enable both Transform Skip (TS) and ISP simultaneously.

a. Alternatively, further, even when ISP mode is enabled (e.g., intrasubpartitionsplit type is not equal to ISP _ NO _ SPLIT), an indication to enable/disable the transition skip mode may be further signaled.

b. Alternatively, the indication of whether to signal the enabling/disabling of the transform skip mode may depend on whether the video content is screen content.

i. In one example, it may depend on the flag signaled in the picture/slice group/slice/brick level.

in one example, if the video content is screen content, an indication to enable/disable the transform skip mode may be signaled. Alternatively, if the video content is camera content, the indication to enable/disable the transform skip mode may be skipped and the TS mode disabled for the ISP codec block.

5. It is suggested that only one quantization parameter and/or one quantization step size and/or one scaling matrix may be allowed for the ISP codec block. That is, all sub-partitions share the same quantization information.

a. In one example, one quantization parameter may be represented by cu _ qp _ delta _ abs and cu _ qp _ delta _ sign _ flag.

b. In one example, the quantization parameter information may be signaled for the ISP codec block only if there is at least one coefficient not equal to zero in at least one sub-partition.

c. In one example, the quantization parameter and/or one quantization step size and/or one scaling matrix may be signaled once for the entire ISP codec block, rather than for each sub-partition.

i. In one example, the information is signaled with the mth sub-partition only if there is at least one coefficient in the mth sub-partition that is not equal to zero.

in one example, the information may be signaled with the first sub-partition in an encoding/decoding order.

in one example, the information may be signaled along with the last sub-partition in coding/decoding order.

in one example, information may be signaled with the mth sub-partition in an encoding/decoding order, where m is no greater than the total number of allowed sub-partitions.

6. The reference samples suggested to be located in the first sub-partition to predict the second sub-partition in the ISP coded block may be further modified (e.g., may be filtered) before being used as a prediction.

a. In one example, whether or not the reference samples are modified (e.g., filtered) before being used as a prediction may depend on the width and/or height of the block.

b. In one example, whether or not the reference samples are modified (e.g., filtered) before being used as a prediction may depend on the intra-prediction mode.

Indications of MaxTbW and/or MaxTbH may be signaled in the sequence/picture/slice group/slice/brick level.

a. In one example, they may be signaled in SPS/VPS/PPS/picture header/slice group header, etc.

Maxtbw and/or MaxTbH may be set to different numbers in different profiles/levels of the video codec standard.

MinTbW and/or MinTbH may be signaled in the sequence/picture/slice/brick level.

9. A hybrid partitioning direction may be enabled for ISP codec blocks, where blocks may be partitioned for horizontal and vertical directions.

a. In one example, a binary value (e.g., intra _ sub _ partitions _ split _ flag) of a partition direction for ISP mode codec may be replaced by an index of the partition direction.

b. In one example, the set of allowed partitioning directions may depend on the block dimension.

i. Alternatively, an indication may be signaled that the set of directions is allowed to be divided.

c. In one example, the set of allowable partition directions may depend on the intra prediction mode.

d. In one example, when W/MaxTbW and H/MaxTbH are both greater than M (e.g., M ═ 1), a hybrid partitioning direction may be enabled in which horizontal and vertical partitioning may occur.

i. An example of the hybrid division direction is shown in fig. 12.

Alternatively, when W/MaxTbW or H/MaxTbH is greater than M (e.g., M ═ 1), the hybrid partitioning direction may be enabled.

in one example, when applying a hybrid ISP, the blocks may be divided horizontally first, then vertically.

1) Alternatively, the blocks may be divided vertically and then horizontally when the hybrid ISP is applied. Fig. 17 shows an example of a hybrid partitioning (also referred to as a quadtree partitioning).

10. Whether and/or how to apply ISPs on blocks may depend on the block dimension W × H and/or the relationship between the maximum and/or minimum transform block dimensions.

a. In one example, if W/MinTbW and H/MinTbH are both equal to 1, then the ISP is disabled.

b. In one example, how the blocks are partitioned may depend on the minimum transform block size.

i. In one example, if W/MinTbW equals K (K >1) and H/MinTbH equals 1, then the ISP may be enabled and vertical partitioning applied.

in one example, if W/MinTbW equals 1 and H/MinTbH equals K (K >1), then ISP may be enabled and horizontal partitioning applied. Alternatively, furthermore, no signaling of the prediction direction is required.

Alternatively, furthermore, no signaling of the prediction direction is required.

Alternatively, in addition, a block may be divided into K sub-partitions.

c. In one example, ISP mode is disabled when W/MaxTbW is greater than 1 or H/MaxTbH is greater than 1.

i. Alternatively, the ISP mode is disabled when W × H/(MaxTbW MaxTbH) is greater than a threshold (e.g., 4).

Alternatively, ISP mode is disabled when both W/MaxTbW and H/MaxTbH are greater than 1.

Alternatively, ISP mode is disabled when W/MaxTbW or H/MaxTbH is greater than a threshold (e.g., 2 or 4).

Alternatively, ISP mode is disabled when both W/MaxTbW and H/MaxTbH are greater than a threshold (e.g., 2 or 4).

d. In one example, ISP mode may be enabled when both W/MaxTbW and H/MaxTbH are greater than (or not less than) the first threshold and not greater than (or less than) the second threshold.

i. Alternatively, ISP mode may be enabled when both W/MaxTbW and H/MaxTbH are greater than the first threshold and less than the second threshold.

in one example, the first threshold and the second threshold are 1 and 4, respectively.

Alternatively, also, signaling of the partition direction (e.g., intra _ sub _ partitions _ split _ flag) may be skipped and the blocks may be partitioned according to certain rules.

1) In one example, quadtree partitioning may be applied first, followed by horizontal binary tree partitioning, and then vertical binary tree partitioning.

2) In one example, the partitioning of one partition tree may be terminated once the width reaches MaxTbW or the height reaches MaxTbH.

a. Alternatively, the partitioning of one partition tree may be terminated once the width reaches MaxTbW and the height reaches MaxTbH.

b. Alternatively, the partitioning of a partition tree may be terminated once the width reaches MaxTbW/M and the height reaches MaxTbH/N, where M and N are two positive integers.

e. When the ISP mode is disabled, signaling of relevant information such as intra _ sub _ modes _ flag is skipped.

11. In one example, more than 4 sub-partitions and/or more than one partitioning direction may be enabled (e.g., for horizontal and vertical partitioning).

i. Alternatively, the above method may be enabled under certain conditions.

in one example, when W/MaxTbW >4 and/or H/MaxTbH > 4.

Regarding the MTS:

12. it is proposed that only two transforms (and corresponding inverse transforms) for the explicit MTS design are reserved. For example, the two transforms may be DCT-II and DST-VII (and corresponding inverse transforms).

a. In one example, there are only two choices in the transformation choice. Alternatively, and in addition, the TS mode may be a third choice, if applicable.

i. In one example, DCT-II is one choice for horizontal and vertical transforms; and the other is DST-VII.

One bit can be coded to indicate whether which of the two transforms is used.

b. Alternatively, there are four choices in the transform selection. Alternatively, and in addition, the TS mode may be the fourth choice, if applicable.

i. The selection comprises the following steps: DCT-II/DST-VII for horizontal and vertical transforms; the joint use of DCT-II and DST-VII, each for horizontal or vertical transforms.

in one example, four choices may be coded with fixed length coding.

Alternatively, four choices may be coded with truncated unary codes.

1) Some example tabulations for four selected bin strings are as follows:

13. the signaling of the proposed allowed transform set and/or the transform index in the explicit MTS may depend on the block dimension.

a. In one example, DCT-II and DST-VII may be allowed for blocks whose width and/or height are not greater than (or less than) a threshold.

b. In one example, DCT-II, DST-VII, and DCT-VIII may be allowed for blocks whose width and/or height are greater than (or not less than) a threshold.

c. Alternatively, in addition, a transform skip mode may be enabled.

d. In one example, the allowed set of transforms may depend on the codec mode.

i. In one example, two transformation bases (TS and DST-VII) may be allowed for the IBC codec block.

in one example, two transform bases (DCT-II and DST-VII) may be allowed, or three transform bases (TS, DCT-II and DST-VII) may be allowed for blocks that are not IBC-coded.

e. How the transformation index is signaled can vary depending on the allowed set of transformations.

14. An indication of the maximum allowed transition size (non-TS mode) used in the MTS may be signaled.

a. In one example, they may be signaled in sequence/picture/slice group/slice/tile level or other types of video unit level.

i. In one example, they may be signaled in SPS/VPS/PPS/picture header/slice group header, etc.

b. In one example, they may not be signaled, but derived from the allowed maximum TS size.

c. In one example, the indication of the maximum allowed transform size (non-TS mode) may control the maximum TS size and the maximum size used in other transform matrices.

i. Alternatively, further, the maximum size of the signal for the non-TS mode and the TS mode separately is not required.

d. In one example, the indication of the maximum allowed transform size may control implicit and explicit MTS transform sizes.

i. Alternatively, also, whether to apply implicit MTS may depend on the signal size.

15. It is proposed to align the maximum allowed transform size used in MTS (non-TS mode) with the maximum allowed block size used in TS mode.

a. In one example, the maximum allowed transform size used in MTS (non-TS mode) and the maximum allowed block size used in TS mode may be the same number.

16. It is proposed that the derivation of the implicit MTS enable flag is independent of the block dimension.

a. Alternatively, in addition, the checking of block sizes in the derivation of the implicit MTS enable flag is skipped.

17. The shared condition check on the block size before signaling MTS information (e.g., transform _ skip _ flag and tu _ MTS _ idx) may be removed.

a. In one example, the MTS information may be further signaled if all of the following sharing conditions are true. Otherwise, no MTS information needs to be signaled.

-tu_cbf_luma[x0][y0]

-treeType！＝DUAL_TREE_CHROMA

-！cu_sbt_flag

b. Alternatively, in addition, when the sharing condition check of other rules (e.g., above) returns true, the condition check of the block dimension compared to the allowed maximum TS size may be applied before signaling the transform _ skip _ flag; and signaling tu _ MTS _ idx is preceded by applying a conditional check of the block dimension compared to the allowed maximum allowed MTS size (e.g., fixed at 32x 32).

18. The shared conditional check of the block dimension before signaling MTS information (e.g., transform _ skip _ flag and tu _ MTS _ idx) remains unchanged, while the conditional check of the block dimension before signaling the transform matrix index (non-TS mode) can be removed.

4.1 example setting of implicit MTS flag

Some proposed changes to VVC working draft version 5 jfet _ N1001_ v2 are described in this example. The underlined parts represent additions to the working draft, while the deleted parts represent suggested deletions.

Typically, the inputs to the transform process for scaling the transform coefficients are:

-specifying a luminance position (xTbY, yTbY) of an upper left sample of the current luminance transform block relative to an upper left luminance sample of the current picture,

a variable nTbW specifying the width of the current transform block,

a variable nTbH specifying the height of the current transform block,

a variable cIdx specifying the color component of the current block,

an array d [ x ] [ y ] of (nTbW) x (nTbH) having scaling transform coefficients of x 0.. nTbW-1, y 0.. nTbH-1.

The output of this process is an (nTbW) x (nTbH) array r [ x ] [ y ] of residual samples, where x ═ 0.. nTbW-1, y ═ 0.. nTbH-1.

The variable implicitimtsenenabled is derived as follows:

-if sps _ mts _ enabled _ flag is equal to 1 and one of the following conditions is met, then imitictsEnable is set equal to 1:

IntraSubPartitionsSplitType not equal to ISP _ NO _ SPLIT

-cu sbt flag is equal to 1 and Max (nTbW, nTbH) is less than or equal toMaxSBTSize

That is, Max (nTbW, nTbH) is compared with MaxSBTSize, instead of fixing the number 32.

-otherwise, implicitMtsEnabled is set equal to 0.

-if cIdx is greater than 0, trTypeHor and trTypeVer are set equal to 0.

Otherwise, if implicitMtsEnabled is equal to 1, then the following applies:

-if intrasubpartitionsplittype does not equal ISP _ NO _ SPLIT, then trTypeHor and trTypeVer are specified according to intraPredMode.

Otherwise, if cu _ sbt _ flag is equal to 1, the trTypeHor and trTypeVer are specified according to cu _ sbt _ horizontal _ flag and cu _ sbt _ pos _ flag.

Else (sps _ explicit _ mts _ intra _ enabled _ flag and sps _ explicit _ mts _ inter _ enabled _ flag equal to 0 andCuPredMode[xTbY][yTbY]equal to MODE _ INTRA) trTypeHor and trTypeVer are derived as follows:

trTypeHor＝(nTbW>＝4&&nTbW<＝16&&nTbW<＝nTbH)？1:0(8 1030)

trTypeVer＝(nTbH>＝4&&nTbH<＝16&&nTbH<＝nTbW)？1:0(8 1031)

else, trTypeHor and trTypeVer are specified in table 813 according to tu _ mts _ idx [ xTbY ] [ yTbY ].

Alternatively, the conditional check that 'cu _ sbt _ flag is equal to 1 and Max (nTbW, nTbH) is less than or equal to 32' when implicitmenabled is determined may be replaced by 'cu _ sbt _ flag equal to 1'.

4.2 example setting of explicit MTS flag

Some proposed changes to VVC working draft version 5 jfet _ N1001_ v2 are described in this example. The underlined parts represent additions to the working draft, while the deleted parts represent suggested deletions. This section provides an example of redundancy check removal during the MTS signaling procedure.

Alternatively, the following may apply:

fig. 16 is a block diagram of the video processing apparatus 1600. Apparatus 1600 may be used to implement one or more of the methods described herein. The apparatus 1600 may be embodied in a smartphone, tablet, computer, internet of things (IoT) receiver, or the like. The apparatus 1600 may include one or more processors 1602, one or more memories 1604, and video processing hardware 1606. The processor(s) 1602 may be configured to implement one or more of the methods described in this document. The memory(s) 1604 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 1606 may be used to implement some of the techniques described in this document in hardware circuits.

Fig. 18 is a flow diagram of a video processing method 1800 in accordance with one or more examples of the present technique. The method 1800 includes, at operation 1802, partitioning a block of video data into sub-blocks using a partitioning mode. The method 1800 includes performing prediction of a sub-block using at least one line of reference video data that is not adjacent to the current sub-block at operation 1804. The method 1800 also includes generating residual signals for the sub-blocks based on the prediction at operation 1806.

Fig. 19 is a flow diagram of a video processing method 1900 in accordance with one or more examples of the present technology. The method 1900 includes, at operation 1902, partitioning a block of video data into sub-blocks using a partitioning mode. The method 1900 includes performing prediction of the sub-blocks by calculating a matrix-vector product based on the reference signal of each sub-block at operation 1904. The method 1900 also includes generating residual signals for the sub-blocks based on the prediction at operation 1906.

Fig. 20 is a flow diagram of a video processing method 2000 in accordance with one or more examples of the present technique. The method 2000 includes, at operation 2002, performing prediction of a block of video data to generate a residual signal. The method 2000 includes performing an explicit transform of the residual signal using one of two transforms at operation 2004. The method 2000 includes encoding an output from the implicit transform at operation 2006.

Additional embodiments and techniques are described in the following examples.

1. A method of video processing, comprising: partitioning a block of video data into sub-blocks using a partitioning pattern; performing prediction of a current sub-block of the sub-blocks using at least one line of reference video data that is not adjacent to the current sub-block; and generating a residual signal of the current sub-block based on the prediction.

2. The method of example 1, wherein all sub-blocks are predicted using the same reference row index of the reference video data.

3. The method of example 1, wherein a first sub-block of the sub-blocks uses a first row index of reference data and remaining sub-blocks use a second row index of reference video data, the second row index corresponding to a row of reference video data that is closest to the sub-block.

4. The method of example 1, wherein a first sub-block of the sub-blocks uses a first row index of reference data and remaining sub-blocks use a second row index of reference video data, the second row index determined based on a partitioning direction of the sub-blocks, a prediction mode, or a dimension of the block.

Other embodiments of examples 1-4 are described in

items

1 and 2 of subsection 4.

5. A method of video processing, comprising: partitioning a block of video data into sub-blocks using a partitioning pattern; performing prediction of the subblocks by calculating a matrix vector product based on a reference signal of each subblock; and generating residual signals of the sub-blocks based on the prediction.

6. The method of example 5, wherein the matrix vector for the sub-block is selected based on an intra mode or a dimension of the sub-block.

Other embodiments of examples 5-6 are described in items 3-4 of subsection 4.

7. A method of video processing, comprising: receiving a block of video data partitioned into sub-blocks using a partitioning pattern; performing prediction of a current sub-block of the sub-blocks using at least one line of reference video data that is not adjacent to the current sub-frame; and reconstructing a current sub-block using the prediction.

8. The method of example 7, wherein all sub-blocks are predicted using the same reference row index of the reference video data.

9. The method of example 7, wherein a first sub-block of the sub-blocks uses a first row index of reference data and remaining sub-blocks use a second row index of reference video data, the second row index corresponding to a row of reference video data that is closest to the sub-block.

10. The method of example 7, wherein a first sub-block of the sub-blocks uses a first row index of reference data and remaining sub-blocks use a second row index of reference video data, the second row index determined based on a partitioning direction of the sub-blocks, a prediction mode, or a dimension of the block.

Other embodiments of examples 7-10 are described in

items

1 and 2 of subsection 4.

11. A method of video processing, comprising: receiving a block of video data partitioned into sub-blocks using a partitioning pattern; performing prediction of the subblocks by calculating a matrix vector product based on a reference signal of each subblock; and reconstructing the sub-blocks using the prediction.

12. The method of example 11, wherein the matrix vector for the sub-block is selected based on an intra mode or a dimension of the sub-block.

Other embodiments of examples 1-4 are described in items 3-4 of subsection 4.

13. A method of video processing, comprising: performing prediction on a block of video data partitioned into sub-blocks using a partitioning mode; and a residual signal based on the predictive transform sub-blocks, wherein a maximum transform block dimension or a minimum transform block dimension is indicated in a level of a sequence, a picture, a slice group, a slice, or a brick in a bitstream representing a block of video data.

14. A method of video processing, comprising: receiving a bitstream representing a block of video data partitioned into sub-blocks using a partitioning pattern; performing an inverse transform on the residual signals of the sub-blocks, wherein a maximum transform block dimension or a minimum transform block dimension is indicated in a sequence, picture, slice group, slice or brick level in the bitstream; and reconstructing the sub-blocks using the output from the inverse transform.

15. The method of example 13 or 14, wherein the maximum transform block dimension or the minimum transform block dimension is set to different values in different profiles, levels, or levels.

Other embodiments of examples 13-15 are described in items 7-8 of subsection 4.

16. A method of video processing, comprising: receiving or transmitting a bitstream representing a block of video data for performing video processing, wherein the block of video data is partitioned into sub-blocks using a partitioning mode, and residual signals of the sub-blocks are quantized in the bitstream, and wherein the sub-blocks share the same quantization information.

17. The method of example 16, wherein the quantization information comprises a quantization parameter, a quantization step size, or a scaling matrix.

18. The method of example 16, wherein the quantization information for all sub-blocks in the block of video data is coded and decoded once in the bitstream.

Other embodiments of examples 16-18 are described in item 5 of subsection 4.

19. A method of video processing, comprising: performing prediction on a block of video data partitioned into sub-blocks using a partitioning mode, wherein reference samples in a first sub-block are modified before being used to perform prediction of a second sub-block; and encoding or reconstructing a block of video data based on the prediction.

Other embodiments of example 19 are described in item 6 of subsection 4.

20. A method of video processing, comprising: performing prediction on a block of video data partitioned into sub-blocks using a partitioning mode, wherein the sub-blocks are partitioned in a plurality of partitioning directions; and encoding or reconstructing a block of video data based on the prediction.

21. The method of example 20, the plurality of segmentation directions determined by dimensions of a block.

Other embodiments of examples 20-21 are described in item 9 of subsection 4.

22. The method of any of examples 1 to 21, wherein the sub-blocks are partitioned based on a minimum transform block dimension or a maximum transform block dimension.

Other embodiments of example 22 are described in items 10 and 11 of subsection 4.

23. A method of video processing, comprising: performing prediction of a block of video data to generate a residual signal; performing an explicit transform of the residual signal using one of two transforms; and encoding the output from the implicit transform.

24. The method of example 23, comprising: the transform options are coded in a bitstream representing the block of video data based on the two transformed variants.

Other embodiments of examples 23-24 are described in items 12-13 of subsection 4.

25. The method of example 22 or 23, comprising: the information about the explicit transformation is signaled without checking the dimensionality of the video data blocks.

Other embodiments of example 25 are described in items 16-18 of subsection 4.

26. A method of video processing, comprising: receiving a block of video data partitioned into one or more sub-blocks; performing an explicit transform of a block of video data using one of two inverse transforms; and reconstructing the block of video data based on the implicit transform.

27. The method of any of examples 23 to 26, wherein the explicit transformation is performed in one or more transformation directions including a horizontal direction and a vertical direction.

28. The method of example 27, wherein different transform directions use different transforms.

29. The method of example 27, wherein different transform directions use the same transform.

Other embodiments of examples 27-29 are described in items 12-13 of subsection 4.

30. The method of any of examples 23 to 29, wherein the maximum allowed transform size is coded in a level of a sequence, a picture, a slice, or a brick in a bitstream representing the block of video data.

31. The method of any of examples 23 to 30, comprising: the maximum allowed transform size is derived based on the allowed maximum transform skip size.

Other embodiments of example 30 are described in items 14-15 of subsection 4.

32. A video processing apparatus comprising a processor configured to perform one or more of examples 1 to 31.

33. A computer-readable medium having code stored thereon, which, when executed by a processor, causes the processor to perform the method of any one or more of examples 1-31.

Fig. 21 is a flow diagram of a video processing method 2100, according to one or more examples of the present technology. The method 2100 includes, at 2102, enabling an intra sub-block partitioning (ISP) mode for a block of video for a transition between the block and a bitstream representation of the block, wherein the block is divided into a plurality of sub-partitions based on the ISP mode; at 2104, a second mode different from the ISP mode of the block is enabled; and at 2106, performing a conversion based on the ISP mode and the second mode.

In some examples, the second mode is a multiple reference row (MRL) mode.

In some examples, in MRL mode, a reference row that is not the closest reference row of the plurality of reference rows may be used for intra prediction of the block.

In some examples, all sub-partitions of a block use the same reference row index to introduce intra prediction of a row block.

In some examples, only the first K sub-partitions of all sub-partitions of the block are intra-predicted using the same reference row index and the remaining sub-partitions are intra-predicted using the closest reference row, K being an integer.

In some examples, K ═ 1.

In some examples, the reference row index is signaled in the bitstream.

In some examples, whether MRL mode is applied to the remaining sub-partitions depends on the partition direction in ISP mode or/and intra prediction mode or/and size of the block.

In some examples, if a block is divided in the horizontal direction in the ISP mode, the MRL mode is applied to the remaining sub-partitions only when the lower left or/and upper neighboring reference samples of the block are used in the intra prediction of the first sub-partition.

In some examples, if a block is divided in the horizontal direction in the ISP mode, the MRL mode is not applied to the remaining sub-partitions when the upper right neighboring reference sample point of the block is used in the intra prediction of the first sub-partition.

In some examples, if a block is divided in the vertical direction in the ISP mode, the MRL mode is applied to the remaining sub-partitions only when the left or/and upper right neighboring reference samples of the block are used in the intra prediction of the first sub-partition.

In some examples, if a block is divided in the vertical direction in the ISP mode, the MRL mode is not applied to the remaining sub-partitions when the lower left adjacent reference samples of the block are used in the intra prediction of the first sub-partition.

In some examples, the indication of ISP mode information is signaled before the signaling of MRL mode related information.

In some examples, the ISP mode information includes at least one of an on/off flag and a division direction, and the MRL mode related information includes a reference row index.

In some examples, when ISP mode is enabled for a block, signaling of MRL related information is skipped.

In some examples, the reference row index is considered to be 0 when ISP mode is enabled for the block.

In some examples, the second mode is a matrix-based intra prediction (MIP) mode.

In some examples, the selection of a matrix for a sub-partition depends on the intra mode and/or the size of the sub-partition.

In some examples, the indication of MIP mode and the indication of ISP mode are signaled for a chunk.

In some examples, the second mode is a Transform Skip (TS) mode.

In some examples, the indication to enable/disable TS mode is further signaled even if ISP mode is enabled.

In some examples, whether to signal an indication to enable/disable TS mode depends on whether the video content of the video is screen content.

In some examples, the indication of whether to signal enabling/disabling of TS mode depends on a flag signaled in at least one of a picture, a slice group, a slice, and a brick level.

In some examples, if the video content is screen content, an indication to enable/disable the transform skip mode is signaled.

In some examples, if the video content is camera content, the indication to transform skip mode is skipped enabled/disabled and TS mode is disabled for the block.

In some examples, all sub-partitions share the same quantization information, including at least one of a quantization parameter, a quantization step size, and a scaling matrix.

In some examples, the quantization parameter is represented by cu _ qp _ delta _ abs and cu _ qp _ delta _ sign _ flag.

In some examples, the quantization information is signaled for the block only if there is at least one coefficient not equal to zero in at least one sub-partition.

In some examples, the quantization information is signaled once for the entire block, rather than signaling the quantization information for each sub-partition.

In some examples, the quantization information is signaled with the mth sub-partition only when there is at least one coefficient not equal to zero in the mth sub-partition, where m is an integer.

In some examples, the quantization information is signaled with the first sub-partition in encoding or decoding order.

In some examples, the quantization information is signaled with the last sub-partition in encoding or decoding order.

In some examples, the quantization information is signaled in encoding or decoding order with the mth sub-partition, where m is an integer no greater than the total number of allowed sub-partitions.

Fig. 22 is a flow diagram of a video processing method 2200 in accordance with one or more examples of the present technique. Method 2200 includes, at 2202, enabling, for a transition between a block of video and a bitstream representation of the block, an intra sub-block partitioning (ISP) mode for the block, wherein the block is divided into a plurality of sub-partitions based on the ISP mode, and reference samples located in a first sub-partition to predict a second sub-partition in the block are further modified before being used as a prediction; at 2204, the conversion is performed based on the ISP mode.

In some examples, the reference samples are filtered before being used as a prediction.

In some examples, whether the reference samples are modified before being used as a prediction depends on the width and/or height of the block.

In some examples, whether the reference sample is modified before being used as a reference depends on the intra prediction mode of the block.

In some examples, the block size of the block is represented by W × H, where W is the block width and H is the block height, the maximum transform block size of the block is represented by MaxTbW × MaxTbH, where MaxTbW and MaxTbH are the maximum transform block width and maximum transform block height, respectively, and the minimum transform block size of the block is represented by MinTbW × MinTbH, where MinTbW and MinTbH are the minimum transform block width and minimum transform block height, respectively.

In some examples, the indication of MaxTbW and/or MaxTbH is signaled in at least one of a sequence, a picture, a slice, a group of slices, a slice, and a brick level.

In some examples, the indication of MaxTbW and/or MaxTbH is signaled in at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), and a Picture Parameter Set (PPS), a picture header, a slice header, and a slice group header.

In some examples, MaxTbW and/or MaxTbH are set to different numbers in different profile levels or levels of the video codec standard.

In some examples, the indication of MinTbW and/or MinTbH is signaled in at least one of a sequence, a picture, a slice, a group of slices, a slice, and a brick level.

In some examples, the indication of MinTbW and/or MinTbH is signaled in at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), and a Picture Parameter Set (PPS), a picture header, a slice header, and a slice group header.

In some examples, MinTbW and/or MinTbH are set to different numbers in different profile levels or levels of the video codec standard.

Fig. 23 is a flow diagram of a video processing method 2300, according to one or more examples of the present technology. Method 2200 includes, at 2302, enabling an intra sub-block partitioning (ISP) mode for a block of video for a transition between the block and a bitstream representation of the block, wherein a hybrid partitioning direction is enabled in the ISP mode and the block is partitioned into a plurality of sub-partitions in horizontal and vertical directions; at 2304, a conversion is performed based on the ISP mode.

In some examples, the binary value of the partition direction for ISP mode coding is replaced by an index of the partition direction.

In some examples, the set of allowed partition directions depends on the block size.

In some examples, the signaling allows for an indication of the set of directions to be divided.

In some examples, the set of allowable partition directions depends on the intra prediction mode of the block.

In some examples, the hybrid partitioning direction is enabled when both W/MaxTbW and H/MaxTbH are greater than M, M being an integer.

In some examples, the hybrid partitioning direction is enabled when W/MaxTbW or H/MaxTbH is greater than M, M being an integer.

In some examples, M ═ 1.

In some examples, the blocks are partitioned by using quadtree partitioning.

In some examples, when applying the hybrid-divide direction, the blocks are first divided horizontally and then divided vertically.

In some examples, when applying the hybrid partitioning direction, the blocks are first partitioned vertically and then horizontally.

In some examples, whether and/or how to apply the ISP pattern on the block depends on the relationship between the block size of the block wxh and/or the maximum transform block size MaxTbW MaxTbH and/or the minimum transform block size MinTbW MinTbH.

In some examples, if W/MinTbW and H/MinTbH are both equal to 1, then ISP mode is disabled for the block.

In some examples, how the blocks are partitioned depends on the minimum transform block size of the block.

In some examples, if W/MinTbW is equal to K and H/MinTbH is equal to 1, then ISP mode is enabled for the block and vertical partitioning is applied to the block, K being an integer greater than 1.

In some examples, if W/MinTbW is equal to 1 and H/MinTbH is equal to K, then ISP mode is enabled for the block and horizontal partitioning is applied to the block, K being an integer greater than 1.

In some examples, the prediction direction need not be signaled.

In some examples, a block is divided into K sub-partitions.

In some examples, ISP mode is disabled for blocks when W/MaxTbW is greater than 1 or H/MaxTbH is greater than 1.

In some examples, ISP mode is disabled for a block when W × H/(MaxTbW) is greater than a threshold, wherein the threshold is 4.

In some examples, ISP mode is disabled for blocks when both W/MaxTbW and H/MaxTbH are greater than 1.

In some examples, ISP mode is disabled for blocks when W/MaxTbW or H/MaxTbH is greater than a threshold, where the threshold is 2 or 4.

In some examples, ISP mode is disabled for blocks when both W/MaxTbW and H/MaxTbH are greater than or equal to a threshold, where the threshold is 2 or 4.

In some examples, ISP mode is disabled for the block when both W/MaxTbW and H/MaxTbH are greater than or equal to the first threshold and less than or equal to the second threshold.

In some examples, ISP mode is disabled for the block when both W/MaxTbW and H/MaxTbH are greater than the first threshold and less than the second threshold.

In some examples, the first threshold is 1 and the second threshold is 4.

In some examples, signaling of the partition direction is skipped and the blocks are partitioned according to certain rules.

In some examples, the quadtree partitioning is applied to the blocks first, then the horizontal binary tree partitioning is applied, and then the vertical binary tree partitioning is applied.

In some examples, the partitioning of one partition tree is terminated once the width reaches MaxTbW or the height reaches MaxTbH.

In some examples, the partitioning of one partition tree is terminated once the width reaches MaxTbW and the height reaches MaxTbH.

In some examples, once the width reaches MaxTbW/M and the height reaches MaxTbH/N, the partitioning of one partition tree is terminated, where M and N are two positive integers.

In some examples, when the ISP mode is disabled for a block, signaling of relevant information including intra _ sub _ modes _ flag is skipped.

In some examples, when W/MaxTbW >4 and/or H/MaxTbH >4, more than 4 sub-partitions and/or more than one partition direction are enabled.

Fig. 24 is a flow diagram of a video processing method 2400 in accordance with one or more examples of the present technique. Method 2400 includes, at 2402, determining, for a transition between a block of video and a bitstream representation of the block, a Multiple Transform Selection (MTS) scheme associated with the block, wherein the MTS scheme is modified to allow for a partial transform and a corresponding inverse transform; at 2404, a conversion is performed based on the determined MTS scheme.

In some examples, the MTS scheme is an explicit MTS, where the transform index of the MTS is signaled in the bitstream of the block.

In some examples, the MTS scheme is modified to allow only two transforms, where the two transforms are DCT-II and DST-VII.

In some examples, the MTS scheme includes two modes in terms of transform selection.

In some examples, when the TS mode is applicable, the MTS scheme includes a third mode of a Transform Skip (TS) mode in addition to the two modes of the transform selection.

In some examples, a first mode of the two modes is DCT-II for horizontal and vertical transforms of the block, and a second mode of the two modes is DST-VII for horizontal and vertical transforms of the block.

In some examples, one bit is coded to indicate which of the two modes is used.

In some examples, the MTS scheme includes four modes in terms of transform selection.

In some examples, when the TS mode is applicable, the MTS scheme includes a fifth mode of a Transform Skip (TS) mode in addition to the two modes of the transform selection.

In some examples, a first mode of the four modes is DCT-II for horizontal and vertical transforms of the block, a second mode of the four modes is DST-VII for horizontal and vertical transforms of the block, a third mode of the four modes is DCT-II for horizontal transforms of the block and DST-VII for vertical transforms of the block, and a fourth mode of the four modes is DST-VII for horizontal transforms of the block and DCT-II for vertical transforms of the block.

In some examples, four modes are coded with fixed length coding.

In some examples, four modes are coded using truncated unary codes.

In some examples, the signaling of allowed transform sets and/or transform indices in an explicit MTS depends on block size.

In some examples, DCT-II and DST-VII are allowed for blocks with widths and/or heights less than or equal to a threshold.

In some examples, DCT-II, DST-VII, and DCT-VIII are allowed for blocks having a width and/or height greater than or equal to a threshold.

In some examples, a Transform Skip (TS) mode is allowed.

In some examples, the allowed transform set depends on the codec mode of the block.

In some examples, a transform set including two transform bases of TS mode and DST-VII is allowed for a block coded in Intra Block Copy (IBC) mode.

In some examples, non-IBC mode coded blocks allow for a set of transforms that includes two transform bases for DCT-II and DST-VII, or three transform bases for TS mode, DCT-II and DST-VII.

In some examples, how to signal that the transformation index changes according to the allowed set of transformations.

In some examples, an indication of a maximum allowed transform size for use in a non-TS mode of the MTS scheme is signaled.

In some examples, the indication is signaled in at least one of a sequence, a picture, a slice group, a slice, a tile level, or other type of video unit level.

In some examples, the indication is signaled in a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), and a Picture Parameter Set (PPS), a picture header, a slice header, and a slice group header.

In some examples, the indication is derived from an allowed maximum TS size.

In some examples, the indication is used to control the maximum TS size and the maximum size used in other transform matrices.

In some examples, the maximum sizes of the non-TS and TS modes need not be signaled separately.

In some examples, the indication of the maximum allowed transform size is used to control the implicit MTS transform size and the explicit MTS transform size.

In some examples, whether to apply implicit MTS depends on the maximum allowed transform size signaled.

In some examples, the maximum allowed transform size used in the non-TS mode of the MTS scheme is aligned with the maximum allowed transform size used in the TS mode.

In some examples, the maximum allowed transform size used in the non-TS mode of the MTS scheme is the same as the maximum allowed transform size used in the TS mode.

In some examples, the MTS scheme is implicit MTS, where the transform matrix of the MTS is directly derived from the transform block size of the block.

In some examples, the derivation of the implicit MTS enable flag indicating whether to enable implicit MTS is independent of the block size of the block.

In some examples, the check of block sizes in the derivation of the implicit MTS enabled flag is skipped.

In some examples, the shared conditional check of block size before signaling MTS information is removed, the MTS information including transform _ skip _ flag and tu _ MTS _ idx.

In some examples, the MTS information is further signaled if all of the following sharing conditions are true:

-tu_cbf_luma[x0][y0]

-treeType！＝DUAL_TREE_CHROMA

-！cu_sbt_flag，

otherwise, MTS information is not signaled.

In some examples, when certain regular shared conditional checks return true, conditional checks of block size compared to the allowed maximum TS size are applied before the transform _ skip _ flag is signaled; and applying a conditional check of the block size compared to the allowed maximum allowed MTS size before signaling tu _ MTS _ idx.

In some examples, the shared conditional check of the block size before signaling the MTS information remains unchanged, while the conditional check of the block size before signaling the transform matrix index used in the non-TS mode of the MTS is removed.

In some examples, the translation generates a block of video from the bitstream representation.

In some examples, the transform generates a bitstream representation from a block of video.

It will be appreciated that the disclosed techniques may be embodied in a video encoder or decoder to improve compression efficiency using techniques that include the use of a reduced-dimension quadratic transform.

The disclosed and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine readable storage device, a machine readable storage substrate, a memory device, a composition of matter effecting a machine readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any subject matter or claims, but rather as descriptions of features specific to particular embodiments of particular technologies. Some features that are described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claim combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described herein should not be understood as requiring such separation in all embodiments.

Only some implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method for processing video, comprising:

for a transition between a block of video and a bitstream representation of the block, enabling an intra sub-block partitioning (ISP) mode for the block, wherein the block is divided into a plurality of sub-partitions based on the ISP mode;

enabling a second mode different from the ISP mode of the block; and

performing the conversion based on the ISP mode and the second mode.

2. The method of claim 1, wherein the second mode is a multiple reference row (MRL) mode.

3. The method of claim 2, wherein, in the MRL mode, a reference row that is not the closest reference row of the plurality of reference rows is available for intra prediction of the block.

4. The method of any of claims 2-3, wherein all sub-partitions of the block use the same reference row index for intra prediction of the block.

5. The method of any of claims 2-3, wherein only the first K sub-partitions of all sub-partitions of the block are intra-predicted using the same reference row index and the remaining sub-partitions are intra-predicted of the block using the closest reference row, K being an integer.

6. The method of claim 5, wherein K-1.

7. The method according to any of claims 4-6, wherein the reference row index is signaled in a bitstream.

8. The method according to any of claims 5-7, wherein whether or not the MRL mode is applied to the remaining sub-partitions depends on the partition direction in ISP mode or/and intra prediction mode or/and size of the block.

9. The method of claim 8, wherein if the block is divided in a horizontal direction in the ISP mode, the MRL mode is applied to the remaining sub-partitions only when the lower-left or/and upper-neighboring reference samples of the block are used in the intra prediction of the first sub-partition.

10. The method of claim 8, wherein if the block is divided in a horizontal direction in the ISP mode, the MRL mode is not applied to the remaining sub-partitions when the upper right neighboring reference samples of the block are used in the intra prediction of the first sub-partition.

11. The method of claim 8, wherein if the block is divided in a vertical direction in the ISP mode, the MRL mode is applied to the remaining sub-partitions only when left or/and upper right neighboring reference samples of the block are used in intra prediction of the first sub-partition.

12. The method of claim 8, wherein if the block is divided in a vertical direction in the ISP mode, the MRL mode is not applied to the remaining sub-partitions when the lower left neighboring reference sample points of the block are used in the intra prediction of the first sub-partition.

13. The method according to any of claims 1-12, wherein the indication of ISP mode information is signaled before the signaling of MRL mode related information.

14. The method of claim 13, wherein the ISP mode information comprises at least one of an on/off flag and a division direction, and the MRL mode related information comprises a reference row index.

15. The method of claim 14, wherein the signaling of the MRL-related information is skipped when ISP mode is enabled for a block.

16. The method of claim 14, wherein the reference row index is considered to be 0 when ISP mode is enabled for a block.

17. The method of claim 1, wherein the second mode is a matrix-based intra prediction (MIP) mode.

18. The method of claim 17, wherein the selection of a sub-partition matrix depends on the intra mode and/or the size of the sub-partition.

19. The method according to claim 17 or 18, wherein an indication of MIP mode and an indication of ISP mode are signalled for the chunks.

20. The method of claim 1, wherein the second mode is a Transform Skip (TS) mode.

21. The method of claim 20, wherein the indication to enable/disable TS mode is further signaled even if ISP mode is enabled.

22. The method of claim 20 or 21, wherein the indication of whether to signal enabling/disabling of TS mode depends on whether video content of the video is screen content.

23. The method of claim 22, wherein the indication of whether TS mode is enabled/disabled is signaled dependent on a flag signaled in at least one of a picture, slice group, slice, and brick level.

24. The method of claim 22, wherein if the video content is screen content, signaling an indication to enable/disable a transform skip mode.

25. The method of claim 22, wherein if the video content is camera content, skipping enabling/disabling an indication of a transform skip mode and disabling a TS mode for the block.

26. The method of any of claims 1-25, wherein all sub-partitions share the same quantization information, including at least one of a quantization parameter, a quantization step size, and a scaling matrix.

27. The method of claim 26, wherein the quantization parameter is represented by cu qp delta abs and cu qp delta sign flag.

28. The method of claim 26, wherein the quantization information is signaled for the block only if there is at least one coefficient not equal to zero in at least one sub-partition.

29. The method of any of claims 26-28, wherein the quantization information is signaled once for an entire block, rather than signaled for each sub-partition.

30. The method of claim 29, wherein the quantization information is signaled with an mth sub-partition only if there is at least one coefficient not equal to zero in the mth sub-partition, where m is an integer.

31. The method of claim 29, wherein the quantization information is signaled together with the first sub-partition in an encoding or decoding order.

32. The method of claim 29, wherein the quantization information is signaled together with a last sub-partition in an encoding or decoding order.

33. The method of claim 29, wherein the quantization information is signaled in coding or decoding order with an mth sub-partition, where m is an integer no greater than the total number of allowed sub-partitions.

34. A method for processing video, comprising:

for a transition between a block of video and a bitstream representation of the block, enabling an intra sub-block partitioning (ISP) mode for the block, wherein the block is divided into a plurality of sub-partitions based on the ISP mode, and reference samples located in a first sub-partition to predict a second sub-partition in the block are further modified before being used as a prediction; and

performing the conversion based on the ISP mode.

35. The method of claim 34, wherein the reference samples are filtered before being used as a prediction.

36. A method according to claim 34 or 35, wherein whether the reference samples are modified before being used as a prediction depends on the width and/or height of the block.

37. The method of claim 34 or 35, wherein whether the reference sample is modified before being used as a reference depends on an intra prediction mode of the block.

38. The method of any of claims 1-37, wherein a block size of the block is represented by W x H, where W is a block width and H is a block height,

the maximum transform block size of the block is represented by MaxTbW × MaxTbH, where MaxTbW and MaxTbH are the maximum transform block width and maximum transform block height, respectively, and

the minimum transform block size of the block is denoted by MinTbW MinTbH, where MinTbW and MinTbH are the minimum transform block width and the minimum transform block height, respectively.

39. The method of claim 38, wherein the indication of MaxTbW and/or MaxTbH is signaled in at least one of a sequence, a picture, a slice, a group of slices, a slice, and a brick level.

40. The method of claim 39, wherein the indication of MaxTbW and/or MaxTbH is signaled in at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), and a Picture Parameter Set (PPS), a picture header, a slice header, and a slice group header.

41. The method of any of claims 38-40, wherein MaxTbW and/or MaxTbH are set to different numbers in different profiles, levels or levels of a video codec standard.

42. The method of claim 38, wherein the indication of MinTbW and/or MinTbH is signaled in at least one of a sequence, a picture, a slice, a group of slices, a slice, and a brick level.

43. The method of claim 42, wherein the indication of MinTbW and/or MinTbH is signaled in at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), and a Picture Parameter Set (PPS), a picture header, a slice header, and a slice group header.

44. The method of any of claims 42-43, wherein MinTbW and/or MinTbH are set to different numbers in different profiles, levels, or levels of a video codec standard.

45. A method for processing video, comprising:

for a transition between a block of video and a bitstream representation of the block, enabling an intra sub-block partitioning (ISP) mode for the block, wherein a hybrid partitioning direction is enabled in the ISP mode and the block is partitioned into a plurality of sub-partitions in horizontal and vertical directions; and

performing the conversion based on the ISP mode.

46. The method of claim 45, wherein the binary value of the partition direction for ISP mode encoding and decoding is replaced by an index of the partition direction.

47. The method of claim 45, wherein the set of allowed partition directions is dependent on block size.

48. The method of claim 45, wherein signaling allows an indication of a set of partitioning directions.

49. The method of claim 45, wherein the set of allowable partition directions depends on an intra prediction mode of the block.

50. The method of claim 45, wherein the hybrid partitioning direction is enabled when both W/MaxTbW and H/MaxTbH are greater than M, M being an integer.

51. The method of claim 45, wherein the hybrid partitioning direction is enabled when W/MaxTbW or H/MaxTbH is greater than M, M being an integer.

52. The method of claim 50 or 51, wherein M-1.

53. The method of any of claims 50-52, wherein the blocks are partitioned by using quadtree partitioning.

54. The method according to any of claims 50-53, wherein when applying the hybrid partitioning direction, the block is first partitioned horizontally and then vertically.

55. The method of any of claims 50-53, wherein when applying the hybrid partitioning direction, the block is partitioned vertically first and then horizontally.

56. The method according to any of claims 38-55, wherein whether and/or how ISP mode is applied on the blocks depends on the relation between block sizes of block WxH and/or maximum transform block sizes MaxTbW MaxTbH and/or minimum transform block sizes MinTbW MinTbH.

57. The method of claim 56, wherein ISP mode is disabled for the block if W/MinTbW and H/MinTbH are both equal to 1.

58. The method of claim 56, wherein how the block is partitioned depends on a minimum transform block size of the block.

59. The method of claim 58, wherein ISP mode is enabled for the block and vertical partitioning is applied to the block if W/MinTbW is equal to K and H/MinTbH is equal to 1, K being an integer greater than 1.

60. The method of claim 58, wherein ISP mode is enabled for the block and horizontal partitioning is applied to the block if W/MinTbW is equal to 1 and H/MinTbH is equal to K, K being an integer greater than 1.

61. The method of claim 59 or 60, wherein no signalling of prediction direction is required.

62. The method of any one of claims 59-61, wherein the block is partitioned into K sub-partitions.

63. The method of claim 56, wherein ISP mode is disabled for the block when W/MaxTbW is greater than 1 or H/MaxTbH is greater than 1.

64. The method of claim 56, wherein ISP mode is disabled for the block when W H/(MaxTbW MaxTbH) is greater than a threshold, wherein the threshold is 4.

65. The method of claim 56, wherein ISP mode is disabled for the block when both W/MaxTbW and H/MaxTbH are greater than 1.

66. The method of claim 56, wherein ISP mode is disabled for the block when W/MaxTbW or H/MaxTbH is greater than a threshold, wherein the threshold is 2 or 4.

67. The method of claim 56, wherein ISP mode is disabled for the block when both W/MaxTbW and H/MaxTbH are greater than or equal to a threshold, wherein the threshold is 2 or 4.

68. The method of claim 56, wherein ISP mode is disabled for the block when W/MaxTbW and H/MaxTbH are both greater than or equal to a first threshold and less than or equal to a second threshold.

69. The method of claim 56, wherein ISP mode is disabled for the block when W/MaxTbW and H/MaxTbH are both greater than a first threshold and less than a second threshold.

70. The method of claim 68 or 69, wherein the first threshold is 1 and the second threshold is 4.

71. The method according to any of claims 68-70, wherein signaling of a partitioning direction is skipped and the blocks are partitioned according to certain rules.

72. The method of claim 71, wherein quadtree partitioning is applied to the block first, then horizontal binary tree partitioning, then vertical binary tree partitioning.

73. The method of claim 71 or 72, wherein partitioning of one partition tree is terminated once the width reaches MaxTbW or the height reaches MaxTbH.

74. The method of claim 71 or 72, wherein partitioning of one partition tree is terminated once the width reaches MaxTbW and the height reaches MaxTbH.

75. The method of claim 71 or 72, wherein partitioning of a partition tree is terminated once the width reaches MaxTbW/M and the height reaches MaxTbH/N, where M and N are two positive integers.

76. The method of any of claims 56-75, wherein signaling of relevant information including intra _ sub _ modes _ flag is skipped when ISP mode is disabled for the block.

77. The method of any of claims 38-76, wherein when W/MaxTbW >4 and/or H/MaxTbH >4, more than 4 sub-partitions and/or more than one partitioning direction are enabled.

78. A method for processing video, comprising:

determining, for a transition between a block of video and a bitstream representation of the block, a Multiple Transform Selection (MTS) scheme associated with the block, wherein the MTS scheme is modified to allow partial transforms and corresponding inverse transforms; and

the conversion is performed based on the determined MTS scheme.

79. The method of claim 78, wherein the MTS scheme is an explicit MTS, wherein a transformation index of the MTS is signaled in a bitstream of the block.

80. The method of claim 78 or 79, wherein the MTS scheme is modified to allow only two transforms, wherein the two transforms are DCT-II and DST-VII.

81. The method of any of claims 78-80, wherein the MTS scheme includes two modes in terms of transform selection.

82. The method of claim 81, wherein the MTS scheme further comprises a third mode of Transform Skip (TS) mode in addition to the two modes of transform selection when TS mode applies.

83. The method of claim 82, wherein a first mode of the two modes is DCT-II for horizontal and vertical transforms of the block, and a second mode of the two modes is DST-VII for horizontal and vertical transforms of the block.

84. The method of claim 83, wherein one bit is coded to indicate which of the two modes is used.

85. The method of any of claims 78-80, wherein the MTS scheme includes four modes in terms of transform selection.

86. The method of claim 85, wherein the MTS scheme further comprises a fifth mode of Transform Skip (TS) mode in addition to the two modes of transform selection when TS mode is applicable.

87. The method of claim 86, wherein a first mode of the four modes is DCT-II for horizontal and vertical transforms of the block, a second mode of the four modes is DST-VII for horizontal and vertical transforms of the block, a third mode of the four modes is DCT-II for horizontal transforms of the block and DST-VII for vertical transforms of the block, and a fourth mode of the four modes is DST-VII for horizontal transforms of the block and DCT-II for vertical transforms of the block.

88. The method of claim 87, wherein the four modes are coded using fixed length coding.

89. The method of claim 87, wherein the four modes are coded using truncated unary codes.

90. The method of claim 78 or claim 79, wherein the signalling of allowed transform sets and/or transform indices in the explicit MTS is dependent on block size.

91. The method of claim 90, wherein DCT-II and DST-VII are allowed for blocks whose width and/or height are less than or equal to a threshold.

92. The method of claim 90, wherein DCT-II, DST-VII, and DCT-VIII are allowed for blocks whose width and/or height are greater than or equal to a threshold.

93. The method of claim 90, wherein a Transform Skip (TS) mode is allowed.

94. The method of claim 90, wherein the allowed set of transforms depends on a codec mode of the block.

95. The method of claim 94, wherein a transform set comprising two transform bases of TS mode and DST-VII is allowed for Intra Block Copy (IBC) mode coded blocks.

96. The method of claim 94, wherein the non-IBC mode coded block allows a set of transforms comprising two transform bases of DCT-II and DST-VII, or allows a set of transforms comprising three transform bases of TS mode, DCT-II and DST-VII.

97. The method of any of claims 90-96, wherein how to signal the transformation index changes according to an allowed set of transformations.

98. The method of any of claims 78-97, wherein an indication of a maximum allowed transform size for use in a non-TS mode of the MTS scheme is signaled.

99. The method of claim 98, wherein the indication is signaled in at least one of a sequence, a picture, a slice group, a slice, a brick level, or other type of video unit level.

100. The method of claim 99, wherein the indication is signaled in a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), and a Picture Parameter Set (PPS), a picture header, a slice header, and a slice group header.

101. The method of claim 98, wherein the indication is derived from an allowed maximum TS size.

102. The method of claim 98, wherein the indication is used to control a maximum TS size and a maximum size used in other transform matrices.

103. The method of claim 102, wherein the maximum sizes of non-TS and TS modes do not need to be signaled separately.

104. The method of claim 98, wherein the indication of a maximum allowed transform size is used to control implicit MTS transform size and explicit MTS transform size.

105. The method of claim 104, wherein whether to apply implicit MTS depends on a maximum allowed transform size signaled.

106. The method of any one of claims 78-105, wherein a maximum allowed transform size used in a non-TS mode of the MTS scheme is aligned with a maximum allowed transform size used in a TS mode.

107. The method of claim 106, wherein a maximum allowed transition size used in a non-TS mode of the MTS scheme is the same as a maximum allowed transition size used in a TS mode.

108. The method of claim 78, wherein the MTS scheme is implicit MTS, wherein a transform matrix of MTS is directly derived from a transform block size of the block.

109. The method of claim 108 wherein the derivation of an implicit MTS enable flag indicating whether implicit MTS is enabled is independent of a block size of the block.

110. The method of claim 109 wherein checking of block sizes in the derivation of the implicit MTS enable flag is skipped.

111. The method of any of claims 78 to 110, wherein shared conditional checking of block sizes prior to signalling MTS information is removed, the MTS information comprising transform _ skip _ flag and tu _ MTS _ idx.

112. The method of claim 111, wherein the MTS information is further signaled if all of the following sharing conditions are true:

-tu_cbf_luma[x0][y0]

-treeType！＝DUAL_TREE_CHROMA

-！cu_sbt_flag，

otherwise, the MTS information is not signaled.

113. The method of claim 111, wherein conditional checking of block size compared to the allowed maximum TS size is applied before signalling transform _ skip _ flag when certain regular sharing conditional checks return true; and applying a conditional check of the block size compared to the allowed maximum allowed MTS size before signaling tu _ MTS _ idx.

114. The method of claim 111 wherein the shared conditional check of block size before signaling MTS information remains unchanged and the conditional check of block size before signaling a transform matrix index for use in non-TS mode of MTS is removed.

115. The method of any of claims 1-114, wherein the converting generates a block of video from a bitstream representation.

116. The method of any of claims 1-114, wherein the converting generates a bitstream representation from a block of video.

117. A device in a video system comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any of claims 1-116.

118. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for performing the method of any of claims 1-116.