RU2807214C2 - Limitation of representation of differential pulse code modulation of coded video quantized residual - Google Patents

Abstract

FIELD: means for processing video data.

SUBSTANCE: for conversion between the current video block and the video bitstream, that the differential encoding mode used for the remainder samples is applied to the current video block. Based on the specified definition, the specified transformation is performed using the differential coding mode. In this case, in the differential coding mode, the differences between the quantized residuals obtained using intra-frame prediction and the quantized residual predictors are represented in a bit stream, and at least one of the reconstructed quantized residuals is obtained based on the sum of one difference and the other reconstructed quantized residual. The values of the reconstructed quantized residuals are limited to a certain range by applying a constraint operation to the reconstructed quantized residuals.

EFFECT: increase of video data processing efficiency.

15 cl, 22 dwg

Description

Field of technology to which the invention relates

This patent document relates to video encoding technologies, devices and systems.

State of the art

Despite advances in video compression, digital video still accounts for the largest amount of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the demand for bandwidth used for digital video is expected to continue to increase.

Disclosure of the invention

This document describes various embodiments and methods in which the secondary transform is used during decoding or encoding of video or images.

A first exemplary video processing method includes determining, based on an applicability rule, that a differential encoding mode is applicable to a conversion between a chrominance video block of a video and a bitstream representation of a chrominance video block, and performing, based on said determination, a conversion between the chrominance video block and representation of a bitstream of a chrominance video block using a differential encoding mode, wherein in the differential encoding mode, a chrominance video block is represented in a bitstream representation using the difference between a quantized remainder of an intra-frame prediction of a chrominance video block and a prediction of the quantized remainder, wherein the intra-frame prediction is performed in the first direction, and the prediction is quantized The remainder is performed in the second direction, and in the differential encoding mode, the difference between the quantized remainder and the prediction of the quantized remainder is represented using a differential pulse code modulation (DPCM) representation.

A second exemplary video processing method includes performing a conversion between a current video block of video and a bitstream representation of the current video block using a differential encoding mode and an intra-frame prediction mode based on a coexistence rule, wherein in the intra-frame prediction mode, a quantized intra-frame prediction residual of the current video block is based on on sample predictions for the current video block, and in differential encoding mode, the current video block is represented in a bitstream representation using the difference between the quantized remainder and the predicted quantized remainder, wherein the difference between the quantized remainder and the predicted quantized remainder is represented using a differential pulse code modulation representation ( DPCM).

A third exemplary video processing method includes converting between a current video block of video and a bitstream representation of the current video block using a differential encoding mode, wherein the current video block is represented in a bitstream representation using a difference between a quantized intra-frame prediction residual of the current video block and the prediction the quantized remainder, wherein the prediction direction of the quantized remainder is determined from the bitstream representation.

A fourth exemplary video processing method includes determining that a differential encoding mode is applicable to a conversion between a current video block of video and a bitstream representation of the current video block, and, based on the determination, performing a conversion between the current video block and a bitstream representation of the current video block. using a differential encoding mode in accordance with an implementation rule, wherein in the differential encoding mode, the current video block is represented in a bitstream representation using the difference between the quantized intra-prediction remainder of the current video block and the prediction of the quantized remainder, and in the differential encoding mode, the difference between the quantized remainder and the prediction of the quantized the remainder is represented using a differential pulse code modulation (DPCM) representation.

A fifth exemplary video processing method includes determining, based on an applicability rule, that a differential encoding mode is applicable to a transformation between a current video block of video and a bitstream representation of the current video block, wherein the current video block is associated with a color component, wherein the applicability rule determines , enabling or disabling the differential encoding mode for the current video block, depending on whether the differential encoding mode is applied to another video video block, and performing, based on the specified definition, a conversion between the current video block and the bitstream representation of the current video block using the differential encoding mode, wherein in the differential encoding mode, the current video block is represented in a bitstream representation using the difference between the quantized intra-prediction remainder of the current video block and the prediction of the quantized remainder, and in the differential encoding mode, the difference between the quantized remainder and the prediction of the quantized remainder is represented using a differential pulse code modulation representation ( DPCM).

In one exemplary aspect, a method for processing video is disclosed. The method includes the step of performing a conversion between a current video block and a bitstream representation of the current video block using a differential encoding mode and selectively using an intra-frame prediction mode based on a coexistence rule; wherein the intra-frame prediction mode is used to generate predictions for samples of the current video block; and the differential encoding mode is used to represent a block of quantized residuals from the pixel predictions using a differential pulse code modulation representation.

In another exemplary aspect, another method of processing video is disclosed. The method includes performing a conversion between a current video block and a bitstream representation of the current video block using a differential encoding mode, wherein a block of quantized residuals from pixel predictions of the current video block is represented using a differential pulse code modulation representation; wherein a first prediction direction or a second differential encoding mode direction is determined from the bitstream representation.

In yet another exemplary aspect, another method of processing video is disclosed. The method includes determining, based on an applicability rule, that a differential encoding mode is applicable to a conversion between a current video block and a bitstream representation of the current video block; and converting between the current video block and the bitstream representation using a differential encoding mode; wherein, in the differential encoding mode, a block of quantized residuals from the intra-frame pixel prediction of the current video block is represented using a differential pulse code modulation representation performed in a residual prediction direction that is different from the horizontal or vertical direction.

In yet another exemplary aspect, another method of processing video is disclosed. The method includes determining that a differential encoding mode is applicable to the conversion between the current video block and a bitstream representation of the current video block; and performing a conversion between the current video block and the bitstream representation using a differential encoding mode implementation rule; wherein, in the differential encoding mode, a block of quantized residuals from the intra-frame pixel prediction of the current video block is represented using a differential pulse code modulation representation performed in a residual prediction direction that is different from the horizontal or vertical direction.

In yet another exemplary aspect, a video processing apparatus is disclosed. The device includes a processor configured to perform the method described above.

In yet another exemplary aspect, a computer-readable storage medium is disclosed. The storage medium stores code for implementing the methods described above by a processor.

These and other aspects are described in this document.

Brief description of drawings

Fig. 1 – illustration of intra-frame block copying.

Fig. 2 is an example of a block encoded in palette mode.

Fig. 3 is an example of using a palette predictor to signal palette entries.

Fig. 4 – examples of horizontal and vertical transverse scanning.

Fig. 5 – example of encoding palette indices.

Fig. 6 is an example of an affine linearly weighted intra-prediction (ALWIP) process.

Fig. 7 is an example of an affine linearly weighted intra-prediction (ALWIP) process.

Fig. 8 is an example of an affine linearly weighted intra-prediction (ALWIP) process.

Fig. 9 is an example of an affine linearly weighted intra-prediction (ALWIP) process.

Fig. 10 is a block diagram of an exemplary hardware platform for implementing the methods described herein.

Fig. 11 is a flowchart of an exemplary video processing method.

Fig. 12 is an example of four merger candidates.

Fig. 13 – exemplary pairs of merge candidates used in video encoding.

Fig. 14 is a block diagram showing an exemplary video processing system in which various methods disclosed herein may be implemented.

Fig. 15 is a block diagram that illustrates an example video encoding system that may use the methods of the present disclosure.

Fig. 16 is a block diagram illustrating an exemplary video encoder.

Fig. 17 is a block diagram illustrating an exemplary video decoder.

Fig. 18-22 are five example flowcharts of example video processing methods.

Carrying out the invention

Section headings are used herein to facilitate understanding and are not intended to limit the embodiments disclosed in a section to that section only. In addition, although some embodiments are described with reference to universal video encoding or other specific video codecs, the disclosed methods are also applicable to other video encoding technologies. In addition, although some embodiments detail video encoding steps, it should be understood that the corresponding decoding steps that override the encoding will be implemented by the decoder. In addition, the term "video processing" includes video encoding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compressed format to another compressed format or at another compressed bit rate.

1. Summary

This patent document relates to video encoding technologies. In particular, it refers to DPCM encoding in video encoding. It can be applied to an existing video encoding standard, such as HEVC, or to a standard (Universal Video Coding) that will be further developed. It may also be used in future video coding standards or video codecs.

2. Initial discussion

Video encoding standards have evolved mainly due to the development of the well-known ITU-T and ISO/IEC standards. ITU-T released H.261 and H.263, ISO/IEC released MPEG-1 and MPEG-4 Visual, and the two organizations jointly released H.262/MPEG-2 Video and H.264/MPEG-4 Advanced. Video coding standards (AVC) and H.265/HEVC [1]. Starting with H.262, video coding standards are based on a hybrid video coding structure that uses temporal prediction plus transform coding. To explore future video encoding technologies beyond HEVC, the Joint Video Experts Team (JVET) was founded in 2015 with VCEG and MPEG. Since then, JVET has been adopted in many new ways and placed into reference software called the Collaborative Exploration Model (JEM) [3,4]. In April 2018, a Joint Video Experts Team (JVET) was established between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) to work on a VVC standard aimed at reducing bit rates by 50% compared to HEVC.

The latest version of the VVC project, i.e. Universal Video Coding (Project 4), can be found at:

phenix.it-sudparis.eu/jvet/doc_end_user/current_document.php?id=5755

The latest version of the VVC reference software, called VTM, can be found at:

vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/tags/VTM-4.0

2.1. Intra-frame block copying

Intra-block copying (IBC), also known as current-picture matching, has been adopted in the HEVC Screen Content Coding (HEVC-SCC) extensions [1] and the current VVC test model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. As shown in FIG. 1, the current block is predicted by the reference block on the same image when IBC is applied. The samples in the reference block must already be reconstructed before the current block is encoded or decoded. Although IBC is not as effective for most camera sequences, it shows significant benefits when encoding to screen content. The reason is that there are many repeating patterns such as icons and text symbols in the screen content image. IBC can effectively eliminate redundancy among these repeating patterns. In HEVC-SCC, an interframe coding unit (CU) can apply IBC if it selects the current picture as a reference picture. In this case, the MV is renamed to a block vector (BV), and the BV is always accurate to an integer number of pixels. To be compatible with the HEVC core profile, the current picture is marked in the decoded picture buffer (DPB) as a "long-term" reference picture. It should be noted that similarly, in multi-view/3D video coding standards, an inter-view reference picture is also labeled as a "long-term" reference picture.

By following the BV to find its reference block, a prediction can be generated by copying the reference block. The remainder can be obtained by subtracting the reference pixels from the original signals. Transformation and quantization can then be applied as with other encoding modes.

Fig. 1 is an illustration of intra-frame block copying.

However, when the reference block is outside the image or overlapped by the current block, or outside the reconstructed area, or outside the valid area limited by some restrictions, the values of some or all of the pixels are not determined. There are actually two solutions to this problem. One of them is to prohibit such a situation, for example, when bitstream matching. Another is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

2.2. IBC in HEVC Screen Content Coding Extensions

In HEVC screen content encoding extensions, when a block uses the current image as a reference, it must ensure that the entire reference block is within the available reconstructed area, as specified in the following specification text:

The offsetX and offsetY variables are obtained as follows:

offsetX = ( ChromaArrayType = = 0 ) ? 0 : ( mvCLX[ 0 ] & 0x7 ? 2 : 0 ) (8-106)

offsetY = ( ChromaArrayType = = 0 ) ? 0 : ( mvCLX[ 1 ] & 0x7 ? 2 : 0 ) (8-107)

The bitstream matching requirement is that when the reference image is the current image, the mvLX luminance motion vector must obey the following constraints:

- When the acquisition process for z-scan order block availability, as specified in clause 6.4.1, is caused by (xCurr, yCurr) set to (xCb, yCb), and the adjacent luminance location (xNbY, yNbY) set to (xPb + (mvLX[ 0 ] >> 2) − offsetX, yPb + ( mvLX[ 1 ] >> 2 ) − offsetY ) as input parameters, the output parameter must be true.

- When the acquisition process for z-scan order block availability, as specified in clause 6.4.1, is caused by (xCurr, yCurr) set to (xCb, yCb), and the adjacent luminance location (xNbY, yNbY) set to (xPb + (mvLX [0] >> 2) + nPbW - 1 + offsetX, yPb + (mvLX [1] >> 2) + nPbH - 1 + offsetY) as input parameters, the output parameter must be true.

- One or both of the following conditions must be met:

- Value ( mvLX[ 0 ] >> 2 ) + nPbW + xB1 + offsetX is less than or equal to 0.

- Value (mvLX[1] >> 2) + nPbH + yB1 + offsetY is less than or equal to 0.

- The following condition must be met:

( xPb + ( mvLX[ 0 ] >> 2 ) + nPbSw − 1 + offsetX) / CtbSizeY − xCurr / CtbSizeY <=

yCurr/CtbSizeY − ( yPb + ( mvLX[ 1 ] >> 2 ) + nPbSh − 1 + offsetY ) / CtbSizeY …(8-108)

This way, it will not happen that the reference block is overlapped by the current block, or the reference block is outside the image. There is no need to populate a reference block or a prediction block.

2.3. IBC in VVC test model

In the current VVC test model, that is, the VTM-4.0 design, the entire reference block must correspond to the current coding tree unit (CTU) and not overlap with the current block. Thus, there is no need to populate the reference block or prediction block. The IBC flag is encoded as the prediction mode of the current CU. Thus, for each CU there are only three prediction modes: MODE_INTRA, MODE_INTER and MODE_IBC.

2.3.1. IBC merge mode

In IBC merge mode, the index pointing to an entry in the IBC merge candidate list is parsed from the bitstream. The compilation of an IBC merger list can be summarized according to the following sequence of steps:

• Stage 1: Obtaining spatial candidates

• Step 2: Inserting HMVP Candidates

• Step 3: Inserting pairwise averaged candidates

When receiving spatial fusion candidates, a maximum of four fusion candidates are selected from the candidates located at the positions shown in FIG. 12. The order of receipt is A ₁ , B ₁ , B ₀ , A ₀ and B ₂ . Position B ₂ is only taken into account when any PU from position A ₁ , B ₁ , B ₀ , A ₀ is not available (for example, due to the fact that it belongs to another slice or tile) or is not encoded in IBC mode. After a candidate at position A ₁ is added, the insertion of the remaining candidates is subjected to a redundancy check, which ensures that candidates with the same motion information are excluded from the list, so that coding efficiency is improved. To reduce computational complexity, not all possible candidate pairs are considered in the redundancy check mentioned. Instead, only the pairs indicated by the arrow in FIG. 13, and a candidate is added to the list only if the corresponding candidate used for redundancy checking does not have the same motion information.

After inserting spatial candidates, if the IBC merge list size is still less than the maximum IBC merge list size, the IBC candidates from the HMVP table can be inserted. After inserting HMVP candidates, a redundancy check is performed.

Finally, the pairwise averaged candidates are inserted into the IBC merge list.

When the reference block identified by the merge candidate is outside the image, or overlaps with the current block, or is outside the reconstructed area, or outside the valid area limited by some restrictions, the merge candidate is called an invalid merge candidate.

It should be noted that invalid merge candidates may be inserted into the IBC merge list.

2.3.2. AMVP IBC mode

In AMVP IBC mode, the AMVP index point for an entry in the AMVP IBC list is parsed from the bitstream. The compilation of the AMVP IBC list can be summarized according to the following sequence of steps:

• Stage 1: Obtaining spatial candidates

○ Check A ₀ , A ₁ until an available candidate is found.

○ Check B ₀ , B ₁ , B ₂ until an available candidate is found.

• Step 2: Inserting HMVP Candidates

• Step 3: Inserting Null Candidates

After inserting spatial candidates, if the AMVP IBC list size is still less than the maximum AMVP IBC list size, the IBC candidates from the HMVP table can be inserted.

Finally, zero candidates are inserted into the IBC AMVP list.

2.4. Palette mode

The basic idea of palette mode is that the samples in the CU are represented by a small set of representative color values. This set is called a palette. In addition, it is also possible to indicate a sample that is outside the palette by signaling a dropout symbol followed by (possibly quantized) component values. This is shown in Fig. 2.

2.5. Palette mode in HEVC Screen Content Coding Extensions (HEVC-SCC)

In palette mode, HEVC-SCC uses a prediction method to encode the palette and index map.

2.5.1. Encoding palette entries

A palette predictor is supported for encoding palette entries. The SPS signals the maximum palette size as well as the palette predictor. In HEVC-SCC palette_predictor_initializer_present_flag is introduced into the PPS. When this flag is 1, entries to initialize the palette predictor are passed in the bitstream. The palette predictor is initialized at the beginning of each CTU row, each slice, and each tile. Depending on the value of palette_predictor_initializer_present_flag, the palette predictor is reset to 0 or initialized using the palette predictor initializer entries signaled in the PPS. In HEVC-SCC, a palette predictor initializer with size 0 was included to allow explicit disabling of palette predictor initialization at the PPS level.

For each entry in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette. This is shown in Fig. 3. Reuse flags are sent using zero movement length encoding. The number of new palette entries is then signaled using an exponential Golomb code of order 0. Finally, the component values for the new palette entries are signaled.

2.5.2. Encoding palette indices

Palette indices are encoded using horizontal and vertical cross scans, as shown in FIG. 4. The scan order is explicitly signaled in the bitstream using palette_transpose_flag. The rest of this subsection assumes that the scan is horizontal.

Palette indices are encoded using two basic palette reference modes: "INDEX" and "COPY_ABOVE". As explained earlier, the drop character is also signaled as "INDEX" mode and is assigned an index equal to the maximum palette size. The mode is signaled using a flag, except for the top line, or when the previous mode was "COPY_ABOVE". The "COPY_ABOVE" mode copies the index of the reference palette in the line above. In "INDEX" mode, the palette index is signaled explicitly. For both the "INDEX" and "COPY_ABOVE" modes, a run value is passed which indicates the number of subsequent samples that are also encoded using the same mode. When a drop symbol is part of a run in "INDEX" or "COPY_ABOVE" mode, the drop component values are signaled for each drop symbol. The encoding of palette indices is shown in FIG. 5.

This syntax order is as follows. First, the number of index values for the CU is signaled. This is followed by signaling the actual index values for the entire CU using truncated binary encoding. In traversal mode, both the number of indexes and the index values are encoded. This results in the crawl bins associated with the index being grouped together. Then the palette sample mode (if necessary) and run are signaled alternately. Finally, the dropout component values corresponding to the dropout samples for the entire CU are grouped together and encoded in a bypass mode.

The optional last_run_type_flag syntax element is signaled after the index values are passed. This syntax element, combined with the number of indices, eliminates the need to signal the run execution value corresponding to the last run in the block.

In HEVC-SCC, palette mode is also enabled for 4:2:2, 4:2:0 and monochrome chroma formats. The signaling of palette entries and palette indexes is virtually identical for all color formats. For non-monochrome formats, each palette entry consists of 3 components. For the monochrome format, each palette entry consists of one component. For subsampled chroma directions, chroma samples are associated with luminance sample indices that are divisible by 2. After reconstructing the palette indices for the CU, if only one component is associated with a sample, only the first component of the palette entry will be used. The only difference in signaling is the values of the dropout components. For each dropout sample, the number of signaled dropout component values may vary depending on the number of components associated with that count.

2.6. Coefficient coding in skip transformation mode

JVET-M0464 and JVET-N0280 propose several modifications to skip-transform (TS) coefficient encoding in order to tailor the remainder encoding to the statistics and characteristics of transform-skip level signals.

Suggested modifications are listed below.

No Last Significant Scan Position: Since the residual signal reflects the spatial residual after prediction, and the TS is not energy compressed by transform, a higher probability is no longer set for trailing zeros or non-significant levels in the lower right corner of the transform block. Thus, in this case, signaling of the last significant scan position is eliminated. Instead, the first subblock processed is the bottom rightmost subblock in the transform block.

Sub-block CBF: Failure to signal the last significant scan position requires that the CBF signaling of the sub-block with coded_sub_block_flag for the TS be changed as follows:

• Due to quantization, the above unimportant sequence may still occur locally within the transform block. Thus, the last significant scan position is removed as described previously, and a coded_sub_block_flag is encoded for all subblocks.

• coded_sub_block_flag for the subblock covering the DC frequency position (top left subblock) is a special case. In Design 3 VVC, the coded_sub_block_flag for this subblock is never signaled and is always assumed to be 1. When the last significant scan position is in another subblock, it means that there is at least one significant level outside the DC subblock. Thus, a DC sub-block can only contain zero/insignificant levels, although the coded_sub_block_flag for that sub-block is assumed to be 1. In the absence of last scan position information in the TS, a coded_sub_block_flag is signaled for each sub-block. This also includes the coded_sub_block_flag for the DC subblock, unless all other coded_sub_block_flag syntax elements are already 0. In this case, the DC's coded_sub_block_flag is assumed to be 1 (inferDcSbCbf = 1). Since there must be at least one significant level in this DC subblock, the sig_coeff_flag syntax element for the first position at (0,0) is not signaled and is set to 1 (inferSbDcSigCoeffFlag = 1), instead of all other sig_coeff_flag syntax elements in this subblock DC were equal to 0.

• Changed context modeling for coded_sub_block_flag. The context model index is calculated as the sum of the coded_sub_block_flag on the left and the coded_sub_block_flag above the current subblock, rather than as a logical division of both.

Context modeling sig_coeff_flag: The local template in the context modeling sig_coeff_flag is modified to include only the neighbor to the left (NB0) and the neighbor above (NB1) of the current scan position. The context model bias is simply the number of significant neighboring positions sig_coeff_flag [NB0] + sig_coeff_flag [NB1]. Consequently, the selection of different context sets is removed depending on the diagonal d in the current transformation block. This results in three context models and one context model set to encode the sig_coeff_flag.

Context modeling of abs_level_gt1_flag and par_level_flag: abs_level_gt1_flag and par_level_flag use a single context model.

abs_remainder encoding: Although the empirical distribution of absolute levels of the skip-remainder residual generally still follows a Laplacian or geometric distribution, there are non-stationarities larger than for the absolute levels of transformation coefficients. In particular, the variance within the sequential implementation window is higher for absolute residual levels. This motivates the following modifications to abs_remainder's syntactic binarization and context modeling:

• Using a higher cutoff value in binarization, that is, the transition point from encoding with sig_coeff_flag, abs_level_gt1_flag, par_level_flag and abs_level_gt3_flag to Rice codes for abs_remainder, and dedicated context models for each bin position gives higher compression efficiency. Increasing the cutoff will result in more "greater than X" flags, such as introducing abs_level_gt5_flag, abs_level_gt7_flag, and so on until the cutoff is reached. The limit itself is set to 5 (numGtFlags = 5).

• The template for obtaining the parameter is modified, that is, only the neighbor to the left and the neighbor above the current scan position are considered similar to the local template for contextual modeling sig_coeff_flag.

coeff_sign_flag context modeling: Due to instability within a sign sequence and the fact that the prediction remainder is often shifted, signs can be encoded using context models even when the global empirical distribution is nearly uniformly distributed. One dedicated context model is used to encode characters, and the character is parsed after sig_coeff_flag to ensure that all context-encoded bins are together.

2.7. Block Differential Pulse Code Modulation (QR-BDPCM) of Quantized Residue

JVET-M0413 proposes quantized residual block differential pulse code modulation (QR-BDPCM) for efficient encoding of screen content.

The prediction directions used in QR-BDPCM can be vertical and horizontal prediction modes. Intra prediction is performed for the entire block by copying the sample in the prediction direction (horizontal or vertical prediction) similar to intra prediction. The remainder is quantized, and the delta (difference) between the quantized remainder and its quantized predictor value (horizontal or vertical) is encoded. This can be described as follows: for a block of size M (rows) × N (columns), let us assume that r _i,j , 0 ≤ i ≤ M–1, 0 ≤ j ≤ N-1 will be the prediction remainder after performing horizontal intra-frame prediction (line-by-line copying the value of the left adjacent pixel across the predicted block) or vertically (line-by-line copying the top adjacent line in the predicted block) using unfiltered samples from the samples at the top or left border of the block. Let Q(r _i,j ), 0 ≤ i ≤ M-1, 0 ≤ j ≤ N-1 denote the version of the quantized remainder r _i,j , where the remainder is the difference between the original block and the predicted block values. The DPCM block is then applied to the quantized remainder samples, resulting in a modified array size M × N with elements . When signaling vertical BDPCM

(2-7-1)

For horizontal prediction, similar rules apply and quantized residual samples are obtained using

(2-7-2)

Quantized remainder samples sent to the decoder.

On the decoder side, the above calculations are performed in reverse order . For the case of vertical prediction

(2-7-3)

For the case of horizontal prediction

(2-7-4)

The inversely quantized residuals Q ^-1 (Q(r _i,j )) are added to the intra-frame block prediction values to obtain the reconstructed sample values.

The main advantage of this scheme is that inverse DPCM can be performed on the fly during coefficient parsing by simply adding a predictor in the form of coefficients that are parsed, or can be performed after parsing.

The changes to the QR-BDPCM draft text are shown as follows.

7.3.6.5 Coding unit syntax

bdpcm_flag[ x0 ][ y0 ] equal to 1 indicates that bdpcm_dir_flag is present in the coding block, including the luma coding block at location (x0, y0)

bdpcm_dir_flag[ x0 ][ y0 ] equal to 0 indicates that the prediction direction to be used in the bdpcm block is horizontal, otherwise vertical.

2.8. Matrix Intra Prediction (MIP)

Matrix intra prediction is also called affine linear weighted intra prediction (ALWIP), in which a weighted matrix is used to obtain the intra prediction signal.

2.8.1. Description of the method

To predict samples of a rectangular block with width W and height H, affine linear-weighted intra-prediction (ALWIP) takes as input one line H of reconstructed neighboring boundary samples to the left of the block and one line W of reconstructed neighboring boundary samples above the block. If reconstructed samples are not available, they are generated in the same way as in normal intra-frame prediction.

Generation of the prediction signal is based on the following three steps:

1. Four samples if W = H = 4, and eight samples in all other cases, are extracted from the boundary samples by averaging.

2. Matrix-vector multiplication followed by bias addition is performed using the averaged samples as input. The result is a reduced prediction signal for the subsampled set of samples in the original block.

3. The prediction signal in the remaining positions is generated from the prediction signal in the subsampled set using linear interpolation, which is a one-step linear interpolation in each direction.

The matrix and displacement vectors required to generate the prediction signal are taken from three sets of matrices S ₀ , S ₁ , S ₂ . The set S ₀ consists of 18 matrices , each of which has 16 rows and 4 columns and 18 vectors offsets, each of size 16. The matrices and offset vectors from this set are used for 4x4 blocks. Set S ₁ consists of 10 matrices , each of which has 16 rows and 8 columns and 10 vectors offsets, each of size 16. The offset matrices and vectors of this set are used for blocks of size 4x8, 8x4, and 8x8. Finally, the set S ₂ consists of 6 matrices , each of which has 64 rows and 8 columns and 6 vectors offsets of size 64. The matrices and offset vectors from this set, or portions of these matrices and offset vectors, are used for all other block shapes.

The total number of multiplications required to compute a matrix-vector multiplication is always less than or equal to 4 W H. In other words, ALWIP modes require no more than four multiplications per sample.

2.8.2. Boundary averaging

At the first stage, the input boundaries And decrease to smaller limits And . Here are both borders And consist of 2 samples in the case of a 4x4 block, and in all other cases both boundaries consist of 4 samples.

In the case of a 4x4 block for , is determined

and is similarly defined .

Otherwise, if the block width W is given as , For , is determined

and is similarly defined .

Two reduced borders And associated with the reduced boundary vector , which is therefore size four for blocks of the 4x4 shape and size eight for blocks of all other shapes. If refers to ALWIP mode, this union is defined as follows:

Finally, a second version of the averaged boundary is required to interpolate the subsampled prediction signal over large blocks. Namely, if And , write W , and for determined

If And , is defined in a similar way.

2.8.3 Generating a reduced prediction signal using matrix-vector multiplication

From the reduced input vector a reduced signal is produced predictions. The last signal is a subsampled block signal with a width and height . Here And are defined as:

Reduced signal The prediction is calculated by calculating the matrix-vector multiplication and adding an offset:

Here A is a matrix that has rows and 4 columns if and 8 columns in all other cases. is a vector of size .

Matrix and vector selected from one of the sets , , in the following way. Index is defined as follows:

In addition, we believe that is defined as follows:

Then if or , And , we believe that And . In case And , let's say is the matrix that is obtained by eliminating each row from , which in case corresponds to the odd x coordinate in the downsampled block, or, in the case , corresponds to the odd y coordinate in the downsampled block.

Finally, the reduced prediction signal is replaced by its transposition in the following cases:

• And

• And

• And

The number of multiplication operations required for the calculation is 4 in the case, since in this case there are 4 columns and 16 rows. In all other cases has 8 columns and lines, and immediately checks what is required in these cases multiplication operations, that is, in these cases to calculate no more than 4 multiplication operations per count are also required.

2.8.4. Illustration of the entire ALWIP process

The entire process of averaging, matrix-vector multiplication, and linear interpolation is illustrated for various shapes in FIG. 6, fig. 7, fig. 8 and fig. 9. It should be noted that the remaining forms are considered as in one of the cases depicted.

1. Given a 4x4 block, ALWIP has four average values along each boundary axis. The resulting four input samples enter into a matrix-vector multiplication. Matrices are taken from the set . After adding the offset, there are 16 final prediction samples. Linear interpolation is not required to generate a prediction signal. Thus, the total is fulfilled multiplications by count.

Fig. 6 is an illustration of ALWIP for 4x4 blocks.

2. Given an 8x4 block, ALWIP has four average values along each boundary axis. The resulting eight input samples go into matrix-vector multiplication. Matrices are taken from the set . This gives 16 samples at odd positions of the prediction block. Thus, the total is fulfilled multiplications by count. After adding offset, these samples are interpolated vertically using the reduced upper bound. Horizontal interpolation is performed using the original left boundary.

Fig. 7 is an illustration of ALWIP for 8x8 blocks.

3. Given an 8x4 block, ALWIP has four average values along the horizontal boundary axis and four original boundary values on the left boundary. The resulting eight input samples go into matrix-vector multiplication. Matrices are taken from the set . This gives 16 samples at the odd horizontal and each vertical position of the prediction block. Thus, in total ( multiplications by count. After adding an offset, these samples are interpolated horizontally using the original left boundary.

Fig. 8 is an illustration of ALWIP for 8x4 blocks.

The transposed case is handled accordingly.

4. Given a 16x16 block, ALWIP has four average values along each boundary axis. The resulting eight input samples go into matrix-vector multiplication. Matrices are taken from the set . This gives 64 samples at odd positions of the prediction block. Thus, in total ( multiplications by count. After adding offset, these samples are interpolated vertically using the eight upper bound averages. Horizontal interpolation follows the use of the original left border.

Fig. 8 is an illustration of ALWIP for 16x16 blocks.

For large forms the procedure is essentially the same, and it is easy to verify that the number of multiplications per count is less than four.

For W×8 blocks at W>8, only horizontal interpolation is necessary, since samples are given at odd horizontal positions and at each vertical position. In this case (multiplication ( per sample are performed in order to calculate the reduced prediction.

Finally, for W×4 blocks with W>8, let us assume that A _k will be the matrix that is obtained by eliminating each row that corresponds to an odd entry on the horizontal axis of the downsampled block. Thus, the output size is 32, and again only horizontal interpolation remains to be done.

To calculate the reduced prediction, multiplication is performed ( per one count. For W = 16 no additional multiplications are required, whereas linear interpolation for W > 16 requires less than 2 multiplications per sample. Thus, the total number of multiplications is less than or equal to four.

Transposed cases are handled accordingly.

2.8.5. One-step linear interpolation

For block size at the prediction signal is obtained from the reduced prediction signal on using linear interpolation. Depending on the block shape, linear interpolation is performed in vertical, horizontal, or both directions. If linear interpolation is to be applied in both directions, it is first applied in the horizontal direction if , and is applied in the vertical direction first, otherwise.

Without loss of generality, consider a block of size at And . One-dimensional linear interpolation is then performed as follows. Without loss of generality, it is sufficient to describe linear interpolation in the vertical direction. First, the reduced prediction signal is expanded to the top using a boundary signal. Let's determine the coefficient vertical upsampling, and write . We then define the extended reduced prediction signal using

A vertical linear interpolated prediction signal is then generated from this extended reduced prediction signal as follows:

For , And .

The bit-shift-only linear interpolation algorithm does not require any multiplication.

2.8.6. Signaling proposed intra-frame prediction modes

For each coding unit (CU) in intra-frame mode, a flag is sent in the bit stream indicating whether or not ALWIP mode should be applied to the corresponding prediction unit (PU). If ALWIP mode is to be used, the index ALWIP mode is signaled using an MPM list with 3 MPMS.

Here, MPM inference is performed using the intra-frame mode of the top and left PUs as follows. There are three fixed tables , , which assign each conventional intra-frame prediction mode an ALWIP mode.

For each PU with width and height index is determined

which specifies from which of three sets the ALWIP parameters should be selected, as in section 1.3 above.

If the above unit prediction is available, belongs to the same CTU as the current PU, and is in intra-frame mode if , and if ALWIP applies to with ALWIP mode , we believe that

If the top PU is reachable, belongs to the same CTU as the current PU and is in intra-frame mode, and if the top PU is in legacy mode intra-frame prediction, we assume that

In all other cases we assume that

,

which means this mode is not available. In the same way, but without the restriction that the left PU must belong to the same CTU as the current PU, the mode is obtained .

Finally, three fixed lists are provided , by default, each containing three different ALWIP modes. From the list default and modes And three separate MPMs are created by replacing -1 with default values and also eliminating duplicates.

2.8.7. Obtaining an adaptive MPM list for traditional intraframe luma and chrominance prediction modes

The proposed ALWIP modes are harmonized with MPM-based coding of traditional intra-frame prediction modes as follows. The processes for obtaining the luma and chrominance MRM list for traditional intra-frame prediction modes use fixed tables , , which provide ALWIP mode mapping with a given PU in one of the traditional intra-frame prediction modes

To obtain a luma MPM list whenever a neighboring luma block that uses ALWIP mode is encountered , this block is processed as if it were using traditional intra-prediction mode . To obtain the MPM chrominance list whenever the current luma block uses the LWIP mode, the same mapping is used to convert the ALWIP mode to the traditional intra-prediction mode.

2.9. Intra-frame chroma coding

In the case of intra-chroma mode encoding, a total of 8 intra-frame modes are allowed for the intra-chroma mode. These modes include five traditional intra-frame modes and three modes based on the inter-component linear model. Chroma DM mode uses the corresponding intra-frame luminance prediction mode. Since I-slices allow a separate block partitioning structure for luma and chrominance components, a single chrominance block can correspond to multiple luma blocks. Thus, the Chroma DM mode directly inherits the intra-frame prediction mode of the corresponding luma block covering the center position of the current chrominance block.

3. Examples of technical problems solved by the disclosed embodiments

Although QR-BDPCM can provide encoding advantages when encoding screen content, it may still have some disadvantages.

1. QR-BDPCM prediction is limited to only horizontal and vertical intra-frame prediction, which may limit the performance of QR-BDPCM prediction.

2. The intra-frame prediction mode is signaled for QR-BDPCM coded blocks, which may increase the overhead of the QR-BDPCM mode.

3. Neighboring information is not taken into account when matching the signaled message with prediction modes in QR-BDPCM mode.

4. QR-BDPCM represents the remainder, supporting only horizontal DPCM and vertical DPCM, which can contain encoding characteristics in a complex residual block.

5. The residual range in QR-BDPCM may exceed the maximum range of other modes other than QR-BDPCM.

6. QR-BDPCM does not take into account the block shape.

7. How to handle chroma when the luma block is encoded using QR-BDPCM is unknown.

4. Examples of options and methods

The list of points below should be considered as examples to explain general concepts. These technical features should not be interpreted in a narrow sense. In addition, these technical features can be combined in any way.

1. Sample prediction in QR-BDPCM encoded blocks can be generated using a matrix intra-prediction (MIP) method.

a) In one example, when both QR-BDPCM and MIP are enabled for the same block, the limitation is that MIP only supports a subset of the allowed modes.

1) In one example, a portion of the enabled modes may include those modes associated with a matrix intra-prediction method that may be associated with a horizontal and/or vertical normal intra-frame mode.

2) In one example, the portion of enabled modes may include only those modes associated with a matrix intra-prediction method that can be associated with a horizontal and/or vertical normal intra-frame mode.

b) In one example, when both QR-BDPCM and MIP are enabled for a single block, all enabled modes in the MIP are supported.

2. Sample prediction in QR-BDPCM encoded blocks can be generated using intra-prediction modes that are different from vertical/horizontal intra-prediction.

a) In one example, samples in QR-BDPCM encoded blocks may be predicted using intra-prediction mode K.

1) In one example, K could be planar mode.

2) In one example, K could be DC mode.

3) In one example, K could be horizontal mode.

4) In one example, K could be vertical mode.

5) In one example, K may be one candidate in the list of most likely modes.

6) In one example, K may be signaled in the bit stream.

b) Allowed intra-prediction modes for QR-BDPCM may be based on

1) the message signaled in the SPS/VPS/PPS/picture header/slice header/tile group header/LCU line/LCU group;

2) the block size of the current block and/or its neighboring blocks;

3) the shape of the current block and/or its neighboring blocks;

4) prediction mode (intra-frame/inter-frame) of neighboring blocks of the current block;

5) mode of intra-frame prediction of neighboring blocks of the current block;

6) indicating the QR-BDPCM modes of the adjacent block of the current block;

7) the current quantization parameter of the current block and/or its neighboring blocks;

8) specifying the color format (for example, 4:2:0, 4:4:4)

9) tree structure of single/double coding; And

10) type of group of slices/tiles and/or type of image.

3. Sample prediction in QR-BDPCM-coded blocks can be generated by non-contiguous samples.

a) In one example, QR-BDPCM may also be enabled for IBC merge mode.

b) In one example, the IBC AMVP mode may also include QR-BDPCM.

c) The block vector used in IBC and QR-BDPCM may be signaled, received or determined in advance.

1) In one example, the IBC mode may be indicated by a motion vector (block vector) and/or a merge index.

2) In one example, the IBC mode may be indicated by a default motion vector.

(1) In one example, the default motion vector could be (-w, 0), where w is a positive integer.

(2) In one example, the default motion vector could be (0, -h), where h is a positive integer.

(3) In one example, the default motion vector could be (-w, -h), where w and h are two positive integers.

3) In one example, the indication of the motion vector used in the IBC and QP-BPDCM coded blocks may be based on:

(1) the message specified in the SPS/VPS/PPS/Image Header/Slice Header/Tile Group Header/LCU Row/LCU Group;

(2) the block size of the current block and/or its neighboring blocks;

(3) the shape of the current block and/or its neighboring blocks;

(4) prediction modes (intra-frame/inter-frame) of neighboring blocks of the current block;

(5) motion vectors of neighboring blocks of the current block;

(6) indicating the QR-BDPCM modes of the adjacent block of the current block;

(7) the current quantization parameter of the current block and/or its neighboring blocks;

(8) specifying the color format (for example, 4:2:0, 4:4:4);

(9) single/dual coding tree structure; And

(10) slice/tile group type and/or image type.

d) In one example, sample prediction in QR-BDPCM mode may be generated by inter-frame prediction means (eg, affine mode, merge mode, and inter-frame mode).

4. The direction indication of the quantized residual prediction in QR-BDPCM can be obtained on the fly.

a) In one example, the indication of the prediction direction of the quantized remainder in the QR-BDPCM may be obtained based on the indication of the current intra-prediction mode.

1) In one example, the direction of the quantized prediction residual in QR-BDPCM can be obtained as vertical when the intra-prediction mode is vertical.

2) In one example, the direction of the quantized prediction residual in QR-BDPCM may be obtained as horizontal when the intra-prediction mode is horizontal.

3) In one example, the direction of the quantized prediction residual in QR-BDPCM may be obtained as vertical when the intra-prediction mode is horizontal.

4) In one example, the direction of the quantized prediction residual in QR-BDPCM may be obtained as horizontal when the intra-prediction mode is vertical.

b) In one example, indicating the prediction direction of the quantized remainder in the QR-BDPCM may be based on:

1) the message specified in the SPS/VPS/PPS/Image Header/Slice Header/Tile Group Header/LCU Line/LCU Group;

2) the block size of the current block and/or its neighboring blocks;

3) the shape of the current block and/or its neighboring blocks;

4) the most probable modes of the current block and/or its neighboring blocks;

5) prediction modes (intra-frame/inter-frame) of neighboring blocks of the current block;

6) mode of intra-frame prediction of neighboring blocks of the current block;

7) motion vectors of neighboring blocks of the current block;

8) indicating the QR-BDPCM modes of the adjacent block of the current block;

9) the current quantization parameter of the current block and/or its neighboring blocks;

10) indicating the color format (for example, 4:2:0, 4:4:4);

11) tree structure of separate/double coding;

12) the type of transformation applied to the current block; And

13) type of group of slices/tiles and/or type of image.

5. The mapping from the signaled index in QR-BDPCM to the intra-frame prediction mode in QR-BDPCM can be based on

a) the message specified in the SPS/VPS/PPS/Image Header/Slice Header/Tile Group Header/LCU Line/LCU Group;

b) the block size of the current block and/or its neighboring blocks;

c) the shape of the current block and/or its neighboring blocks;

d) the most probable modes of the current block and/or its neighboring blocks;

e) prediction modes (intra-frame/inter-frame) of neighboring blocks of the current block;

f) intra-frame prediction mode of neighboring blocks of the current block;

g) motion vectors of neighboring blocks of the current block;

h) indicating the QR-BDPCM modes of the adjacent block of the current block:

1) the current quantization parameter of the current block and/or its neighboring blocks:

j) specifying the color format (eg 4:2:0, 4:4:4);

k) single/dual coding tree structure;

l) the type of transformation applied to the current block; And

m) slice/tile group type and/or image type

6. In QR-BDPCM, the quantized residual is predicted in the horizontal and vertical directions. It is proposed to predict the quantized remainder in directions other than the vertical and horizontal directions. Let's pretend that means that the quantized remainder denotes the quantized remainder after the remainder prediction process.

a) In one example, 45 degree QR-BDPCM may be supported.

1) In one example, DPCM may be performed in a 45 degree direction, where can be obtained If available.

b) In one example, 135 degree QR-BDPCM may be supported.

1) In one example, DPCM may be performed in the 135 degree direction, where can be obtained If available

c) In one example, any directions may be supported in QR-BDPCM.

1) In one example, can be received if available .

(1) In one example, m and/or n may be signaled in the bitstream.

(2) In one example, m and/or n may be integers and may be based on:

(3) the message specified in the SPS/VPS/PPS/Image Header/Slice Header/Tile Group Header/LCU Row/LCU Group.

(4) i and/or j;

(5) the block size of the current block and/or its neighboring blocks;

(6) the shape of the current block and/or its neighboring blocks;

(7) the most probable modes of the current block and/or its neighboring blocks;

(8) prediction modes (intra-frame/inter-frame) of neighboring blocks of the current block;

(9) mode of intra-frame prediction of neighboring blocks of the current block;

(10) motion vectors of neighboring blocks of the current block;

(11) indicating the QR-BDPCM modes of the adjacent block of the current block;

(12) the current quantization parameter of the current block and/or its neighboring blocks;

(13) specifying the color format (for example, 4:2:0, 4:4:4);

(14) single/dual coding tree structure; And

(15) slice/tile group type and/or image type;

7. QR-BDPCM can be applied to chrominance blocks (such as Cb/Cr or B/R color components).

a) In one example, the allowed intra-frame prediction directions for QR-BDPCM-coded luma and chrominance blocks may be the same, for example, only in the horizontal and vertical directions.

b) In one example, the permitted prediction methods for QR-BDPCM-coded luma and chrominance blocks may be the same, eg, IBC/inter-prediction/horizontal and vertical intra-prediction modes.

c) In one example, the allowed residual prediction direction for QR-BDPCM-coded luma and chrominance blocks may be the same.

d) In one example, the residual prediction direction for a chrominance QR-BDPCM may be derived from the residual prediction direction for a corresponding luminance block.

1) In one example, the corresponding luma block may be a combined luma block.

2) In one example, the corresponding luma block may be a luma block containing a combined sample in the upper left corner of the chrominance block.

3) In one example, the corresponding luma block may be a luma block containing a superimposed sample of a centrally located chrominance block sample.

e) In one example, CCLM and QR-BDPCM could not be applied to the same chroma block.

1) Alternatively, CCLM can also be applied to QR-BDPCM encoded blocks.

f) In one example, a chrominance residual co-coding method (eg, cb and cr co-coding) and QR-BDPCM cannot be applied to the same chroma block.

8. The recovered quantized remainder in QR-BDPCM can be limited to a certain range.

a) One example might add the constraint that all differences of quantized residuals (e.g. in Equations 2-7-1 and 2-7-2) may be within a certain range.

b) In one example, a constraint could be added that the entire reconstructed quantized remainder (e.g. in Equations 2-7-3 and 2-7-4) may be within a certain range.

c) In one example, the constraint operation may be applied to differences of quantized residuals (e.g. in Equations 2-7-1 and 2-7-2), so that the recovered quantized residue can be within a certain range.

d) In one example, the constraint operation may be applied to the recovered differences of the quantized residuals (e.g. in Equations 2-7-3 and 2-7-4), so that the recovered quantized residue may be within a certain range.

e) In one example, the constraint operation can be defined as (x < min? min: (x > max? max: x))).

f) In one example, the constraint operation can be defined as ((x <= min? min: (x >= max? max: x)).

g) In one example, the constraint operation can be defined as (x < min? min: (x >= max? max: x)).

h) In one example, the constraint operation can be defined as (x <= min? min: (x > max? max: x)).

1) In one example, min and/or max can be negative or positive.

j) In one example, min is set to -32768 and max is set to 32767.

1) Alternatively, min and/or max may depend on the inverse quantization range for non-QR-BDPCM encoded blocks.

2) Alternatively, min and/or max may depend on the bit depth of the input sample/recovered sample.

3) Alternatively, min and/or max may depend on whether lossless encoding is used.

(1) In one example, min and/or max may depend on transquant_bypass_enabled_flag.

(2) In one example, min and/or max may depend on cu_transquant_bypass_flag.

k) In one example, min and/or max may be based on:

1) the message specified in the SPS/VPS/PPS/Image Header/Slice Header/Tile Group Header/LCU Line/LCU Group;

2) the block size of the current block and/or its neighboring blocks;

3) the shape of the current block and/or its neighboring blocks;

4) the most probable modes of the current block and/or its neighboring blocks;

5) prediction modes (intra-frame/inter-frame) of neighboring blocks of the current block;

6) mode of intra-frame prediction of neighboring blocks of the current block;

7) motion vectors of neighboring blocks of the current block;

8) indicating the QR-BDPCM modes of the adjacent block of the current block;

9) the current quantization parameter of the current block and/or its neighboring blocks;

10) indicating the color format (for example, 4:2:0, 4:4:4);

11) tree structure of separate/double coding;

12) the type of transformation applied to the current block;

13) type of group of slices/tiles and/or type of image;

9. For a block, QR-DPCM can be applied from the last row/column to the first row/column.

a) In one example, when the remainder prediction direction is horizontal, the remainder of the (i+1)th column may be used to predict the remainder of the ith column.

b) In one example, when the remainder prediction direction is vertical, the remainder of the (i+1)th row may be used to predict the remainder of the i-th row.

10. QR-DPCM can be applied to a subset of a block.

a) In one example, when the direction of the residual prediction is horizontal, QR-DPCM is not applied to the leftmost columns of the residuals.

b) In one example, when the direction of the residual prediction is vertical, QR-DPCM is not applied to the topmost k rows of the residuals.

c) In one example, when the direction of the residual prediction is horizontal, QR-DPCM is not applied to the rightmost columns of the k residuals.

d) In one example, when the direction of the residual prediction is vertical, QR-DPCM is not applied to the bottommost k rows of the residuals.

e) The k value described above may be a target value based on:

1) the message specified in the SPS/VPS/PPS/Image Header/Slice Header/Tile Group Header/LCU Line/LCU Group;

2) the block size of the current block and/or its neighboring blocks;

3) the shape of the current block and/or its neighboring blocks;

4) the most probable modes of the current block and/or its neighboring blocks;

5) prediction modes (intra-frame/inter-frame) of neighboring blocks of the current block;

6) intra-frame prediction mode of the current block;

7) mode of intra-frame prediction of neighboring blocks of the current block;

8) motion vectors of neighboring blocks of the current block;

9) indicating the QR-BDPCM modes of the adjacent block of the current block;

10) the current quantization parameter of the current block and/or its neighboring blocks;

11) indicating the color format (for example, 4:2:0, 4:4:4);

12) tree structure of single/double coding;

13) the type of transformation applied to the current block;

14) type of group of slices/tiles and/or type of image;

11. QR-DPCM can be applied segment by block

a) In one example, when the residual prediction direction is vertical, and N=nK, the residual prediction can be performed as

.

b) In one example, when the residual prediction direction is horizontal, and M=mK, the residual prediction may be performed as

.

12. Enabling/disabling QR-DPCM for one color component may occur because it is associated with another color component.

a) In one example, for a chrominance block, the inclusion of QR-DPCM may depend on the use of a QR-DPCM associated with one or more representative blocks within the associated luminance block.

1) In one example, a representative block may be defined in the same way as a block that is used to obtain the DM.

2) In one example, if a representative block in a combined luma block is QR-DPCM encoded and the current chroma block is encoded in DM mode, QR-DPCM may also be enabled for the current chroma block.

b) Alternatively, QR-DPCM may be signaled for chrominance components to indicate use.

1) In one example, a single flag may be signaled to indicate the use of two chrominance components.

2) Alternatively, two flags may be signaled, respectively, to indicate the use of two chroma components.

3) In one example, when a chroma block is encoded using certain modes (such as CCLM), signaling indicating the use of QR-DPCM is skipped.

13. The above methods may also be applicable to other DPCM/QR-DPCM variants.

5. Literature references

[1] ITU-T and ISO/IEC, “High efficiency video coding”, Rec. ITU-T H.265 | ISO/IEC 23008-2 (02/2018).

[2] B. Bross, J. Chen, S. Liu, Versatile Video Coding (Draft 4), JVET-M1001, Jan.2019

In fig. 10 is a block diagram of a video processing apparatus 1000. Apparatus 1000 may be used to implement one or more of the methods described herein. The device 1000 may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. The device 1000 may include one or more processors 1002, one or more storage devices 1004, and video processing hardware 1006. One or more processors 1002 may be configured to implement one or more of the methods described herein. One or more storage devices 1004 may be used to store data and code used to implement the methods and technologies described herein. Video processing hardware 1006 may be used to implement, in hardware circuits, some of the technologies described herein.

In fig. 11 is a flowchart of an exemplary video processing method 1100. Method 1100 includes performing 1102 a transformation between the current video block and a bitstream representation of the current video block using a differential encoding mode and selectively using an intra-frame prediction mode based on a coexistence rule. The intra-frame prediction mode is used to generate predictions for samples of the current video block. The differential encoding mode is used to represent a block of quantized residuals from pixel predictions using a differential pulse code modulation representation.

In fig. 14 is a block diagram illustrating an example of a video processing system 1400 in which various technologies disclosed herein may be implemented. Various implementations may include some or all of the components of the system 1400. The system 1400 may include an input 1402 for receiving video content. Video content may be received in a raw or uncompressed format, such as 8- or 10-bit multi-pixel values, or may be presented in a compressed or encoded format. Input 1402 may represent a network interface, a peripheral bus interface, or a storage device interface. Examples of a network interface include wired interfaces such as Ethernet, passive optical network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

System 1400 may include an encoding component 1404 that may implement various encoding or encryption methods described herein. The encoding component 1404 allows the average video bit rate from the input 1402 to the output of the encoding component 1404 to be reduced to create an encoded representation of the video. Therefore, encoding technologies are sometimes called video compression technologies or video transcoding technologies. The output of encoding component 1404 may be either stored or transmitted via a connected link, as represented by component 1406. The stored or transmitted bitmap (or encoded) representation of the video received at input 1402 may be used by component 1408 to generate pixel values or display video that sent to display interface 1410. The process of producing user-viewable video from a bitstream representation is sometimes called video decompression. In addition, although certain video processing operations are referred to as "encoding" operations or means, it should be understood that the encoding means or operations are used in the encoder, and the corresponding decoding means or operations that reverse the encoding results will be performed by the decoder.

Examples of the peripheral bus interface or display interface may include a universal serial bus (USB), a high-definition multimedia interface (HDMI), or a display port (Displayport), etc. Examples of storage device interfaces include SATA (Serial Advanced Technology), PCI, IDE interface, etc. The technologies described herein may be implemented in various electronic devices, such as mobile phones, laptops, smartphones, or other devices that are capable of digital data processing and/or video display.

Some embodiments of the disclosed technology include deciding or determining whether to enable a video processing facility or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode when processing a video block, but may not necessarily change the resulting bitstream based on the use of the tool or mode. That is, the conversion from a video block to a video bitstream representation will use the video processing facility or mode, when enabled, based on the decision or determination. In another example, when the video processing tool or mode is enabled, the decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from a video bitstream representation to a video block will be performed using the video processing engine or mode that has been enabled based on the decision or determination.

Some embodiments of the disclosed technology include deciding or determining whether to disable a video processing facility or mode. In one example, when a video processing tool or mode is disabled, the encoder will not use the tool or mode when converting a video block to a video bitstream representation. In another example, when the video processing tool or mode is disabled, the decoder will process the bitstream knowing that the bitstream has not been modified using the video processing tool or mode that was disabled based on the decision or determination.

In fig. 15 is a block diagram that illustrates an example video encoding system 100 that may use the methods of the present disclosure. As shown in FIG. 15, the video encoding system 100 may include a source device 110 and a destination device 120. The source device 110, which may be referred to as a video encoder, produces encoded video data. A destination device 120, which may be referred to as a video decoder, may decode encoded video data produced by a source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input/output (I/O) interface 116.

Video source 112 may include a source such as a video capture device, an interface for receiving video data from a video content provider, and/or a computer graphics system for generating video data, or a combination of such sources. Video data may contain one or more images. Video encoder 114 encodes video data from video source 112 to produce a bitstream. The bit stream may include a sequence of bits that form an encoded representation of video data. The bitstream may include encoded images and associated data. A coded image is a coded representation of an image. Associated data may include sequence parameter sets, image parameter sets, and other syntactic structures. I/O interface 116 may include a modulator/demodulator (modem) and/or a transmitter. The encoded video data may be transmitted directly to the recipient device 120 via I/O interface 116 via network 130a. The encoded video data may also be stored on storage medium/server 130b for access by recipient device 120.

Destination device 120 may include an input/output interface 126, a video decoder 124, and a display device 122.

I/O interface 126 may include a receiver and/or a modem. I/O interface 126 may receive encoded video data from source device 110 or media/server 130b. Video decoder 124 may decode encoded video data. The display device 122 may display the decoded video data to the user. Display device 122 may be integrated with target device 120 or may be external to target device 120 that is configured to interface with the external display device.

Video encoder 114 and video decoder 124 may operate in accordance with a video compression standard, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVM) standard, and other existing and/or additional standards.

In fig. 16 is a block diagram illustrating an example of video encoder 200, which may be video encoder 114 in the system 100 illustrated in FIG. 15.

Video encoder 200 may be configured to perform any or all of the methods of the present disclosure. In the example shown in FIG. 16, video encoder 200 includes a variety of functional components. The methods described in the present disclosure may be shared among various components of video encoder 200. In some examples, the processor may be configured to perform any or all of the methods described in the present disclosure.

Functional components of the video encoder 200 may include a division block 201, a prediction block 202, which may include a mode selection block 203, a motion estimation block 204, a motion compensation block 205 and an intra-frame prediction block 206, a residual generation block 207, a transform block 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy encoding unit 214.

In other examples, video encoder 200 may include more or fewer functional components, or different functional components. In an example, prediction block 202 may include an intra-block copy (IBC) block. The IBC block may perform prediction in an IBC mode in which at least one reference image is an image on which the current video block is located.

In addition, some components, such as motion estimation block 204 and motion compensation block 205, can be implemented with a high degree of integration, but in the example shown in FIG. 16, they are presented separately for ease of explanation.

The division unit 201 may divide the image into one or more video blocks. Video encoder 200 and video decoder 300 may support different video block sizes.

The mode selection unit 203 may select one of the encoding modes, intra-frame or inter-frame, for example, based on the error results, and provide the resulting intra-frame or inter-frame coded block to a residual generation unit 207 to generate the residual block data and to a recovery unit 212 to reconstruct the encoded block with for use as a reference image. In some example, the mode selector 203 may select a combination of an intra-frame and inter-frame prediction (CIIP) mode in which the prediction is based on an inter-frame prediction signal and an intra-frame prediction signal. The mode selector 203 may also select a motion vector resolution (eg, sub-pixel or integer pixel precision) for the block in the case of inter-frame prediction.

To perform inter-frame prediction for the current video block, motion estimator 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 to the current video block. Motion compensation unit 205 may determine a predicted video block for the current video block based on motion information and decoded image samples from buffer 213 that are different from the image associated with the current video block.

The motion estimation unit 204 and the motion compensation unit 205 may perform different operations for the current video block, for example, depending on whether the current video block is in an I-slice, a P-slice, or a B-slice.

In some examples, motion estimator 204 may perform unidirectional prediction for the current video block, and motion estimator 204 may retrieve reference images from list 0 or list 1 for the reference video block for the current video block. The motion estimator 204 may then generate a reference index that indicates a reference image in list 0 or list 1 that contains the reference video block, and a motion vector that indicates a spatial offset between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current block based on the reference video block specified in the motion information of the current video block.

In other examples, motion estimator 204 may perform bidirectional prediction for the current video block, motion estimator 204 may look up reference images in list 0 for a reference video block for the current video block, and can look up reference images in list 1 for another reference video block for the current video block. Motion estimator 204 may then generate reference indices that indicate reference pictures in list 0 and list 1 containing reference video blocks and motion vectors that indicate spatial offsets between the reference video blocks and the current video block. The motion estimation unit 204 may output reference indices and motion vectors of the current video block as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video blocks specified in the motion information of the current video block.

In some examples, motion estimator 204 may output a complete set of motion information for the decoder decoding process.

In some examples, motion estimator 204 may not output the full set of motion information for the current video. Rather, motion estimator 204 may signal motion information of the current video block with reference to motion information of another video block. For example, motion estimator 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of an adjacent video block.

In one example, motion estimator 204 may indicate, in a syntax structure associated with the current video block, a value that indicates to video decoder 300 that the current video block has the same motion information as another video block.

In another example, motion estimator 204 may identify, in the syntax structure associated with the current video block, another video block and a motion vector difference (MVD). The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the specified video block. Video decoder 300 may use the motion vector of the specified video block and the difference of the motion vectors to determine the motion vector of the current video block.

As discussed above, video encoder 200 may predictively signal a motion vector. Two examples of prediction signaling techniques that may be implemented by video encoder 200 include enhanced motion vector prediction (AMVP) and merge mode signaling.

The intra-frame prediction unit 206 may perform intra-frame prediction for the current video block. When intra-frame prediction unit 206 performs intra-frame prediction on the current video block, intra-frame prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same frame. The prediction data for the current video block may include the predicted video block and various syntax elements.

Residual generating unit 207 may generate residual data for the current video block by subtracting (eg, indicated by a minus sign) the predicted video block(s) of the current video block from the current video block. The current video block residual data may include residual video blocks that correspond to various sample components in the current video block.

In other examples, there may be no residual data for the current video block for the current video block, for example, in a skip mode, and the residual generating unit 207 may not perform the subtraction operation.

Transform processing unit 208 may generate one or more video blocks of transform coefficients for the current video block by applying one or more transforms to the remainder video block associated with the current video block.

After transform processing unit 208 produces a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.

The inverse quantization unit 210 and the inverse transform unit 211 may apply the inverse quantization and inverse transform to the transform coefficient video block, respectively, to recover the remainder video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more predicted video blocks generated by prediction unit 202 to obtain a reconstructed video block associated with the current block for storage in buffer 213.

After the recovery unit 212 recovers the video block, a contour filtering operation may be performed to reduce video block artifacts in the video block.

Entropy encoding block 214 may receive data from other functional components of video encoder 200. When entropy encoding block 214 receives data, entropy encoding block 214 may perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream that includes entropy encoding -encoded data.

In fig. 17 is a block diagram illustrating an example of video decoder 300, which may be video decoder 114 in the system 100 illustrated in FIG. 16.

Video decoder 300 may be configured to perform any or all of the methods of the present disclosure. In the example shown in FIG. 17, video decoder 300 includes a variety of functional components. The methods described in the present disclosure may be shared among various components of the video decoder 300. In some examples, the processor may be configured to perform any or all of the methods described in the present disclosure.

In the example shown in FIG. 17, video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a restoration unit 306, and a buffer 307. In some examples, the video decoder 300 may perform a decoding step, in generally the reverse of the encoding step described with respect to video encoder 200 (FIG. 16).

The entropy decoding unit 301 may extract the encoded bit stream. The coded bitstream may include entropy-coded video data (eg, coded blocks of video data). The entropy decoding unit 301 can decode the entropy-coded video data, and from the entropy-decoded video data, the motion compensation unit 302 can determine motion information including motion vectors, motion vector precision, reference picture list indices, and other motion information. The motion compensation unit 302 may, for example, determine such information by performing AMVP and merging mode.

Motion compensation block 302 may create motion compensated blocks, possibly performing interpolation based on interpolation filters. Identifiers for interpolation filters that must be used with sub-pixel precision can be included in syntax elements.

Motion compensation block 302 may use interpolation filters that are used by video encoder 20 during encoding of a video block to calculate interpolated values for sub-integer pixels of the reference block. The motion compensation block 302 may determine the interpolation filters used by the video encoder 200 in accordance with the received syntax information and use the interpolation filters to create prediction blocks.

Motion compensation block 302 may use some syntax information to determine the block sizes used to encode the frame(s) and/or slice(s) of the encoded video sequence, partition information that describes how each image macroblock of the encoded video sequence is divided into partitions, modes , indicating how each section is encoded, one or more reference frames (and lists of reference frames) for each inter-frame coded block, and other information for decoding the coded video sequence.

Intra-frame prediction block 303 may use intra-frame prediction modes, such as those received in a bitstream, to form a prediction block from spatially contiguous blocks. The inverse quantization unit 303 performs inverse quantization, that is, the inverse operation of quantization, on the quantized video block coefficients provided in the bit stream and decoded by the entropy decoding unit 301. The inverse transform block 303 applies the inverse transform.

The recovery unit 306 may add the remainder blocks with the corresponding prediction blocks generated by the motion compensation unit 202 or the intra-frame prediction unit 303 to generate decoded blocks. If necessary, the deblocking filter can also be used to filter the decoded blocks to remove blocking artifacts. The decoded video blocks are then stored in a buffer 307, which provides reference blocks for subsequent motion compensation/intra prediction and also produces decoded video for presentation on the display device screen.

In some embodiments, in ALWIP mode or MIP mode, the prediction block for the current video block is determined by row and column averaging followed by matrix multiplication, followed by interpolation to determine the prediction block.

In fig. 18 is a flowchart of an exemplary video processing method 1800. Method 1800 includes determining 1802, based on an applicability rule, that a differential coding mode is applicable to the conversion between a video chrominance video block and a bitstream representation of the chrominance video block. Operation 1804 includes performing, based on the determination, a transformation between a chroma video block and a bitstream representation of the chroma video block using a differential encoding mode, where, in the differential encoding mode, the chroma video block is represented in a bitstream representation using the difference between the quantized intra-prediction residual of the video block chrominance and quantized residual prediction, wherein intra-frame prediction is performed in a first direction and quantized residual prediction is performed in a second direction, and where, in differential coding mode, the difference between the quantized residual and the quantized residual prediction is represented using a differential pulse code modulation (DPCM) representation ).

In some embodiments of method 1800, a first direction or a second direction is determined from a vertical direction and a horizontal direction. In some embodiments of method 1800, intra-frame prediction is performed in a first direction, which is the same as the intra-frame prediction performed on a luma video block corresponding to a chrominance video block. In some embodiments of method 1800, the allowed intra-prediction mode for a luma video block is the same mode that is used to perform intra-prediction for a chrominance video block corresponding to the luma video block. In some embodiments of method 1800, the second direction allowed for a quantized remainder of a chrominance video block is the same as the direction used for another quantized remainder of a luminance video block corresponding to the chroma video block. In some embodiments of method 1800, the second direction used to predict the quantized residual is based on the second direction in which additional quantized residuals are predicted for a luma video block corresponding to a chrominance video block. In some embodiments of method 1800, a luma video block is combined with a chrominance video block. In some embodiments of method 1800, the luma video block includes a first sample that is aligned with a second sample located in the upper left corner of the chrominance video block. In some embodiments of method 1800, the luminance video block includes a first sample that is aligned with a second sample located in the center of the chrominance video block.

In some embodiments of method 1800, the applicability rule specifies that the differential encoding mode does not apply to a chroma video block when the chrominance video block is encoded in a cross-component linear model (CCLM) coding mode. In some embodiments of method 1800, the applicability rule specifies that a differential coding mode applies to a chroma video block when the chroma video block is encoded in a intercomponent linear model (CCLM) coding mode. In some embodiments of method 1800, the applicability rule specifies that the differential encoding mode does not apply to a chrominance video block when the chroma video block is encoded using a chrominance residual co-coding method. In some embodiments of method 1800, a video chrominance block is associated with a blue component or a red component.

In fig. 19 is a flowchart of an exemplary video processing method 1900. Method 1900 includes performing 1902 a transformation between the current video block of video and a bitstream representation of the current video block using a differential encoding mode and a coexistence rule-based intra-prediction mode, where, in the intra-prediction mode, the quantized intra-prediction remainder of the current video block is based on sample predictions for of the current video block, where, in differential encoding mode, the current video block is represented in a bitstream representation using the difference between the quantized remainder and the predicted quantized remainder, and where the difference between the quantized remainder and the predicted quantized remainder is represented using a differential pulse code modulation (DPCM) representation .

In some embodiments of method 1900, the intra-prediction mode is a matrix intra-prediction (MIP) mode in which sample predictions for the current video block are determined by performing, on the previously encoded video samples, a boundary downsampling operation followed by a matrix-vector multiplication operation. and which is selectively followed by an upsampling operation. In some embodiments of method 1900, the coexistence rule limits the MIP mode to a subset of allowed modes for the MIP mode. In some embodiments of method 1900, part of the set of enabled modes includes a horizontal intra-frame mode and/or a vertical intra-frame mode. In some embodiments of method 1900, a portion of the set of enabled modes includes only the horizontal intra-frame mode and/or the vertical intra-frame mode. In some embodiments of method 1900, the coexistence rule indicates that the MIP mode supports all allowed modes for the MIP mode. In some embodiments of method 1900, in intra-frame prediction mode, sample predictions for the current video block are determined by making predictions in a non-horizontal or non-vertical direction. In some embodiments of method 1900, the intra-frame prediction mode is a planar mode or a constant current prediction mode. In some embodiments of method 1900, the intra-prediction mode is a vertical prediction mode or a horizontal prediction mode. In some embodiments of method 1900, an intra-frame prediction mode is selected from a list of most probable modes (MPM). In some embodiments of method 1900, the intra-frame prediction mode is identified by a field in the bitstream representation. In some embodiments of method 1900, an intra-prediction mode is selected from a plurality of enabled intra-prediction modes.

In some embodiments of method 1900, a plurality of enabled intra-frame prediction modes are indicated in a message signaled in: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a picture header, a slice header, a tile group header, the line with the largest coding unit (LCU) or group of LCUs. In some embodiments of method 1900, the set of allowed intra-frame prediction modes is based on the block size of either the current video block or an adjacent video block of the current video block. In some embodiments of method 1900, the set of allowed intra-frame prediction modes is based on the shape of either the current video block or an adjacent video block of the current video block. In some embodiments of method 1900, the plurality of allowed intra-prediction modes is based on the intra-prediction mode of a neighboring video block of the current video block. In some embodiments of method 1900, the plurality of allowed intra-prediction modes is based on the inter-prediction mode of a neighboring video block of the current video block. In some embodiments of method 1900, the plurality of allowed intra-frame prediction modes is based on an indication of whether an adjacent video block of the current video block is encoded using a differential encoding mode. In some embodiments of method 1900, the set of allowed intra-frame prediction modes is based on the value of the quantization parameter used for the current video block or for an adjacent video block of the current video block. In some embodiments of method 1900, the set of allowed intra-frame prediction modes is based on the color format used to encode the current video block. In some embodiments of method 1900, the plurality of allowed intra-frame prediction modes is based on whether a single or dual tree coding structure is used to encode the current video block. In some embodiments of method 1900, the set of enabled intra-prediction modes is based on the slice, or tile group type, or image type associated with the current video block.

In some embodiments of method 1900, in intra-frame prediction mode, sample predictions for the current video block are determined from non-adjacent samples in an adjacent video block of the current video block. In some embodiments of method 1900, the intra-frame prediction mode comprises an intra-frame block copy merging mode, and where the coexistence rule indicates that the differential coding mode is enabled for the IBC merging mode. In some embodiments of method 1900, the intra-frame prediction mode comprises an enhanced intra-block copy motion vector prediction mode, and wherein the coexistence rule indicates that the differential encoding mode is enabled for the enhanced intra-block copy motion vector prediction mode. In some embodiments of method 1900, the intra-frame prediction mode includes an intra-frame block copy mode, and the motion vector used in the intra-frame block copy mode and the differential encoding mode is indicated by a field in the bitstream representation, or is obtained or determined in advance.

In some embodiments of method 1900, the intra-frame block copy mode is indicated by a motion vector and/or a merge index. In some embodiments of method 1900, the intra-frame block copy mode is indicated by a default motion vector. In some embodiments of method 1900, the default motion vector is (-w, 0), where w is a positive integer. In some embodiments of method 1900, the default motion vector is (0, -h), where h is a positive integer. In some embodiments of method 1900, the default motion vector is (-w, -h), where w and h are positive integers. In some embodiments of method 1900, a field that indicates a motion vector is included in the message, a message signaled in: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a picture header, a slice header, a header tile group, Largest Coding Unit (LCU) string, or LCU group. In some embodiments of method 1900, the motion vector specified in the field is based on the block size of either the current video block or an adjacent video block of the current video block. In some embodiments of method 1900, the motion vector specified in the field is based on the shape of either the current video block or an adjacent video block of the current video block. In some embodiments of method 1900, the motion vector indicated in the field is based on the inter-frame prediction mode or intra-frame prediction mode of a neighboring video block of the current video block. In some embodiments of method 1900, the motion vector indicated in the field is based on the motion vectors of an adjacent video block of the current video block. In some embodiments of method 1900, the motion vector indicated in the field is based on an indication of whether an adjacent video block of the current video block is encoded using a differential encoding mode.

In some embodiments of method 1900, the motion vector specified in the field is based on the value of the quantization parameter of the current video block or an adjacent video block of the current video block. In some embodiments of method 1900, the motion vector specified in the field is based on the color format used to encode the current video block. In some embodiments of method 1900, the motion vector specified in the field is based on whether a single or dual tree encoding structure is used to encode the current video block. In some embodiments of method 1900, the motion vector specified in the field is based on a slice, or tile group type, or image type associated with the current video block. In some embodiments of method 1900, in intra-frame prediction mode, sample predictions for the current video block are determined from non-adjacent samples in an adjacent video block of the current video block using inter-frame prediction means.

In some embodiments of method 1900, a coexistence rule specifies a mapping of a signaled index associated with a differential encoding mode to an intra-frame prediction mode. In some embodiments of method 1900, the mapping is indicated in a message signaled in: Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Slice Header, Tile Group Header, Largest One Row coding (LCU) or LCU group. In some embodiments of method 1900, the matching is based on the block size of either the current video block or a neighboring video block of the current video block. In some embodiments of method 1900, the matching is based on the shape of either the current video block or a neighboring video block of the current video block. In some embodiments of method 1900, the matching is based on the most probable modes (MPM) of the current video block or a neighboring video block of the current video block. In some embodiments of method 1900, the matching is based on the inter-prediction mode or intra-prediction mode of a neighboring video block of the current video block. In some embodiments of method 1900, the matching is based on motion vectors of an adjacent video block of the current video block. In some embodiments of method 1900, the matching is based on an indication of whether an adjacent video block of the current video block is encoded using a differential encoding mode. In some embodiments of method 1900, the matching is based on the value of a quantization parameter of the current video block or a neighboring video block of the current video block. In some embodiments of method 1900, the mapping is based on the color format used to encode the current video block. In some embodiments of method 1900, the matching is based on whether a single or dual coding tree structure is used to encode the current video block. In some embodiments of method 1900, the matching is based on the type of transform applied to the current video block. In some embodiments of method 1900, the matching is based on the type of slice, or group of tiles, or image type associated with the current video block.

In some embodiments of method 1900, the quantized residual prediction is performed in a residual prediction direction that is different from the horizontal direction or the vertical direction. In some embodiments of method 1900, the remainder prediction direction is a 45 degree direction. In some embodiments of method 1900, where indicates the quantized remainder, where indicates the counts of the quantized remainder, and where can be obtained using in response to availability . In some embodiments of method 1900, the direction of the residual prediction is 135 degrees. In some embodiments of method 1900, where indicates the quantized remainder, where indicates the counts of the quantized remainder, and where can be obtained using in response to availability .

In some embodiments of method 1900, the quantized remainder prediction is performed in the direction of a remainder prediction that is based on the first value and the second value. In some embodiments of method 1900, where indicates the quantized remainder, where indicates the counts of the quantized remainder, where can be obtained using in response to availability , where m is the first value, and n is the second value, where i is the third value indicating the first position along the plurality of rows of the current video block, and where j is the fourth value indicating the second position along the plurality of columns of the current video block. In some embodiments of the method 1900, m and n are signaled in a bitstream representation. In some embodiments of the method 1900, m and n are integers.

In some embodiments of the method 1900, m and n are indicated in a message included in: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a picture header, a slice header, a tile group header, a line largest coding unit (LCU) or LCU group. In some embodiments of method 1900, m is based on the third value for i, and n is based on the fourth value for j. In some embodiments of method 1900, the first value for m and the second value for n are based on the block size of either the current video block or an adjacent video block of the current video block. In some embodiments of method 1900, the first value for m and the second value for n are based on the shape of either the current video block or an adjacent video block of the current video block. In some embodiments of method 1900, the first value for m and the second value for n are based on the most probable modes (MPM) of the current video block or a neighboring video block of the current video block. In some embodiments of method 1900, the first value for m and the second value for n are based on the inter-prediction mode or intra-prediction mode of a neighboring video block of the current video block. In some embodiments of method 1900, the first value for m and the second value for n are based on the motion vectors of a neighboring video block of the current video block. In some embodiments of method 1900, the first value for m and the second value for n are based on an indication of whether an adjacent video block of the current video block is encoded using a differential encoding mode. In some embodiments of method 1900, the first value for m and the second value for n are based on the value of the quantization parameter of the current video block or an adjacent video block of the current video block. In some embodiments of method 1900, the first value for m and the second value for n are based on the color format used to encode the current video block. In some embodiments of method 1900, the first value for m and the second value for n are based on whether a single encoding tree or a dual encoding tree is used to encode the current video block. In some embodiments of method 1900, the first value for m and the second value for n are based on the slice, tile group type, or image type associated with the current video block.

In fig. 20 is a flowchart of an exemplary video processing method 2000. Method 2000 includes performing 2002 a transformation between a current video block of video and a bitstream representation of the current video block using a differential encoding mode, in which the current video block is represented in a bitstream representation using the difference between the quantized intra-prediction residual of the current video block and the quantized residual prediction, where the direction The quantized remainder prediction is derived from the bitstream representation.

In some embodiments of method 2000, the prediction direction of the quantized remainder is implicitly obtained from the intra prediction mode used for intra prediction. In some embodiments of method 2000, the prediction direction of the quantized remainder is output as a direction that is the same as the direction used for the intra prediction mode. In some embodiments of method 2000, the direction is vertical or horizontal. In some embodiments of method 2000, the quantized remainder prediction direction may be a first direction that is different from the second direction used for the intra-prediction mode. In some embodiments of method 2000, the first direction is vertical and the second direction is horizontal. In some embodiments of method 2000, the first direction is horizontal and the second direction is vertical.

In some embodiments of method 2000, the quantized remainder prediction direction is obtained based on a message signaled in: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a picture header, a slice header, a tile group header, the line with the largest coding unit (LCU) or group of LCUs. In some embodiments of method 2000, the prediction direction of the quantized remainder may be obtained based on the block size of either the current video block or a neighboring video block of the current video block. In some embodiments of method 2000, the quantized residual prediction direction is inferred based on the shape of either the current video block or a neighboring video block of the current video block. In some embodiments of method 2000, the quantized residual prediction direction is inferred based on the most probable modes (MPMs) of the current video block or a neighboring video block of the current video block. In some embodiments of method 2000, the prediction direction of the quantized remainder is derived based on the inter-prediction mode or intra-frame prediction mode of a neighboring video block of the current video block. In some embodiments of method 2000, the quantized residual prediction direction is obtained based on the motion vectors of an adjacent video block of the current video block. In some embodiments of method 2000, the quantized remainder prediction direction is inferred based on an indication of whether an adjacent video block of the current video block is encoded using a differential encoding mode. In some embodiments of method 2000, the quantized remainder prediction direction is obtained based on the value of the quantization parameter of the current video block or a neighboring video block of the current video block. In some embodiments of method 2000, the quantized remainder prediction direction is obtained based on the color format used to encode the current video block. In some embodiments of method 2000, the quantized remainder prediction direction is obtained based on whether a single or dual tree encoding structure is used to encode the current video block. In some embodiments of method 2000, the prediction direction of the quantized remainder is inferred based on the type of transform applied to the current video block. In some embodiments of method 2000, the quantized remainder prediction direction is obtained based on the slice, or tile group type, or image type associated with the current video block.

In fig. 21 is a flowchart of an exemplary video processing method 2100. Method 2100 includes determining 2102 that a differential encoding mode is applicable to a conversion between the current video block of video and a bitstream representation of the current video block, and performing 2104, based on the determination, a conversion between the current video block and the bitstream representation of the current video block using the differential encoding mode. according to an implementation rule where, in differential encoding mode, the current video block is represented in a bitstream representation using the difference between the quantized intra-prediction residual of the current video block and the prediction of the quantized remainder, and where, in differential encoding mode, the difference between the quantized remainder and the prediction of the quantized the remainder is represented using a differential pulse code modulation (DPCM) representation.

In some embodiments of method 2100, the implementation rule specifies that the quantized remainder values are limited within a range of values. In some embodiments of method 2100, the implementation rule specifies that the difference values between the quantized remainder and the predicted quantized remainder are limited within a range of values. In some embodiments of method 2100, the implementation rule specifies that a constraint operation is applied to the difference between the quantized remainder and the predicted quantized remainder, and the constraint operation constrains the values of the difference between the quantized remainder and the predicted quantized remainder within a range of values. In some embodiments of method 2100, the implementation rule specifies that a constraint operation is applied to the quantized remainder, and the constraint operation constrains the values of the quantized remainder within a range of values.

In some embodiments of method 2100, the difference between the quantized remainder and the predicted quantized remainder is determined in accordance with any of the following equations:

,

And

,

Where describes the difference between the quantized remainder and the prediction of the quantized remainder,

where Q(r _i,j ) describes the values of the quantized remainder, where M and N, respectively, describe the rows and columns of the current video block, and where i and j, respectively, describe the positions along the rows and columns of the current video block.

In some embodiments of method 2100, the quantized remainder is determined in accordance with any of the following equations:

,

And

,

Where And describe the difference between the quantized remainder and the predicted quantized remainder, where Q(r _i,j ) describes the quantized remainder, where M and N, respectively, describe the rows and columns of the current video block, and where i and j, respectively, describe the positions along the rows and columns current video block.

In some embodiments of method 2100, each value of the values is constrained by a constraint operation according to the following constraint rule: the value is a minimum value in response to a value less than the minimum value, the value is a maximum value in response to a value greater than the maximum value. In some embodiments of method 2100, each value of the values is constrained by a constraint operation according to the following constraint rule: the value is a minimum value in response to the value being less than or equal to the minimum value, and is a maximum value in response to the value being greater than or equal to the maximum meaning. In some embodiments of method 2100, each value of the values is constrained by a constraint operation according to the following constraint rule: the value is a minimum value in response to the value being less than the minimum value, and the value is a maximum value in response to the value being greater than or equal to the maximum value. . In some embodiments of method 2100, each value of the values is constrained by a constraint operation according to the following constraint rule: the value is a minimum value in response to the value being less than or equal to the minimum value, and is a maximum value in response to the value being greater than the maximum value.

In some embodiments of method 2100, the minimum value and/or maximum value is or is a positive value or a negative value. In some embodiments of method 2100, the minimum value is -32768 and the maximum value is 32767. In some embodiments of method 2100, the minimum value and/or maximum value is based on a different range of values that relates to inverse quantization for a video block that is not encoded in the differential coding. In some embodiments of method 2100, the minimum value and/or maximum value are based on the bit depth of the input quantized residue block sample or the reconstructed sample of the reconstructed quantized residue block. In some embodiments of method 2100, the minimum value and/or maximum value are based on whether the current video block is encoded or decoded using a lossless encoding method. In some embodiments of method 2100, the minimum value and/or maximum value is based on an indication of whether a transquantum bypass flag is turned on, where whether a lossless encoding method is used is indicated by a transquantum bypass flag. In some embodiments of method 2100, the minimum value and/or maximum value is based on an indication of whether a coding unit (CU) transquantum bypass flag is turned on, where whether a lossless encoding method is used at the CU level is indicated by the CU transquantum bypass flag. In some embodiments of method 2100, the minimum value and/or maximum value is based on a message signaled in: sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS), picture header, slice header, tile group header elements, Largest Coding Unit (LCU) string, or LCU group.

In some embodiments of method 2100, the minimum value and/or maximum value are based on the block size of either the current video block or an adjacent video block of the current video block. In some embodiments of method 2100, the minimum value and/or maximum value are based on the shape of either the current video block or an adjacent video block of the current video block. In some embodiments of method 2100, the minimum value and/or maximum value are based on the most probable modes (MPM) of the current video block or a neighboring video block of the current video block. In some embodiments of method 2100, the minimum value and/or maximum value are based on the inter-prediction mode or intra-frame prediction mode of a neighboring video block of the current video block. In some embodiments of method 2100, the minimum value and/or maximum value are based on the motion vectors of a neighboring video block of the current video block. In some embodiments of method 2100, the minimum value and/or maximum value are based on an indication of whether an adjacent video block of the current video block is encoded using a differential encoding mode. In some embodiments of method 2100, the minimum value and/or maximum value are based on the value of the quantization parameter of the current video block or an adjacent video block of the current video block. In some embodiments of method 2100, the minimum value and/or maximum value are based on the color format used to encode the current video block. In some embodiments of method 2100, the minimum value and/or maximum value are based on whether a single or dual tree encoding structure is used to encode the current video block.

In some embodiments of method 2100, the minimum value and/or maximum value are based on the type of transform applied to the current video block. In some embodiments of method 2100, the minimum value and/or maximum value are based on the slice, tile group type, or image type associated with the current video block. In some embodiments of method 2100, the applicability rule specifies that the quantized remainder prediction is performed in the direction from the last row or last column of the current video block to the first row or first column of the current video block. In some embodiments of method 2100, in response to the direction being horizontal, the prediction of the value of the quantized remainder in the i-th column of the current video block is the value of the quantized remainder in the (i+1)-th column of the current video block. In some embodiments of method 2100, in response to the direction being vertical, the prediction of the quantized remainder value in the i-th row of the current video block is the quantized remainder value in the (i+1)-th row of the current video block.

In some embodiments of method 2100, the applicability rule specifies that the differential coding mode applies to a subset or portion of the current video block. In some embodiments of method 2100, the applicability rule specifies that the differential encoding mode does not apply to the k leftmost columns of the current video block when the quantized remainder prediction is performed in the horizontal direction. In some embodiments of method 2100, the applicability rule specifies that the differential encoding mode does not apply to the topmost lines of the current video block when quantized remainder prediction is performed in the vertical direction. In some embodiments of method 2100, the applicability rule specifies that the differential encoding mode does not apply to the rightmost columns of the current video block when quantized remainder prediction is performed in the horizontal direction. In some embodiments of method 2100, the applicability rule specifies that the differential encoding mode does not apply to the lowest k lines of the current video block when the quantized remainder prediction is performed in the vertical direction. In some embodiments of method 2100, the value of k is a given value.

In some embodiments of method 2100, the value of k is indicated in a message signaled in: Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Slice Header, Tile Group Header, Line with Largest coding unit (LCU) or LCU group. In some embodiments of method 2100, the value of k is based on the block size of either the current video block or an adjacent video block of the current video block. In some embodiments of method 2100, the value of k is based on the shape of either the current video block or a neighboring video block of the current video block. In some embodiments of method 2100, the value of k is based on the most probable modes (MPM) of the current video block or a neighboring video block of the current video block. In some embodiments of method 2100, the value of k is based on the inter-prediction mode or intra-prediction mode of a neighboring video block of the current video block. In some embodiments of method 2100, the value of k is based on the intra-prediction mode of the current video block. In some embodiments of method 2100, the value of k is based on motion vectors of an adjacent video block of the current video block. In some embodiments of method 2100, the value of k is based on an indication of whether an adjacent video block of the current video block is encoded using a differential encoding mode.

In some embodiments of method 2100, the value of k is based on the value of the quantization parameter of the current video block or an adjacent video block of the current video block. In some embodiments of method 2100, the value of k is based on the color format used to encode the current video block. In some embodiments of method 2100, the value of k is based on whether a single encoding tree or a dual encoding tree is used to encode the current video block. In some embodiments of method 2100, the value of k is based on the type of transform applied to the current video block. In some embodiments of method 2100, the value of k is based on the type of slice or tile group or image type associated with the current video block.

In some embodiments of method 2100, the implementation rule specifies that the differential coding mode is applied on a segment-by-segment basis to the current video block. In some embodiments of method 2100, the implementation rule specifies that the quantized residual samples are determined in accordance with the following formula when the quantized residual prediction is performed in the vertical direction, and when N=nK:

,

Where describes the difference between the quantized remainder and the predicted quantized remainder, where Q(r _i,j ) describes the quantized remainder, where M and N, respectively, describe the rows and columns of the current video block, where i and j, respectively, describe the positions along the rows and columns of the current video block, where n and K are integers.

In some embodiments of method 2100, the implementation rule specifies that the quantized residual samples are determined in accordance with the following formula when the quantized residual prediction is performed in the horizontal direction, and when M = mK:

,

Where describes the samples of the quantized remainder, where Q(r _i,j ) describes the quantized remainder, where M and N, respectively, describe the rows and columns of the current video block, where i and j, respectively, describe the positions along the rows and columns of the current video block, and where m and K are integers.

In fig. 22 is a flowchart of an exemplary video processing method 2200. Method 2200 includes determining 2202, based on an applicability rule, that a differential encoding mode is applicable to the transformation between the current video block of video and the bitstream representation of the current video block, where the current video block is associated with a color component, and where the applicability rule specifies that is included or differential encoding mode is disabled for the current video block, depending on whether differential encoding mode is applied to another video video block. Method 2200 also includes performing 2204, based on the determination, a transformation between the current video block and a bitstream representation of the current video block using a differential encoding mode, where, in the differential encoding mode, the current video block is represented in a bitstream representation using the difference between the quantized remainder of the intraframe prediction of the current video block and prediction of the quantized remainder, and where, in differential coding mode, the difference between the quantized remainder and the prediction of the quantized remainder is represented using a differential pulse code modulation (DPCM) representation.

In some embodiments of method 2200, the current video block is a chroma video block, the applicability rule specifies whether the differential encoding mode is enabled for the chroma video block based on whether the differential encoding mode is applied to one or more representative luminance video blocks, and one or more representative video blocks brightness are combined with the color video block. In some embodiments of method 2200, one or more representative luma video blocks are defined in the same manner as that used to derive the resulting mode for intra-frame prediction for the chrominance video block. In some embodiments of method 2200, the current video block is a chroma video block, the applicability rule specifies that a differential encoding mode is enabled for a chroma video block when the resulting mode is used for intra-frame prediction of a chroma video block, and a differential encoding mode is enabled for one or more representative luminance blocks. and one or more representative luma video blocks are combined with the chrominance video block. In some embodiments of method 2200, the applicability rule specifies whether the differential encoding mode is on or off for the current video block in one or more syntax elements included in the bitstream representation.

In some embodiments of method 2200, one syntax element indicates whether the differential encoding mode is enabled or disabled for two color components, and the two color components include a color component associated with the current video block. In some embodiments of method 2200, two syntax elements indicate whether the differential encoding mode is enabled or disabled for two color components, respectively, and the two color components include a color component associated with the current video block. In some embodiments of method 2200, the applicability rule specifies that the bitstream representation excludes a syntax element that indicates whether the differential encoding mode is enabled or disabled for the current video block when a particular mode is used for intra-frame prediction of the current video block. In some embodiments of method 2200, the particular mode includes a cross-component linear model (CCLM).

The following list of examples is a description of additional embodiments.

1. A video processing method, comprising: performing a conversion between a current video block and a bitstream representation of the current video block using a differential encoding mode and selectively using an intra-frame prediction mode based on a coexistence rule; where the intra-frame prediction mode is used to generate predictions for samples of the current video block; and wherein a differential encoding mode is used to represent a block of quantized residuals from the pixel predictions using a differential pulse code modulation representation.

2. The method according to Example 1, wherein the intra prediction mode is a matrix intra prediction (MIP) mode, and wherein the coexistence rule limits the MIP to a portion of the allowed MIP modes.

3. The method according to example 2, in which part of the allowed modes includes horizontal or vertical normal intra-frame mode.

Additional embodiments of Examples 1-3 are described in Section 4, Clause 1. For example, the differential encoding mode may represent the current version of the QR-BDPCM encoding mode.

4. The method according to Example 1, wherein the intra-frame prediction mode includes prediction in a non-horizontal or non-vertical direction.

5. The method according to Example 1 or 4, wherein the intra prediction mode is a planar mode or a constant current prediction mode.

6. The method according to Example 1 or 4, wherein the intra prediction modes are a vertical prediction mode or a horizontal prediction mode.

7. The method of Example 1 or 4, wherein the intra-frame prediction mode is identified by a field in the bitstream representation.

8. The method according to Example 1 or 4, in which the intra-frame prediction mode depends on the block size of the current video block or a neighboring block.

9. The method according to Example 1 or 4, in which the intra-prediction mode depends on the shape of the current block or a neighboring block.

10. The method according to Example 1 or 4, wherein the intra-prediction mode depends on whether the current video block or an adjacent video block is encoded using inter-frame prediction or intra-frame prediction.

11. The method according to Example 1 or 4, wherein the intra-frame prediction mode depends on whether the adjacent video block is encoded using a differential encoding mode.

12. The method according to Example 1 or 4, wherein the intra-frame prediction mode depends on the value of the quantization parameter used for the current video block or an adjacent video block.

13. The method according to Example 1 or 4, wherein the intra-frame prediction mode depends on the color format used to encode the current video block.

14. The method according to Examples 1 or 4, wherein the intra-frame prediction mode depends on whether a single tree encoding structure or a dual tree encoding structure is used to encode the current video block.

Additional embodiments of Examples 4-14 are presented in paragraph 2 of section 4.

15. The method according to example 1, in which the formation of predictions for samples of the current video block is performed from non-adjacent samples in the adjacent video area.

16. The method according to Example 1, wherein the intra-frame prediction mode comprises an intra-frame block copy merging mode.

17. The method according to Example 1, wherein the intra-frame prediction mode comprises an improved intra-frame block motion vector prediction mode.

18. The method as in any one of Examples 15-17, wherein the intra-prediction mode is indicated by a block vector or a merge index.

Additional embodiments of Examples 15-18 are presented in paragraph 3 of section 4.

19. The method of Example 1, wherein the coexistence rule specifies a mapping from a signaled index in a differential encoding mode to a field-based intra-frame prediction mode in the bitstream representation.

20. The method of Example 1, wherein the coexistence rule specifies a mapping from the signaled index in the differential encoding mode to the intra-frame prediction mode based on the size of the current video block or a neighboring block.

21. The method according to Example 1, wherein the coexistence rule specifies a mapping from the signaled index in the differential encoding mode to the intra-frame prediction mode based on the shape of the current video block or a neighboring block.

22. The method according to Example 1, wherein the coexistence rule specifies a mapping from a signaled index in a differential encoding mode to an intra-frame prediction mode based on the prediction mode of the current video block or a neighboring block.

23. The method of Example 1, wherein the coexistence rule specifies a mapping from the signaled index in the differential encoding mode to the intra-frame prediction mode based on the most likely modes of the current video block or an adjacent block.

24. The method of Example 1, wherein the coexistence rule specifies a mapping from a signaled index in a differential encoding mode to an intra-frame prediction mode based on motion vectors of the current video block or an adjacent block.

25. The method of Example 1, wherein the coexistence rule specifies a mapping from the signaled index in the differential encoding mode to the intra-frame prediction mode based on whether an adjacent block is encoded using the differential encoding mode.

26. The method of Example 1, wherein the coexistence rule specifies a mapping from a signaled index in a differential encoding mode to an intra-frame prediction mode based on a quantization parameter used by the current video block or a neighboring block.

27. The method of Example 1, wherein the coexistence rule specifies a mapping from the signaled index in the differential encoding mode to the intra-frame prediction mode based on the color format of the current video block.

28. The method of Example 1, wherein the coexistence rule specifies a mapping from the signaled index in the differential encoding mode to the intra-frame prediction mode based on whether the current video block uses a single or dual encoding tree.

29. The method of Example 1, wherein the coexistence rule specifies a mapping from a signaled index in a differential encoding mode to an intra-frame prediction mode based on a transform applied to the current video block.

30. The method of Example 1, wherein the coexistence rule specifies a mapping from a signaled index in a differential encoding mode to an intra-frame prediction mode based on a slice type, a tile group type, or an image type of the current video block.

Additional embodiments of examples 19-30 are presented in paragraph 2 of section 4.

31. A video processing method, comprising: performing a conversion between a current video block and a bitstream representation of the current video block using a differential encoding mode, in which a block of quantized residuals from pixel predictions of the current video block is represented using a differential pulse code modulation representation; where the first prediction direction or the second differential encoding mode direction is derived from the bitstream representation.

32. The method according to Example 31, wherein the first pixel prediction direction is implicitly obtained from an intra-prediction mode used for prediction.

33. The method according to Example 32, in which the second direction of the differential encoding mode may be the same direction as the first prediction direction.

34. The method according to Example 31, wherein the second direction is obtained from an intra prediction mode used for prediction.

35. The method according to Example 31, wherein the second direction is obtained from the size of the current video block or neighboring block or the shape of the current video block or neighboring block.

36. The method according to example 31, in which the second direction is obtained from the motion vectors of the adjacent block.

37. The method according to example 31, in which the second direction is obtained from the most probable modes of the current video block or an adjacent block.

38. The method according to Example 31, wherein the second direction is obtained from the neighboring block prediction mode.

39. The method according to Example 31, wherein the second direction is obtained from an intra-adjacent block prediction mode.

40. The method of Example 31, wherein the second direction is derived from whether the adjacent block uses a differential encoding mode.

Additional embodiments of examples 31-40 are presented in paragraph 4 of section 4.

41. A video processing method, comprising: determining, based on an applicability rule, that a differential encoding mode is applicable to a conversion between a current video block and a bitstream representation of the current video block; and performing a conversion between the current video block and the bitstream representation using a differential encoding mode;

where, in differential encoding mode, a block of quantized residuals from the intra-frame pixel prediction of the current video block is represented using a differential pulse code modulation representation performed in a residual prediction direction that is different from the horizontal or vertical direction.

42. The method according to Example 41, wherein the remainder prediction direction is the direction at an angle of 45 degrees.

43. The method according to Example 41, wherein the remainder prediction direction is 135 degrees.

44. The method according to Example 41, wherein the direction of the residual prediction is related to a field in the bitstream representation or the size of the current video block or neighboring block or the shape of the current video block or neighboring block.

Additional embodiments of examples 41-44 are presented in paragraph 6 of section 4.

45. The method of Example 41, wherein the applicability rule specifies the use of a differential encoding mode due to the fact that the current video block is a chrominance block.

46. The method according to Example 45, wherein the applicability rule further specifies that the residual prediction direction for the current video block is the same direction as for the luminance block corresponding to the current video block.

47. The method of Example 41, wherein the applicability rule specifies the use of differential coding due to the fact that the current video block does not use a cross-component linear model (CCLM) coding mode.

Additional embodiments of Examples 45-47 are presented in paragraph 7 of section 4.

48. The method according to Example 41, wherein the applicability rule specifies to obtain the applicability of the differential encoding mode for one color component from the applicability of the differential encoding mode for another color component.

Additional embodiments of Example 48 are presented in section 4, paragraph 12.

49. A method of processing video, comprising: determining that a differential encoding mode is applicable to a conversion between a current video block and a bitstream representation of the current video block; and performing a conversion between the current video block and the bitstream representation using a differential encoding mode implementation rule; wherein, in the differential encoding mode, a block of quantized residuals from the intra-frame pixel prediction of the current video block is represented using a differential pulse code modulation representation performed in a residual prediction direction that is different from the horizontal or vertical direction.

50. The method of Example 49, wherein the implementation rule specifies a limit on the values of a block of quantized residues within a range.

51. The method of Example 49, wherein the implementation rule specifies the use of the constraint to obtain a block of quantized residues.

Additional embodiments of examples 49-51 are presented in paragraph 8 of section 4.

52. The method of Example 49, wherein the implementation rule specifies that the prediction is performed from the last line of the current video block to the first line of the current video block.

53. The method of Example 49, wherein the implementation rule specifies that the prediction is made from the last column of the current video block to the first column of the current video block.

Additional embodiments of examples 52-53 are presented in paragraph 9 of section 4.

54. The method of Example 49, wherein the implementation rule specifies that the differential coding mode is applied to only a subset of the current video block.

55. The method of Example 54, in which the subset eliminates the k left columns of the remainders, where k is an integer that is less than the width of the block in pixels.

56. The method of Example 54, in which the subset eliminates the top k rows of the remainders, where k is an integer that is less than the height of the block in pixels.

Additional embodiments of examples 54-56 are presented in paragraph 10 of section 4.

57. The method of Example 49, wherein the implementation rule specifies the application of a differential coding mode on a segment-by-segment basis to the transform.

Additional embodiments of Example 57 are presented in paragraph 11 of section 4.

58. A video processing device comprising a processor configured to implement one or more of Examples 1-57.

59. A computer-readable storage medium having code stored thereon, wherein the code, when executed by a processor, causes the processor to implement the method set forth in any one or more examples 1-57.

In the list of examples herein, the term "transformation" may refer to deriving a bitstream representation for the current video block or deriving the current video block from the bitstream representation. The bitstream representation need not represent a contiguous group of bits and may be divided into bits that are included in header fields, or into codewords that represent information about the encoded pixel values.

In the above examples, the applicability rule may be predefined and known to the encoders and decoders.

It should be understood that the disclosed methods may be implemented in video encoders or decoders to improve compression efficiency using methods that include the use of various implementation rules regarding the use of a differential encoding mode in intra-frame encoding, as described herein.

The disclosed and other solutions, examples, embodiments, modules and functional operations described herein may be implemented in digital electronic circuits or in computer software, firmware or hardware, including the structures disclosed herein and their structural equivalents, or a combination of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, that is, one or more computer program instruction modules encoded on a computer-readable storage medium for execution or control of operation from a data processing device. A computer-readable storage medium may be a computer-readable storage device, a computer-readable storage medium, a memory device, a composition of matter providing a propagated machine-readable signal, or a combination of one or more of these. The term "data processing device" covers all hardware devices, devices and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The device may include, in addition to hardware, code that creates an execution environment for the computer program in question, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these. A propagation signal is an artificially generated signal, such as an electrical, optical, or electromagnetic machine-generated signal, that is generated to encode information for transmission to a suitable receiving device.

A computer program (also known as a program, software, application, script, or code) can be written in any form of a programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file on a file system. A program may be stored in part of a file that contains other programs or data (for example, one or more scripts stored in a document markup language), in a single file dedicated to the program in question, or in multiple coherent files (for example, files that store one or multiple modules, subprograms, or pieces of code). A computer program can be deployed for execution on a single computer or on multiple computers that are located in one location or distributed across multiple sites and interconnected by a communications network.

The processes and logical flows described herein may be executed by one or more programmable processors executing one or more computer programs to perform functions by manipulating input data and producing output data. In addition, processes and logic flows may also be executed, and the device may also be implemented as a special purpose logic circuit, such as an FPGA (Field Programmable Gate Array) or ASIC (Application Application Integrated Circuit).

Processors suitable for executing a computer program include, by way of example, both general purpose and special purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor only accepts instructions and data to read memory or RAM, or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. In general, a computer will also include, or be operatively coupled to receive data from or transmit data to, or both, one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks. However, the computer does not necessarily have to have such devices. Computer-readable media suitable for storing computer program instructions and data include all types of non-volatile memory, data storage devices and storage devices, including, by way of example, semiconductor memory devices such as, for example, EPROMs, EEPROMs and flash memory devices; magnetic disks, such as internal hard drives or removable drives; magneto-optical disks; and CD ROMs and DVD-ROMs. The processor and memory may be complemented by or incorporated into specialized logic circuitry.

Although this patent document contains many specific details, they should not be considered as limiting the scope of the subject matter of the invention or anything that may be claimed, but rather they should be considered as a description of functions that may be specific to particular embodiments of particular methods. Certain features described herein in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of one embodiment may also be implemented in multiple embodiments individually or in any suitable subcombination. Moreover, although features have been described above as operating in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be excluded from the combination, and the claimed combination may be directed to a subcombination or a variation of a subcombination.

Likewise, although operations are depicted in the drawings in a particular order, this should not be understood to require that such operations be performed in a particular order stated or in a sequential order, or that all illustrated operations be performed to achieve the desired results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood to require such separation in all embodiments.

Only a few implementations and examples have been described above, and other implementations, improvements and changes may be made based on what is described and illustrated in this patent document.

Claims (34)

1. A method for processing video data, comprising the steps of: determining, for conversion between the current video video block and the video bitstream, that a differential encoding mode used for the remainder samples is applied to the current video block; And perform, based on said definition, said transformation using a differential encoding mode, wherein, in the differential encoding mode, the differences between the quantized residuals obtained by intra-frame prediction and the quantized residual predictors are represented in a bit stream, wherein at least one of the reconstructed quantized residuals is obtained based on the sum of one difference and the other reconstructed quantized residual, and the values of the reconstructed quantized residuals are limited to a certain range by applying a constraint operation to the reconstructed quantized residuals. 2. The method according to claim 1, in which each of the values of the reconstructed quantized residuals is limited by a restriction operation as follows: the value is equal to the minimum value in response to the value being less than the minimum value, and value is equal to the maximum value in response to the value being greater than the maximum value. 3. The method of claim 2, wherein the minimum value is a negative value and the maximum value is a positive value. 4. The method of claim 3, wherein the minimum value is set to -32768 and the maximum value is set to 32767. 5. The method of claim 1, wherein the application or non-application of the differential encoding mode to the current video block is indicated by one or more syntax elements included in the bitstream. 6. The method of claim 1, wherein the single syntax element indicates whether the differential encoding mode is applied to the two chrominance components. 7. The method of claim 6, wherein a single syntactic element is excluded from the bitstream in response to using a particular mode for intra-frame prediction of two chroma components. 8. The method of claim 7, wherein the defined mode includes an intercomponent linear model. 9. The method according to claim 1, in which predictors of quantized residuals are generated at the sample level. 10. The method of claim 9, wherein the differential encoding mode is applied in the horizontal direction or in the vertical direction. 11. The method according to claim 1, in which, at the conversion stage, the current video block is encoded into a bitstream. 12. The method according to claim 1, in which, at the conversion stage, the current video block is decoded from the bit stream. 13. A video data processing device containing a processor and non-volatile memory with instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to execute: determining, for conversion between the current video video block and the video bitstream, that the differential encoding mode used for the remainder samples is applied to the current video block; And performing, based on said definition, said transformation using a differential encoding mode, wherein, in the differential encoding mode, the differences between the quantized residuals obtained by intra-frame prediction and the quantized residual predictors are represented in a bit stream, wherein at least one of the reconstructed quantized residuals is obtained based on the sum of one difference and the other reconstructed quantized residual, and the values of the reconstructed quantized residuals are limited to a certain range by applying a constraint operation to the reconstructed quantized residuals. 14. A non-volatile computer-readable storage medium on which instructions are stored that cause the processor to execute: determining, for conversion between the current video video block and the video bitstream, that the differential encoding mode used for the remainder samples is applied to the current video block; And performing, based on said definition, said transformation using a differential encoding mode, wherein, in the differential encoding mode, the differences between the quantized residuals obtained by intra-frame prediction and the quantized residual predictors are represented in a bit stream, wherein at least one of the reconstructed quantized residuals is obtained based on the sum of one difference and the other reconstructed quantized residual, and the values of the reconstructed quantized residuals are limited to a certain range by applying a constraint operation to the reconstructed quantized residuals. 15. A method for storing a video bitstream, comprising the steps of: determining that the differential encoding mode used for the residual samples is applied to the current video video block; generating a video bitstream based on the specified definition using a differential encoding mode; And store the video bitstream in a non-volatile computer-readable storage medium, wherein, in the differential encoding mode, the differences between the quantized residuals obtained by intra-frame prediction and the quantized residual predictors are represented in a bit stream, wherein at least one of the reconstructed quantized residuals is obtained based on the sum of one difference and the other reconstructed quantized residual, and the values of the reconstructed quantized residuals are limited to a certain range by applying a constraint operation to the reconstructed quantized residuals.