RU2810950C2 - Joint coding of palette mode indication - Google Patents

Abstract

FIELD: video processing tools.

SUBSTANCE: prediction mode for the specified block, based on one or more valid prediction modes, including at least a palette mode, is determined for conversion between a video domain block and a data bit stream representation of the video. The palette mode indication is determined according to the prediction mode. Conversion is carried out based on the specified one or more valid prediction modes.

EFFECT: increased efficiency of video encoding.

19 cl, 34 dwg

Description

Field of technology to which the invention relates

This document concerns video and image encoding technologies.

State of the art

Digital video occupies the largest share of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and presenting video on a display increases, bandwidth requirements for digital video use are expected to continue to grow.

Disclosure of the invention

The proposed technology can be used in embodiments of video or image decoders or encoders that employ palette mode encoding.

In one exemplary aspect, a method for processing video is provided. The method comprises converting between a video domain of a video and a representation of the video as a stream of data bits. The data bitstream representation is processed according to a first format rule that determines whether to report a first palette mode indication for the specified block in question, and a second format rule that determines the position of the first indication relative to the second prediction mode use indication for the specified block.

In one exemplary aspect, a method for processing video is provided. The method comprises determining, for conversion between a video domain block and a data bitstream representation of the video, a prediction mode based on one or more valid prediction modes, which include at least a palette mode, for the specified block. The palette mode usage indication is determined according to the prediction mode. The method also includes performing a conversion based on said one or more valid prediction modes.

According to another exemplary aspect, a video processing method is provided. The method comprises converting between a video block and a representation of the video as a stream of data bits. The data bitstream representation is processed in accordance with a formatting rule that defines a first palette mode use indication and a second intra block copy (IBC) mode use indication, which are signaled in dependence on each other.

According to another exemplary aspect, a video processing method is provided. The method comprises determining, for conversion between a video block and a bitstream video representation, whether there is an indication of the use of a palette mode in the bitstream representation based on the size of the specified block; and perform the conversion based on the specified definition.

According to another exemplary aspect, a video processing method is provided. The method comprises determining, for conversion between a video block and a data bitstream representation of the video, whether an indication is available for using an intraframe block copy (IBC) mode in the data bitstream representation based on the size of the specified block; and perform the conversion based on the specified definition.

According to another exemplary aspect, a video processing method is provided. The method comprises determining, for conversion between a video block and a data bitstream representation of the video, whether a palette mode is valid for said block, based on a second indication for a video area containing said block; and perform the conversion based on the specified definition.

According to another exemplary aspect, a video processing method is provided. The method comprises determining, for conversion between a video block and a data bit stream representation of the video, whether an intra-block copy (IBC) mode is valid for the specified block, based on a second indication for a video area containing the specified block; and perform the conversion based on the specified definition.

According to another exemplary aspect, a video processing method is provided. The method comprises determining that a palette mode should be used to process a transform unit, encoding block, or region, wherein the use of the palette mode is coded separately from the prediction mode, and further processing the transform unit, encoding block, or region using the palette mode.

According to another exemplary aspect, a video processing method is provided. The method comprises determining, for a current video block, that a sample associated with one input palette position of a palette mode has a first bit depth different from a second bit depth associated with the current video block, and performing based on said at least one input palette position, further processing of the current video block.

According to yet another exemplary aspect, another video processing method is proposed. The method comprises performing a conversion between a current video block of an image or video and a data bitstream representation of the video, wherein information about whether an intra-frame block copy mode is used in the conversion is reported in the data bitstream representation or inferred based on encoding conditions of the current video block, and the intra-frame block copying mode contains encoding of the current video block from another video block in the image.

According to yet another exemplary aspect, another video processing method is proposed. The method includes the steps of determining whether to apply a deblocking filter in the process of transforming the current video block of an image or video, wherein the current video block is encoded using palette mode encoding, in which the current video block is represented using representative sample values that are less than the total number of pixels in the current video block, and perform a transformation such that the deblocking filter is applied in the case where it is determined that the deblocking filter is to be applied.

According to yet another exemplary aspect, another video processing method is proposed. The method comprises the steps of determining a quantization procedure or an inverse quantization procedure for use in the conversion between a current video block of an image or video and a representation of the video as a stream of data bits, wherein the current video block is encoded using palette mode encoding, wherein the current video block is represented using values representative samples, the number of which is less than the total number of pixels in the current video block, and perform a transformation based on the specified definition of the quantization procedure or the inverse quantization procedure.

According to yet another exemplary aspect, another video processing method is proposed. The method comprises determining, for conversion between a current video block of a video containing a plurality of video blocks, and a data bitstream representation of the video, that the current video block is a palette-mode encoded block; based on the specified definition, a procedure for constructing a list of the most probable modes is performed by considering the current video block as a block encoded using intra-frame prediction, and a transformation is performed based on the result of the list construction procedure; wherein the block encoded in palette mode is encoded or decoded using the palette or representative sample values.

According to yet another exemplary aspect, another video processing method is proposed. The method contains

According to yet another exemplary aspect, another video processing method is proposed. The method comprises determining, for conversion between a current video block of a video containing a plurality of video blocks, and a data bitstream representation of the video, that the current video block is a palette-mode encoded block; based on the specified definition, performing procedures for constructing a list of the most probable mode by considering the current video block as a block encoded without using intra-frame prediction, and performing a transformation based on the result of the list construction procedure; wherein the block encoded in palette mode is encoded or decoded using the palette or representative sample values.

According to yet another exemplary aspect, another video processing method is proposed. The method comprises determining, for conversion between a current video block of a video containing a plurality of video blocks, and a data bitstream representation of the video, that the current video block is a palette-mode encoded block; based on the specified definition, performing list construction procedures by considering the current video block as an unavailable block, and performing a transformation based on the result of the list construction procedure; wherein the block encoded in the palette encoding mode is encoded or decoded using the palette or representative sample values.

According to yet another exemplary aspect, another video processing method is proposed. The method includes the steps of determining, during the conversion between the current video block and the data bitstream representation of the current video block, that the current video block is a block encoded in palette mode, determining, based on the fact that the current video block is a block encoded in palette mode, the range of context-sensitive encoding bins used for the transformation, and performs the transformation based on the range of context-sensitive encoding bins.

According to yet another exemplary aspect, the method described above may be implemented by a video encoding device including a processor.

In yet another exemplary aspect, the methods may be implemented as processor-executable instructions and stored on a computer-readable program medium.

These and other aspects are further described herein.

Brief description of drawings

Fig. 1 shows an example of intra-frame block copying.

Fig. Figure 2 shows an example of a block encoded in palette mode.

Fig. Figure 3 shows an example of using a palette predictor to pass input palette positions to a form.

Fig. 4 shows an example of horizontal and vertical cross-scanning.

Fig. 5 shows an example of encoding palette indices.

Fig. 6 is a block diagram of an example of a video processing apparatus.

Fig. 7 shows a block diagram of an example implementation of a video encoder.

Fig. 8 is a logic diagram of an example of a video processing method.

Fig. 9 shows an example of pixels involved in filter on/off decision and strong/weak filter selection.

Fig. 10 shows an example of binarization of four modes.

Fig. 11 shows an example of binarization of four modes.

Fig. 12 shows examples of 67 directions for intra-frame prediction.

Fig. 13 shows examples of adjacent video blocks.

Fig. 14 shows examples of ALF filter shapes (chrominance: 5x5 diamond, luminance: 7x7 diamond).

Fig. 15(a) shows examples of calculations of the subsampled Laplace operator for the vertical gradient.

Fig. 15(b) shows examples of calculations of the subsampled Laplace operator for a horizontal gradient.

Fig. 15(c) shows examples of calculations of the subsampled Laplace operator for a diagonal gradient.

Fig. 15(d) shows examples of calculations of the subsampled Laplace operator for a diagonal gradient.

Fig. 16 shows examples of modified block classification by virtual boundaries.

Fig. 17 shows examples of modified ALF filtering for the luminance component at virtual boundaries.

Fig. 18 shows an example of four one-dimensional (1-D) 3-pixel structures for classifying pixels in an edge offset (EO) structure.

Fig. 19 shows an example of four stripes grouped together and represented by the position of the starting strip.

Fig. 20 shows an example of the top and left adjacent blocks used to output weights for the CIIP prediction mode.

Fig. 21 shows an example architecture for luma display with chrominance scaling.

Fig. 22 shows an example of the scan order for a 4x4 block.

Fig. 23 shows another example of the scan order for a 4x4 block.

Fig. 24 is a block diagram of an example of video processing in which the proposed technology can be implemented.

Fig. 25 shows a logic diagram representation of a video processing method in accordance with the proposed technology.

Fig. 26 shows a logic diagram representation of another video processing method in accordance with the proposed technology.

Fig. 27 shows another logic diagram of another video processing method in accordance with the proposed technology.

Fig. 28 shows another logic diagram of another video processing method in accordance with the proposed technology.

Fig. 29 shows another logic diagram of another video processing method in accordance with the proposed technology.

Fig. 30 shows another logic diagram of another video processing method in accordance with the proposed technology.

Fig. 31 shows another different logic diagram of another video processing method in accordance with the proposed technology.

Carrying out the invention

This document proposes various techniques that can be used by decoding devices to decode an image or video data bitstream to improve the quality of the enhanced or decoded digital video or images. For brevity, the term "video" is used here to cover both a sequence of images (traditionally called a "video") and individual images. Moreover, the video encoder may also implement these techniques during the encoding process to reconstruct decoded frames used for further encoding.

The section headings in this document are used for ease of understanding and are not intended to limit the options and techniques to those sections only. Therefore, options from one section can be combined with options from other sections.

1. General Provisions

This document applies to video encoding technologies. In particular, the document relates to palette mode encoding using a base color representation in video encoding. The document can be applied to existing video coding standards, such as High Efficiency Video Coding (HEVC) or the Versatile Video Coding (VVC) standard, which is not yet complete. It may also be applicable to future video processing standards or video codecs.

2. Initial discussion

Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards. ITU-T has released H.261 and H.263, ISO/IEC has released MPEG-1 and MPEG-4 Visual, and the two have jointly released H.262/MPEG-2 Video and H.264/ MPEG-4 Advanced Video Coding (AVC) and H.265/HEVC. Since the H.262 standard, video coding standards have been based on a hybrid video coding structure using time prediction plus transformation coding. To research future video encoding technologies that will be developed after HEVC encoding technology, the Video Coding Experts Group (VCEG) and the Motion Picture Experts Group (MPEG) jointly founded the Joint Video Exploration Team in 2015 (JVET)). Since then, the JVET team has developed many techniques and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) formed the Joint Video Expert Team (JVET) to work on the VVC standard, with the goal of reducing required data bit rate by 50% compared to HEVC encoding.

In fig. 7 is a block diagram of an example implementation of a video encoder. In fig. 7 shows that the encoder implementation has a built-in feedback loop in which the video encoder also performs video decoding functions (reconstructing a compressed representation of video data for use in encoding subsequent video data).

2.1 Intra-frame block copying mode

Intra-block copy (IBC) mode, also called using the current image as a reference, has been adopted in the HEVC Screen Content Coding extensions (HEVC-SCC) and in the test model of today's version of the VVC coding standard (VTM-4.0). ). The IBC copy principle extends the concept of motion compensation from inter-prediction coding to intra-prediction coding. As shown in FIG. 1, when IBC copying is applied, the current block is predicted based on a reference block from the same image. The samples from the reference block must already be reconstructed before the current block is encoded or decoded. Although the IBC copy mode is not as efficient as most camcorder-captured sequences, it does show significant gains in encoding screen content. The reason for this is the presence of large groups of repeating structures, such as icons and text characters, in the display of screen content. The IBC copy mode can effectively eliminate the redundancy between these repeating structures. In the HEVC-SCC standard, a coding unit (CU) using inter-picture prediction can apply the IBC copy mode if it selects the current picture as the reference picture. In this case, the motion vector (MV) is renamed a block vector (BV), and this BV vector is always accurate to an integer pixel. To ensure compatibility with the main HEVC encoding profile, the current picture is marked as a “long-term” reference picture in the Decoded Picture Buffer (DPB). It should be noted that similarly, in multi-view/3D video coding standards, an inter-view reference picture is also labeled as a “long-term” reference picture.

By following the BV vector to find the reference block, the prediction can be generated by copying the reference block. The remainder can be obtained by subtracting the reference pixels from the original signals. Transformation and quantization can then be applied, as in other encoding modes.

However, when the reference block is outside the image, or is superimposed on the current block, or is outside the reconstructed region, or is outside the actual region limited by some limiting factors, some or all of the pixel values are undefined. There are basically two technical solutions to deal with this problem. One solution is to prevent this situation from happening, for example by matching data bit streams. Another solution is to apply padding to undefined pixel values. The following subsections describe these technical solutions in detail.

2.2 IBC Copy Mode in HEVC Screen Content Coding Extensions

In extensions with screen content encoding in the HEVC encoding standard, when any block uses the current image as a reference, it must ensure that the entire reference block is within the available reconstructed region, as indicated by the following special text (bold):

Thus, the case where the reference block overlaps the current block or the reference block is outside the image will not occur. Therefore, there is no need to “fill in” the reference or forecast block.

2.3 Copying IBC in Test Model for VVC Encoding

In today's VVC encoding test model, such as the VTM-4.0 draft document, the entire reference block must be in the current coding tree unit (CTU) and must not overlap the current block. Thus, there is no need to "fill in" the reference or forecast block. The IBC copy flag is encoded to indicate the prediction mode for the current CU. Thus, there are a total of three forecast modes, MODE_INTRA, MODE_INTER and MODE_IBC for each CU.

2.3.1 Copy mode with merge (IBC Merge)

In the IBC merge copy mode, an index indicating an input position in the candidate list for this IBC merge copy mode is extracted by parsing from the data bit stream. The list construction procedure for IBC copy mode with merge can be summarized according to the following sequence of steps:

Stage 1: Formation of spatial candidates

Step 2: Inserting Candidates for HMVP Prediction

Step 3: Inserting pairwise averaged candidates

When generating spatial merging candidates, a maximum of four merging candidates are selected from the pool of candidates located at the positions shown in the drawings. Candidates are selected in the following order: A _1, B _1, B _0, A ₀ and B ₂ . Position B ₂ is only taken into account when any of the PU units that should be in positions A ₁ , B ₁ , B ₀ , A ₀ is not available (for example, because this unit belongs to another slice or tile) or is not encoded in IBC copy mode. After adding a candidate at position A _{1 ,} the addition of the remaining candidates must be done with redundancy control, which ensures that candidates with the same motion information are excluded from the list, so that coding efficiency is improved. To reduce computational complexity, not all possible candidate pairs are considered in the redundancy check process mentioned above. In contrast, only pairs connected by an arrow are considered, as shown in the drawings, and any candidate is added to the list only if the corresponding candidate used for redundancy control does not have the same motion information.

After inserting spatial candidates, if the size of the IBC join copy mode candidate list is still less than the maximum size of the IBC join copy mode candidate list, the IBC copy mode candidates from the HMVP prediction table can be inserted. When inserting candidates from the HMVP prediction table, redundancy checks are performed.

Finally, the pairwise averaged candidates are added to the candidate list for the IBC copy mode with pooling.

When the reference block identified by a merging candidate is outside the image, or is superimposed on the current block, or is outside the reconstructed region, or is outside the valid region limited by some limiting factors, this merging candidate is called an invalid merging candidate.

Note that invalid merge candidates may be inserted into the candidate list for IBC copy mode with merge.

2.3.2 AMVP Prediction Mode with IBC Copy

In the IBC copy AMVP prediction mode (hereinafter referred to as the IBC AMVP mode), an AMVP prediction index indicating an input position in the IBC AMVP mode list is obtained by parsing from the data bit stream. The construction of the IBC AMVP mode list can be summarized according to the following sequence of steps:

Stage 1: Formation of spatial candidates

Check A0, A1 until an available candidate is found.

Check B0, B1, B2 until an available candidate is found.

Step 2: Inserting Candidates for HMVP Prediction

Step 3: Inserting Null Candidates

After inserting spatial candidates, if the list size for an IBC AMVP mode is still less than the maximum list size for that IBC AMVP mode, candidates for the IBC copy mode from the HMVP prediction table can be inserted.

Finally, zero candidates are inserted into this list for the IBC AMVP mode.

2.4 Palette mode

The basic idea behind palette mode is that samples in a CU are represented by a small set of representative color values. This set is called a palette. It is also possible to indicate an off-palette sample by signaling a dropout symbol followed by (possibly quantized) component values. This is illustrated in FIG. 2.

2.5 Palette mode in the HEVC encoding standard with Screen Content Coding Extensions (HEVC-SCC)

In palette mode in the HEVC-SCC standard, prediction is used to encode the palette and index map.

2.5.1 Coding input palette positions

To encode input palette positions, a palette predictor is supported. The maximum palette size, as well as the palette predictor, is reported as an alarm in the SPS set. In the HEVC-SCC encoding standard, the palette_predictor_initializer_present_flag flag is inserted into the PPS set. When this flag is 1, the input positions for initializing the palette predictor are signaled in the data bitstream. The palette predictor is initialized at the beginning of each line of CTUs, each slice, and each tile. Depending on the value of the palette_predictor_initializer_present_flag, the palette predictor is set to 0 or initialized using the palette predictor initializer input positions signaled in the PPS set. In the HEVC-SCC coding standard, a palette predictor initializer of size 0 was enabled to allow explicit de-initialization of the palette predictor at the PPS set level.

For each input position in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette. This is illustrated in Fig. 3. Reuse flags are transmitted using run-of-zero length encoding. Thereafter, the number of new input positions is signaled using an exponential Golomb code of order 0. Finally, the component values for the new palette input positions are signaled.

2.5.2 Encoding palette indices

Palette indices are encoded using horizontal and vertical transverse scanning passes, as shown in FIG. 4. The scan order is signaled in the data bitstream using the palette_transpose_flag. The rest of this subsection assumes that the scan is horizontal.

Palette indices are encoded using two main modes for palette samples: 'index mode' ('INDEX') and 'copy top mode' 'COPY_ABOVE'. As explained earlier, the drop symbol is also signaled as 'INDEX' mode and assigned an index equal to the maximum palette size. The mode indication is signaled using a flag, except in the case of the top line or in the case where the previous mode was the 'COPY_ABOVE' mode. In 'COPY_ABOVE' mode, the palette index is copied from the row above. In 'INDEX' mode, the palette index is explicitly signaled. For both modes 'INDEX' and 'COPY_ABOVE' the run length is reported in signaling form, which specifies the number of consecutive samples also encoded in the same mode. When a hit symbol is part of a series in 'INDEX' or 'COPY_ABOVE' mode, for each hit symbol the component values of that symbol are signaled. The encoding of palette indices is illustrated in FIG. 5.

The syntactic order for this is as follows. First, the number of index values for the CU is reported. This is followed by signaling the actual index values for the entire CU using truncated binary encoding. Both the number of indexes and the index values are encoded in bypass mode. This groups index-related bypass sections together. The palette sample mode (if necessary) and burst size are then signaled in an interleaved manner. The dropout component values corresponding to the dropout samples for the entire CU are then grouped together and bypass encoded.

An additional syntax element, last_run_type_flag, is signaled after the index values are signaled. This syntax element, in combination with the number of indices, eliminates the need to signal the run value corresponding to the last run in the block.

In the HEVC-SCC encoding standard, palette mode is also enabled for 4:2:2, 4:2:0 and black and white color formats. The signaling of input palette positions and palette indices is almost identical for all color formats. In the case of non-black and white (non-monochrome) formats, each input palette position consists of 3 components. For the black and white format, each input palette position consists of one component. For subsampled chroma directions, the chroma samples are associated with the luminance sample indices, which are divisible by 2. After reconstructing the palette indices for the CU, if only one component is associated with any sample, only the first component of the input palette position is used. The only difference in signaling is only for the values of the dropped components. For each outlier sample, the number of outlier component values signaled may vary depending on the number of components associated with that sample.

In the VVC coding standard, two coding tree structures are used when encoding intra-frame prediction slices, so that the luma component and the two chrominance components can have different palettes and palette indices. In addition, the two color components share the same palette and the same palette indices.

2.6 Deblocking scheme in the VVC coding standard

Note that in the following descriptions, the symbol pN _M denotes the N-th count in the M-th row on the left side relative to the vertical edge or the N-th count in the M-th column on the top side relative to the horizontal edge, the symbol qN _M denotes the N-th count the Mth row on the right side relative to the vertical edge or the Nth count in the Mth column on the bottom side relative to the horizontal edge. An example of the symbols pN _M and qN _M is shown in FIG. 9.

Note that in the following descriptions, the symbol p _N denotes the Nth count in a row on the left side relative to the vertical edge or the Nth count in a column on the top side relative to the horizontal edge, the symbol q _N denotes the Nth count in a row on the right side relative to the vertical edge or the Nth count in the column on the bottom side relative to the horizontal edge.

The decision to turn the filter on/off is made for four rows as one unit. Fig. 9 illustrates the pixels involved in filter on/off decisions. The 6 pixels in the two red frames for the first four rows are used to determine whether the filter is on/off for the 4 rows. The 6 pixels in the two red frames for the second four rows are used to determine whether the filter is on/off for those second four rows.

In some embodiments, the vertical edges in the image are filtered first. The horizontal edges of the image are then filtered using as input the samples modified by filtering the vertical edges. The vertical and horizontal edges in the CTBs of each CTU are processed separately based on the coding units. The vertical edges of the coding blocks in a coding unit are filtered starting from an edge on the left side of the coding blocks and moving through the edges towards the right side of the coding blocks in their geometric order. The horizontal edges of the coding blocks in a coding unit are filtered starting from the top edge of the coding blocks and moving through the edges towards the bottom side of the coding blocks in their geometric order.

2.6.1 Boundary decision

Filtering is applied to the boundaries of 8x8 blocks. In addition, it must be a transform block boundary or an encoding subblock boundary (eg, due to the use of affine motion prediction, ATMVP). For objects that are not such boundaries, filtering is not activated.

2.6.2 Calculation of boundary rigidity

For a transform block boundary/encoding subblock boundary, if it is located in an 8x8 grid, it can be filtered and setting the boundary hardness parameter bS[ xD _i ][ yD _j ] (where [ xD _i ][ yD _j ] denotes the coordinates) for that edge defined as follows:

- If samples p ₀ or q ₀ are in a coding block in a coding unit that is encoded in intra-prediction mode, the parameter bS[ xD _i ][ yD _j ] is set to 2.

- Otherwise, if the block edge is also a transform block edge, and sample p ₀ or q ₀ is in a transform block having one or more levels of non-zero transformation coefficients, the parameter bS[ xD _i ][ yD _j ] is set to 1.

- Otherwise, if the prediction mode for the encoding subblock containing sample p ₀ is different from the prediction mode for the encoding subblock containing sample q ₀ , the parameter bS[ xD _i ][ yD _j ] is set to 1.

- Otherwise, if one or more of the following conditions is true, the parameter bS[ xD _i ][ yD _j ] is set to 1:

- Both subblocks - the encoding subblock containing sample p ₀ and the encoding subblock containing sample q ₀ - are encoded in the IBC copy prediction mode, and the absolute difference between the horizontal or vertical components of the motion vectors used in predicting the two encoding subblocks is not less 4 units of measurement of brightness component counts in quarters.

- To predict the encoding subblock containing the sample p ₀ , reference images or a number of motion vectors that are different from those images and the number of vectors used in predicting the encoding subblock containing the sample q ₀ are used.

Note 1 – The determination of whether the reference pictures used for the specified two encoding subblocks are the same or different is based only on which pictures are referenced as the reference, regardless of whether prediction is made using an index pointing to list 0 reference pictures, or using an index pointing to reference picture list 1, and regardless of whether the indexed position in the reference picture list is different.

Note 2 – The number of motion vectors used to predict a coding subblock in which the top-left sample covers the point ( xSb, ySb ) is PredFlagL0[ xSb ][ ySb ] + PredFlagL1[ xSb ][ ySb ].

- One motion vector is used to predict the encoding subblock containing sample p ₀ , and one motion vector is used to predict the encoding subblock containing sample q ₀ , and the absolute difference between the horizontal or vertical components of the motion vectors used is not less than 4 in quarters of the luminance sample unit component.

- Two motion vectors and two different reference pictures are used to predict a coding subblock containing sample p ₀ , two motion vectors for the same two reference pictures are used to predict a coding subblock containing sample q ₀ , and the absolute difference between the horizontal or vertical components of the two motion vectors used to predict the two encoding subblocks for the same reference image of at least 4 in quarter units of luminance component samples.

- Two motion vectors for the same reference picture are used to predict a coding subblock containing sample p ₀ , two motion vectors for the same reference picture are used to predict a coding subblock containing sample q ₀ , and both of the following conditions are true:

- The absolute difference between the horizontal or vertical components of motion vectors from list 0 used in predicting two coding subblocks is not less than 4 in quarter units of luminance component samples, or the absolute difference between the horizontal or vertical components of motion vectors from list 1 used in predicting two coding subblocks, not less than 4 in quarters of the unit of measurement of the brightness component samples.

- The absolute difference between the horizontal or vertical components of the motion vector from list 0 used in predicting the coding subblock containing sample p ₀ and the motion vector from list 1 used in predicting the coding subblock containing sample q ₀ is not less than 4 in quarters of a unit of measurement samples of the luminance component, or the absolute difference between the horizontal or vertical components of the motion vector from list 1, used when predicting a coding subblock containing sample p ₀ , and the motion vector from list 0, used when predicting a coding subblock containing sample q ₀ , not less than 4 in quarters of the unit of measurement of brightness component counts.

- Otherwise, the variable bS[ xD _i ][ yD _j ] is set to 0.

Tables 2-1 and 2-2 summarize the rules for calculating boundary stiffness (BS).

Table 2-1. Boundary hardness (when SPS IBC mode is not activated)

A priority Conditions Y U V 5 At least one of the adjacent blocks is encoded using intra-frame prediction 2 2 2 4 The boundary of the TU unit and at least one of the neighboring blocks has non-zero transformation ratios 1 1 1 3 The reference images or number of MV vectors (1 for unidirectional prediction, 2 for bidirectional prediction) for adjacent blocks differ 1 N/A N/A 2 The absolute difference between motion vectors for the same reference image, belonging to adjacent blocks, is not less than one unit of brightness reading 1 N/A N/A 1 Other 0 0 0

Table 2-2. Boundary hardness (when SPS IBC mode is activated)

A priority Conditions Y U V 8 At least one of the adjacent blocks is encoded using intra-frame prediction 2 2 2 7 The boundary of the TU unit and at least one of the neighboring blocks has non-zero transformation ratios 1 1 1 6 The prediction modes for adjacent blocks are different (for example, one of the modes is an IBC copy mode, and one of the modes is an inter-frame prediction mode) 1 5 Both the IBC copy mode and the absolute difference between the motion vectors belonging to adjacent blocks are not less than one unit of brightness reading 1 N/A N/A 4 The reference images or number of MV vectors (1 for unidirectional prediction, 2 for bidirectional prediction) for adjacent blocks differ 1 N/A N/A 3 The absolute difference between motion vectors for the same reference image, belonging to adjacent blocks, is not less than one unit of brightness reading 1 N/A N/A 1 Other 0 0 0

2.6.3 Release decision for luminance component

This subsection describes the release decision procedure.

Wider - stronger luminance filters are used only when all three conditions - Condition1, Condition2 and Condition3 - are TRUE.

Condition 1 is the "large block condition". This condition determines whether samples on the P-side and Q-side belong to large blocks, as represented by the variables bSidePisLargeBlk and bSideQisLargeBlk, respectively. These variables bSidePisLargeBlk and bSideQisLargeBlk are defined as follows.

bSidePisLargeBlk = ((edge type is vertical and sample p ₀ belongs to CU unit with width >= 32) | | (edge type is horizontal and sample p ₀ belongs to CU unit with height >= 32))? TRUE: FALSE

bSideQisLargeBlk = ((edge type is vertical and sample q ₀ belongs to CU unit with width >= 32) | | (edge type is horizontal and sample q ₀ belongs to CU unit with height >= 32))? TRUE: FALSE

Based on the variables bSidePisLargeBlk and bSideQisLargeBlk, condition 1 is defined as follows.

Condition1 = (bSidePisLargeBlk || bSidePisLargeBlk) ? TRUE: FALSE

Next, if Condition 1 is true, Condition 2 must be further tested. First, the following variables are defined:

- dp0, dp3, dq0, dq3 are first determined, as in the HEVC encoding standard

- if (side p is not less than 32)

dp0 = ( dp0 + Abs( p5 ₀ − 2 * p4 ₀ + p3 ₀ ) + 1 ) >> 1

dp3 = ( dp3 + Abs( p5 ₃ − 2 * p4 ₃ + p3 ₃ ) + 1 ) >> 1

- if (side q is not less than 32)

dq0 = ( dq0 + Abs( q5 ₀ − 2 * q4 ₀ + q3 ₀ ) + 1 ) >> 1

dq3 = ( dq3 + Abs( q5 ₃ − 2 * q4 ₃ + q3 ₃ ) + 1 ) >> 1

Condition2 = (d < β) ? TRUE: FALSE

where d= dp0 + dq0 + dp3 + dq3, as shown in section 2.2.4.

If Condition1 and Condition2 are satisfied, next check whether any of the blocks use subblocks:

If (bSidePisLargeBlk)

If (block mode P == SUBBLOCKMODE)

Sp =5

otherwise

Sp =7

otherwise

Sp=3

If (bSideQisLargeBlk)

If (block mode Q == SUBBLOCKMODE)

Sq =5

otherwise

Sq =7

otherwise

Sq = 3

Finally, if Condition 1 and Condition 2 are valid, the proposed deblocking method will check condition 3 (the strong filter condition for a large block), which is defined as follows.

For Condition3 (Strong Filter Condition) – StrongFilterCondition, define the following variables:

dpq is defined as in the HEVC encoding standard.

sp ₃ = Abs( p ₃ − p ₀ ), defined as in the HEVC encoding standard

if (side p is not less than 32)

if(Sp==5)

sp ₃ = ( sp ₃ + Abs( p ₅ − p ₃ ) + 1) >> 1

otherwise

sp ₃ = ( sp ₃ + Abs( p ₇ − p ₃ ) + 1) >> 1

sq ₃ = Abs( q ₀ − q ₃ ), defined as in the HEVC encoding standard

if (side q is not less than 32)

If(Sq==5)

sq ₃ = ( sq ₃ + Abs( q ₅ − q ₃ ) + 1) >> 1

otherwise

sq ₃ = ( sq ₃ + Abs( q ₇ − q ₃ ) + 1) >> 1

As in the HEVC encoding standard, StrongFilterCondition = (dpq less than ( β >> 2 ), sp ₃ + sq ₃ less than ( 3*β >> 5 ), and Abs( p ₀ − q ₀ ) less than ( 5 * t _C + 1 ) >> 1) ? TRUE: FALSE.

2.6.4 Stronger luminance deblocking filter (designed to work with larger blocks)

When samples on either side of the boundary belong to a large block, a bilinear filter is applied. The fact that a reference belongs to a large block is established when the width is >= 32 for a vertical edge, and when the height is >= 32 for a horizontal edge.

The bilinear filter is described below.

Located on the block boundary, samples p _i for i=0 by Sp-1 and q _i for j=0 by Sq-1 (pi and qi represent the i-th sample in the row for filtering the vertical edge, or the i-th sample in the column for horizontal edge filtering) in the deblocking filter described above according to the HEVC coding standard is then replaced by linear interpolation as follows:

-

-

where the terms tcPD _i and tcPD _j define the position-dependent truncation described in Section 2.3.6 and the parameters g _j , f _i , Middle _s,t , P _s and Q _s are given in Table 2-3:

Table 2-3. Release filters with long branches

Sp, Sq
7, 7
(side p: 7,
side
q: 7)


7, 3
(p side: 7
side q: 3)


3, 7
(p side: 3
side q: 7)


7, 5
(p side: 7
side q: 5)


5, 7
(p side: 5
side q: 7)


5, 5
(p side: 5
side q: 5)


5, 3
(p side: 5
side q: 3)


3, 5
(p side: 3
side q: 5)

2.6.5 Release control for the chrominance component

Strong filters for the chrominance component are used on both sides of the border between blocks. Here, the chrominance filter is selected when both sides of the chroma edge are at least 8 (chroma position), and the following three-condition solution is satisfied: The first condition applies to the edge hardness as well as the large block. The proposed filter can be applied when the width or height of the block, orthogonally intersecting the edge of the block, is not less than 8 in the region of the chrominance component samples. The second and third conditions are basically the same as the deblocking decisions for the luminance component in the HEVC coding standard, namely the on/off decision and the strong filter decision, respectively.

In the first solution, the boundary strength (bS) is modified to filter out the chrominance component, as shown in Table 2-2. The conditions in Table 2-2 are checked sequentially. If any of the conditions are satisfied, then the remaining conditions with lower priority are skipped.

Chroma deblocking filtering is performed when the hardness bS is 2, or when the hardness bS is 1 and a large block boundary is detected.

The second and third conditions are basically the same as the solution for the strong luminance filter in the HEVC coding standard, viz.

For the second condition:

The parameter d is determined as for the luminance component deblocking filter in the HEVC coding standard.

The second condition will be true (TRUE) when the parameter d is less than β.

The third StrongFilterCondition is defined as follows:

dpq is defined as in the HEVC encoding standard.

sp ₃ = Abs( p ₃ − p ₀ ), defined as in the HEVC coding standard

sq ₃ = Abs( q ₀ − q ₃ ), defined as in the HEVC encoding standard

As in the HEVC encoding standard, StrongFilterCondition = (dpq less than ( β >> 2 ), sp ₃ + sq ₃ less than ( β >> 3 ), and Abs( p ₀ − q ₀ ) less than ( 5 * t _C + 1 ) >> 1)

2.6.6 Strong chrominance deblocking filter

The following strong deblocking filter is defined for the chrominance component:

p ₂ ′= (3*p ₃ +2*p ₂ +p ₁ +p ₀ +q ₀ +4) >> 3

p ₁ ′= (2*p ₃ +p ₂ +2*p ₁ +p ₀ +q ₀ +q ₁ +4) >> 3

p ₀ ′= (p ₃ +p ₂ +p ₁ +2*p ₀ +q ₀ +q ₁ +q ₂ +4) >> 3

The proposed filter for the chrominance component performs deblocking of the chrominance component on a 4x4 grid of samples.

2.6.7 Position-dependent truncation

The position-dependent truncation parameter tcPD is applied to the output samples after a luminance filtering procedure involving strong and long filters modifying the 7, 5 and 3 samples at the boundary. Assuming a distribution of quantization error, it is proposed to increase the truncation level for samples for which a higher level of quantization noise is expected and therefore for which a larger deviation of the reconstructed sample value from the true value of this sample is expected.

For each of the P or Q boundaries filtered using an asymmetric filter, depending on the results of the decision procedure in section 2.3.3, select a table of position-dependent thresholds from two tables (i.e., tables Tc7 and Tc3 below), transmitted to the decoding device as additional information:

Tc7 = { 6, 5, 4, 3, 2, 1, 1};
Tc3 = { 6, 4, 2 };

tcPD = (Sp == 3) ? Tc3: Tc7;

tcQD = (Sq == 3) ? Tc3: Tc7;

For a P or Q edge filtered using a short asymmetric filter, a position-dependent threshold with a smaller amplitude is applied:

Tc3 = { 3, 2, 1 };

After determining the threshold value, truncation of the filtered sample values p' _i and q' _i is performed, in accordance with the truncation parameters tcP and tcQ:

p'' _i = Clip3(p' _i + tcP _i , p' _i – tcP _i , p' _i );

q'' _j = Clip3(q' _j + tcQ _j , q' _j – tcQ _j , q' _j );

where p' _i and q' _i are the filtered sample values, p'' _i and q'' _j are the sample output values after truncation, and tcP _i are the truncation thresholds derived from the tc and parameter values used in the VVC encoding standard parameters tcPD and tcQD. The Clip3 function is a clipping function as specified in the VVC coding standard.

2.6.8 Regulation of subblock release

To allow parallel friendly release using both long filters and subblock release, the long filter functions limit modification to at most 5 samples on the side using subblock release (AFFINE mode or ATMVP prediction or DMVR refinement), as shown in Section brightness control for long filters. In addition, the subblock release procedure is adjusted such that the boundaries of subblocks on an 8x8 grid close to a CU or an implicit TU boundary are limited to modification by at most two samples on each side.

The following applies to subunit boundaries that are not co-located with a CU unit boundary.

If (block mode Q == SUBBLOCKMODE && edge !=0) {

if (!(implicitTU && (edge == (64 / 4))))

if (edge == 2 || edge == (orthogonalLength - 2) || edge == (56 / 4) || edge == (72 / 4))

Sp = Sq = 2;

otherwise

Sp = Sq = 3;

otherwise

Sp = Sq = bSideQisLargeBlk ? 5:3

}

Where an edge parameter of 0 corresponds to a CU boundary, an edge parameter of 2 or orthogonalLength-2 corresponds to a subblock boundary 8 counts from the CU boundary, etc. Where implicit TU is true if implicit TU partitioning is used.

2.6.9 Line buffer limit 4CTU/2CTU for luma/chrominance component

Horizontal edge filtering is limiting to Sp = 3 for luma, Sp = 1 and Sq = 1 for chroma when the horizontal edge is aligned with the CTU boundary.

2.7 Intra-prediction coding in the VVC coding standard

To capture the arbitrary edge directions present in natural video, the number of intra-frame prediction directional modes in the VTM5 document is increased from 33, as used in HEVC encoding, to 65. New directional modes that are not present in the HEVC encoding standard are indicated by red dotted arrows in fig. 12, and planar mode and DC mode remain the same. This denser distribution of directional intra-prediction modes is applicable for all block sizes and for both luminance and chrominance intra-prediction modes.

In the VTM5 document, several conventional angle intra-prediction modes are adaptively replaced by wide-angle intra-prediction modes for non-square blocks. Wide-angle intra-frame prediction is described in Section 3.3.1.2.

In the HEVC coding standard, each block encoded in intra-prediction mode is square in shape, and the length of each side is a power of 2. Thus, no partitioning operations are required to generate an intra-prediction predictor using DC mode. In the VTM5 document, blocks can be rectangular in shape, which generally necessitates the use of a split operation for each block. To avoid splitting operations for DC prediction, only the long side is used to calculate the average for non-square blocks.

To keep the complexity of generating the most probable mode (MPM) list low, a 6-MPM intra-prediction mode option is used by taking into account two adjacent available intra-prediction modes. When constructing the list of MPM modes, the following three aspects are taken into account:

i. Default intra-prediction mode

ii. Adjacent intra-frame prediction modes

iii. Inferred intra-frame prediction modes

The unified 6-MPM list is used for intra-prediction blocks whether MRL (multiple reference line) and ISP coding tools are used or not. The list of MPM modes is constructed based on the intra-prediction modes for the left or top adjacent blocks. Assuming that the mode for the block on the left is designated Left and the mode for the block above is designated Above, the unified MPM mode list is constructed as follows (Blocks on the left and above are shown in Fig. 13).

- When a neighboring block is unavailable, it defaults to planar intra-prediction mode.

- If both Left and Above modes are non-corner modes:

○ List of MPM modes → {Planar, DC, V, H, V-4, V+4}

- If one of the Left and Above modes is a corner mode and the other is a non-corner mode:

○ Setting the “Max” mode as a “higher” mode from the Left and Above modes

○ List of MPM modes → {Planar, Max, DC, Max -1, Max +1, Max -2}

- If both modes – Left and Above, are angular modes and at the same time different:

○ Setting Max mode as a “higher” mode from Left and Above modes

○ if the difference between the Left and Above modes is in the range from 2 to 62 inclusive

▪ List of MPM modes →{Planar, Left, Above, DC, Max -1, Max +1}

○ Otherwise

▪ List of MPM modes → {Planar, Left, Above, DC, Max -2, Max +2}

- If both modes – Left and Above, are angular modes and at the same time the same:

○ List of MPM modes → {Planar, Left, Left -1, Left +1, DC, Left -2}

In addition, the first section (bit) of the index codeword of the mpm list is encoded using context-sensitive binary arithmetic coding (CABAC). A total of three contexts are used, corresponding to whether the current intra-prediction block is an MRL-mode-enabled block, an ISP-mode-enabled block, or a normal intra-prediction block.

During the generation process of the list of 6 MPM modes, pruning is applied to eliminate duplicate modes so that only unique modes can be included in the list of MPM modes. For entropy encoding of 61 modes not included in the list of MPM modes, a truncated binary code (TBC) is used.

To encode the color component in the intra-frame prediction mode, a total of 8 intra-frame prediction modes are allowed. The combination of these modes contains five traditional intra-frame prediction modes and three cross-component linear model (CCLM) modes (CCLM, LM_A and LM_L). The signaling procedures and the formation of color component coding modes are given in Table 2-4. The chrominance component coding modes directly depend on the intra-frame prediction mode for the corresponding luminance component block. Since separate structures for luma and chroma components are activated in I-slices, one chroma block can correspond to multiple luma blocks. Therefore, for the DM mode for the chrominance component, the intra-frame prediction mode for the corresponding luminance component block covering the center of the current chrominance component block is directly “inherited”.

Table 2-4. Derivation of the chrominance prediction mode from the luminance mode when the cclm_ mode is activated

Color prediction mode Corresponding mode of intra-frame luminance component prediction 0 50 18 1 X
( 0 <= X <= 66 ) 0 66 0 0 0 0 1 50 66 50 50 50 2 18 18 66 18 18 3 1 1 1 66 1 4 81 81 81 81 81 5 82 82 82 82 82 6 83 83 83 83 83 7 0 50 18 1 X

2.8 Quantized residual Block Differential Pulse-code Modulation (QR-BDPCM)

In JVET-M0413, a Quantized Residue Block Differential Pulse Code Modulation (QR-BDPCM) method is proposed for efficient encoding of screen content.

The prediction directions used in QR-BDPCM modulation may correspond to vertical and horizontal prediction modes. Intra-frame prediction is carried out for the entire block by copying samples in the prediction direction (horizontal or vertical prediction). The remainder is quantized and the difference between the quantized remainder and the quantized value of its predictor (horizontal or vertical) is encoded. This can be described as follows: For a block of size M (rows) × N (columns), let r _i,j , 0 ≤ i ≤ M-1, 0 ≤ j ≤ N-1 represent the prediction residual after performing intra-frame prediction in horizontally (copying the value of the left-adjacent pixel across the predicted block row by row) or in the vertical direction (copying the top-adjacent row to each row in the predicted block) using unfiltered samples from above or to the left of the samples at the block boundary. Let Q(r _i,j ), 0 ≤ i ≤ M-1, 0 ≤ j ≤ N-1 denote the quantized version of the remainder r _i,j , where the remainder is the difference between the values from the original block and the predicted block. The block's DPCM modulation is then applied to the quantized residual samples, resulting in a modified matrix size M × N with elements . When BDPCM modulation in the vertical direction is reported as an alarm:

(2-7-1)

For horizontal forecasting, similar rules are applied, and quantized residual samples are obtained using the formula

(2-7-2)

Quantized samples the remainder is transferred to the decoding device.

At the decoder side, the above calculations are inverted to obtain Q(r _i,j ), 0 ≤ i ≤ M-1, 0 ≤ j ≤ N-1. For the case of vertical forecasting,

(2-7-3)

For horizontal case,

(2-7-4)

The inversely quantized residuals, Q ^-1 (Q(r _i,j )), are added to the predicted values in the block during intra-frame prediction to obtain the reconstructed sample values.

The main advantage of this scheme is that DPCM inverse modulation can be performed in real time (“on the fly”) during coefficient parsing simply by adding a predictor while or after parsing those coefficients.

2.9 Adaptive loop filter

In the VTM5 document, an Adaptive Loop Filter (ALF) is used, adapting this filter on a block basis. For the luminance component, one of 25 filters is selected for each 4x4 block based on the direction and activity of local gradients.

2.9.1 Filter shape

In the VTM5 document, two diamond-shaped filters are used (as shown in Fig. 14). A 7x7 diamond shape is used for the luminance component, and a 5x5 diamond shape is used for the chrominance components.

2.9.2 Block classification

For the luminance component, each of the 4x4 blocks is classified and assigned to one of 25 classes. The classification index C is derived, based on the directionality D and the quantized activity value Â, as follows:

C=5D+Â … (2-9-1)

To calculate the parameters D and Â, first calculate the gradients for the horizontal, vertical and two diagonal directions using the one-dimensional (1-D) Laplace operator:

(2-9-2)

(2-9-3)

(2-9-4)

(2-9-5)

Where the subscripts i and j denote the coordinates of the top left sample in the 4x4 block and R(i,j) denotes the reconstructed one with coordinates (i,j).

To reduce the complexity of block classification, the calculation of a subsampled one-dimensional Laplace operator is used. As shown in FIG. 15 (a)-(d), the same subsampled positions are used to compute gradients in all directions.

Then the maximum and minimum values of the gradients in the horizontal and vertical directions are set as:

, (2-9-6)

The maximum and minimum values of the gradient in two diagonal directions are set as:

, (2-9-7)

To derive the directivity value D, these values are compared one with the other and with two threshold values t ₁ and t ₂ :

Stage 1. If both ratios And are true, the directionality D is set to 0.

Stage 2. If , then continue from Stage 3; otherwise continue from Step 4.

Stage 3. If , D is set equal to 2; otherwise, D is set equal to 1.

Stage 4. If , D is set equal to 4; otherwise, D is set to 3.

The activity value A is calculated as:

(2-9-8)

The value of A is then quantized into the range 0 – 4, inclusive, and the quantized value is denoted as Â.

For color components in the image, no classification methods are used, i.e. a single set of ALF filter coefficients is applied for each chrominance component.

2.9.3. Geometric transformation of filter coefficients and truncation levels

Before each 4x4 block of luminance component is filtered, geometric transformations, such as rotation or flipping around a diagonal or vertical axis, are applied to the filter coefficients f(k,l) and filter truncation levels c(k,l), depending on gradient value calculated for the block in question. This is equivalent to applying such a transformation to the filter support area. The idea is to make the different blocks to which ALF filtering is applied more similar to one another by aligning their directions.

Three types of geometric transformations are introduced, including flipping around the diagonal, flipping around the vertical and rotation:

Flipping around a diagonal: (2-9-9)

Flip around vertical: ,

(2-9-10)

Turn: , (2-9-11)

where K denotes the filter size and denote the coordinates of the filter coefficients, so that item (0,0) is in the upper left corner and item (K-1, K-1) is in the lower right corner. These transformations are applied to the filtering coefficients f(k,l) and to the truncation levels c(k,l) depending on the values of the gradients calculated for the block in question. The relationships between the transformation and the four gradients for the four directions are summarized in the following table.

Table 2-5. Relationship between gradients calculated for one block and transformations

Gradient values Transformation g _d2 < g _d1 and g _h < g _v No transformation g _d2 < g _d1 and g _v < g _h Flipping around a diagonal g _d1 < g _d2 and g _h < g _v Flip around vertical g _d1 < g _d2 and g _v < g _h Turn

2.9.4 Filter parameter signaling

In the VTM5 document, ALF filter parameters are reported as signaling in the Adaptation Parameter Set (APS). In one APS set, up to 25 groups of luma filter coefficients and truncation value indices and up to one group of chrominance filter coefficients and truncation value indices can be signaled. To reduce bit overhead, filter coefficients for different classification categories can be combined. In the slice header, the indexes of the APS sets used for the current slice are signaled.

The truncation value indices decoded from the APS set allow the truncation values to be determined using a table of such luma truncation values and a table of such truncation values for the chrominance component. These truncation values depend on the internal bit depth. More precisely, the table of truncation values for the luma component and the table of truncation values for the chrominance component are obtained using the following formulas M:

, (2-9-12)

(2-9-13)

where B is equal to the internal bit depth and N is 4, which denotes the number of valid truncation values according to the VTM5.0 document.

The filtering procedure can be controlled at the CTB block level. A flag is always signaled to indicate whether ALF filtering is applicable to the luminance component CTB. The luminance component CTB may select a filter coefficient group from a plurality of 16 fixed filter coefficient groups and filter coefficient groups from APS sets. The filter coefficient group index is signaled to the luminance component CTB to indicate which coefficient group is being applied. 16 fixed groups of filter coefficients are preset and hard-coded in both the encoder and the decoder.

These filter coefficients are quantized at a rate of 128. In order to limit the complexity of the multiplication, the matching of the data bit streams is applied such that the value of the coefficient not in the central position must be in the range −2 ⁷ to 2 ⁷ − 1, inclusive. The coefficient located in the central position is not reported in the data bit stream and is considered equal to 128.

2.9.5 Filtering procedure

At the decoder side, when the ALF filter is enabled for the CTB block, each sample R(i,j) in the CU is filtered, resulting in a sample value R'(i,j) as shown below.

(2-9-14)

Where denotes the decoded filter coefficients, denotes the truncation function and denotes the decoded truncation parameters. Variables k and l vary between And , where L denotes the length of the filter. The truncation function has the form , which corresponds to the function

2.9.6 Virtual boundary filtering procedure to reduce row buffer

The VTM5 document uses modified block classification and filtering on samples near the horizontal boundaries of the CTUs to reduce the line buffer requirements of the ALF filter. For this purpose, the virtual boundary is defined as a row by shifting the horizontal boundary of the CTU with “N” samples, as shown in FIG. 16, where N is equal to 4 for the luma component and equal to 2 for the chrominance component.

The modified block classification is applied to the luminance component as shown in FIG. 2 – 11. To calculate the one-dimensional (1D) gradient of the Laplace operator for a 4x4 block above the virtual boundary, only the samples located above the virtual boundary are used. Similarly, to calculate the one-dimensional (1D) gradient of the Laplace operator for a 4x4 block below the virtual boundary, only the samples located below the virtual boundary are used. The quantization of the activity value A is accordingly scaled to account for the reduced number of samples used to compute the one-dimensional (1D) gradient of the Laplace operator.

For the filtering procedure, a symmetrical filling operation is used on the virtual boundaries for both the luminance and chrominance components. As shown in FIG. 17, when the filtered sample is located below the virtual boundary, the neighboring samples located above this virtual boundary are filled. At the same time, the corresponding readings on other sides are also filled symmetrically.

2.10 Nonlinear Adaptive Offset (SAO) Filtering Procedure

A sample adaptive offset (SAO) nonlinear filtering procedure is applied to the reconstructed samples after the deblocking filter using an offset specified for each CTB by the encoder. The HM model encoder first decides whether to apply the SAO filtering procedure to the current slice. If SAO filtering is applied to the slice in question, each CTB is classified as belonging to one of five SAO types, as shown in Table 2-6. The concept of SAO filtering is to classify pixels into categories and reduce distortion by adding bias to the pixels of each category. The SAO operation contains an Edge Offset (EO), which uses the edge offset to classify pixels into SAO types 1 through 4, and a Band Offset (BO), which uses pixel intensity to classify pixels into SAO types 5 . Each applicable CTB has SAO offset parameters, which collectively contain the sao_merge_left_flag flag, the sao_merge_up_flag flag, the SAO offset type, and four offset values. If the sao_merge_left_flag is 1, the current CTB block will reuse the SAO offset type and offsets inherent to the left CTB block. If the sao_merge_up_flag is 1, the current CTB block will reuse the SAO offset type and offsets inherent to the CTB block above.

Table 2-6. SAO Offset Type Specifications

Type SAO Type of adaptive offset that should be used Number of categories 0 No 0 1 1-D offset of the edge of the pattern by 0-degrees 4 2 1-D offset of the edge of the design by 90 degrees 4 3 1-D offset of the edge of the pattern by 135 degrees 4 4 1-D offset pattern edge by 54 degrees 4 5 stripe offset 4

2.10.1. How each SAO offset type works

Edge bias uses four one-dimensional (1-D) 3-pixel patterns to classify the current pixel p by taking into account edge direction information, as shown in FIG. 18. From left to right, these offsets are: 0 degrees, 90 degrees, 135 degrees and 45 degrees.

Each CTB block is classified into one of five categories according to Table 2-7.

Table 2-7. Pixel classification rule for EO offset

Category Condition Meaning 0 None of the following Pretty much monotonous 1 p < 2 neighbors Local minimum 2 p < 1 neighbor && p == 1 neighbor edge 3 p > 1 neighbor && p == 1 neighbor edge 4 p > 2 neighbors Local maximum

Band offset (BO) classifies all pixels in the region of one CTB block into 32 equal bands by using the five most significant bits of the pixel value as the band index. In other words, the pixel intensity range is divided into 32 equal segments from zero to the maximum intensity value (for example, 255 for 8-bit pixels). Four adjacent stripes are grouped together and each group is designated by its leftmost position, as shown in FIG. 19. The encoder searches all positions to detect a group with maximum distortion reduction by compensating for the offset of each band.

2.11 Combined intra-interprediction (CIIP) mode

In the VTM5 document, when a CU is encoded in merge mode, if the CU contains at least 64 luminance samples (in other words, the product of the CU width and the CU height is at least 64), and if both the CU width and the height CUs less than 128 luminance samples signal an additional flag to indicate whether combined inter/intra prediction (CIIP) mode is applied to the current CU. As the name indicates, CIIP prediction combines an inter-prediction signal with an intra-frame prediction signal. The inter prediction signal in the CIIP prediction mode, P _inter , is obtained using the same inter prediction procedure as that used in the conventional combining mode; and the _intra prediction signal P is obtained using a conventional planar mode intra prediction procedure. The intra- and inter-prediction signals are then combined using weighted averaging, where weights are calculated depending on the coding modes for the top and left adjacent blocks (shown in FIG. 20) as follows:

- If the upper neighbor is available and encoded in intra-prediction mode, then the isIntraTop parameter is set to 1, otherwise this isIntraTop parameter is set to 0;

- If the neighbor on the left is available and encoded in intra-prediction mode, then the isIntraLeft parameter is set to 1, otherwise this isIntraLeft parameter is set to 0;

- If (isIntraLeft + isIntraLeft) is 2, then wt is set to 3;

- Otherwise, if (isIntraLeft + isIntraLeft) is 1, then wt is set to 2;

- Otherwise, the wt parameter is set to 1.

CIIP forecasting is carried out as follows:

… (3-1)

2.12 Luminance Display with Chroma Scaling (LMCS)

In the VTM5 document, an encoding tool called luma mapping with chroma scaling (LMCS) is added as a new processing unit before edge filtering. The LMCS mode has two main components: 1) in-circuit display of the brightness component based on adaptive piecewise linear models; 2) application of luminance component-dependent scaling of the remainder of the chrominance component for chrominance components. Fig. 21 shows an implementation architecture of the LMCS mode from the point of view of a decoder. The light blue shaded blocks in Fig. 21 indicate that processing is applied to the displayed area; This includes inverse quantization, inverse transformation, intra-frame luminance prediction, and summation of luminance prediction with the remainder of the luminance component. The unshaded blocks in Fig. 21 indicate where processing is applied in the original (ie, unmapped) region; and these include edge filters such as deblocking filters, ALF filters and SAO filters, motion compensated prediction, intra-frame chrominance prediction, summing the chrominance prediction with the chrominance residual, and storing the decoded pictures as reference pictures. Blocks shaded in light yellow in Fig. 21 represent new function blocks for the LMCS mode, including forward and inverse luminance signal mapping and a luminance-dependent chrominance scaling procedure. Like most other tools in the VVC coding standard, LMCS mode can be enabled/disabled at the sequence level using the SPS set flag.

3. Examples of problems solved by options

A single palette flag is typically used to indicate whether the palette mode is in use for the current CU, which may have various restrictions and variations regarding entropy encoding. However, in previous video encoding standards, the question of how best to encode the palette flag has not yet been fully explored.

Palette samples may have visual artifact if processed through a post-contour filtering procedure.

The palette scanning order can be improved for non-square blocks.

4. Examples of options

The following sentence descriptions should be considered as examples to clarify general concepts. These sentences should not be interpreted in a narrow sense. Further, these offers can be combined in any way.

1. The palette mode usage indication for the transform unit/prediction unit/coding block/area may be encoded separately from the prediction mode.

a. In one example, the prediction mode indication may be encoded before the palette use indication.

i. Additionally, as an alternative, the palette usage indication may be conditionally signaled based on the prediction mode.

1. In one example, when the prediction mode is an intra-frame block copy mode (ie, MODE_IBC mode), transmission of palette mode use indication signaling may be skipped. Additionally, as an alternative, the palette usage indication may be false when the current prediction mode is MODE_IBC.

2. In one example, when the prediction mode is an inter-prediction mode (ie, MODE_INTER), signaling of a palette mode usage indication may be skipped. Additionally, as an alternative, the palette usage indication may be false when the current prediction mode is MODE_INTER.

3. In one example, when the prediction mode is an intra prediction mode (ie, MODE_INTRA), the palette mode usage indication signaling may be skipped. Additionally, as an alternative, the palette usage indication may be false when the current prediction mode is MODE_INTRA.

4. In one example, when the prediction mode is a skip mode (ie, the skip flag is 1), transmission of the palette mode use indication signaling may be skipped. Additionally, as an alternative, the palette usage indication may be false when the current CU is in skip mode.

5. In one example, when the prediction mode is an intra prediction mode (ie, MODE_INTRA), the palette mode use indication may be signaled. Moreover, as an alternative, transmission of the palette usage indication signaling may be skipped when the prediction mode is an inter-frame prediction mode or an intra-frame block copy mode.

a) Additionally, as an alternative, when the prediction mode is an intra-prediction mode and is not a pulse-code modulation (PCM) mode, palette mode use may be signaled.

b) Additionally, as an alternative, when the prediction mode is an intra-prediction mode, the palette mode usage indication may be signaled before the PCM modulation mode usage indication. In one example, when the palette mode is used, signaling that the PCM modulation mode is being used may be skipped.

c) Additionally, alternatively, when the prediction mode is an inter-prediction mode or an intra-frame block copy mode, signaling indicating the use of the palette mode may be skipped.

2. In one example, when the prediction mode is an inter-prediction mode (eg, MODE_INTER), an indication of use of the palette mode may be signaled.

a) Alternatively, when the prediction mode is an intra-prediction mode, signaling indicating the use of the palette mode may be skipped.

3. In one example, when the prediction mode is an intra-frame block copy mode, an indication of the use of the palette mode may be signaled. Moreover, alternatively, when the prediction mode is an inter-prediction mode or an intra-prediction mode, signaling indicating the use of the palette mode may be skipped.

ii. Additionally, as an alternative, signaling indicating the use of a palette mode may be conditionalized based on the image/slice/tile group type.

b. In one example, the prediction mode indication may be encoded after the palette mode use indication.

c. In one example, signaling indicating the use of a palette mode may be transmitted when the prediction mode is INTRA or INTER_MODE.

i. In one example, the palette mode use indication may be encoded after the skip flag, the prediction mode indication, and the PCM mode flag.

ii. In one example, the palette mode use indication may be encoded after the skip flag, the prediction mode indication, and before the PCM mode flag.

iii. In one example, when the current block is encoded in intra-prediction mode, signaling indicating the palette mode and IBC copy mode may then be transmitted.

1. In one example, a one-bit flag may be signaled to indicate whether palette mode or IBC copy mode is being reported.

2. In one example, transmission of one-bit flag signaling may be skipped under certain conditions, such as block sizes and whether IBC copy mode or palette mode is enabled for a single tile/group of tiles/slice/image/sequence.

d. In one example, an indication of the prediction mode (such as whether an intra-prediction mode or an inter-prediction mode is used) may be encoded first, followed by conditional signaling of whether the mode is a palette mode or not.

i. In one example, when the prediction mode is an intra-frame prediction mode, another flag may be further signaled to indicate whether the mode is a palette mode or not.

1. In one example, said "other flag" may be signaled when the palette mode is activated for one unit of video data (eg, sequence/image/tile group/tile).

2. In one example, the specified "other flag" may be signaled, provided the block size is appropriate.

3. Additionally, as an alternative, if the mode is not a palette mode, one flag may be further signaled to indicate whether the mode is a PCM modulation mode or not.

4. In one example, said "other flag" may be context encoded according to neighboring block information. Alternatively, this "other flag" may be context encoded according to only one context. Alternatively, this "other flag" could be encoded "bypass", i.e. without context.

ii. Alternatively, when the prediction mode is an inter-frame prediction mode, another flag may be further signaled to indicate whether the mode is an IBC copy mode.

1. In one example, the specified "other flag" may be signaled when the IBC copy mode is activated for one unit of video data (eg, sequence/image/tile group/tile).

2. In one example, the specified "other flag" may be signaled, provided the block size is appropriate

2. It is proposed to add a palette mode as an additional candidate for the role of prediction mode. The palette mode use indication may be determined/signalled based on the prediction mode, for example, as discussed in the example of option 1 above. In some embodiments, it is not necessary to signal the palette mode use indication separately from the prediction mode indication.

a. In one example, the plurality of prediction modes may comprise an intra-prediction mode, an intra-block copy mode, and a palette mode for intra-predictive slices/I-frames/intra-predictive-coding tile groups.

b. Alternatively, the plurality of prediction modes may comprise an intra-prediction mode, a palette mode for intra-predictive slices/I-frames/sub-intra-predictive tile groups.

c. In one example, the plurality of prediction modes may comprise an intra-prediction mode, an intra-frame block copy mode, and a palette mode for 4x4 blocks.

d. In one example, a plurality of prediction modes may comprise an intra-prediction mode, an inter-prediction mode, an intra-block copy mode, and a palette mode for intra-prediction-codable slices/P and/or B-frames/intra-prediction-coding tile groups.

e. In one example, a plurality of prediction modes may comprise an intra-prediction mode, an inter-prediction mode, an intra-frame block copy mode for intra-predictive slices/P and/or B-frames/sub-intra-predictive tile groups.

f. Alternatively, the plurality of prediction modes may comprise at least two of the following list of modes: an intra-prediction mode, an inter-prediction mode, an intra-frame block copy mode, and a palette mode.

g. In one example, the inter-frame prediction mode may not be included in the set of prediction modes for 4x4 blocks.

h. In one example, when the block is not encoded in the skip mode (which is a special case of the inter-prediction mode), the prediction mode index may be context-encoded using different sections (bits). In some embodiments, transmission of one or more sections (bits) as signaling may be skipped due to some condition such as block size (eg, 4x4) or the prediction mode is not activated (eg, IBC copy mode is not activated, so the corresponding sections (digits) are skipped).

i. In one example, the binary representation (binarization) of these four modes is defined as: intra-prediction mode (1), inter-prediction mode (00), IBC copy mode (010), and palette mode (011). Three sections (bits) are used here, and each bit corresponds to the value of one section (bit).

ii. In one example, the binarization of these four modes is defined as: intra prediction mode (10), inter prediction mode (00), IBC copy mode (01) and palette mode (11), as shown in FIG. 10. Two sections (bits) are used here, and each bit corresponds to the value of one section (bit)

iii. In one example, if the current slice is an intra-prediction encoded slice and the IBC copy mode is not enabled in the SPS set, the binarization of palette mode and intra-prediction mode is defined as: palette mode (1) and intra-prediction mode (0). One section (bit) is used here.

iv. In one example, if the current slice is not an intra-prediction encoded slice and the IBC copy mode is not enabled in the SPS set, the binarization of the palette mode, inter-prediction modes, and intra-prediction modes is defined as: intra-prediction mode (1), inter-prediction mode ( 00) and palette mode (01). Here two sections (bits) are used, and each bit corresponds to the value of one section (bit).

v. In one example, if the current slice is an intra-prediction mode encoded slice and the IBC copy mode is enabled in the SPS, the binarization of palette mode and intra-prediction mode is defined as: IBC (1) copy mode, palette (01) mode, and intra-prediction mode forecast (00). Here two sections (bits) are used, and each bit corresponds to the value of one section (bit).

vi. In one example, the binarization of these four modes is defined as: inter-prediction mode (1), intra-prediction mode (01), IBC copy mode (001), and palette mode (000). Three sections (bits) are used here, and each bit corresponds to the value of one section (bit).

vii. In one example, the binarization of these four modes is defined as: intra prediction mode (1), inter prediction mode (01), IBC copy mode (001) and palette mode (000). Three sections (bits) are used here, and each bit corresponds to the value of one section (bit).

viii. In one example, the binarization of these four modes is defined as: inter-prediction mode (0), intra-prediction mode (10), IBC copy mode (111) and palette mode (110), as shown in FIG. 11. Three sections (bits) are used here, and each bit corresponds to the value of one section (bit).

3. IBC palette mode/copy mode usage indication signaling may be dependent on other mode information.

a. In one example, the palette mode use indication may be signaled when the current prediction mode is an intra prediction mode and is not an IBC copy mode.

b. In one example, an indication of use of the IBC copy mode may be signaled when the current prediction mode is an intra prediction mode and is not a palette mode.

4. Exactly how to convey mode information as an alarm may depend on the slice/image/type of tile group.

a. In one example, when considering an I-slice/tile group encoded using intra-prediction, one flag may be signaled to indicate whether the mode in question is an IBC copy mode. If the mode is not an IBC copy mode, another flag may optionally be transmitted to indicate whether the mode is a palette mode or an intra-prediction mode.

b. In one example, when considering an I-slice/tile group encoded using intra-prediction, one flag may be signaled to indicate whether the mode in question is an intra-prediction mode. If the mode is not an intra-frame prediction mode, another flag may be optionally transmitted to indicate whether the mode is a palette mode or an IBC copy mode.

5. The palette mode usage indication may be signaled and/or generated based on the following conditions.

a. dimensions of the current block

i. In one example, the palette mode use indication may be signaled only for blocks for which the width*height product is not greater than a threshold value, such as 64*64.

ii. In one example, the palette mode use indication may be signaled only for blocks for which both the width and height are at least equal to a threshold value, such as 64

iii. In one example, the palette mode use indication may be signaled only for blocks for which all of the following conditions are true:

1. width and/or height not less than a threshold value, such as 16;

2. width and/or height not greater than a threshold value such as 32 or 64

iv. In one example, the palette mode use indication may be signaled only for blocks whose width is equal to their height (i.e. square blocks)

b. prediction mode for the current block

c. Current quantization parameter for the current block

d. Palette mode flag for adjacent blocks

e. Intra-frame block copying mode flags for adjacent blocks

f. Color format display (such as 4:2:0, 4:4:4)

g. Separate/dual coding tree structure

h. Slice/Tile Group Type and/or Image Type

6. The IBC copy mode usage indication may be signaled and/or received based on the following conditions.

a. dimensions of the current block

i. In one example, the use of IBC copy mode may be signaled only for blocks for which both the height and width are less than 128

b. prediction mode for the current block

c. Current quantization parameter for the current block

d. Palette mode flag for adjacent blocks

e. Intra-frame block copying mode flags for adjacent blocks

f. Color format display (such as 4:2:0, 4:4:4)

g. Separate/dual coding tree structure

h. Slice/Tile Group Type and/or Image Type

7. Palette mode can be considered as an intra-prediction mode (e.g MODE_INTRA) in the release decision process.

a. In one example, if the samples on the p side or the q side are encoded in palette mode, the boundary hardness is set to 2.

b. In one example, if the samples on both the p-side and the q-side are encoded in palette mode, the boundary hardness is set to 2

c. Alternatively, the palette mode can be considered as an inter-prediction mode (eg, MODE_INTER) in the release decision process.

8. Palette mode can be considered as a separate mode (eg MODE_PLT) in the release decision process.

a. In one example, if the samples on the p side or the q side are encoded in palette mode, the boundary hardness is set to 0.

i. Alternatively, if samples on one side are encoded in palette mode, the edge hardness is set to 0.

b. In one example, if samples on the p side are encoded in IBC copy mode and samples on the q side are encoded in palette mode, the boundary hardness is set to 1, and vice versa.

c. In one example, if samples on the p side are encoded in intra-prediction mode and samples on the q side are encoded in palette mode, the boundary hardness is set to 2, and vice versa.

9. Palette mode can be considered as a block-skipping transformation mode during the release process

a. Alternatively, the palette mode can be thought of as the block's BDPCM modulation mode during the deblocking process.

10. Indication of palette mode use for a block may be signaled and/or derived from a flag at the slice/tile group/image level

a. In one example, this indicates whether a fractional motion vector difference (MVD) is valid in merge with motion vector difference (MMVD, also called UMVE) mode and/or in adaptive mode. adaptive motion vector resolution (AMVR), (for example, the slice_fracmmvd_flag flag). Additionally, as an alternative, if the slice_fracmmvd_flag indicates that the fractional difference MVD mode is activated, signaling indicating use of the palette mode is skipped and the palette mode is considered not to be activated.

b. In one example, this flag indicates whether palette mode is enabled for a slice/tile group/image. Further, alternatively, when such a flag indicates that the palette mode is not activated, signaling indicating the use of the palette mode is skipped for the block, and the palette mode is considered not activated.

11. Indication of the use of Intra Block Copy (IBC) mode for a block can be signaled and/or inferred based on a flag at the slice/tile group/image level.

a. In one example, the flag indicates whether fractional motion vector difference (MVD) is valid in fusion motion vector difference (MMVD, also called UMVE) mode and/or adaptive motion vector resolution (AMVR) mode (e.g., slice_fracmmvd_flag) . Also, as an alternative, if the slice_fracmmvd_flag indicates that the MVD fractional difference mode is activated, signaling of the IBC copy mode indication is skipped and the IBC copy mode is considered not to be activated.

b. In one example, this flag indicates whether IBC copy mode is enabled for the slice/tile group/image. Also, alternatively, when this flag indicates that the IBC copy mode is not activated, IBC copy mode use signaling for the block is skipped and the IBC copy mode is considered not to be activated.

12. The sample associated with one input palette position may have a different bit depth from the internal bit depth and/or from the bit depth of the original/reconstructed samples.

a. In one example, if we denote the bit depth associated with a sample as N, the following may apply:

i. In one example, N may be an integer (eg, 8).

ii. In one example, N may be greater than the internal bit depth and/or the bit depth of the original/reconstructed samples.

iii. In one example, N may be less than the internal bit depth and/or the bit depth of the original/reconstructed samples.

iv. In one example, N may depend on

1. Dimensions of the current block

2. Current quantization parameter for the current block

3. Color format indication (such as 4:2:0, 4:4:4)

4. Separate/dual coding tree structure

5. Slice/Tile Group Type and/or Image Type

6. Number of input palette positions

7. Number of predicted input palette positions

8. Color component index

b. In one example, samples associated with multiple input palette positions may have different bit depths.

i. In one example, let C0, C1 be the two input positions of the current palette and may have bit depths equal to b0 and b1, respectively. Here b0 may not be equal to b1

1. In one example, b0 may be larger/smaller than the internal bit depth and/or bit depth of the original/reconstructed samples, and/or b1 may be larger/smaller than the internal bit depth and/or bit depth of the original/reconstructed samples.

c. In one example, in palette mode, samples can be reconstructed according to offset sample values associated with input palette positions.

i. In one example, the samples can be reconstructed by left shifting the samples at the input palette positions by M bits.

ii. In one example, the reconstructed value could be (C<<M) + (1<<(M-1)), where C denotes the input palette position.

iii. In one example, the samples can be reconstructed by right shifting the samples at the input palette positions by M bits.

iv. In one example, the reconstructed value may be a truncation ((C+(1<<(M-1)))>>M, 0, (1<<N)-1), where C denotes the input palette position and N denotes bit depth reconstruction.

v. Additionally, as an alternative, in one example, the value of M may depend on the difference in bit depths between the samples associated with the input palette positions and the internal bit depth of the reconstructed samples/original samples.

1. In one example, the number M may be equal to the internal bit depth minus the bit depth of the samples at the input palette positions

2. In one example, the number M may be equal to the bit depth of the samples in the input palette positions minus the internal bit depth

3. In one example, the number M may be equal to the bit depth of the input samples minus the bit depth of the samples in the input palette positions

4. In one example, the number M may be equal to the bit depth of the samples in the input palette positions minus the bit depth of the original samples.

5 In one example, the number M may be equal to the bit depth of the reconstructed samples minus the bit depth of the samples at the input palette positions

6. In one example, the number M may be equal to the bit depth of the samples in the input palette positions minus the bit depth of the reconstructed samples

vi. In one example, the number M may be an integer (eg, 2).

vii. Additionally, as an alternative, in one example, the number M may depend on

1. Dimensions of the current block

2. Current quantization parameter for the current block

3. Color format indication (such as 4:2:0, 4:4:4)

4. Separate/dual coding tree structure

5. Slice/Tile Group Type and/or Image Type

6 Number of palette input positions

7. Number of predicted input palette positions

8. Reference position in block/image/slice/tile

9. Color component index

vii. In one example, the sample reconstruction process may use a lookup table operation based on the samples at the input palette positions.

1. In one example, values from the lookup table may be signaled as part of an SPS set/VPS set/PPS set/picture header/slice header/tile group header/LCU row/LCU group.

2. In one example, the values from the lookup table may be output as part of an SPS set/VPS set/PPS set/image header/slice header/tile group header/LCU row/LCU group.

13. The signaled/output quantization parameter (QP) for blocks encoded in palette encoding mode may first be modified before being used to output outlier pixels/samples, such as by truncation.

a. In one example, the applicable QP parameter range for blocks encoded in the palette encoding mode can be considered in the same manner as in the skip transformation mode and/or in the BDPCM modulation mode.

b. In one example, the applicable QP parameter for blocks encoded in palette encoding mode may be revised to be equal to max(Qp, 4 + T), where T is an integer and Qp is a signaled or output quantization parameter for the corresponding block.

i. In one example, the parameter T may be a predetermined threshold value.

ii. In one example, the T parameter may be equal to (4 + min_qp_prime_ts_ minus4), where the min_qp_prime_ts_ minus4 parameter may be signaled.

14. The method for encoding outlier samples/symbols can be unified regardless of whether the transformation and quantization bypass mode is activated or not.

a. In one example, the outlier sample may be transmitted as a fixed length signaling.

b. In one example, the outlier sample may be transmitted as a fixed length signaling using N bits.

i. In one example, the number N may be an integer (for example, 8 or 10) and may depend on

1. The message is sent as signaling as part of an SPS set/VPS set/PPS set/Image header/Slice header/Tile group header/LCU string/LCU group.

2. Internal bit depth

3. Input bit depth

4. Dimensions of the current block

5. Current quantization parameter of the current block

6. Color format indication (such as 4:2:0, 4:4:4)

7. Separate/dual coding tree structure

8. Slice/Tile Group Type and/or Image Type

c. In one example, the code length for signaling the outlier pixel/sample may depend on the internal bit depth.

i. Alternatively, the code length for signaling the outlier pixel/sample may depend on the input bit depth.

d. In one example, the length of the code for signaling the outlier pixel/sample may depend on the quantization parameter.

i. In one example, the code length for signaling the outlier pixel/sample may be f(Qp)

1. In one example, the function f can be defined as (internal bit length – (Qp - 4)/6).

15. Quantization and/or inverse quantization procedures for blocks encoded in the palette encoding mode and for blocks encoded in the palette encoding mode can be defined in different ways.

a. In one example, right-shifting bits can be used to quantize the outlier sample instead of using the quantization procedure for transformation coefficients or residuals.

b. In one example, left-shifting the bits can be used to inversely quantize the outlier sample instead of using the inverse quantization procedure for transform coefficients or residuals.

c. On the encoder side, the following may apply:

i. In one example, the value of a dropped pixel may be signaled as f(p, Qp), where p denotes the pixel/sample value.

ii. In one example, the function f can be defined as p>>((Qp-4)/6), where p denotes the pixel/sample value and Qp denotes the quantization parameter.

iii. In one example, the value of a dropped pixel may be signaled as p>>N, where p denotes the pixel/sample value.

1. In one example, the number N may be an integer (for example, 2) and may depend on

a) The message is sent as signaling as part of an SPS set/VPS set/PPS set/Image header/Slice header/Tile group header/LCU string/LCU group.

b) Internal bit depth

c) Input bit depth

d) Dimensions of the current block

e) Current quantization parameter of the current block

f) Color format indication (such as 4:2:0, 4:4:4)

g) Separate/dual coding tree structure

h) Slice/tile group type and/or image type

d. On the decoder side the following may apply:

i. In one example, the value of the outlier pixel/sample may be signaled as f(bd,p,Qp)

1. In one example, the function f can be defined as clip(0, (1<<(bd-(Qp-4)/6))-1, (p + (1<<(bd-1))) >>((Qp-4)/6)).

ii. In one example, the value of the dropped pixel/sample can be reconstructed as f(p,Qp), where p denotes the decoded value of the dropped pixel/sample.

1. In one example, the function f can be defined as p<<((Qp-4)/6)

iii. In one example, the value of the dropped pixel/sample can be reconstructed as f(bd,p,Qp), where p denotes the decoded value of the dropped pixel/sample.

1. In one example, the clipping function can be defined as clip(0, (1<<bd)-1, p<<((Qp-4)/6))

iv. In the above examples, the clipping function clip(a,i,b) can be defined as (i < a ? a : (i > b ? b : i)).

v. In the above examples, the clipping function clip(a,i,b) can be defined as (i <= a ? a : (i >= b ? b : i)).

vi. In the above examples, p may denote a pixel/sample value, bd may denote an internal bit depth or input bit depth, and Qp denotes a quantization parameter.

16. A block encoded in palette mode can be treated as a block encoded in intra-prediction mode (eg, MODE_INTRA) during the Most Probable Mode (MPM) list generation procedure.

a. In one example, when sampling intra-prediction modes of adjacent (adjacent or non-adjacent) blocks during MPM mode list construction, if an adjacent block (e.g., left or top) is encoded in palette mode, it can be considered a block encoded in normal mode intra-frame prediction, (for example, MODE_INTRA) with the default mode.

i. In one example, the default mode may be DC/PLANAR/VER/HOR mode.

ii. In one example, the default mode may be one of the intra-prediction modes.

iii. In one example, the default mode signaling may be conveyed as part of a DPS set/SPS set/VPS set/PPS set/APS set/picture header/slice header/tile group header/Largest coding unit (LCU) )/Coding Unit (CU)/String of LCU/Group of LCU/TU/Block of PU/Video Coding Unit.

17. A palette-mode encoded block may be treated as a non-intra-prediction block (eg, treated as a MODE_PLT block) during most probable mode (MPM) list construction.

a. In one example, when sampling intra-prediction modes of neighboring blocks during MPM mode list construction, if an adjacent block (e.g., left or top) is encoded in palette mode, it can be treated in the same or similar manner as blocks encoded in inter-frame mode. forecasting.

b. In one example, when sampling intra-frame prediction modes of neighboring blocks during MPM mode list construction, if an adjacent block (e.g., left or top) is encoded in palette mode, it can be treated in the same or similar manner as blocks encoded in copy mode IBC.

18. A luma block encoded in palette mode and corresponding to a chrominance block encoded in DM mode can be interpreted as a block having a default intra-prediction mode.

a. In one example, a corresponding luma block encoded into palette mode may be considered an intra-predictive block (eg, MODE_PLT) when the chrominance block is encoded in DM mode.

b. In one example, the default forecast mode may be DC/PLANAR/VER/HOR mode.

c. In one example, the default prediction mode may be one of the intra-prediction modes.

d. In one example, the default prediction mode signaling may be transmitted as part of a DPS set/SPS set/VPS set/PPS set/APS set/picture header/slice header/tile group header/largest coding unit (LCU)/coding unit (CU)/LCU Lines/LCU Groups/TU Units/PU Units/Video Coding Units.

19. A block encoded in palette mode can be considered as an inaccessible block during list construction for history-based motion vector prediction (HMVP), MERGE mode, and/or advanced prediction mode advanced motion vector prediction (AMVP).

a. In one example, an inaccessible block may be a block for which there is no motion information or its motion information cannot be used to predict other blocks.

b. In one example, a palette-mode encoded block may be considered an intra-prediction block (eg, MODE_INTRA) or a palette-mode block (eg, MODE_PLT) during the list building process for HMVP prediction, MERGE, and/or prediction modes AMVP.

i. Alternatively, in one example, when retrieving neighboring block motion information during list construction for HMVP prediction, MERGE, and/or AMVP prediction modes, the palette mode-encoded neighboring block may be considered to be a block with an invalid support index.

ii. Alternatively, in one example, when retrieving neighboring block motion information during list building for HMVP, MERGE, and/or AMVP prediction modes, the palette-mode-encoded neighboring block may be considered an inter-prediction block having support index equal to 0.

iii. Alternatively, in one example, when retrieving neighboring block motion information during list construction for HMVP, MERGE, and/or AMVP prediction modes, the palette-mode-encoded neighboring block may be considered an inter-prediction block having zero motion vector.

20. The decision about how to treat a block encoded in palette mode (for example, whether to use the methods described above and/or how to apply them) can be based on:

a. Video content (such as screen content or native content)

b. Message sent as signaling as part of DPS Set/SPS Set/VPS Set/PPS Set/APS Set/Picture Header/Slice Header/Tile Group Header/Largest Coding Unit (LCU)/Coding Unit (CU)/String of LCUs/ Group of LCU Units/TU Unit/Block of PU Units/Video Coding Unit

c. Position of CU unit/PU unit/TU unit/block/video coding unit

d. Sizes of the current block and/or adjacent blocks

e. Shape of the current block and/or its neighboring blocks

f. Color format indication (such as 4:2:0, 4:4:4, RGB or YUV)

g. Coding tree structure (such as double tree or single tree)

h. Slice/Tile Group Type and/or Image Type

i. Color component (for example, may only apply to the luminance component and/or chrominance component)

j. Time Layer ID

k. Profiles/Levels/Tieres of the standard

21. Sections (bits) with context-sensitive encoding for a block encoded in palette encoding mode may be limited within a certain range.

a. In one example, a counter is associated with the block to record how many sections (bits) have been subjected to context-sensitive encoding. When the count in the counter exceeds the threshold, bypass encoding is used instead of context-sensitive encoding.

i. Alternatively, NumColorComp counters can be assigned to record how many sections (bits) have been subjected to context-sensitive encoding for each color component. The NumColorComp number represents the number of color components to be encoded in one block (eg, for one CU unit in YUV format, the NumColorComp number is set to 3).

ii. Alternatively, the counter may be initialized to zero and, after encoding one section (bit) in a context-sensitive manner, the number in the counter is incremented by one.

b. Alternatively, the counter may be initialized to some value greater than zero (eg, W*H*K), and after encoding one bit in a context-sensitive manner, the number in the counter is decremented by one. When the count is less than or equal to T, bypass encoding is used instead of context-sensitive encoding.

i. In one example, the value of T is set to 0 or 1.

ii. In one example, the value of T is set in accordance with the decoded information or the number of encoding passes or the like.

c. In one example, blocks encoded in the palette encoding mode may have the same or different threshold values compared to blocks encoded in the TS mode, or to blocks encoded in the non-TS mode, in terms of sections with context-sensitive encoding .

i. In one example, the number of context-sensitive encoding sections (bits) for a block encoded in palette encoding mode may be set to (W*H*T), where W and H represent the width and height of one block, respectively, and T is an integer. In one example, the T number is set to the same as that used for blocks encoded in the TS mode, such as 1.75 or 2.

ii. In one example, the number of context-sensitive encoding sections (bits) for a block encoded in palette encoding mode can be set to (W*H*NumColorComp*T), where W and H represent the width and height of one block, respectively; NumColorComp denotes the number of color components to be encoded for one block (for example, for one CU unit in YUV format, the NumColorComp number is set to 3), and T is an integer. In one example, the T number is set to the same as that used for blocks encoded in the TS mode, such as 1.75 or 2.

d. In one example, the threshold for blocks encoded in palette mode may be less than for blocks encoded in TS mode, or for blocks encoded in non-TS mode, in terms of sections (bits) with context-sensitive encoding .

e. In one example, the threshold for blocks encoded in palette mode may be greater than for blocks encoded in TS mode, or for blocks encoded in non-TS mode, in terms of sections (bits) with context-sensitive encoding .

22. A palette-mode encoded block may be treated as a non-intra prediction mode encoded block (eg, considered as a block with a prediction mode set to MODE_PLT) during the CIIP prediction mode intra-prediction neighbor counting process.

a. In one example, when sampling intra-prediction modes for neighboring blocks during intra-prediction neighbor counting in CIIP prediction mode, if an adjacent block (eg, left and/or top) is encoded in palette mode, it may be considered the same or in the same way as blocks encoded in inter-frame prediction mode.

b. In one example, when sampling intra-prediction modes for neighboring blocks during intra-prediction neighbor counting in CIIP prediction mode, if an adjacent block (eg, left and/or top) is encoded in palette mode, it may be considered the same or in the same way as blocks encoded in IBC copy mode.

c. Alternatively, a block encoded in palette mode can be considered as an intra-prediction block during the CIIP prediction mode intra-prediction neighbor counting process.

23. It is suggested to skip pre-filtering and/or post-filtering procedures for samples encoded in palette mode.

a. In one example, samples encoded in palette mode may not be released.

b. In one example, samples encoded in palette mode may not be subject to offset compensation during the SAO process.

c. In one example, samples encoded in palette mode may not be filtered during ALF filtering.

i. In one example, the classification during the ALF filtering process may skip samples encoded in palette mode

d. In one example, the LMCS mode may not be activated for samples encoded in palette mode.

24. It is proposed to add more scanning orders to the palette mode.

a. In one example, the reverse horizontal cross-scan order may be used, defined as follows.

i. In one example, the scanning direction for odd lines may be left to right.

ii. In one example, the scanning direction for even lines may be from right to left.

iii. In one example, the scan order for a 4x4 block may be as shown in FIG. 22.

b. In one example, a reverse vertical cross-scan order may be used, defined as follows.

i. In one example, the scanning direction for odd lines may be from top to bottom.

ii. In one example, the scanning direction for even lines may be from bottom to top.

iii. In one example, the scan order for a 4x4 block may be as shown in FIG. 23.

25. The combination of valid scan orders may depend on the block shape.

a. In one example, when the width-to-height ratio of a block is greater than a threshold value, only the horizontal transverse scan order and the reverse horizontal transverse scan order may be applied.

i. In one example, the threshold value is 1.

ii. In one example, the threshold value is 4.

b. In one example, when the height to width ratio of a block is greater than a threshold value, only the vertical lateral scan order and the reverse vertical lateral scan order may be applied.

i. In one example, the threshold value is 1.

ii. In one example, the threshold value is 4.

26. It is proposed that in QR-BDPCM modulation mode, only one intra-prediction direction and/or one scanning direction is allowed.

a. In one example, only the vertical direction is allowed for a block whose width is greater than its height.

b. In one example, only the horizontal direction is allowed for a block whose width is less than its height.

c. In one example, a direction indication in the QR-BDPCM modulation mode may be output for a non-square block.

i. In one example, furthermore, the direction indication in the QR-BDPCM modulation mode may indicate the vertical direction for a block whose width is greater than its height.

ii. In one example, furthermore, the direction indication in the QR-BDPCM modulation mode may indicate a horizontal direction for a block whose width is less than its height.

27. The methods according to paragraphs 24, 25 and 26 can only be applied to blocks for which w*Th >= h or h*Th >= w, where w and h denote the width and height, respectively, and Th denotes the threshold value.

a. In one example, the Th value is an integer (for example, 4 or 8) and may be based on

i. Video content (such as screen content or natural content)

ii. Message sent as signaling as part of DPS Set/SPS Set/VPS Set/PPS Set/APS Set/Picture Header/Slice Header/Tile Group Header/Largest Coding Unit (LCU)/Coding Unit (CU)/String of LCUs/ Group of LCU Units/TU Unit/Block of PU Units/Video Coding Unit

iii. Position of CU unit/PU unit/TU unit/block/video coding unit

iv. Sizes of the current block and/or adjacent blocks

v. Shape of the current block and/or its neighboring blocks

vi. Color format indication (such as 4:2:0, 4:4:4, RGB or YUV)

vii. Coding tree structure (such as double tree or single tree)

viii. Slice/Tile Group Type and/or Image Type

ix. Color component (for example, may only apply to the luminance component and/or chrominance component)

x. Time Layer ID

xi. Profiles/Levels/Tieres of the standard

5. Additional options

5.1 Option 1

This section shows an example of an embodiment in which the bitstream representation of video can be modified from the basic bitstream syntax. Changes are highlighted in bold italics.

5.2 Option #2

This option describes the modeType variable. The newly added text is in bold italic.

A modeType variable may be used to specify whether the intra-prediction, IBC copy, palette mode, and inter-prediction modes (MODE_TYPE_ALL) can be used, or whether only the intra-prediction, palette mode, and inter-prediction modes can be used. IBC copying (MODE_TYPE_INTRA), or whether only inter-frame prediction coding modes (MODE_TYPE_INTER) can be used for coding units within a coding tree node.

5.3 Option #3

This option describes the syntax of the encoding unit. In this embodiment, the pred_mode_plt_flag flag is signaled after the pred_mode_ibc_flag flag. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

7.3.7.5 Coding unit syntax

5.4 Option #4

This option describes the syntax of the encoding unit. In this embodiment, the pred_mode_plt_flag is signaled after the pred_mode_ibc_flag, and the pred_mode_plt_flag is signaled only when the current prediction mode is the MODE_INTRA mode. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

7.3.7.5 Coding unit syntax

5.5 Option #5

This option describes the syntax of the encoding unit. In this embodiment, the pred_mode_ibc_flag is signaled after the pred_mode_plt_flag. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

7.3.7.5 Coding unit syntax

5.6 Option #6

This option describes the syntax of the encoding unit. In this embodiment, the pred_mode_ibc_flag is signaled after the pred_mode_plt_flag, and the pred_mode_plt_flag is signaled only when the current prediction mode is the MODE_INTRA mode. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

7.3.7.5 Coding unit syntax

5.7 Option #7

This option describes the syntax of the encoding unit. In this embodiment, the pred_mode_plt_flag flag and the pred_mode_ibc_flag flag are signaled when the prediction mode is MODE_INTRA mode. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

7.3.7.5 Coding unit syntax

5.8 Option #8

This option describes the syntax of the encoding unit. In this embodiment, the pred_mode_plt_flag flag and the pred_mode_ibc_flag flag are signaled when the prediction mode is not MODE_INTRA mode. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

7.3.7.5 Coding unit syntax

5.9 Option #9

This option describes the syntax of the encoding unit. In this embodiment, the pred_mode_plt_flag flag and the pred_mode_ibc_flag flag are signaled when the prediction mode is MODE_INTER. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

7.3.7.5 Coding unit syntax

5.10 Option #10

This variant describes the semantics of the pred_mode_plt_flag flag. The newly added text is in bold italic.

5.11 Option #11

This variant describes the semantics of the pred_mode_plt_flag flag. The newly added text is in bold italic.

5.12 Option #12

This option describes how to determine the hardness of a boundary. The newly added text is in bold italic.

8.8.3.5 Procedure for determining boundary filtration strength

The inputs to this procedure are:

recPicture matrix of image samples,

position ( xCb, yCb ) specifying the top-left sample of the current encoding block relative to the top-left sample of the current image,

variable nCbW specifying the width of the current coding block,

variable nCbH specifying the height of the current encoding block,

edgeType variable specifying whether the vertical (EDGE_VER) or horizontal (EDGE_HOR) edge is filtered,

variable cIdx, which specifies the color component of the current encoding block,

two-dimensional (nCbW)x(nCbH) edgeFlags matrix.

The output of this procedure is a two-dimensional (nCbW)x(nCbH) matrix bS, which defines the filtering strength of the boundary.

...

The variable bS[ xD _i ][ yD _j ] is defined as follows:

If cIdx is 0 and both samples p ₀ and q ₀ are in a coding block for which the intra_bdpcm_flag is 1, the value of bS[ xD _i ][ yD _j ] is set to 0.

Otherwise, if a sample p ₀ or q ₀ is in a coding block from a coding unit that is encoded in the intra-prediction mode, the parameter bS[ xD _i ][ yD _j ] is set to 2.

Otherwise, if the block edge is also a transform block edge and sample p ₀ or q ₀ is in an encoding block for which the ciip_flag is 1, bS[ xD _i ][ yD _j ] is set to 2.

Otherwise, if the block edge is also a transform block edge and sample p ₀ or q ₀ is in a transform block containing one or more non-zero transformation levels, bS[ xD _i ][ yD _j ] is set to 1.

Otherwise, if the prediction mode of the encoding subblock containing sample p ₀ is different from the prediction mode of the encoding subblock containing sample q ₀ , the parameter bS[ xD _i ][ yD _j ] is set to 1.

Otherwise, if cIdx is 0 and one or more of the following conditions are true, bS[ xD _i ][ yD _j ] is set to 1:

Both subblocks – encoding subblock containing sample p₀, and subblock encoding containing sample q₀, are encoded in the IBC copy prediction mode, and the absolute difference between the horizontal or vertical components of the motion vectors used in the prediction of the two encoding subblocks is not less than 4 in the quarter unit of luminance component samples.

To predict the encoding subblock containing sample p ₀ , different reference images or a different number of motion vectors are used compared to that used to predict the encoding subblock containing sample q ₀ .

Note 1 – The determination of whether the reference pictures used for two encoding subblocks are the same or different is based only on which pictures are referenced, regardless of whether the prediction is generated using index to reference picture list 0 or index to list 1 reference pictures, and regardless of whether the indexed positions in the reference picture list are different.

Note 2 – The number of motion vectors used to predict a name encoding subblock with a top-left sample covering the point ( xSb, ySb ) is equal to PredFlagL0[ xSb ][ ySb ] + PredFlagL1[ xSb ][ ySb ].

One motion vector is used to predict the encoding subblock. containing sample p ₀ , and one motion vector is used to predict a coding subblock containing sample q ₀ , and the absolute difference between the horizontal or vertical components of the motion vectors used is not less than 4 in units of luminance component samples.

Two motion vectors and two different reference pictures are used to predict a coding subblock containing sample p ₀ , two motion vectors for the same two reference pictures are used to predict a coding subblock containing sample q ₀ , and the absolute difference between the horizontal or vertical components of the two vectors motion used to predict said two encoding subblocks for the same reference image of at least 4 in units of quarter luminance samples.

Two motion vectors for the same reference picture are used to predict a coding subblock containing sample p ₀ , two motion vectors for the same reference picture are used to predict a coding subblock containing sample q ₀ , and both of the following conditions are true:

The absolute difference between the horizontal or vertical components of the motion vectors from list 0 used to predict the specified two coding subblocks is not less than 4 in units of quarter luminance samples, or the absolute difference between the horizontal or vertical components of the motion vectors from list 1 used to predict the specified two coding subblocks, not less than 4 in quarters of the unit of measurement of the brightness component samples.

The absolute difference between the horizontal or vertical components of the motion vector from list 0 used to predict the encoding subblock containing sample p ₀ and the motion vector from list 1 used to predict the encoding subblock containing sample q ₀ is not less than 4 in quarters of the sample unit brightness component, or the absolute difference between the horizontal or vertical components of the motion vector from list 1 used to predict the encoding subblock containing sample p ₀ and the motion vector from list 0 used to predict the encoding subblock containing sample q ₀ not less than 4 in quarters of the unit of measurement of the brightness component samples.

Otherwise, the variable bS[ xD _i ][ yD _j ] is set to 0.

5.13a Option #13a

This option describes how to determine the hardness of a boundary. The newly added text is in bold italic.

8.8.3.5 Procedure for determining boundary filtration strength

The inputs to this procedure are:

recPicture matrix of image samples,

position ( xCb, yCb ) specifying the top-left sample of the current encoding block relative to the top-left sample of the current image,

variable nCbW specifying the width of the current coding block,

variable nCbH specifying the height of the current encoding block,

edgeType variable specifying whether the vertical (EDGE_VER) or horizontal (EDGE_HOR) edge is filtered,

variable cIdx, which specifies the color component of the current encoding block,

two-dimensional (nCbW)x(nCbH) edgeFlags matrix.

The output of this procedure is a two-dimensional (nCbW)x(nCbH) matrix bS, which defines the filtering strength of the boundary.

...

The variable bS[ xD _i ][ yD _j ] is defined as follows:

If cIdx is 0 and both samples p ₀ and q ₀ are in a coding block for which the intra_bdpcm_flag is 1, the value of bS[ xD _i ][ yD _j ] is set to 0.

Otherwise, if a sample p ₀ or q ₀ is in a coding block from a coding unit that is encoded in the intra-prediction mode, the parameter bS[ xD _i ][ yD _j ] is set to 2.

Otherwise, if the block edge is also a transform block edge and sample p ₀ or q ₀ is in an encoding block for which the ciip_flag is 1, bS[ xD _i ][ yD _j ] is set to 2.

Otherwise, if the block edge is also a transform block edge and sample p ₀ or q ₀ is in a transform block containing one or more non-zero transformation levels, bS[ xD _i ][ yD _j ] is set to 1.

Otherwise, if the prediction mode of the encoding subblock containing sample p ₀ is different from the prediction mode of the encoding subblock containing sample q ₀ , the parameter bS[ xD _i ][ yD _j ] is set to 1.

Otherwise, if cIdx is 0 and one or more of the following conditions are true, bS[ xD _i ][ yD _j ] is set to 1:

Both subblocks – encoding subblock containing sample p₀, and subblock encoding containing sample q₀, are encoded in the IBC copy prediction mode, and the absolute difference between the horizontal or vertical components of the motion vectors used in the prediction of the two encoding subblocks is not less than 4 in a quarter unit of luminance component samples.

To predict the encoding subblock containing sample p ₀ , different reference images or a different number of motion vectors are used compared to that used to predict the encoding subblock containing sample q ₀ .

Note 1 – The determination of whether the reference pictures used for two encoding subblocks are the same or different is based only on which pictures are referenced, regardless of whether the prediction is generated using index to reference picture list 0 or index to list 1 reference pictures, and also regardless of whether the indexed positions in the reference picture list are different.

Note 2 – The number of motion vectors used to predict a name encoding subblock with a top-left sample covering the point ( xSb, ySb ) is equal to PredFlagL0[ xSb ][ ySb ] + PredFlagL1[ xSb ][ ySb ].

One motion vector is used to predict the encoding subblock. containing sample p ₀ , and one motion vector is used to predict a coding subblock containing sample q ₀ , and the absolute difference between the horizontal or vertical components of the motion vectors used is not less than 4 in units of luminance component samples.

Two motion vectors and two different reference pictures are used to predict a coding subblock containing sample p ₀ , two motion vectors for the same two reference pictures are used to predict a coding subblock containing sample q ₀ , and the absolute difference between the horizontal or vertical components of the two vectors motion used to predict said two encoding subblocks for the same reference image of at least 4 in units of quarter luminance samples.

Two motion vectors for the same reference picture are used to predict a coding subblock containing sample p ₀ , two motion vectors for the same reference picture are used to predict a coding subblock containing sample q ₀ , and both of the following conditions are true:

The absolute difference between the horizontal or vertical components of the motion vectors from list 0 used to predict the specified two coding subblocks is not less than 4 in units of quarter luminance samples, or the absolute difference between the horizontal or vertical components of the motion vectors from list 1 used to predict the specified two coding subblocks, not less than 4 in quarters of the unit of measurement of the brightness component samples.

The absolute difference between the horizontal or vertical components of the motion vector from list 0 used to predict the encoding subblock containing sample p ₀ and the motion vector from list 1 used to predict the encoding subblock containing sample q ₀ is not less than 4 in quarters of the sample unit brightness component, or the absolute difference between the horizontal or vertical components of the motion vector from list 1 used to predict the encoding subblock containing sample p ₀ and the motion vector from list 0 used to predict the encoding subblock containing sample q ₀ not less than 4 in quarters of the unit of measurement of the brightness component samples.

Otherwise, the variable bS[ xD _i ][ yD _j ] is set to 0.

5.13b Option #13b

This option describes the encoding and reconstruction of outlier samples. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

[[The list levelScale[ ] is specified as levelScale[ k ] = { 40, 45, 51, 57, 64, 72 } with k = 0..5.]]

The following applies:

[[tmpVal = ( PaletteEscapeVal[ cIdx ][ xCb + xL ][ yCb + yL ] *
levelScale[ qP%6 ] ) << ( qP / 6 ) + 32 ) >> 6 (8-77)

recSamples[ x ][ y ] = Clip3( 0, ( 1 << bitDepth ) − 1, tmpVal ) (8-78)]]

5.14 Option #14

Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

8.4.5.3 Decoding procedure for palette mode

The inputs to this procedure are:

- position ( xCb, yCb ), specifying the top-left sample of the brightness component of the current block relative to the top-left sample of the current image,

- the startComp variable specifying the first color component in the palette table,

- variable cIdx, which specifies the color component of the current block,

- two variables nCbW and nCbH, specifying the width and height of the current block, respectively.

The output of the procedure is the matrix recSamples[ x ][ y ], for x = 0.. nCbW − 1, y = 0.. nCbH – 1, specifying the reconstructed values of the block samples.

Depending on the value of the cIdx variable, the nSubWidth and nSubHeight variables are defined as follows:

- If the cIdx variable is 0, the nSubWidth variable is set to 1, and the nSubHeight variable is set to 1.

- Otherwise, the nSubWidth variable is set to SubWidthC and the nSubHeight variable is set to SubHeightC.

A block ( nCbW x nCbH ) of the recSamples matrix of reconstructed samples at position ( xCb, yCb ) is represented by recSamples[ x ][ y ] with x = 0..nCTbW − 1 and y = 0..nCbH − 1, and the value of recSamples[ x ] [ y ] for each x in the range 0 to nCbW − 1, inclusive, and each y in the range 0 to nCbH − 1, inclusive, is defined as follows:

- The variables xL and yL are defined as follows:

xL = palette_transpose_flag ? x * nSubHeight : x * nSubWidth (8-234)

yL = palette_transpose_flag ? y * nSubWidth : y * nSubHeight (8-235)

- The bIsEscapeSample variable is defined as follows:

- If PaletteIndexMap[ xCb + xL ][ yCb + yL ] is equal to MaxPaletteIndex and palette_escape_val_present_flag is 1, bIsEscapeSample is set to 1.

- Otherwise, bIsEscapeSample is set to 0.

- If bIsEscapeSample is 0, the following applies:

recSamples[ x ][ y ] = CurrentPaletteEntries[ cIdx ][ PaletteIndexMap[ xCb + xL ][ yCb + yL ] ] (8-236)

- Otherwise, if the cu_transquant_bypass_flag is 1, the following applies:

recSamples[ x ][ y ] = PaletteEscapeVal[ cIdx ][ xCb + xL ][ yCb + yL ] (8-237)

- Otherwise (bIsEscapeSample is 1 and cu_transquant_bypass_flag is 0), the following ordered steps apply:

1. The procedure for determining the quantization parameters, as specified in clause 8.7.1, is invoked from the position (xCb, yCb) specifying the top-left sample of the current block relative to the top-left sample of the current image.

[Ed. (BB): The QP parameters are already defined at the beginning of the decoding procedure of an intra-prediction encoded CU, so there is no need to define them again in this subclause. While this is similar to the HEVC v4 SCC encoding standard, I think this redundancy can be eliminated. Please confirm. ]

QpPrimeTsMin = 4 + min_qp_prime_ts_ minus4

3. The bitDepth variable is defined as follows:

bitDepth = ( cIdx = = 0 ) ? BitDepth _Y : BitDepth _C (8-241)

4. The list levelScale[ ] is specified as levelScale[ k ] = { 40, 45, 51, 57, 64, 72 } with k = 0..5.

[Ed. (BB): For CU units encoded not in palette mode, the levelScale parameter depends on rectNonTsFlag, should this be applied here too?]

5. The following applies:

tmpVal = ( PaletteEscapeVal[ cIdx ][ xCb + xL ][ yCb + yL ] *
levelScale[ qP%6 ] ) << ( qP / 6 ) + 32 ) >> 6 (8-242)

recSamples[ x ][ y ] = Clip3( 0, ( 1 << bitDepth ) − 1, tmpVal ) (8-243)

When one of the following conditions is true:

- the variable cIdx is equal to 0 and the parameter numComps is equal to 1;

- variable cIdx is equal to 3;

the PredictorPaletteSize[startComp] variable and the PredictorPaletteEntries matrix are defined or modified as follows:

for( i = 0; i < CurrentPaletteSize[ startComp ]; i++ )
for( cIdx = startComp; cIdx < (startComp + numComps); cIdx++ )
newPredictorPaletteEntries[ cIdx ][ i ] = CurrentPaletteEntries[ cIdx ][ i ]
newPredictorPaletteSize = CurrentPaletteSize[ startComp ]
for( i = 0; i < PredictorPaletteSize && newPredictorPaletteSize <PaletteMaxPredictorSize; i++ )
if( !PalettePredictorEntryReuseFlags[ i ] ) {
for( cIdx = startComp; cIdx < (startComp + numComps); cIdx++ ) (8-244)
newPredictorPaletteEntries[ cIdx ][ newPredictorPaletteSize ] =
PredictorPaletteEntries[ cIdx ][ i ]
newPredictorPaletteSize++
}
for( cIdx = startComp; cIdx < ( startComp + numComps ); cIdx++ )
for( i = 0; i <newPredictorPaletteSize; i++ )
PredictorPaletteEntries[ cIdx ][ i ] = newPredictorPaletteEntries[ cIdx ][ i ]
PredictorPaletteSize[ startComp ] = newPredictorPaletteSize

The data bitstream matching requirement is that the value of PredictorPaletteSize[ startComp ] must be in the range 0 to PaletteMaxPredictorSize, inclusive.

5.15 Option #15

Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

8.4.2 Procedure for determining the intra-frame prediction mode for the luminance component

...

- Otherwise (both skip_intra_flag[ xPb ][ yPb ] and DimFlag[ xPb ][ yPb ] are 0), the IntraPredModeY[ xPb ][ yPb ] parameter is determined through the following ordered steps:

1. Adjacent positions ( xNbA, yNbA ) and ( xNbB, yNbB ) are set equal to ( xPb − 1, yPb ) and ( xPb, yPb − 1 ), respectively.

2. For X to be replaced by A or B, the candIntraPredModeX variables are defined as follows:

▪ The procedure for determining the availability for a block in z-scan order, as specified in clause 6.4.1, is invoked with position ( xCurr, yCurr ) set to ( xPb, yPb ) and adjacent position ( xNbY, yNbY ) set to ( xNbX, yNbX ) as input, and the output is assigned to the variable availableX.

▪ The candidate intra-prediction mode candIntraPredModeX is defined as follows:

▪ If availableX is FALSE, candIntraPredModeX is set to INTRA_DC.

▪ [[– Otherwise, if CuPredMode[ xNbX ][ yNbX ] is not equal to MODE_INTRA or pcm_flag[ xNbX ][ yNbX ] is equal to 1 or , candIntraPredModeX is set to INTRA_DC, ]]

▪ Otherwise, if X is equal to B and yPb − 1 is less than ( ( yPb >> CtbLog2SizeY ) << CtbLog2SizeY ), candIntraPredModeB is set equal to INTRA_DC.

▪ Otherwise, if IntraPredModeY[ xNbX ][ yNbX ] is greater than 34, candIntraPredModeX is set equal to INTRA_DC.

...

5.16 Option #16

Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

8.4.2 Procedure for determining the intra-frame prediction mode for the luminance component

The inputs to this procedure are:

- position ( xCb , yCb ) of the luminance component, specifying the top-left sample of the current luminance coding block relative to the top-left sample of the luminance component of the current image,

- variable cbWidth, specifying the width of the current coding block in brightness samples,

- variable cbHeight, specifying the height of the current coding block in brightness samples.

This procedure defines the IntraPredModeY[ xCb ][ yCb ] intra-prediction mode for luminance samples.

1. For X to be replaced by A or B, the candIntraPredModeX variables are defined as follows:

- The candidate intra-prediction mode candIntraPredModeX is defined as follows:

- If one or more of the following conditions are true, candIntraPredModeX is set equal to INTRA_PLANAR.

- The availableX variable is FALSE.

- CuPredMode[ xNbX ][ yNbX ] is not equal to MODE_INTRA.

- intra_mip_flag[ xNbX ][ yNbX ] is equal to 1.

- X is equal to B and yCb − 1 is less than ( ( yCb >> CtbLog2SizeY ) << CtbLog2SizeY ).

- Otherwise, candIntraPredModeX is set equal to IntraPredModeY[ xNbX ][ yNbX ].

...

The variable IntraPredModeY[ x ][ y ] with x = xCb..xCb + cbWidth − 1 and y = yCb..yCb + cbHeight − 1 is set equal to IntraPredModeY[ xCb ][ yCb ].

5.17 Option #17

Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

8.4.3 Procedure for determining the intra-frame prediction mode for the luminance component

The inputs to this procedure are:

- position (xCb, yCb) of the luminance component, specifying the top-left sample of the current luminance coding block relative to the top-left sample of the luminance component in the current image,

- variable cbWidth, which specifies the width of the current coding block in luminance component samples,

- variable cbHeight, which specifies the height of the current coding block in luminance component samples.

In this procedure, the IntraPredModeY[ xCb ][ yCb ] mode of intra-frame luminance component prediction is determined.

1. For X to be replaced by A or B, the candIntraPredModeX variables are defined as follows:

- The procedure for determining availability for a block, as specified in Article 6.4.X, [Ed. (BB): A procedure for checking the availability of neighboring blocks has yet to be developed] is invoked with position ( xCurr, yCurr ) set to ( xCb, yCb ) and neighboring position ( xNbY, yNbY ) set to ( xNbX, yNbX ) as input data, and the output is assigned to the variable availableX.

- The candidate intra-prediction mode candIntraPredModeX is defined as follows:

- If one or more of the following conditions are true, candIntraPredModeX is set equal to [[INTRA_PLANAR]] INTRA_DC.

- The availableX variable is FALSE.

- CuPredMode[ xNbX ][ yNbX ] is not equal to MODE_INTRA.

- The flag intra_mip_flag[ xNbX ][ yNbX ] is equal to 1.

- X is equal to B and yCb − 1 is less than ( ( yCb >> CtbLog2SizeY ) << CtbLog2SizeY ).

- Otherwise, candIntraPredModeX is set equal to IntraPredModeY[ xNbX ][ yNbX ].

...

The variable IntraPredModeY[ x ][ y ] with x = xCb..xCb + cbWidth − 1 and y = yCb..yCb + cbHeight − 1 is set equal to IntraPredModeY[ xCb ][ yCb ].

5.18 Option #18

Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

8.4.3 Procedure for determining the intra-frame prediction mode for the luminance component

The inputs to this procedure are:

- position (xCb, yCb) of the luminance component, specifying the top-left sample of the current luminance coding block relative to the top-left sample of the luminance component in the current image,

- variable cbWidth, which specifies the width of the current coding block in luminance component samples,

- variable cbHeight, which specifies the height of the current coding block in luminance component samples.

In this procedure, the IntraPredModeY[ xCb ][ yCb ] mode of intra-frame luminance component prediction is determined.

1. For X to be replaced by A or B, the candIntraPredModeX variables are defined as follows:

- The candidate intra-prediction mode candIntraPredModeX is defined as follows:

- If one or more of the following conditions are true, candIntraPredModeX is set equal to [[INTRA_PLANAR]] INTRA_DC.

- The availableX variable is FALSE.

- CuPredMode[ xNbX ][ yNbX ] is not equal to MODE_INTRA.

- The flag intra_mip_flag[ xNbX ][ yNbX ] is equal to 1.

- X is equal to B and yCb − 1 is less than ( ( yCb >> CtbLog2SizeY ) << CtbLog2SizeY ).

- Otherwise, candIntraPredModeX is set equal to IntraPredModeY[ xNbX ][ yNbX ].

...

The variable IntraPredModeY[ x ][ y ] with x = xCb..xCb + cbWidth − 1 and y = yCb..yCb + cbHeight − 1 is set equal to IntraPredModeY[ xCb ][ yCb ].

5.19 Option #19

Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

5.20 Option #20

Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

5.21 Option #21

Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

5.22 Option #22

This option describes the syntax of the encoding unit. In this embodiment, the pred_mode_plt_flag flag is signaled after the pred_mode_ibc_flag flag. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

7.3.7.5 Coding unit syntax

5.23 Option #23

Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

5.24 Option #24

This option describes the syntax of the encoding unit. In this embodiment, the pred_mode_plt_flag is signaled after the pred_mode_ibc_flag. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

7.3.7.5 Coding unit syntax

5.25 Option #25

This option describes the syntax of the encoding unit. In this embodiment, the palette syntax is signaled if the current prediction mode is MODE_PLT mode. Newly added text is in bold italic, and deleted text is marked “[[ ]]”.

7.3.7.5 Coding unit syntax

coding_unit( x0, y0, cbWidth, cbHeight, cqtDepth, treeType, modeType ) { Descriptor chType = treeType = = DUAL_TREE_CHROMA? 10 if( slice_type != I | | sps_ibc_enabled_flag | | sps_palette_enabled_flag) { if( treeType != DUAL_TREE_CHROMA &&
!( ( ( cbWidth = = 4 && cbHeight = = 4 ) | | modeType = = MODE_TYPE_INTRA )
&& !sps_ibc_enabled_flag ) ) cu_skip_flag[ x0 ][ y0 ] ae(v) if( cu_skip_flag[ x0 ][ y0 ] = = 0 && slice_type != I
&& !( cbWidth = = 4 && cbHeight = = 4 ) && modeType = = MODE_TYPE_ALL ) pred_mode_flag ae(v) if( ( ( slice_type = = I && cu_skip_flag[ x0 ][ y0 ] = =0 ) | |
( slice_type != I && ( CuPredMode[ chType ][ x0 ][ y0 ] != MODE_INTRA | |
( cbWidth = = 4 && cbHeight = = 4 && cu_skip_flag[ x0 ][ y0 ] = = 0 ) ) ) &&
cbWidth <= 64 && cbHeight <= 64 && modeType != MODE_TYPE_INTER &&
sps_ibc_enabled_flag && treeType != DUAL_TREE_CHROMA ) pred_mode_ibc_flag ae(v) if( ( ( ( slice_type = = I | | ( cbWidth = = 4 && cbHeight = = 4 ) | | sps_ibc_enabled_flag ) &&
CuPredMode[ x0 ][ y0 ] = = MODE_INTRA ) | |
( slice_type != I && !( cbWidth = = 4 && cbHeight = = 4 ) && !sps_ibc_enabled_flag
&& CuPredMode[ x0 ][ y0 ] != MODE_INTRA ) ) && sps_palette_enabled_flag &&
cbWidth <= 64 && cbHeight <= 64 &&&& cu_skip_flag[ x0 ][ y0 ] == 0 &&
modeType != MODE_INTER ) pred_mode_plt_flag ae(v) } } if([[CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTRA | |]]
CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_PLT ) { if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_LUMA ) { if( pred_mode_plt_flag ) { if( treeType = = DUAL_TREE_LUMA ) palette_coding( x0, y0, cbWidth, cbHeight, 0, 1 ) else /* SINGLE_TREE */ palette_coding( x0, y0, cbWidth, cbHeight, 0, 3 ) } else { ... } ... }

5.26 Option #26

This option describes the procedure for determining the intra-prediction mode for the chrominance component. The newly added text is in bold italic.

Procedure for determining the intra-frame prediction mode for the color component

The inputs to this procedure are:

- position ( xCb, yCb ) of the luminance component, specifying the top-left sample of the current chrominance coding block relative to the top-left sample of the luminance component in the current image,

- variable cbWidth, which specifies the width of the current coding block in luminance component samples,

- variable cbHeight, which specifies the height of the current coding block in luminance component samples.

In this procedure, the IntraPredModeC[ xCb ][ yCb ] mode of intra-frame chrominance prediction is determined.

The corresponding lumaIntraPredMode mode for intra-frame luminance prediction is determined as follows:

- If the intra_mip_flag[ xCb ][ yCb ] is 1, the lumaIntraPredMode parameter is set to INTRA_PLANAR.

- Otherwise, if the parameter CuPredMode[ 0 ][ xCb ][ yCb ] is equal to MODE_IBC , the lumaIntraPredMode parameter is set to INTRA_DC.

- Otherwise, the lumaIntraPredMode parameter is set to IntraPredModeY[ xCb + cbWidth / 2 ][ yCb + cbHeight / 2 ].

...

5.27 Option #27

This option describes an image reconstruction procedure with a display procedure for luminance component samples. The newly added text is in bold italic.

Image reconstruction with the procedure for displaying the brightness component. The inputs to this procedure are:

- position ( xCurr, yCurr ) of the upper left sample of the current block relative to the upper left sample of the current image,

- variable nCurrSw, specifying the width of the block,

- variable nCurrSh, specifying the height of the block,

- matrix (nCurrSw)x(nCurrSh) of predSamples samples, specifying the predicted samples of the brightness component for the current block,

- matrix (nCurrSw)x(nCurrSh) of resSamples samples, specifying samples of the residual brightness component for the current block.

The output of this procedure is a matrix of reconstructed recSamples samples of the brightness component of the image.

The matrix (nCurrSw)x(nCurrSh) of the mapped predicted samples of the brightness component predMapSamples is defined as follows:

- If one of the following conditions is true, the samples predMapSamples[ i ][ j ] are set equal to predSamples[ i ][ j ] for i = 0..nCurrSw − 1, j = 0..nCurrSh − 1:

- the variable CuPredMode[ 0 ][ xCurr ][ yCurr ] is equal to MODE_INTRA.

- the variable CuPredMode[ 0 ][ xCurr ][ yCurr ] is equal to MODE_IBC.

- the variable CuPredMode[ 0 ][ xCurr ][ yCurr ] is equal to MODE_INTER and the flag ciip_flag[ xCurr ][ yCurr ] is equal to 1.

- Otherwise, the parameter (CuPredMode[ 0 ][ xCurr ][ yCurr ] is equal to MODE_INTER and the ciip_flag[ xCurr ][ yCurr ] is equal to 0) is applied as follows:

...

5.28 Option #28

This option describes an example of scan orders corresponding to Example 24 in Section 4.

The input to this procedure is the block width blkWidth and the block height blkHeight.

The output of this procedure is the matrices hReverScan[ sPos ][ sComp ] and vReverScan[ sPos ][ sComp ]. The hReverScan matrix represents the horizontal reverse scan order, and the vReverScan matrix represents the vertical reverse scan order. The sPos matrix index specifies the scan position, ranging from 0 to ( blkWidth * blkHeight ) − 1, inclusive. A matrix index sComp of 0 specifies the horizontal component, and a matrix index sComp of 1 specifies the vertical component. Depending on the value of blkWidth and blkHeight, the hTravScan and vTravScan matrices are determined as follows:

i = 0
for(y = 0; y <blkHeight; y++ )

{
if( y % 2 != 0 ) {
for( x = 0; x <blkWidth; x++ ) {
hReverScan[ i ][ 0 ] = x
hReverScan[i][1] = y
i++
}

}
else

{

for( x = blkWidth − 1; x >= 0; x− − ) {
hReverScan[i][0] = x
hReverScan[i][1] = y
i++
}

}

}

i = 0
for( x = 0; x <blkWidth; x++ )

{
if( x % 2 != 0 )

{
for( y = 0; y <blkHeight; y++ ) {
vReverScan[ i ][ 0 ] = x
vReverScan[i][1] = y
i++
}

}
else

{

for( y = blkHeight − 1; y >= 0; y− − ) {
vReverScan[i][0] = x
vReverScan[i][1] = y
i++
}

}

}

In fig. 6 is a block diagram of a video processing apparatus 600. Apparatus 600 may be used to implement one or more of the methods described herein. Device 600 may be implemented as a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. Device 600 may include one or more processors 602, one or more storage devices 604, and video processing hardware 606. Processor(s) 602 may be configured to implement one or more of the methods described herein. Storage device(s) 604 may be used to store data and code used to implement the methods and technologies described herein. Video processing hardware 606 may be used to implement, in hardware, some of the technologies described herein.

In fig. 8 illustrates a logic diagram for a video processing method 800. This method 800 comprises determining (805) that a palette mode should be used to process a transform unit, coding block, or region, applying the palette mode code separately from the prediction mode, and performing (810) further processing of the transform unit, coding block, or region using the mode palettes.

With respect to method 800, some examples of palette mode encoding and its use are described in Section 4 of this document.

With respect to method 800, a block of video may be encoded in a video data bitstream, in which bit efficiency can be achieved using a data bitstream generation rule related to palette mode encoding.

The proposed methods may include an option in which the prediction mode indication is encoded before the palette mode use indication.

The proposed methods may include an option in which the use of the palette mode is signaled conditionally in the form of signaling based on the predictive mode.

The proposed methods may include an embodiment in which the prediction mode is an intra-frame block copy mode, and signaling indicating the use of the palette mode is omitted.

The proposed methods may include an embodiment that determines that the palette mode use indication is false based on the fact that the current prediction mode is an intra-frame block copy mode.

The proposed methods may include an embodiment in which the prediction mode is an inter-frame prediction mode and signaling indicating the use of the palette mode is omitted.

The proposed methods may include an embodiment that determines that the palette mode usage indication is false based on the fact that the current prediction mode is an inter-frame prediction mode.

The proposed methods may include an embodiment in which the prediction mode is an intra-frame prediction mode, and signaling indicating the use of the palette mode is omitted.

The proposed methods may include an embodiment that determines that the palette mode usage indication is false based on the fact that the current prediction mode is an intra-prediction mode.

The proposed methods may include an embodiment in which the prediction mode is an intra-frame prediction mode, and signaling indicating the use of the palette mode is omitted.

The proposed methods may include an option in which the prediction mode is an intra-frame block copy mode, and carry out signaling indicating the use of the palette mode.

The proposed methods may include an option in which the palette mode use indication is provided as an alarm based on the image, slice, or tile group type.

The proposed methods may include an option in which a palette mode is added as a candidate for a prediction mode.

The proposed methods may include an option in which the set of prediction modes may contain one or more modes from the following list: intra-prediction mode, intra-frame block copy mode, or palette modes for intra-prediction slices, inter-frame prediction slices, I-frames, P-frames , B-frames or tile groups with intra-frame prediction.

The proposed methods may include an option in which the set of prediction modes contains two or more modes from the following list: intra-prediction mode, inter-frame prediction mode, intra-frame block copy mode, or palette mode.

The proposed methods may include an option in which the use of a palette mode is indicated by signaling or determined based on some condition.

The proposed methods may include an option in which the specified condition may be one or more items from the following list: dimensions of the current block, prediction mode for the current block, quantization parameter (QP) for the current block, palette mode flag for adjacent blocks, intraframe mode flag copying for adjacent blocks, color format indication, split or dual coding tree structure, or slice type, or group type, or image type.

The proposed methods may include an option wherein the indication of use of the palette mode is signaled or determined based on a flag at the slice level, a flag at the tile group level, or a flag at the image level.

The proposed methods may include an embodiment in which the use of intra-frame block copy mode is signaled or determined based on a slice-level flag, a tile-group-level flag, or an image-level flag.

Taking into account items 6 - 9 described in the previous section, some options may preferably use the following technical solutions.

One technical solution may comprise a video processing method comprising performing a conversion between a current video block from a single video image and a representation of the video in question in the form of a stream of data bits, where information about whether the intra-frame block copy mode is used for this conversion is signaled to composition of the representation in the form of a stream of data bits or determined based on the encoding condition of the current video block; where the intra-frame block copy mode contains the encoding of the current video block from another video block in the image. The following features can be implemented in various options:

- where the specified encoding condition contains the dimensions of the current video block.

- where the specified encoding condition contains a prediction mode for the current video block or a quantization parameter used in the conversion for the current video block.

Taking into account items 13 - 15 described in the previous section, some options may preferably implement the following technical solutions.

The technical solution may comprise a method for determining whether to apply a deblocking filter in the process of converting a current video block from a single video image, where the current video block is encoded using palette mode encoding, in which the current video block is represented using representative sample values, the number of which is less than the total number of pixels in the current video block; and performing a transformation using the deblocking filter when the result of said determination indicates that the deblocking filter should be applied.

Another solution may comprise a video processing method of defining a quantization procedure or an inverse quantization procedure for use in the conversion process between a current video block from a single video image and a representation of the video in question in the form of a stream of data bits, where the current video block is encoded using encoding mode a palette in which the current video block is represented using representative sample values, the number of which is less than the total number of pixels of the current video block; and performing a transformation based on said definition of a quantization procedure or an inverse quantization procedure. A set of additional features may include:

- where the quantization procedure or inverse quantization procedure selected by the specified determination for the current video block is different from another quantization procedure or from another inverse quantization procedure applied to another video block encoded in a manner other than palette mode encoding.

- where the conversion procedure involves encoding the current video block and turning it into a representation in the form of a stream of data bits.

- where the conversion procedure comprises decoding a representation in the form of a stream of data bits to generate the current video block of the video in question.

- where the transformation procedure uses a decision procedure identical to another decision procedure used to transform another video block encoded using intra-frame prediction.

It should be understood that the described technologies may be implemented in video encoders or video decoders to improve compression efficiency using improved coding tree structures.

Taking into account items 16 – 21 in the previous section, some technical solutions could be as follows:

A video processing method, comprising: determining, for converting between a current video block of a video containing a plurality of video blocks, and a representation of this video in the form of a stream of data bits, that the current video block is a block encoded in a palette encoding mode; based on the above definition, performing a procedure for constructing a list of the most probable modes by considering the current video block as a block encoded using intra-frame prediction, and performing a transformation based on the result of the list construction procedure; where a block encoded in the palette encoding mode is encoded or decoded using the palette or representative sample values.

The above method, where the list construction procedure considers the adjacent palette-mode encoded block as an intra-prediction block in the default mode.

A video processing method, comprising: determining, for converting between a current video block of a video containing a plurality of video blocks, and a representation of this video in the form of a stream of data bits, that the current video block is a block encoded in a palette encoding mode; based on the above definition, performing a procedure for constructing a list of the most probable modes by considering the current video block as a block not encoded using intra-frame prediction, and performing a transformation based on the result of the listing procedure; where a block encoded in the palette encoding mode is encoded or decoded using the palette or representative sample values.

The above method, in which the list construction procedure considers an adjacent block encoded in the palette mode as an intra-prediction encoded block when selecting an intra-prediction mode for the adjacent block encoded in the palette encoding mode.

A video processing method, comprising: determining, for converting between a current video block of a video containing a plurality of video blocks, and a representation of this video in the form of a stream of data bits, that the current video block is a block encoded in a palette encoding mode; based on the above definition, performing a list construction procedure by considering the current video block as an unavailable block, and performing a conversion based on the result of the list construction procedure; where a block encoded in the palette encoding mode is encoded or decoded using the palette or representative sample values.

The above method, where the list construction procedure is designed to predict the motion vector based on the history.

The above method where the list construction procedure is applied to the MERGE mode or the advanced motion vector prediction mode.

The above methods, where the determination procedure further comprises determining based on the content of the video.

The above methods where the definition procedure corresponds to a field in a representation in the form of a stream of data bits.

A video processing method, comprising: determining, during a conversion between a current video block and a representation of that current video block in the form of a data bitstream, that the current video block is a block encoded in a palette encoding mode, determining, based on the fact that the current a video block is a block encoded in palette encoding mode, a range of sections (bits) with context-sensitive encoding used for conversion; and performing a conversion based on this range of sections (bits) with context-sensitive encoding.

The above method, where sections (bits) of the current video block falling outside this range are encoded using a bypass mode encoding method or decoded using a bypass mode decoding method in the conversion process

The above methods where the conversion procedure comprises encoding the video and converting it into a representation in the form of a stream of data bits.

The above methods where the conversion procedure comprises decoding a representation in the form of a stream of data bits to generate a video.

In fig. 24 is a block diagram showing an example of a video processing system 2400 in which the various methods described herein may be implemented. Various implementations may include some or all of the components of the system 2400. The system 2400 may include an input 2402 for receiving video content. This video content may be received in its original or uncompressed format, 8 or 10-bit multi-pixel values, or may be in a compressed or encoded format. This input 2402 may be a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired bus interfaces such as Ethernet, passive optical network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

System 2400 may include an encoding component 2404 capable of implementing various encoding techniques described herein. The encoding component 2404 may reduce the average bit rate of video data from the input 2402 to the output of the encoding component 2404 to obtain an encoded representation of the video. Therefore, these encoding methods are sometimes called video compression methods or video transcoding methods. The output of encoding component 2404 may be either stored or transmitted via attached communications media, represented here by component 2406. The stored or transmitted (or encoded) representation of the video in the form of a stream of data bits received as input 2402 may be used by component 2408 to generate values. pixels or display video transmitted to display interface 2410. The procedure of generating user-viewable video from its representation in the form of a stream of data bits is sometimes called video decompression (expansion). In addition, although some video processing operations are referred to as "encoding" operations or tools, it should be understood that the encoding tools or operations are used in the encoding devices and the corresponding decoding tools or operations that reverse the encoding results will be performed by the decoding device.

Examples of a peripheral bus interface or display interface may be a universal serial bus (USB) or a high definition multimedia interface (HDMI) or Displayport, etc. Examples of storage interfaces include the serial advanced technology attachment (SATA) interface, the peripheral device interface (PCI), the IDE interface, and other similar interfaces. The methods described herein may be implemented in a variety of electronic devices, such as mobile phones, laptop computers, smartphones, or other devices capable of digital processing and/or displaying video.

In fig. 25 is a logic diagram of a video processing method 2500 in accordance with the proposed technology. Method 2500 comprises, at step 2510, converting between a video region block of a video clip and a representation of that video in the form of a stream of data bits. This representation in the form of a data bit stream is processed according to a first format rule specifying whether to signal a first palette mode use indication for a block, and a second format rule specifying the position of the first indication relative to a second prediction mode use indication for that block.

In some embodiments, the video domain comprises a transform unit, an encoding unit, a prediction unit, or a video domain. In some embodiments, the second indication of use of the prediction mode is located in the video representation in the form of a stream of data bits before the first indication of use of the palette mode.

In some embodiments, a first indication of palette mode use is conditionally included in the video representation in the form of a data bitstream based on a second indication of prediction mode use. In some embodiments, the first indication of using a palette mode is omitted in the video representation in the form of a data bit stream if the second indication of using a prediction mode is a prediction mode using intra-block copy (IBC). In some embodiments, the first indication of using a palette mode is omitted in the video representation in the form of a data bit stream if the second indication of using a prediction mode indicates an inter-frame prediction mode. In some embodiments, the first indication of using a palette mode is omitted in the video representation in the form of a data bitstream if the second indication of using a prediction mode indicates an intra-frame prediction mode. In some embodiments, the first indication of using a palette mode is skipped in the video representation in the form of a data bitstream if the second indication of using a prediction mode is a skip mode. In some embodiments, omitting the first indication of use of the palette mode in the video representation in the form of a data bit stream means that the palette mode is not used.

In some embodiments, the first palette mode use indication is encoded into a data bit stream if the second prediction mode use indication is an IBC copy prediction mode. In some embodiments, the first palette mode use indication is encoded into a data bitstream if the second prediction mode use indication indicates an intra-prediction mode. In some embodiments, the prediction mode is not a pulse-code modulation (PCM) mode. In some embodiments, the first indication of use of the palette mode is encoded before the indication of use of the PCM modulation mode in the video representation in the form of a data bitstream. In some embodiments, an indication of the use of a PCM modulation mode is provided in the video representation in the form of a stream of data bits. In some embodiments, the IBC copy mode indication is encoded and converted into a data bit stream representation. In some embodiments, if an intra-prediction mode is used, a flag in the bitstream representation indicates that the bitstream representation is signaling an indication of the palette mode or the IBC copy mode. In some embodiments, this flag is passed based on a condition characterizing the block, this condition comprising the size of the block, an indication of whether IBC copy mode is enabled for an area associated with that block, or whether palette mode is activated for a specified area associated with that block.

In some embodiments, the first palette mode use indication is encoded into a data bit stream if the second prediction mode use indication indicates an inter-prediction mode. In some embodiments, the first palette mode use indication is encoded after at least one of a skip mode indication, a prediction mode use indication, or a PCM modulation mode use indication. In some embodiments, the first indication of using a palette mode is encoded after the indication of a skip mode or prediction mode and is encoded before the indication of using a PCM modulation mode.

In some embodiments, the first palette mode use indication is positioned before the second prediction mode use indication in the data bit stream representation. In some embodiments, the first palette mode use indication is positioned after the second prediction mode use indication, this second prediction mode use indication indicating an intra- or inter-frame prediction mode in the video representation in the form of a data bit stream. In some embodiments, the first indication of palette mode use is provided as an alarm based on the image, slice, or tile group type. In some embodiments, the first palette mode use indication includes a first flag indicating that the palette mode is enabled for the block. In some embodiments, a first indication of palette mode use is conventionally included in the video presentation in the form of a data bitstream based on a first flag indicating that the palette mode is enabled at the sequence level, image level, tile group level, or tile level. In some embodiments, another flag indicating the PCM modulation mode for the block is included in the data bitstream representation if the palette mode for the block in question is not activated. In some embodiments, the first flag is subject to context-sensitive encoding based on information about one or more neighboring blocks relative to the current block. In some embodiments, this first flag is encoded without context information from one or more neighboring blocks relative to the current block.

In some embodiments, the second indication of use of the prediction mode includes a second flag indicating the prediction mode. In some embodiments, if a second flag in the bitstream representation indicates that the prediction mode is an inter-frame prediction mode, the bitstream representation further includes a third flag indicating that the intra-frame block copy mode is enabled. In some embodiments, if a second flag in the bitstream representation indicates that the prediction mode is an intra-frame prediction mode, the bitstream representation further includes a third flag indicating that the intra-frame block copy mode is enabled. In some embodiments, the third flag is conditionally included in the representation in the form of a data bitstream based on block sizes.

In some embodiments, the block in question is a coding unit, and the second flag in the data bit stream representation indicates that the prediction mode is an intra-prediction mode. In some embodiments, said first flag is conditionally included in a data bit stream representation based on the size of the block.

In fig. 26 is a logic diagram representation of a video processing method 2600 in accordance with the proposed technology. Method 2600 comprises, at step 2610, determining, for conversion between a block of video region within a video and a data bitstream representation of that video, a prediction mode based on one or more valid prediction modes, including at least a palette mode for the block in question. The palette mode usage indication is determined in accordance with the prediction mode. Method 2600 comprises, at step 2620, performing a conversion based on the specified determination.

In some embodiments, the plurality of one or more valid prediction modes comprises an intra-frame prediction mode. In some embodiments, the plurality of one or more valid prediction modes comprises an intraframe block copy (IBC) mode. In some embodiments, the plurality of one or more valid prediction modes comprises an inter-frame prediction mode.

In some embodiments, the video region in question comprises an intra-prediction encoded slice, an intra-prediction encoded image, or a group of intra-prediction encoded tiles. In some embodiments, the plurality of one or more valid prediction modes comprises an intra-prediction mode, an intra-frame block copy mode, and a palette mode.

In some embodiments, the video region in question comprises an inter-prediction encoded slice, an inter-prediction encoded image, or a group of inter-prediction encoded tiles, a P-slice, a B-tear, a P-frame, or a B-frame. In some embodiments, the plurality of one or more valid prediction modes comprises an intra-prediction mode, an intra-frame block copy mode, a palette mode, and an inter-frame prediction mode.

In some embodiments, the block is 4x4 in size. In some embodiments, the inter-frame prediction mode is excluded from the set of one or more valid prediction modes if the block in question is 4x4 in size.

In some embodiments, the data bit stream representation comprises at least a prediction mode index representing one or more valid prediction modes if the block in question is not encoded in skip mode, where the prediction mode index is represented using one or more sections.

In some embodiments, the prediction mode index is represented using three binary sections (bits), where a value of "1" in the first section indicates an intra-prediction mode, a value of "0" in the first section, and a value of "0" in the second section indicates an inter-prediction mode. the value "0" in the first section, the value "1" in the second section and the value "0" in the third section indicates the IBC copy mode, and the value "0" in the first section", the value "1" in the second section and the value "1" the third section indicates the palette mode.

In some embodiments, the prediction mode index is represented using two binary sections (bits), where a value of "1" in the first section and a value of "0" in the second section indicate an intra-frame prediction mode, a value of "0" in the first section and a value of "0" in the second section indicates an inter-frame prediction mode, and a value of “0” in the first section and a value of “1” in the second section indicates an IBC copy mode, and a value of “1” in the first section and a value of “1” in the second section indicates a palette mode.

In some embodiments, the prediction mode index is represented using a single binary section (bit), if the current video slice is an intra-prediction encoded slice and the IBC copy mode is not enabled, the first value "0" in this section indicates the intra-prediction mode and the second value The "1" in this section indicates palette mode.

In some embodiments, the prediction mode index is represented using two binary sections (bits) if the current video slice is not an intra-prediction encoded slice and the IBC copy mode is not enabled, where a value of "1" in the first section indicates the intra-prediction mode , a value of "0" in the first section and a value of "0" in the second section denotes an inter-frame prediction mode, and a value of "0" in the first section and a value of "1" in the second section denotes a palette mode. In some embodiments, the prediction mode index is represented using two binary sections (bits) if the current video slice is an intra-prediction encoded slice and the IBC copy mode is enabled, where the value "1" in the first section indicates the IBC copy mode value A "0" in the first section and a value of "1" in the second section indicates a palette mode, and a value of "0" in the first section and a value of "0" in the second section indicates an intra-frame prediction mode. In some embodiments, the indication of use of the IBC copy mode is conveyed as signaling in a sequence parameter set (SPS) in a data bit stream representation.

In some embodiments, the prediction mode index is represented using three binary sections (bits),

where the value "1" in the first section indicates the inter-prediction mode, the value "0" in the first section and the value "1" in the second section indicates the intra-prediction mode, the value "0" in the first section, the value "0" in the second section and the value The "1" in the third section denotes the IBC copy mode, and the "0" value in the first section, the "0" value in the second section, and the "0" value in the third section denotes palette mode.

In some embodiments, the prediction mode index is represented using three binary sections (bits),

where the value "1" in the first section indicates the intra-prediction mode, the value "0" in the first section and the value "1" in the second section indicates the inter-prediction mode, the value "0" in the first section, the value "0" in the second section and the value The "1" in the third section denotes the IBC copy mode, and the "0" value in the first section, the "0" value in the second section, and the "0" value in the third section denotes palette mode.

In some embodiments, the prediction mode index is represented using three binary sections (bits), where a value of "0" in the first section indicates an inter-prediction mode, a value of "1" in the first section, and a value of "0" in the second section indicates an intra-prediction mode. the value "1" in the first section, the value "1" in the second section and the value "1" in the third section indicate the IBC copy mode, and the value "1" in the first section, the value "1" in the second section and the value "0" in The third section indicates the palette mode.

In some embodiments, signaling regarding one or more binary sections (bits) in the data bit stream representation is omitted if a certain condition is satisfied. In some embodiments, this condition contains the size of the block in question. In some embodiments, this condition includes an indication that a certain prediction mode is not activated, in which case the binary section (bit) corresponding to that prediction mode is skipped in the data bit stream representation.

In fig. 27 is a logic diagram representation of a video processing method 2700 in accordance with the present technology. This method 2700 comprises, at step 2710, converting between a block of video and a representation of that video in the form of a stream of data bits. This representation in the form of a data bit stream is processed in accordance with a formatting rule specifying that a first indication of palette mode use and a second indication of intraframe block copy (IBC) mode use are signaled depending on the other.

In some embodiments, this formatting rule specifies that a first indication is signaled in a data bitstream representation if the prediction mode for the block in question matches the first prediction mode that is not an IBC copy mode. In some embodiments, this formatting rule specifies that the second indication is signaled in a data bitstream representation if the prediction mode for the block in question matches the first prediction mode that is not a palette mode. In some embodiments, said first prediction mode is an intra-prediction mode.

In fig. 28 is a logic diagram representation of a video processing method 2800 in accordance with the present technology. This method 2800 comprises, at step 2810, determining, for conversion between a block of video and a bitstream representation of that video, the presence of a palette mode usage indication in the bitstream representation based on the size of the block in question. Method 2800 also includes, at step 2820, performing a conversion based on the specified determination.

In fig. 29 is a logic diagram representation of a video processing method 2900 in accordance with the present technology. This method 2900 comprises, at step 2910, determining, for conversion between a block of video and a data bitstream representation of that video, the presence of an indication of the use of an intraframe block copy (IBC) mode in the data bitstream representation based on the size of the block in question. Method 2900 also includes, at step 2920, performing a conversion based on the specified determination. In some embodiments, the block size indication comprises at least one of the number of samples per block, block width, or block height.

In some embodiments, the indication is signaled in a data bit stream representation if the block width is not greater than a threshold. In some embodiments, the indication is signaled in a data bit stream representation if the block height is not greater than a threshold value. In some embodiments, the specified threshold value is 64.

In some embodiments, the indication is signaled in a data bit stream representation if the width and height of the block in question is greater than a threshold value. In some embodiments, this threshold value is 4. In some embodiments, the indication is provided as an alarm in a data bit stream representation if the number of samples in the block in question is greater than the threshold value. In some embodiments, this threshold value is 16. In some embodiments, the indication is signaled in a data bit stream representation if the width of the block in question is equal to the height of that block.

In some embodiments, the indication is not present in the data bit stream representation if (1) the width of the block in question is greater than a first threshold, (2) the height of that block is greater than a second threshold, or (3) the number of samples in the block in question is not greater than the third threshold . In some embodiments, the first threshold value and the second threshold value are 64. In some embodiments, the third threshold value is 16.

In some embodiments, said determination is further based on a certain characteristic associated with the block in question. In some embodiments, this characteristic includes an indication of the prediction mode for the block. In some embodiments, this characteristic comprises a quantization parameter for the block. In some embodiments, this characteristic contains a palette flag for a block adjacent to the block in question. In some embodiments, this characteristic contains an IBC copy mode flag for a block adjacent to the block in question. In some embodiments, this characteristic includes an indication of the color format for the block in question. In some embodiments, this characteristic contains the coding tree structure for the block in question. In some embodiments, this characteristic comprises an indication of the slice group type, tile group type, or image type for the block in question.

In fig. 30 is a logic diagram representation of a video processing method 3000 in accordance with the present technology. This method 3000 comprises, at step 3010, determining, for conversion between a block of video and a data bitstream representation of that video, whether a palette mode is valid for the block in question, based on a second indication for the video area containing that block. Method 3000 also includes, at step 3020, performing a conversion based on the specified determination.

In some embodiments, the video area contains a slice, a group of tiles, or an image. In some embodiments, the data bit stream representation eliminates an explicit indication of whether the palette mode is valid if the second indication indicates that fractional motion vector difference is enabled. In some embodiments, the second indication is in the form of a flag present in the representation in the form of a stream of data bits. In some embodiments, the second indication indicates whether a palette mode is activated for the video area. In some embodiments, the data bit stream representation eliminates an explicit indication of whether the palette mode is valid if the second indication indicates that the palette mode is not enabled for the video region. In some embodiments, the palette mode is invalid for the block in question if the representation in the form of a stream of data bits precludes an explicit indication of whether the palette mode is valid.

In fig. 31 is a logic diagram representation of a video processing method 3100 in accordance with the present technology. This method 3100 comprises, at step 3110, determining, for conversion between a block of video and a representation of that video in the form of a data bit stream, whether intra-block copy (IBC) mode is valid for the block in question, based on a second indication for the video area containing that block. Method 3100 also includes, at step 3120, performing a conversion based on the specified determination.

In some embodiments, said video area comprises a slice, a group of tiles, or an image. In some embodiments, the data bit stream representation eliminates an explicit indication of whether the IBC copy mode is valid if the second indication indicates that fractional motion vector difference is enabled. In some embodiments, the second indication is in the form of a flag present in the representation in the form of a stream of data bits. In some embodiments, the second indication indicates whether the IBC copy mode is activated for the video region in question. In some embodiments, the data bit stream representation eliminates an explicit indication of whether the IBC copy mode is valid if the second indication indicates that the IBC copy mode is not enabled for the video region in question. In some embodiments, the IBC copy mode is invalid for a block if the data bit stream representation precludes an explicit indication of whether the IBC copy mode is valid.

In some embodiments, the conversion procedure generates the current block from a data bit stream representation. In some embodiments, the conversion procedure generates a bitstream representation of the data from the current block.

Some embodiments of the proposed technology involve making a decision or determination that a video processing tool or mode should be activated. In one example, when a video processing tool or mode is enabled, the encoder will use that tool or mode when processing a block of video, but may not necessarily modify the resulting data bit stream based on the use of the tool or mode. In other words, the conversion procedure from a block of video to a representation of that video in the form of a stream of data bits will use the specified video processing tool or mode when it is activated based on the decision or determination made. In another example, when a video processing tool or mode is activated, the decoding device will process the data bit stream knowing that the data bit stream has been modified based on the video processing tool or mode. In other words, the conversion from a representation of this video in the form of a stream of data bits to a block of video will be carried out using said video processing tool or mode based on said mode or definition.

Some embodiments of the proposed technology involve making a decision or determination that a tool or video processing mode should be deactivated. In one example, when a video processing tool or mode is not enabled, the encoder will not use that tool or mode when converting a block of video into a data bitstream representation of that video. In another example, when a video processing tool or mode is not activated, the decoding device will process the data bit stream knowing that the data bit stream has not been modified using said video processing tool or mode that was activated based on the decision or determination made.

The proposed and other solutions, examples, embodiments, modules and functional operations described in this document may be implemented in digital electronic circuitry or in computer downloadable software, firmware or hardware, including the structures discussed herein and their structural equivalents, or in combination of one or more of the listed components. The considered and other options can be implemented in one or more computer software products, i.e. in one or more modules of computer program instructions encoded on a computer-readable storage medium for execution by a data processing device or for controlling the operation of that data processing device. The computer-readable medium may be a computer-readable storage device, a computer-readable storage substrate, a storage device, a composition of objects generating a computer-readable propagating signal, or a combination of one or more of the above. The term "data processing device" covers all devices, apparatus and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. Such a device may contain, in addition to hardware, code that provides an environment for executing the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these. A propagating signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal, generated to encode information for transmission to a suitable receiving device.

A computer program (also known as a program, software, software application, script or code) may be written in any programming language, including compiled or interpreted languages, and may be deployed in any form, including a stand-alone program or in form of a module, component, subroutine, or other unit suitable for use in a computer environment. A computer program does not necessarily correspond to a file on a file system. A program may be stored in a portion of a file that also contains other programs or data (for example, one or more scripts are stored in a markup language document), in a single file specifically dedicated only to the program in question, or in multiple coordinated files (for example, files that save one or more modules, subroutines or code fragments). A computer program may be deployed for execution on a single computer or on multiple computers located at a single location or distributed over multiple locations and connected through a communications network.

The procedures and logic described herein may be implemented by one or more programmable processors executing one or more computer programs to implement functions by operating on input data and generating output data. These procedures and logic circuits may also be implemented by, and the hardware may also be implemented in, the form of special-purpose logic circuitry, such as a field programmable gate array (FPGA) or application specific integrated circuit (ASIC). circuit)).

Processors suitable for executing a computer program include, for example, general purpose and special purpose microprocessors and any one or more digital computer processors of any type. In general, the processor will receive instructions and data from read only memory and/or random access memory. The main elements of a computer are a processor for executing commands and one or more storage devices for storing commands and data. In general, a computer will also include, or be operatively coupled to receive data and/or transmit data, one or more mass storage devices for storing data, such as magnetic devices, magneto-optical disks, or optical disks. However, the computer does not need to have such devices. Computer-readable media for storing computer program instructions and data includes all forms of non-volatile storage devices and storage media, including, for example, semiconductor storage devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable memory (EPROM). EEPROM) and flash memory devices; magnetic disks, such as built-in hard drives or removable drives; magneto-optical disks; and CD ROMs and DVD-ROMs. The processor and storage device can be augmented or embedded in special-purpose logic circuitry.

Although this patent document contains many specific details, they should not be construed as limiting the scope of any subject matter of the invention or what may be claimed, but merely as a description of features that may be specific to particular embodiments of particular methods. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of one embodiment may also be implemented in multiple embodiments individually or in some suitable subcombination. Moreover, although features may be described above as operating in certain combinations, and even originally stated as such, one or more features from a claimed combination may, in some cases, be excluded from such combination, and the claimed combination may be made into a subcombination or variation subcombinations.

Likewise, although operations are shown in a particular order in the drawings, this should not be understood to require that such operations always be performed in the particular order shown, or in a sequential order, or that all operations shown be performed in order to achieve the desired result. Moreover, the separation of the various system components in the embodiments described herein should not be understood to require such separation in all embodiments.

Only a small number of implementations and examples are described herein, but other implementations, improvements and variations may be made based on what is described and illustrated herein.

Claims (35)

1. A method for processing video data, comprising the steps of: determining, for conversion between the current video video block and the video data bit stream, that a first prediction flag is included in the data bit stream, wherein the current video block is not skipped, wherein the first prediction flag indicates whether an inter-frame prediction mode or an intra-frame prediction mode is applied; determining that a second prediction flag or a third prediction flag is further included in the data bit stream, at least based on the value of the first prediction flag, wherein the second prediction flag indicates whether the first prediction mode is applied, wherein in the first prediction mode, prediction samples are obtained from the values reconstructed samples of the same image that contains the current video block, wherein a third prediction flag indicates whether a second prediction mode is applied, wherein in the second prediction mode, the reconstructed samples are represented by a plurality of representative color values, and the plurality of representative color values comprises at least one of 1) palette predictors, 2) outlier samples, or 3) palette information included in the data bitstream; And performing a conversion between the first video block and the data bit stream. 2. The method of claim 1, wherein the second prediction flag is further included in the data bit stream if the first prediction flag indicates that an inter-frame prediction mode is being applied. 3. The method of claim 1 or 2, wherein the third prediction flag is further included in the data bit stream if the first prediction flag indicates that an intra-frame prediction mode is being applied. 4. Method according to any one of paragraphs. 1 to 3, in which the third prediction flag is encoded in a context-sensitive manner using a single context. 5. Method according to any one of paragraphs. 1 to 4, wherein the inclusion of the second prediction flag and the third prediction flag in the data bitstream is based on the size of the current video block. 6. Method according to any one of paragraphs. 1 to 5, wherein the inclusion of a second prediction flag in the data bit stream is further based on the fact that the enable flag associated with the first prediction mode is at a higher level, and the inclusion of a third prediction flag in the data bit stream is further based on the fact that the activation flag related to the second prediction mode is at a higher level, wherein said higher level may be a sequence level, an image level, a tile group level, or a tile group level. 7. Method according to any one of paragraphs. 1 to 6, in which the first bin and the second bin are used to indicate which mode from the intra-prediction mode, the inter-prediction mode, the first prediction mode and the second prediction mode is applied to the current video block, where M is equal to the value of the first bin and N is equal to the value of the second bin, Moreover, in the case when (M, N) = (0, 0), the interframe prediction mode is applied to the current video block. 8. The method according to claim 7, wherein in the case where (M, N) = (0, 1), the first prediction mode is applied to the current video block. 9. The method according to claim 7 or 8, wherein in the case where (M, N) = (0, 0) or (0, 1), the first bin corresponds to the first prediction flag, and the second bin corresponds to the second prediction flag, when In this case, the third prediction flag is not included in the data bit stream. 10. Method according to any one of paragraphs. 7 – 9, in which in the case when (M, N) = (1, 0), the intra-frame prediction mode is applied to the current video block. 11. Method according to any one of paragraphs. 7 – 10, in which in the case when (M, N) = (1, 1), the second prediction mode is applied to the current video block. 12. Method according to any one of paragraphs. 7 – 11, in which in the case where (M, N) = (1, 0) or (1, 1), the first bin corresponds to the first prediction flag, and the second bin corresponds to the third prediction flag, while the second prediction flag is not included into a stream of data bits. 13. Method according to any one of paragraphs. 7 - 12, in which in the case where (M, N) = (0, 0), (0, 1) or (1, 1), the third bin indicating whether to apply the third prediction mode is excluded from the data bit stream wherein in the third prediction mode, the difference between the quantized remainder and the prediction of the quantized remainder is represented in the data bit stream as a PCM representation. 14. Method according to any one of paragraphs. 7 – 13, in which the third bin indicating whether to apply the third prediction mode is included in the data bit stream in the case where (M, N) = (1, 0), while in the third prediction mode the difference between the quantized remainder and the prediction the quantized remainder is represented in the data bit stream as a representation using pulse code modulation. 15. Method according to any one of paragraphs. 1–14, in which the conversion procedure involves encoding the current video block into a stream of data bits. 16. The method as claimed in any one of claims 1 to 14, wherein the conversion procedure comprises decoding the current video block from the data bit stream. 17. A video data processing device comprising a processor and a non-volatile storage device with commands written therein, wherein the commands, when executed by the processor, cause the processor to execute: determining, for conversion between the current video video block and the video data bit stream, that a first prediction flag is included in the data bit stream, wherein the current video block is not skipped, wherein the first prediction flag indicates whether an inter-frame prediction mode or an intra-frame prediction mode is applied; determining that a second prediction flag or a third prediction flag is further included in the data bit stream, at least based on the value of the first prediction flag, wherein the second prediction flag indicates whether the first prediction mode is applied, wherein in the first prediction mode, predicted samples are obtained from values of reconstructed samples of the same image that contains the current video block, wherein a third prediction flag indicates whether a second prediction mode is applied, wherein in the second prediction mode, the reconstructed samples are represented by a plurality of representative color values, and the plurality of representative color values comprises at least one of 1 ) palette predictors, 2) outlier samples, or 3) palette information included in the data bit stream; And performing a conversion between the first video block and the data bit stream. 18. A non-volatile computer-readable storage medium that stores instructions that, when executed by a processor, cause the processor to execute: determining, for conversion between the current video video block and the video data bit stream, that a first prediction flag is included in the data bit stream, wherein the current video block is not skipped, wherein the first prediction flag indicates whether an inter-frame prediction mode or an intra-frame prediction mode is applied; determining that a second prediction flag or a third prediction flag is further included in the data bit stream, at least based on the value of the first prediction flag, wherein the second prediction flag indicates whether the first prediction mode is applied, wherein in the first prediction mode, predicted samples are obtained from values of reconstructed samples of the same image that contains the current video block, wherein a third prediction flag indicates whether a second prediction mode is applied, wherein in the second prediction mode, the reconstructed samples are represented by a plurality of representative color values, and the plurality of representative color values comprises at least one of 1 ) palette predictors, 2) outlier samples, or 3) palette information included in the data bitstream; And performing a conversion between the first video block and the data bit stream. 19. A method for storing a video data bit stream, comprising the steps of: determining, for conversion between the current video video block and the video data bit stream, that a first prediction flag is included in the data bit stream, wherein the current video block is not skipped, wherein the first prediction flag indicates whether an inter-frame prediction mode or an intra-frame prediction mode is applied; determining that a second prediction flag or a third prediction flag is further included in the data bit stream, at least based on the value of the first prediction flag, wherein the second prediction flag indicates whether the first prediction mode is applied, wherein in the first prediction mode, prediction samples are obtained from the values reconstructed samples of the same image that contains the current video block, wherein a third prediction flag indicates whether a second prediction mode is applied, wherein in the second prediction mode, the reconstructed samples are represented by a plurality of representative color values, and the plurality of representative color values comprises at least one of 1) palette predictors, 2) outlier samples, or 3) palette information included in the data bitstream; performing a conversion between the first video block and the data bit stream; storing the stream of data bits on a non-volatile computer-readable medium for recording information.

Images