CN113812160B - Maximum possible mode list construction for screen content codec - Google Patents

Abstract

The construction of a list of the maximum possible modes for screen content codec is described. In an exemplary aspect, a method for video processing includes: for a transition between a current block of video and a bitstream representation of the current block, building two or more maximum likelihood mode (MPM) lists of intra-coding modes of the current block, wherein the two or more MPM lists include at least a first MPM list built with a first building method and a second MPM list built with a second building method different from the first building method; and performing conversion using one MPM list selected from the constructed two or more MPM lists.

Description

Maximum possible mode list construction for screen content codec

Cross Reference to Related Applications

The present application claims in time the priority and benefit of international patent application number PCT/CN2019/086114 filed on 5.9 and 9 days of 2019, in accordance with applicable regulations of patent laws and/or paris convention. The entire disclosure of international patent application number PCT/CN2019/086114 is incorporated by reference as part of the disclosure of this application.

Technical Field

The present application relates to video encoding and decoding techniques, devices, and systems.

Background

Despite advances in video compression, digital video still occupies the largest bandwidth usage on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth requirements for digital video usage are expected to continue to increase.

Disclosure of Invention

Apparatus, systems, and methods relating to digital video coding, and more particularly, to adaptive loop filtering for video coding are described. The method may be applied to existing video codec standards, such as High Efficiency Video Codec (HEVC), as well as future video codec standards, such as Versatile Video Codec (VVC), or codecs.

Video codec standards have been developed primarily by developing the well-known ITU-T and ISO/IEC standards. ITU-T developed h.261 and h.263, ISO/IEC developed MPEG-1 and MPEG-4 vision, and two organizations jointly developed the h.262/MPEG-2 video, h.264/MPEG-4 Advanced Video Codec (AVC), and h.265/HEVC standards. Since h.262, video codec standards have been based on hybrid video codec structures in which temporal prediction plus transform coding has been employed. To explore future video codec technologies beyond HEVC, VCEG and MPEG have jointly established a joint video exploration team (jfet) in 2015. Since then, jfet has adopted many new approaches and applied it to reference software known as the Joint Exploration Model (JEM). A joint video expert group (jfet) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) was created 4 months 2018 to work against the VVC standard with the goal of 50% bit rate reduction compared to HEVC.

In one representative aspect, a method for video processing may be provided using the disclosed techniques that includes constructing a first mode list of intra-coding modes based on intra-modes of neighboring blocks of a current video block; constructing a second mode list of intra-frame codec modes based on the first mode list; and performing intra-mode encoding and decoding on the current video block using at least one of the first mode list or the second mode list.

In another representative aspect, the disclosed techniques can be used to provide a method for video processing, the method comprising: a mode list of intra-coding modes is constructed to include candidate modes based on intra-modes of neighboring blocks of the current video block, wherein constructing the mode list includes locating the candidate modes at locations in the mode list based on types of the candidate modes.

In one representative aspect, a method for video processing can be provided using the disclosed techniques. The method includes constructing two or more Maximum Possible Mode (MPM) lists of intra-coding modes of a current block of the video for a transition between the current block and a bitstream representation of the current block, wherein the two or more MPM lists include at least a first MPM list constructed using a first construction method and a second MPM list constructed using a second construction method different from the first construction method; and performing conversion using one MPM list selected from the constructed two or more MPM lists.

In yet another representative aspect, the above-described method is embodied in the form of processor-executable code and stored in a computer-readable program medium.

In yet another representative aspect, an apparatus configured or operable to perform the above-described method is disclosed. The apparatus may include a processor programmed to perform such a method.

In yet another exemplary aspect, a video decoding apparatus implementing the video decoding method described herein is disclosed.

The above and other aspects and features of the disclosed technology are described in more detail in the accompanying drawings, description and claims.

Drawings

Fig. 1 shows an example of intra block copy.

Fig. 2 shows an example of a block coded in a palette mode.

Fig. 3 shows an example of signaling palette entries using palette predictor signalling.

Fig. 4 shows an example of horizontal and vertical traversal scans for the coding palette indices.

Fig. 5 shows a codec example of palette indices.

FIG. 6 illustrates an example of a multi-type tree partitioning pattern.

Fig. 7 shows an example of the positions of the samples used to derive α and β.

Fig. 8 shows an example of luminance mapping with a chromaticity scaling architecture.

Fig. 9 shows an example of 67 intra prediction modes.

Fig. 10 shows an example of the left and upper neighboring blocks of the current block.

Fig. 11 shows an example of four reference lines adjacent to a prediction block.

Fig. 12a shows an example of a subdivision for 4x8 and 8x4 CUs, and fig. 12b shows an example of a subdivision for CUs other than 4x8, 8x4 and 4x 4.

Fig. 13a and 13b illustrate a flowchart of an example method for schema list construction in accordance with the disclosed techniques.

Fig. 14 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described herein.

Fig. 15 is a flow chart of an example method for video encoding and decoding.

Detailed Description

Video codec methods and techniques are ubiquitous in modern technology as the demand for higher resolution video is increasing. Video codecs typically include electronic circuitry or software that compresses and decompresses digital video, and are continually being improved to provide higher codec efficiency. The video codec converts uncompressed video into a compressed format, or vice versa. There is a complex relationship between video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of the encoding and decoding algorithms, the sensitivity to data loss and errors, the ease of editing, random access and end-to-end delay. Compression formats typically conform to standard video compression specifications, such as the High Efficiency Video Codec (HEVC) standard (also known as h.265 or MPEG-H part 2), the upcoming Versatile Video Codec (VVC) standard, or other current and/or future video codec standards.

Embodiments of the disclosed technology may be applied to existing video codec standards (e.g., HEVC, h.265) and future standards to improve runtime performance. The section headings are used herein to enhance the readability of the description, and are not in any way limiting the discussion or embodiments (and/or implementations) to only the corresponding sections.

1 intra block copy

Intra Block Copy (IBC), also known as current picture reference, has been used in HEVC screen content codec extensions (HEVC-SCC) and current VVC test model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. As shown in fig. 1, when IBC is applied, the current block is predicted by a reference block in the same picture. The samples in the reference block must have been reconstructed before the current block is encoded or decoded. Although IBC is not as efficient for most camera capture sequences, it exhibits significant codec gain for screen content. The reason is that there are a large number of repeated patterns, such as icons and text characters, in the screen content picture. IBC can effectively eliminate redundancy between these repeated patterns. In HEVC-SCC, an inter-codec Coding Unit (CU) may apply IBC if the IBC selects the current picture as its reference picture. In this case, the MV is renamed to a Block Vector (BV) which always has integer pixel precision. For compatibility with primary grade HEVC, a current picture is marked as a "long-term" reference picture in a Decoded Picture Buffer (DPB). It should be noted that inter-view reference pictures are also labeled as "long-term" reference pictures in various view/3D video codec standards similarly.

Following BV finds its reference block, a prediction can be generated by copying the reference block. The residual may be obtained by subtracting the reference pixel from the initial signal. Transformation and quantization may then be applied as in other codec modes.

However, when the reference block is outside the picture, or overlaps the current block, or outside the reconstructed region, or outside the active region limited by some constraint, no part or all of the pixel values are defined. Basically, there are two schemes to deal with such problems. One solution is to not allow such a situation, for example in bitstream consistency. Another approach is to apply padding for those undefined pixel values. The following subsections describe schemes in detail.

1.2 IBC in HEVC screen content codec extensions

In the screen content codec extension of HEVC, when a block uses the current picture as a reference, it should be ensured that the entire reference block is in the available reconstruction area, as shown in the following specification text.

Variables offsetX and offsetY are derived as follows:

offsetX＝(ChromaArrayType＝＝0)？0:(mvCLX[0]&0x72:0)

[ equation 1]

offsetY＝(ChromaArrayType＝＝0)？0:(mvCLX[1]&0x72:0)

[ equation 2]

One requirement for bitstream consistency is that the luma motion vector mvLX should obey the following constraints when the reference picture is in the current picture:

-upon invoking the derivation procedure of the z-scan order block availability specified in clause 6.4.1, with (xCurr, yCurr) set equal to (xCb, yCb), and with adjacent luminance positions (xNbY, yNbY) set equal to (xpb+ (mvLX [0] > > 2) -offsetX, yPb + (mvLX [1] > > 2) -offsetY) as inputs, the output should be equal to true.

-upon invoking the derivation procedure of the z-scan order block availability specified in clause 6.4.1, wherein the adjacent luminance positions (xNbY, yNbY) set equal to (xCb, yCb) and set equal to (xpb+ (mvLX [0] > > 2) +npbw-1+offsetx, ypb+ (mvLX [1] > > 2) +npbh-1+offsety) should be equal to true as inputs.

One or both of the following conditions should be true:

the value of (mvLX [0] > 2) +nPbW+xB1+offsetX is less than or equal to 0.

The value of (mvLX [1] > 2) +nPbH+yB1+offsetY is less than or equal to 0.

The following conditions should be true:

(xPb+ (mvLX [0] > > 2) +nPbSh-1+offsetX)/CtbSizeY-xCurr/CtbSizeY < = yCurr/CtbSizeY- (yPb + (mvLX [1] > > 2) +nPbSh-1+offsetY)/CtbSizeY [ equation 3]

Thus, a case where the reference block overlaps with the current block or the reference block is outside the picture does not occur. No padding of the reference block or the prediction block is required.

1.3 IBC in VCC test model

In the current VCC test model, i.e., the VTM-4.0 design, the entire reference block should utilize the current Codec Tree Unit (CTU) and not overlap with the current block. Thus, there is no need to pad the reference block or the prediction block. The IBC flag is encoded and decoded as a prediction mode of the current CU. Thus, there are three prediction MODEs for each CU, MODE_INTRA, MODE_INTER and MODE_IBC.

Ibc Merge mode

In IBC Merge mode, an index is parsed from the bitstream pointing to an entry in the IBC Merge candidate list. The construction of the IBC merge list can be summarized according to the following sequence of steps:

step 1: derivation of spatial candidates

Step 2: insertion of HMVP candidates

Step 3: insertion of pairwise average candidates

In deriving the spatial merge candidates, up to four merge candidates are selected among candidates located at the positions indicated by A1, B0, A0, and B2. The deduced order is A1, B0, A0 and B2. Position B2 is only considered when any PU of positions A1, B0, A0 is not available (e.g. because it belongs to another slice or tile) or IBC mode codec is not utilized. After adding the candidates at the position A1, redundancy check is performed on the insertion of the remaining candidates, the redundancy check ensuring that candidates having the same motion information are excluded from the list, thereby improving the codec efficiency.

After inserting the spatial candidates, IBC candidates from the HMVP table may be inserted if the IBC merge list size is still smaller than the maximum IBC merge list size. Redundancy check is performed when inserting HMVP candidates.

Finally, pairs of average candidates are inserted into the IBC merge list.

A merge candidate is said to be an invalid merge candidate when the reference block identified by the merge candidate is outside the picture, or overlaps the current block, or is outside the reconstructed region, or is outside the active region limited by some constraint.

Note that an invalid merge candidate may be inserted into the IBC merge list.

1.3.2 IBC AMVP mode

In IBC AMVP mode, an AMVP index pointing to an entry in the IBC AMVP list is parsed from the bitstream. The construction of the IBC AMVP list can be summarized according to the following sequence of steps:

step 1: derivation of spatial candidates

A0, A1 are checked until a usable candidate is found.

B0, B1, B2 are checked until a usable candidate is found.

Step 2: insertion of HMVP candidates

Step 3: zero candidate insertion

After inserting the spatial candidates, IBC candidates from the HMVP table may be inserted if the IBC AMVP list size is still smaller than the maximum IBC AMVP list size.

Finally, zero candidates are inserted into the IBC AMVP list.

1.4 Adaptive Motion Vector Resolution (AMVR)

In HEVC, a Motion Vector Difference (MVD) is signaled in units of quarter luma samples (between the motion vector of the CU and the predicted motion vector) when use_integer_mv_flag is equal to 0 in the slice header. In VVC, a CU-level Adaptive Motion Vector Resolution (AMVR) scheme is introduced. AMVR allows MVD codecs of CUs to be decoded with different precision. Depending on the mode of the current CU (normal AMVP mode or affine AVMP mode), the MVD of the current CU may be adaptively selected as follows:

Normal AMVP mode: quarter-luminance samples, integer-luminance samples, or four-luminance samples.

Affine AMVP mode: quarter luminance samples, integer luminance samples, or 1/16 luminance samples.

If the current CU has at least one non-zero MVD component, a MVD resolution indication at the CU level is conditionally signaled. If all MVD components (i.e., horizontal and vertical MVDs for reference list L0 and reference list L1) are zero, then the quarter luma sample MVD resolution is inferred.

For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter-luminance-sample MVD precision is used for the CU. If the first flag is 0, no further signaling is required and the quarter-luma sample MVD precision is used for the current CU. Otherwise, a second flag is signaled to indicate whether integer luminance samples or four luminance samples MVD precision is used for the normal AMVP CU. The same second flag is used to indicate whether integer luminance samples or 1/16 luminance sample MVD precision is used for affine AMVP CU. To ensure that the reconstructed MVs have the desired precision (quarter, integer, or quad luma samples), the motion vector predictors of the CU will be rounded to the same precision as the MVDs before they are added together. The motion vector predictors are rounded towards zero (i.e., the positive infinity is rounded towards negative motion vector predictors and the negative infinity is rounded towards positive motion vector predictors).

The encoder uses the RD check to determine the motion vector resolution for the current CU. To avoid performing three CU-level RD checks for each MVD resolution at all times, only RF checks of MVD precision outside of quarter-luminance samples are conditionally invoked. For the normal AVMP mode, the RD cost for the quarter-luminance sample MVD precision and the integer-luminance sample MV precision is first calculated. Then, the RD cost of the integer-sample MVD precision is compared with the quarter-luminance-sample MVD precision to determine whether the RD cost of the four-luminance-sample MVD precision needs to be further checked. And skipping RD checking of the MVD precision of the four brightness samples when the RD cost of the MVD precision of the quarter brightness samples is far smaller than the MVD precision of the integer brightness samples. For the affine AMVP mode, if the affine inter mode is not selected after checking the rate distortion costs of the affine merge/skip mode, the quarter-luminance sample MVD precision normal AMVP mode, and the quarter-luminance sample MVD precision affine AMVP mode, the 1/16-luminance sample MV precision and the 1-pixel MV precision affine inter mode are not checked. Further, affine parameters obtained in the quarter-luminance sample MV precision affine inter mode are used as starting search points in the 1/16-luminance sample and quarter-luminance sample MV precision affine inter modes.

1.5 Palette mode in HEVC screen content codec extension (HEVC-SCC)

The basic idea behind palette mode is to represent samples in a CU by a small set of representative color values. This set is called a palette. The escape symbol may also be signaled, followed by a (possibly quantized) component value, to indicate the samples outside the palette. This is illustrated in fig. 2.

1.5.1 codec for palette entries

To encode and decode palette entries, a palette predictor is maintained. The maximum sizes of the palette and palette predictors are signaled in the SPS. In HEVC-SCC, a palette_predictor_initiator_present_flag is introduced in PPS. When this flag is 1, an entry for initializing the palette predictor is signaled in the bitstream. The palette predictor is initialized at the beginning of each CTU row, each stripe, and each tile. The palette predictor is reset to 0 according to the value of palette predictor initializer presentflag or initialized using a signaled palette predictor initializer entry in PPS. In HEVC-SCC, a palette predictor initializer of size 0 is enabled to allow explicit disabling of palette predictor initialisation at the PPS level.

For each entry in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette. This is illustrated in fig. 3. The reuse flag is sent using run-length codec of zero. After this, the number of new palette entries is signaled using an exponential Golomb code of order 0. Finally, the component values for the new palette entries are signaled.

1.5.2 coding and decoding of palette indices

The palette index is encoded using horizontal and vertical traversal scans as shown in fig. 4. The scan order is explicitly signaled in the bitstream using a palette _ transfer _ flag. For the remainder of the section, the scan is assumed to be horizontal.

Two palette-sampling modes are used: 'INDEX' and 'COPY_ABOVE', codec palette indices. As previously described, the escape symbol is also signaled as an 'INDEX' mode and is assigned an INDEX equal to the maximum palette size. In addition to the top row, or when the previous mode is 'copy_above', the mode is signaled using a flag. In the 'copy_above' mode, palette indices for samples in the upper row are copied. In the 'INDEX' mode, palette indices are explicitly signaled. For both the 'INDEX' and 'copy_above' modes, a run value is signaled that specifies the number of subsequent samples that also use the same mode codec. When the escape symbol is part of operation in the 'INDEX' or 'COPY_ABOVE' mode, an escape component value is signaled for each escape symbol. The coding of the palette indices is shown in fig. 5.

This syntax order is accomplished as follows. First, the number of index values of the CU is signaled. After this, the actual index value of the entire CU is signaled using truncated binary codec. Both the number of indexes and the index value are encoded and decoded in bypass mode. This divides the index-dependent bypass binary bits together. The palette sample pattern (if necessary) and run are then signaled in an interleaved fashion. Finally, the component escape values corresponding to the escape samples of the entire CU are split together and encoded in bypass mode.

After signaling the index value, an additional syntax element, last_run_type_flag, is signaled. This syntax element, in combination with the number of indexes, eliminates the need to signal the run value corresponding to the last run in the block.

In HEVC-SCC, palette modes are also enabled for 4:2:2, 4:2:0, and monochrome chroma formats. The signaling of palette entries and palette indices is nearly the same for all chroma formats. For non-monochromatic formats, each palette entry is composed of 3 components. For a monochrome format, each palette entry is composed of a single component. For the undersampled chroma direction, the chroma samples are associated with a luma sample index that may be divided by 2. After reconstructing the palette index for the CU, only the first component of the palette entry is used if the sample has only a single component associated with it. The only difference in signaling is for the escape component values. For each escape sample, the number of signaled escape component values may be different depending on the number of components associated with the sample.

1.6 transform coefficient codec in Skip mode

Several modifications have been proposed to the coefficient codec in the Transform Skip (TS) mode in order to adjust the residual codec for the statistics and signal characteristics of the transform skip level.

The proposed modifications are listed below.

No last significant scan position:since the residual signal reflects the spatial residual after prediction and no energy compaction is performed by the transform for TS, no higher probability of trailing zero or insignificant levels at the lower right corner of the transform block is given anymore. The last valid scanning position signaling is then omitted in this case.

Sub-block CBF:omitting the last valid scanning position signaling requires modifying the sub-block CBF signaling with coded sub-block flag for TS as follows.

The aforementioned invalid sequences may still occur locally inside the transform block due to quantization. Then, the last valid scanning position is eliminated as described above and the coded sub_block_flag is coded for all sub-blocks, excluding the case where all CG (except the last CG) have zero coefficients, so that the coded sub_block_flag does not need to be coded for the last CG.

Special cases are given for coded sub block flag of the sub block covering the DC frequency position (upper left sub block). In VVC draft 3, the coded_sub_block_flag for that sub-block is never signaled and is always inferred to be equal to 1. When the last active scanning position is located in another sub-block, this means that there is at least one active level outside the DC sub-block. Thus, a DC sub-block may contain only zero/non-active levels, although the coded sub-block flag for that sub-block is inferred to be equal to 1. In case that there is no last scanning position information in the TS, the coded sub block flag for each sub block is signaled.

This also includes coded_sub_block_flag for the DC sub-block except when all other coded_sub_block_flag syntax elements have been equal to 0. In this case, it is inferred that DC coded_sub_block_flag is equal to 1 (inperdcsbcbf=1). Since there is at least one active level in this DC sub-block, the sig_coeff_flag syntax element for the first position at (0, 0) is not signaled and is derived equal to 1 (inpersbdcssigcoeff flag=1), instead of if all other sig_coeff_flag syntax elements in this DC sub-block are equal to 0.

The context modeling for coded sub block flag is changed. The context model index is calculated as the sum of the coded sub block flag to the left of the current sub block and the coded sub block flag above, and the logical separation of the two.

sig_coeff_flag context modeling:the local template in the sig_coeff_flag context modeling is modified to include only the neighbor (NB 0) to the left of the current scan position and the neighbor (NB 1) above. The context model offset is the important neighbor sig_coeff_flag [ NB0 ]]+sig_coeff_flag[NB1]Is a number of (3). Thus, the selection of a different set of contexts depending on the diagonal d within the current transform block is eliminated. Three context models and a single set of context models are thus obtained for encoding and decoding sig_coeff_flag.

abs_level_gt1_flag and par_level_flag context modeling: a single context model is employed for abs_level_gt1_flag and par_level_flag.

abs_remain codec: although the empirical distribution of the absolute level of the transform skip residual typically still matches the laplace or geometric distribution, there is a greater steady state of unevenness than the absolute level of the transform coefficients. In particular, the variation within the window of successive implementations is higher for the absolute level of the residual. This facilitates the following modifications of abs_domain syntax binarization and context modeling:

a higher cut-off value is used in binarization, i.e. from encoding and decoding with sig_coeff_flag, abs_level_gt1_flag, par_level_flag and abs_level_gt3_flag to the conversion point of the Rice code for abs_remain, and a higher compression efficiency is obtained for the dedicated context model for each binary bit position. Increasing the cut-off value will get more "greater than X" flags, e.g. introducing abs_level_gt5_flag, abs_level_gt7_flag, etc., until the cut-off value is reached. The cut-off value itself is fixed to 5 (numgtflag=5).

The templates for the price parameter derivation are modified, i.e. only the neighbors to the left and above of the current scan position are considered, similar to the local templates modeled for the sig_coeff_flag context.

coeff_sign_flag context modeling:due to the unsteadiness inside the symbol sequence and the fact that the prediction residuals are often biased, the symbol can be encoded and decoded using a context model even when the global empirical distribution is almost uniform. The symbols are encoded using a single dedicated context model and parsed after sig_coeff_flag to hold all context codec bits together.

1.7 quantization residual Block differential pulse code modulation (QR-BDPCM)

Quantization residual block differential pulse code modulation (QR-BDPCM) is proposed to efficiently codec screen content.

The prediction directions used in QR-BDPCM may be a vertical prediction mode and a horizontal prediction mode.Intra prediction is performed on the entire block by performing sample copying in a prediction direction (horizontal or vertical prediction) similar to intra prediction. The residual is quantized and the difference between the quantized residual and its predictor (horizontal or vertical) quantized value is encoded and decoded. This can be described by: let r_ (i, j), 0.ltoreq.i.ltoreq.M-1, 0.ltoreq.j.ltoreq.N-1 for a block of size M (row). Times.N (column) be prediction residuals after intra prediction is performed using unfiltered samples from upper or left block boundary samples either horizontally (copy left neighboring pixel values line by line within the prediction block) or vertically (copy top neighboring lines to each line in the prediction block). Let Q (r_ (i, j)) 0.ltoreq.i.ltoreq.M-1, 0.ltoreq.j.ltoreq.N-1 denote quantized versions of the residual r_ (i, j), where the residual is the difference between the original block and the predicted block value. Then, the block DPCM is applied to quantized residual samples to obtain a modified MxN array R of elements Signaling vertical BDPCM:

for horizontal prediction, a similar rule applies, and residual quantization samples are obtained by the following equation:

quantized residual samplesTo the decoder.

On the decoder side, the above calculations are reversed to produce Q (r _i,j ) I is more than or equal to 0 and less than or equal to M-1, j is more than or equal to 0 and less than or equal to N-1. In the case of a vertical prediction,

in the case of a horizontal situation,

inverse-quantizing residual, Q ^-1 (Q(r _i,j ) To the intra block predictor to generate reconstructed sample values.

The main benefit of this approach is that it works simultaneously during coefficient parsing, simply by adding predictors when parsing the coefficients, or it can be done after parsing.

Draft text changes for QR-BDPCM are shown below.

7.3.6.5 codec unit syntax

/>

Bdpcm_flag [ x0] [ y0] equal to 1 specifies that bdpcm_dir_flag is present in a codec unit including a luma codec block at a position (x 0, y 0)

Bddcm dir flag x0 y0 equal to 0 specifies that the prediction direction to be used in the bddcm block is horizontal, otherwise vertical.

1.8 dividing structure

In HEVC, various local characteristics are accommodated by dividing CTUs into CUs using a quadtree structure (denoted as coding tree). At the leaf CU level it is decided whether to use inter-picture (temporal) prediction or intra-picture (spatial) prediction for coding the picture region. Each leaf CU may be further divided into one, two, or four PUs according to the division type of the PU. In one PU, the same prediction process is applied and the relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU partition type, the leaf CU may be partitioned into Transform Units (TUs) according to another quadtree structure similar to the codec tree of the CU. An important feature of the HEVC structure is that it has multiple partitioning concepts, including CUs, PUs, and TUs.

In VVC, a quadtree with a nested multi-type tree using binary and ternary partition segmentation structures replaces the concept of multiple partition unit types, i.e. it eliminates the separation of CU, PU and TU concepts unless the size is too large for maximum transform length, which is required by the CU and supports greater flexibility of the CU partition shape. In the coding tree structure, a CU may be square or rectangular. A Codec Tree Unit (CTU) is first partitioned by a quadtree (also known as quadtree) structure. The quadtree leaf nodes may then be further partitioned by a multi-type tree structure. As shown in fig. 6, there are four division types in the multi-type tree structure, vertical binary division (split_bt_ver), horizontal binary division (split_bt_hor), vertical trigeminal division (split_tt_ver), and horizontal trigeminal division (split_tt_hor). The multi-type leaf nodes are called Codec Units (CUs), and unless the CU is too large for the maximum transform length, such segmentation is used for prediction and transform processing without any further segmentation. This means that in most cases, in quadtrees with nested multi-type tree codec block structures, the CU, PU and TU have the same block size. An anomaly occurs when the maximum supported transform length is less than the width or height of the color component of the CU. In addition, the luminance and chrominance components have separate partition structures on the I-slice.

1.9 Cross-component Linear model prediction

To reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VTM4, for which chroma samples are predicted based on luma samples of the same CU reconstructed using a linear model as follows:

pred _C (i，j)＝α·rec _L ' (i, j) +beta [ equation 8 ]]

Where the predicted chroma samples in a CU are represented, and the downsampled reconstructed luma samples in the same CU are represented. The linear model parameters α and β are derived from the relationship between luminance and chrominance values from two samples, a luminance sample having a minimum sample value and a maximum sample value within a downsampled set of adjacent luminance samples, and its corresponding chrominance sample. The linear model parameters α and β are obtained according to the following equation.

β＝Y _b -α·X _b [ equation 10 ]]

Wherein Y is _a And X _a The luminance value and the chrominance value representing the luminance sample having the maximum luminance sample value. And X is _b And Y _b Respectively representing the luminance value and the chrominance value of the luminance sample having the smallest luminance sample value. Fig. 8 shows examples of left and upper samples and the positions of the samples of the current block involved in the CCLM mode.

2.10 Luminance Mapping (LMCS) with chroma scaling

In VTM4, a codec tool called luma map with chroma scaling (LMCS) is added as a new processing block before the loop filter. LMCS has two main components: 1) A loop map of luminance components based on an adaptive piece-by-piece linear model; 2) For the chrominance components, a luminance dependent chrominance residual scaling is applied. Figure 8 shows the LMCS architecture from the decoder perspective. The shaded blocks in fig. 8 indicate where in the mapping domain the process is applied; these include inverse quantization, inverse transformation, intra-luminance prediction, and adding luminance prediction along with luminance residual. The unshaded blocks in fig. 8 indicate where the process is applied in the original (i.e., unmapped) domain; these include loop filters such as deblocking, ALF and SAO, motion compensated prediction, chroma intra prediction, adding chroma prediction along with chroma residuals, and storing decoded pictures as reference pictures. The light yellow shaded blocks in fig. 8 are new LMCS functional blocks, including positive and inverse mapping of luminance signals and luminance dependent chroma scaling procedures. Like most other tools in VVCs, LMCS may be enabled/disabled at the sequence level using SPS flags.

2.11 intra mode codec with 67 intra prediction modes

To capture any edge direction present in natural video, the number of directional intra modes in VTM4 is extended from 33 (as used in HEVC) to 65. The new orientation mode that is not in HEVC is depicted in fig. 9 as a red dashed arrow, with the plane and DC modes remaining the same. These denser directional intra prediction modes are applicable to all block sizes as well as both luma and chroma intra predictions.

Regardless of whether MRL and ISP codec tools are applicable, a unified 6-MPM list is presented for intra blocks. Constructing MPM list based on left intra mode

/>

/>

/>

/>

Syntax elements intra_luma_mpm_flag [ x0] [ y0], intra_luma_not_player_flag [ x0] [ y0], intra_luma_mpm_idx [ x0] [ y0], and intra_luma_mpm_remain [ x0] [ y0] specify an intra prediction mode for luma samples. The array index x0, y0 specifies the position (x 0, y 0) of the top-left luma sample of the considered codec block relative to the top-left luma sample of the picture. When intra_luma_mpm_flag [ x0] [ y0] is equal to 1, intra prediction modes are inferred from neighboring intra prediction codec units according to clause 8.4.2.

When there is no intra_luma_mpm_flag [ x0] [ y0] (e.g., ISP enabled, or MRL enabled (reference index > 0)), it is inferred to be equal to 1.

In the absence of intra_luma_not_player_flag [ x0] [ y0] (e.g., MRL enabled), it is inferred to be equal to 1.

Derivation of 8.4.2 luma intra prediction modes

Inputs to this process are:

a luminance location (xCb, yCb) specifying a top left sample of the current luma codec block relative to a top left sample of the current picture,

a variable cbWidth, specifying the width of the current codec block in the luma samples,

the variable cbHeight specifies the height of the current codec block in the luma samples.

In this process, the luminance intra prediction mode IntraPredModeY [ xCb ] [ yCb ] is derived.

Table 8-1 specifies the value and associated name of the intra prediction mode IntraPredModeY [ xCb ] [ yCb ].

TABLE 8-1-specification of intra prediction modes and associated names

/>

NOTE-INTRA prediction modes intra_lt_cclm, intra_l_cclm, and intra_t_cclm apply only to chroma components.

IntraPredModeY [ xCb ] [ yCb ] is derived as follows:

-if intra_luma_not_player_flag [ xCb ] [ yCb ] is equal to 1, the following ordered steps:

1. the adjacent positions (xNbA, yNbA) and (xNbB, yNbB) are set equal to (xCb-1, yCb+cbHeight-1) and (xCb +cbWidth-1, yCb-1), respectively.

2. For the case where X is replaced by A or B, the variable candIntraPredModeX is derived as follows:

-availability derivation procedure for blocks specified in clause 6.4.X [ ed. (BB): the adjacent block availability check procedure tbd is invoked, wherein a position (xCurr, yCurr) equal to (xCb, yCb) and an adjacent position (xNbY, yNbY) equal to (xNbX, yNbX) are set as inputs, and an output is assigned to availableX.

-deriving a candidate intra prediction mode candIntraPredModeX as follows:

-candIntraPredModeX is set equal to INTRA_PLANAR if one or more of the following conditions are true.

The variable availableX is equal to false.

CuPredMode [ xNbX ] [ yNbX ] is not equal to MODE_INTRA, ciip_flag [ xNbX ] [ yNbX ] is not equal to 1.

Pcm_flag [ xNbX ] [ yNbX ] is equal to 1.

X is equal to B and yCb-1 is less than ((yCb > > CtbLog2 SizeY) < < CtbLog2 SizeY).

Otherwise, candIntraPredModeX is set equal to IntraPredModeY [ xNbX ] [ yNbX ].

3. candModeList [ x ], where x=0.4, is derived as follows.

-deriving candModeList [ x ], where x=0..4, if candIntraPredModeB is equal to candIntraPredModeA and candIntraPredModeA is greater than intra_dc.

candModeList [0] = candIntraPredModeA [ equation 11]

candModeList [1] =2+ ((candIntraPredModeA+61)% 64) [ equation 12]

candModeList [2] =2+ ((candIntraPredModeA-1)% 64) [ equation 13]

candModeList [3] = INTRA_DC [ equation 14]

candModeList [4] =2+ ((candIntraPredModeA+60)% 64) [ equation 15]

Otherwise, if candIntraPredModeB is not equal to candIntraPredModeA, candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC, the following applies:

the variables minAB and maxAB are derived as follows:

minab=min (candIntraPredModeA, candIntraPredModeB) [ equation 16]

maxab=max (candIntraPredModeA, candIntraPredModeB) [ equation 17]

-deriving candModeList [ x ], where x=0..4, if candIntraPredModeA and candIntraPredModeB are both greater than intra_dc.

candModeList [0] = candIntraPredModeA [ equation 18]

candModeList [1] = candIntraPredModeB [ equation 19]

candModeList [2] = INTRA_DC [ equation 20]

-if maxAB-minAB is in the range of 2 to 62 (inclusive), the following applies:

candModeList [3] =2+ ((maxAB+61)% 64) [ equation 21]

candModeList [4] =2+ ((maxAB-1)% 64) [ equation 22]

Otherwise, the following applies:

candModeList [3] =2+ ((maxAB+60)% 64) [ equation 23]

candModeList [4] =2+ ((maxAB)% 64) [ equation 24]

-else (candIntraPredModeA or candIntraPredModeB is greater than intra_dc), deriving candModeList [ x ], where x=0..4:

candModeList [0] = maxAB [ equation 25]

candModeList [1] = INTRA_DC [ equation 26]

candModeList [2] =2+ ((maxAB+61)% 64) [ equation 27]

candModeList [3] =2+ ((maxAB-1)% 64) [ equation 28]

candModeList [4] =2+ ((maxAB+60)% 64) [ equation 29]

Otherwise, the following applies:

candModeList [0] = INTRA_DC [ equation 30]

candModeList [1] = INTRA_ANGALAR 50 [ equation 31]

candModeList [2] = INTRA_ANGALAR 18 [ equation 32]

candModeList [3] = INTRA_ANGALAR 46 [ equation 33]

candModeList [4] = INTRA_ANGALAR 54 [ equation 34]

4. IntraPredModeY [ xCb ] [ yCb ] is derived by applying the following procedure:

-if intra_luma_mpm_flag [ xCb ] [ yCb ] is equal to 1, intrapredmodey [ xcb ] [ yCb ] is set equal to candModeList [ intra_luma_mpm_idx [ xCb ] [ yCb ].

Otherwise, deducing IntraPredModeY [ xCb ] [ yCb ] by applying the following sequential steps:

1) Where candModeList [ i ] is greater than candModeList [ j ], i=0..3, and for each i, j= (i+1)..4, then the two values are exchanged as follows:

(candModeList [ i ], candModeList [ j ])=Swap (candModeList [ i ], candModeList [ j ]) [ equation 35]

2) IntraPredModeY [ xCb ] [ yCb ] is derived by the following sequential steps:

IntraPredModeY [ xCb ] [ yCb ] is set equal to intra_luma_mpm_remain [ xCb ] [ yCb ].

The value of IntraPredModeY [ xCb ] [ yCb ] is incremented by 1

For i equal to 0 to 4 (inclusive), the value of IntraPredModeY [ xCb ] [ yCb ] is incremented by 1 when IntraPredModeY [ xCb ] [ yCb ] is greater than or equal to candModeList [ i ].

Otherwise (intra_luma_not_player_flag [ xCb ] [ yCb ] equals 0), intraPredModeY [ xCb ] [ yCb ] is set equal to intra_planet.

The variable IntraPredModeY [ x ] [ y ] is set equal to IntraPredModeY [ xCb ] [ yCb ], where x=xcb.. xCb +cbwidth-1 and y=ycb.. yCb +cbheight-1.

1.12 Multi-reference line (MRL) intra prediction

Multiple Reference Line (MRL) intra prediction uses more reference lines for intra prediction. In fig. 11, an example of 4 reference lines is depicted, in which the samples of segments a and F are not taken from the reconstructed neighboring samples, but are padded with the nearest samples from segments B and E, respectively. HEVC intra prediction uses the nearest reference line (i.e., reference line 0). In the MRL, additional lines (reference line 1 and reference line 3) are used.

The index (mrl _idx) of the selected reference line is signaled and used to generate an intra predictor. For reference lines idx greater than 0, only additional reference line patterns are included in the MPM list, and only MPM indexes are signaled without remaining patterns. The reference line index is signaled before the intra prediction mode, and in case of signaling the non-zero reference line index, the plane and DC modes are excluded from the intra prediction mode.

For the first line of blocks inside the CTU, the MRL is disabled to prevent the use of extended reference samples outside the current CTU line. Also, the PDPC is disabled when the extra line is used.

1.13 Intra-frame subdivision (ISP)

An intra sub-segmentation (ISP) tool vertically or horizontally divides a luma intra prediction block into 2 or 4 sub-segmentations according to a block size. For example, the minimum block size of an ISP is 4x8 (or 8x 4). If the block size is greater than 4x8 (or 8x 4), the corresponding block is divided by 4 sub-divisions. Fig. 12 shows two examples of possibilities. All sub-partitions satisfy the condition of having at least 16 samples.

TABLE 2-1-entropy CODEC coefficient set size

Block size	Coefficient group size
		1×N,N≥16	1×16
N×1,N≥16	16×1
		2×N,N≥8	2×8
N×2,N≥8	8×2
		All other possible mxn cases	4×4

For each sub-partition, reconstructed samples are obtained by adding residual signals to the prediction signal. Here, the residual signal is generated through processes such as entropy decoding, inverse quantization, and inverse transformation. Thus, the reconstructed sample value of each sub-partition may be used to generate a prediction of the next sub-partition, and each sub-partition is iteratively processed. In addition, the first subdivision to be processed is the subdivision containing the left-hand sample of the CU, and then continues downwards (horizontal division) or to the right (vertical division). As a result, the reference points for generating the sub-divided prediction signals are located only on the left and upper sides of the line. All sub-partitions share the same intra mode. The following is a summary of the ISP interactions with other codec tools.

Multiple Reference Line (MRL): if the block has an MRL index other than 0, then the ISP codec mode will be inferred to be 0, so ISP mode information will not be sent to the decoder.

Entropy coding coefficient group size: the size of the modified entropy encoded sub-blocks has been modified such that they have 16 samples in all possible cases, as shown in table 2-1. Note that the new size only affects the blocks produced by the ISP, with one of the scales being smaller than 4 samples. In all other cases, the coefficient set remains on a 4 x 4 scale.

CBF codec: at least one of the sub-partitions is assumed to have a non-zero CBF. Thus, if n is the number of subdivisions and the first subdivision has produced zero CBF, then the CBF of the nth subdivision is inferred to be 1.

MPM use: in the ISP mode codec block, the MPM flag will be inferred to be 1, the MPM list modified to exclude DC mode and prioritize horizontal intra mode for ISP level, and vertical intra mode for vertical division.

Transforming the size constraint: all ISP transforms with a length of more than 16 points use DCT-II.

PDPC: when the CU uses ISP codec mode, the PDPC filter will not be applied to the resulting subdivision.

MTS flag: if the CU uses ISP codec mode, the MTS CU flag will be set to 0 and will not be sent to the decoder. Thus, the decoder will not perform RD testing for each resulting subdivision, for different available transforms. The transform selection of ISP mode will instead be fixed and selected according to the intra mode utilized, the order of processing and the block size. Thus, no signaling is required. For example, let t _H And t _V For horizontal and vertical transforms selected for w×h sub-partitions, respectively, where w is width and h is height. Then, the transformation is selected according to the following rules:

if w=1 or h=1, there is no horizontal or vertical transformation, respectively.

If w=2 or w >32, =DCT-II

If h=2 or h >32, =dct-II

Otherwise, the transformation is selected as in Table 2-2.

Table 2-2-transform selection depends on intra mode

Disadvantages of the prior embodiments

The MPM list is designed for natural sequences. On screen content codec, the vertical mode and the horizontal mode are two main modes, and thus different MPM list construction may be required in screen content codec. Also, on mixed content, more than one MPM list may be required, which may be able to switch based on whether the current block is a screen content block.

Exemplary method for maximum possible Pattern List construction for Screen content codec

The following should be considered as an example explaining the general concept. These examples are provided only to facilitate an understanding of the disclosed technology and should not be construed in a narrow manner. Furthermore, the embodiments may be combined in any manner.

1. Two or more MPM list construction methods may be used for conventional intra-mode codec.

A. In one example, the first MPM list may be a current list and the second MPM list may be derived from the first list.

I. In one example, to generate the second MPM list, the first MPM list may be generated first, after which reordering may be applied to those patterns in the first MPM list.

a) In one example, the plane and/or DC mode may be placed at different locations in the list.

in one example, to generate the second MPM list, the first MPM list may be generated first, after which one or more modes in the list may be replaced.

a) In one example, the planar and/or DC modes may be replaced with angular modes (e.g., horizontal or vertical).

b) In one example, some modes other than the vertical mode and/or the horizontal mode may be replaced by the vertical mode and/or the horizontal mode.

b. In one example, the second MPM list may adhere to the logic of the first MPM list, but replace some non-angular modes with angular modes.

c. In one example, whether more than one MPM list is allowed may be signaled in the SPS/VPS/PPS/picture header/slice group header/maximum codec unit (LCU)/Codec Unit (CU)/LCU row/LCU level group.

d. In one example, the number of allowed MPM lists may be predefined or signaled to the decoder.

i. In one example, the number of MPM lists is a fixed value, e.g., 2.

in one example, the number of MPM lists may be signaled to the decoder in SPS/VPS/PPS/picture header/slice group header/Largest Codec Unit (LCU)/Codec Unit (CU)/LCU row/LCU level groups.

e. In one example, one MPM list is used for natural content codec and the other is used for screen content codec.

f. In one example, which MPM list to use may be based on

i. Video content (e.g., screen content or natural content)

Signaling messages in SPS/VPS/PPS/picture header/slice header/group header/Largest Codec Unit (LCU)/Codec Unit (CU)/LCU row/LCU level group or other video units.

a) In one example, a flag may be signaled in the slice header to indicate a list to be selected for encoding and decoding the conventional intra prediction mode.

b) In one example, the MPM list may depend on a message informing of video content information, such as screen content or camera capture content.

Prediction Unit (PU) location

Block scale of current block and/or its neighboring blocks

v. block shape of current block and/or neighboring blocks thereof

Prediction modes (intra/inter) of neighboring blocks of the current block

Intra prediction mode of neighboring block of current block

Motion vector of neighboring block of current block

Indication of QR-BDPCM mode for neighboring blocks of current block

x. indication of QR-BDPCM mode of transform type of current block (e.g., indicated by tu_mts_idx)

Current quantization parameters for a current block and/or its neighboring blocks

Indication of color format (e.g., 4:2:0, 4:4:4)

xiii codec tree structure

Slice/slice group type and/or picture type

2. The MPM list may be switched at the slice/picture/CU/LCU group/block level. Whether to switch MPM list may be based on

a. Video content (e.g., screen content or natural content)

b. Signaling messages in SPS/VPS/PPS/Picture header/slice group header/maximum codec unit (LCU)/Codec Unit (CU)/LCU line/LCU level groups

c. Position of Prediction Unit (PU)

d. Block scale of current block and/or its neighboring blocks

e. Block shape of current block and/or its neighboring block

f. Prediction modes (intra/inter) of neighboring blocks of a current block

g. Intra prediction mode of neighboring blocks of current block

h. Motion vector of neighboring block of current block

i. Indication of QR-BDPCM mode for neighboring blocks of current block

j. Indication of QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu_mts_idx)

k. Current quantization parameters for a current block and/or its neighboring blocks

Indication of color format (e.g. 4:2:0, 4:4:4)

m. coding and decoding tree structure

n, slice/group type and/or picture type

O. color component (e.g. luminance or chrominance)

3. In construction of the MPM list, a specific angle mode (e.g., a vertical mode and/or a horizontal mode) may always be in the MPM list.

a. In one example, the particular angle pattern may be inserted at a given location, e.g., the first, in the MPM list.

b. In one example, the vertical mode and the horizontal mode may always be sequentially at the first two positions in the MPM list.

c. In one example, the horizontal mode and the vertical mode may always be sequentially at the first two positions in the MPM list.

d. In one example, the MPM list may always be { INTRA_ANGULAR50, INTRA_ANGULAR18, DC, plane, INTRA_ANGULAR34, INTRA_ANGULAR66}

e. In one example, the MPM list may always be { INTRA_ANGULAR18, INTRA_ANGULAR50, plane, DC, INTRA_ANGULAR34, INTRA_ANGULAR66}

f. In one example, the MPM list may always be { INTRA_ANGULAR50, INTRA_ANGULAR18, plane, DC, INTRA_ANGULAR34, INTRA_ANGULAR66}

g. In one example, the MPM list may always be { plane, DC, intra_anguar 18, intra_anguar 50, intra_anguar 34, intra_anguar 66}.

h. In one example, if neither the left-side neighbor INTRA mode nor the upper neighbor INTRA mode is available, the MPM list may be { INTRA_ANGULAR50, INTRA_ANGULAR18, DC, planar, INTRA_ANGULAR34, INTRA_ANGULAR66}

i. In one example, if neither the left-side neighbor INTRA mode nor the upper neighbor INTRA mode is available, the MPM list may be { INTRA_ANGULAR50, INTRA_ANGULAR18, DC, planar, INTRA_ANGULAR34, INTRA_ANGULAR2}

j. In one example, if the left-side neighbor INTRA mode and the top-neighbor INTRA mode are the same, and this mode is a non-ANGULAR INTRA mode, the MPM list may be { DC, planar, intra_anguar 50, intra_anguar 18, intra_anguar 34, intra_anguar 66}

k. In one example, if the left neighboring intra mode and the upper neighboring intra mode are the same, and such mode a is an angular intra mode, an MPM list may be constructed based on the direction of a.

i. In one example, if a > =2 and a < =18, the MPM list may always be { intra_anguar 18, intra_anguar 2, intra_anguar 50, DC, plane, intra_anguar 66}

in one example, if a > =18 and a < =34, the MPM list may always be { intra_number 18, intra_number 34, intra_number 50, DC, plane, intra_number 2}

in one example, if a > = 34 and a < = 50, the MPM list may always be { intra_number 50, intra_number 34, intra_number 18, DC, plane, intra_number 66}

in one example, if a > =50 and a < =66, the MPM list may always be { intra_number 50, intra_number 66, intra_number 18, DC, plane, intra_number 34}

In one example, the left-side neighboring intra mode and the upper-neighboring intra mode are different modes, one of which is a non-angular mode.

i. In one example, the MPM list may additionally be { DC, planar, INTRA_ANGULAR50, INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR66}

In one example, the left-side neighboring intra mode and the upper neighboring intra mode are different modes, both of which are angular modes.

I. In one example, the left Max represents the maximum between these two intra modes.

in one example, if Max > =2 and Max < =18, the MPM list may be { intra_anguar 18, intra_anguar 2, intra_anguar 50, DC, plane, intra_anguar 66}

in one example, if Max > =18 and Max < =34, the MPM list may be { intra_number 18, intra_number 34, intra_number 50, DC, plane, intra_number 2}

in one example, if Max > =34 and Max < =50, the MPM list may be { intra_number 50, intra_number 34, intra_number 18, DC, plane, intra_number 66}

In one example, if Max > =50 and Max < =66, the MPM list may be { intra_number 50, intra_number 66, intra_number 18, DC, plane, intra_number 34}

N. in one example, at the top of the current design on MPM list construction, vertical and/or horizontal modes may replace one existing mode in the MPM list.

i. In one example, in the MPM list, the mode having the direction closest to the vertical mode may be replaced with the vertical mode.

in one example, in the MPM list, the mode having the direction closest to the horizontal mode may be replaced with the horizontal mode.

In one example, the location of the vertical and/or horizontal modes in the MPM list may be based on

i. Signaling messages in SPS/VPS/PPS/Picture header/slice group header/maximum codec unit (LCU)/Codec Unit (CU)/LCU line/LCU level groups

Prediction Unit (PU) location.

Block scale of current block and/or its neighboring blocks

Block shape of current block and/or neighboring blocks

Prediction modes (intra/inter) of neighboring blocks of the current block

Intra prediction modes for neighboring blocks of a current block

Motion vector of neighboring block of current block

Indication of QR-BDPCM mode for neighboring blocks of current block

indication of QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu_mts_idx)

x current quantization parameters of current block and/or its neighboring blocks

Indication of color format (e.g., 4:2:0, 4:4:4)

xii coding tree structure

Strip/slice group type and/or picture type

The intra mode of the rdpcm may be set to a mode other than the first mode in the MPM list. In this case, a prediction signal may be generated from one intra mode, and the stored intra modes may correspond to each other. The stored intra modes may be used to predict intra modes of successive codec blocks. In some embodiments, the stored intra mode may be different from the intra mode used in the intra prediction process of the current block.

a. In one example, the intra mode of the RDPCM codec block is set to the second MPM candidate (e.g., corresponding to intra_luma_mpm_idx=0)

b. In one example, the intra mode of the RDPCM codec block is set to any MPM candidate on the MPM list other than the first (e.g., corresponding to intra_luma_mpm_idx=0)

C. In one example, whether and/or how the above methods are applied may be based on

i. Signaling messages in SPS/VPS/PPS/Picture header/slice group header/maximum codec unit (LCU)/Codec Unit (CU)/LCU line/LCU level groups

Prediction Unit (PU) location.

Block scale of current block and/or its neighboring blocks

Block shape of current block and/or neighboring blocks

Prediction modes (intra/inter) of neighboring blocks of the current block

Intra prediction modes for neighboring blocks of a current block

Motion vector of neighboring block of current block

Indication of QR-BDPCM mode for neighboring blocks of current block

indication of QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu_mts_idx)

x current quantization parameters of current block and/or its neighboring blocks

Indication of color format (e.g., 4:2:0, 4:4:4)

xii coding tree structure

Strip/slice group type and/or picture type

5. The location of the flat pattern may not always be the first in the MPM list.

a. In one example, the flat pattern may be placed in the last position in the MPM list.

b. In one example, the flat pattern may be placed anywhere in the MPM list except for the first one.

c. In one example, the flat mode may not be allowed to be placed in the first position in the MPM list.

d. In one example, whether and/or how the above methods are applied may be based on

i. The position of the Prediction Unit (PU) is signaled in the SPS/VPS/PPS/picture header/slice group header/maximum codec unit (LCU)/Codec Unit (CU)/LCU row/LCU level group.

Block scale of current block and/or its neighboring blocks

Block shape of current block and/or neighboring blocks

Prediction modes (intra/inter) of neighboring blocks of the current block

Intra prediction modes for neighboring blocks of a current block

Motion vector of neighboring block of current block

Indication of QR-BDPCM mode for neighboring blocks of current block

indication of QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu_mts_idx)

x current quantization parameters of current block and/or its neighboring blocks

Indication of color format (e.g., 4:2:0, 4:4:4)

xii coding tree structure

Strip/slice group type and/or picture type

Binarization of mpm index (e.g., intra_luma_mpm_idx) may not use truncated unary binarization.

a. In one example, the codeword length of the MPM index (e.g., intra_luma_mpm_idx) may be fixed.

Signaling of mpm index (e.g., intra_luma_mpm_idx) may use Exp-Golomb binarization of order K.

i. In one example, K is an integer (e.g., 2) and may be based on

a) The location of the Prediction Unit (PU).

b) Block scale of current block and/or its neighboring blocks

c) Block shape of current block and/or its neighboring block

d) Prediction modes (intra/inter) of neighboring blocks of a current block

e) Intra prediction mode of neighboring blocks of current block

f) Motion vector of neighboring block of current block

g) Indication of QR-BDPCM mode for neighboring blocks of current block

h) Indication of QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu_mts_idx)

i) Current quantization parameters for a current block and/or its neighboring blocks

j) Indication of color format (e.g., 4:2:0, 4:4:4)

k) Coding and decoding tree structure

l) slice/slice group type and/or picture type

7. For the indication of MPM candidates in the codec list, the first flag may be encoded followed by the remaining index. The first flag is used to indicate whether the selected intra prediction mode is the same as or different from the first MPM candidate in the list.

The semantics of intra_luma_not_player_flag may be interpreted as whether the intra prediction mode is not the first MPM candidate in the MPM list, but is not interpreted as whether it is not a plane mode.

b. Alternatively, the first flag may be conditionally encoded and decoded.

i. In one example, if the block represents screen content, the first flag may not be encoded.

c. Alternatively, in addition, if the selected mode is not the first MPM candidate in the list, signaling of the remaining MPM index (e.g., intra_luma_mpm_idx) may be further signaled.

8. The vertical and horizontal intra-prediction in the proposed MPM list construction process may correspond to directly copying the reference samples in the intra-prediction process.

a. In one example, vertical intra prediction may be generated by directly copying reference samples in the same column.

b. In one example, horizontal intra prediction may be generated by directly copying reference samples in the same row.

c. In one example, the indication of whether to apply the above predictions may be based on

i. Video content (e.g., screen content or natural content)

Signalling messages in SPS/VPS/PPS/Picture header/stripe header/group of slice header/maximum codec unit (LCU)/Codec Unit (CU)/LCU line/LCU level group

Predicting the location of a unit (PU).

Block scale of current block and/or its neighboring blocks

v. block shape of current block and/or neighboring blocks thereof

Prediction modes (intra/inter) of neighboring blocks of the current block

Intra prediction mode of neighboring block of current block

Motion vector of neighboring block of current block

indication of QR-BDPCM mode for neighboring blocks of current block

x. indication of QR-BDPCM mode of transform type of current block (e.g., indicated by tu_mts_idx)

Current quantization parameters for a current block and/or its neighboring blocks

Indication of color format (e.g., 4:2:0, 4:4:4)

xiii independent/double codec tree structure

Slice/slice group type and/or picture type

xv. color component (e.g. may be applied only to chrominance components or luminance components)

9. The above method can also be applied to other intra prediction methods, such as affine intra prediction methods (also known as matrix-based intra prediction methods), intra sub-division prediction (ISP), multiple Reference Lines (MRL), and the like.

Fig. 13a and 13b show a flow chart of an exemplary method of video processing. Referring to fig. 13, the method 1300 includes, at step 1302, constructing a first mode list of intra-coding modes based on intra-modes of neighboring blocks of the current video block. The method 1300 further includes, at step 1304, building a second mode list of intra-coding modes based on the first mode list. The method 1300 also includes, at step 1306, performing intra-mode codec for the current video block using at least one of the first mode list or the second mode list.

In some embodiments, wherein the first mode list and the second mode list are MPM (maximum likelihood mode) lists. In some implementations, building the second pattern list includes reordering patterns of the first pattern list. In some implementations, building the second list of modes includes replacing one or more modes of the first list of modes. In some embodiments, the second mode list has the same logic as the first mode list. In some embodiments, the method further comprises: an indication of whether both the first mode list and the second mode list are allowed is included in the bitstream representation of the current video block. In some implementations, whether to use the first mode list or the second mode list is determined based on messages in SPS, VPS, PPS, picture header, slice group header, largest Codec Unit (LCU), coding Unit (CU), LCU row, LCU group, or other video unit. In some implementations, performing intra-mode codec includes switching from the first mode list to the second mode list or switching from the second mode list to the first mode list. In some implementations, the switching occurs at the stripe, slice, picture, CU, LCU, LCU group or block level. In some embodiments, the switching is based on at least one of: a. video content, b. a signaled message in SPS, VPS, PPS, picture header, slice group header, largest Codec Unit (LCU), codec Unit (CU), LCU row, LCU group, c. a position of Prediction Unit (PU), d. a block size of current video block and/or neighboring block, e. a block shape of current video block and/or neighboring block, f. a prediction mode of neighboring block of current video block, g. an intra prediction mode of neighboring block of current video block, h. a motion vector of neighboring block of current video block, i. an indication of QR-BDPCM mode of neighboring block of current video block, j. an indication of QR-BDPCM mode of transform type of current video block, k. a current quantization parameter of current video block and/or neighboring block, l. an indication of color format, m. a codec tree structure, n. slice group header and/or picture type, or o. color component. In some implementations, each of the first and second pattern lists includes a pattern index, and binarization of the pattern index is performed without truncated unary binarization. In some embodiments, at least one of the first mode list and the second mode list is applied to perform an intra prediction method, an affine intra prediction method, an intra sub-division prediction (ISP), or a multi-reference line (MRL).

Referring to fig. 13b, the method 1320 includes constructing a mode list of intra-coding modes based on intra modes of neighboring blocks of the current video block to include the candidate mode, wherein constructing the mode list includes locating the candidate mode at a position in the mode list based on a type of the candidate mode. In some implementations, locating the candidate pattern includes locating a particular angle pattern at a given location in the pattern list. In some implementations, locating the candidate patterns includes locating the vertical pattern and the horizontal pattern, or the horizontal pattern and the vertical pattern, in the first two positions in the pattern list in order. In some implementations, locating the candidate mode includes locating an intra-mode of RDPCM (residual differential pulse code modulation) to a location in the mode list other than the first location. In some embodiments, the prediction signal is generated from an intra mode. In some embodiments, the intra mode of the subsequent codec block is predicted using another intra mode. In some implementations, locating the candidate pattern includes locating the planar pattern to a location in the pattern list other than the first location. In some implementations, the pattern list includes a pattern index, and binarization of the pattern index is performed without truncated unary binarization. In some embodiments, the method further comprises indicating whether the selected intra-prediction mode is the same as or different from the first candidate mode in the mode list. In some embodiments, the method further comprises generating intra-prediction samples based on the mode list by copying reference samples in the same column or row. In some implementations, the mode list is applied to perform an intra prediction method, an affine intra prediction method, an intra sub-division prediction (ISP), or a multi-reference line (MRL).

Fig. 14 is a block diagram of a video processing apparatus 1200. The device 1200 may be used to implement one or more methods described herein. The device 1200 may be included in a smartphone, tablet, computer, internet of things (IoT) receiver, or the like. The apparatus 1200 may include one or more processors 1202, one or more memories 1204, and video processing hardware 1206. The processor 1202 may be configured to implement one or more of the methods described herein (including, but not limited to, methods 1100 and 1150). Memory(s) 1204 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 1206 may be used to implement some of the techniques described in this document in hardware circuitry.

In some embodiments, the video codec method may be implemented using an apparatus implemented on a hardware platform as described in connection with fig. 14.

Fig. 15 is a flow chart of an example method 1500 of video processing. The method 1500 includes, at 1502, building two or more maximum likelihood mode (MPM) lists of intra-coding modes for a current block of video for a transition between the current block and a bitstream representation of the current block, wherein the two or more MPM lists include at least a first MPM list built using a first building method and a second MPM list built using a second building method different from the first building method; and at 1504, performing conversion using one MPM list selected from the constructed two or more MPM lists.

In some examples, the first MPM list is a current MPM list of the current block, and the second MPM list is derived from the first MPM list.

In some examples, a first MPM list is first generated, and a second MPM list is generated by reordering those patterns in the first MPM list.

In some examples, the planar mode and/or the DC mode are placed in different locations in the first MPM list.

In some examples, a first MPM list is first generated, and a second MPM list is generated by replacing one or more patterns in the first MPM list.

In some examples, the planar mode and/or the DC mode are replaced with an angular mode.

In some examples, the angular mode includes at least one of a horizontal mode and a vertical mode.

In some examples, some modes other than the horizontal mode and the vertical mode in the first MPM list are replaced with the horizontal mode and/or the vertical mode.

In one example, the second MPM list complies with logic of the first MPM list, wherein some non-angular modes are replaced with angular modes.

In some examples, whether to allow two or more MPM lists is signaled in at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a picture header, a slice group header, a maximum codec unit (LCU), a Codec Unit (CU), LCU rows, and LCU group levels.

In some examples, the number of allowed MPM lists is predefined or signaled to the decoder.

In some examples, the number of MPM lists is a fixed value.

In some embodiments, the fixed value is 2.

In some examples, the number of MPM lists is signaled in at least one of VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU rows, and LCU group level.

In some examples, one of the first MPM list and the second MPM list is for natural content codec and the other is for screen content codec.

In some examples, which MPM list is selected depends on one or more of:

i. video content including at least one of screen content and natural content;

signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

predicting the location of the unit (PU);

block dimensions of the current block and/or its neighboring blocks;

v. block shape of current block and/or its neighboring blocks;

prediction modes including intra mode and inter mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

Motion vectors of neighboring blocks of the current block;

indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks of the current block;

x. an indication of QR-BDPCM mode of the transform type of the current block;

current quantization parameters of the current block and/or its neighboring blocks;

indication of the color format of the current block;

codec tree structure;

banding, slice group type, and/or picture type.

In some examples, a flag is signaled in the slice header to indicate a list to be selected for encoding and decoding the normal intra prediction mode.

In some examples, the MPM list is dependent on a message indicating video content information including at least one of screen content or camera capture content.

In some examples, the color format of the current block is 4:2:0 or 4:4:4.

In some examples, the MPM list may be allowed to be switched at a stripe, tile, picture, CU, LCU, LCU group or block level.

In some examples, whether to switch the MPM list depends on at least one of:

video content comprising at least one of screen content and natural content;

signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

xviii. Predicting the location of the unit (PU);

block dimensions of the current block and/or its neighboring blocks;

xx. block shapes of the current block and/or its neighboring blocks;

prediction modes including intra mode and inter mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

xxiii motion vectors of neighboring blocks of the current block;

xxiv an indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

xxv. an indication of QR-BDPCM mode of the transform type of the current block;

current quantization parameters of the current block and/or its neighboring blocks;

xxvii an indication of the color format of the current block;

xxviii codec tree structure;

xxix, slice group type and/or picture type;

xxx color components including at least one of a luma component and a chroma component;

in some examples, the particular angle mode is always in the MPM list during construction of the MPM list.

In some examples, the particular angular mode is a vertical mode and/or a horizontal mode.

In some examples, the particular angle pattern is inserted at a given location in the MPM list.

In some examples, the particular angle pattern is inserted at a first location in the MPM list.

In some examples, the vertical mode and the horizontal mode are always in the first two positions in the MPM list in order.

In some examples, the horizontal mode and the vertical mode are always in the first two positions in the MPM list in order.

In some examples, the MPM list is always { intra_angle 50, intra_angle 18, DC, plane, intra_angle 34, intra_angle 66}.

In some examples, the MPM list is always { intra_angle 18, intra_angle 50, plane, DC, intra_angle 34, intra_angle 66}.

In some examples, the MPM list is always { intra_angle 50, intra_angle 18, plane, DC, intra_angle 34, intra_angle 66}.

In some examples, the MPM list is always { plane, DC, intra_number 18, intra_number 50, intra_number 34, intra_number 66}.

In some examples, if neither the left-side neighbor INTRA mode nor the upper neighbor INTRA mode of the current block is available, the MPM list is { intra_anguar 50, intra_anguar 18, DC, plane, intra_anguar 34, intra_anguar 66}.

In some examples, if neither the left-side neighbor INTRA mode nor the upper neighbor INTRA mode of the current block is available, the MPM list is { intra_anguar 50, intra_anguar 18, DC, plane, intra_anguar 34, intra_anguar 2}.

In some examples, if the left-neighboring INTRA mode and the above-neighboring INTRA mode of the current block are the same, and this mode is a non-ANGULAR INTRA mode, the MPM list is { DC, planar, intra_anguar 50, intra_anguar 18, intra_anguar 34, intra_anguar 66}.

In some examples, if the left-neighboring intra mode and the upper-neighboring intra mode of the current block are the same, and such mode a is an angular intra mode, an MPM list is constructed based on the direction of such mode a.

In some examples, if a > =2 and a < =18, the MPM list is always { intra_number 18, intra_number 2, intra_number 50, DC, plane, intra_number 66}.

In some examples, if a > =18 and a < =34, the MPM list is always { intra_number 18, intra_number 34, intra_number 50, DC, plane, intra_number 2}.

In some examples, if a > =34 and a < =50, the MPM list is always { intra_number 50, intra_number 34, intra_number 18, DC, plane, intra_number 66}.

In some examples, if a > =50 and a < =66, the MPM list is always { intra_number 50, intra_number 66, intra_number 18, DC, plane, intra_number 34}.

In some examples, if the left-side neighboring INTRA mode and the above-neighboring INTRA mode of the current block are different modes, and one of them is a non-ANGULAR mode, the MPM list is { DC, planar, intra_anguar 50, intra_anguar 18, intra_anguar 34, intra_anguar 66}.

In some examples, the left-side neighboring intra mode and the upper-neighboring intra mode of the current block are different modes and both are angular modes, where Max represents the maximum value between the two intra modes.

In some examples, if Max > =2 and Max < =18, the MPM list is { intra_number 18, intra_number 2, intra_number 50, DC, plane, intra_number 66}.

In some examples, if Max > =18 and Max < =34, the MPM list may be { intra_number 18, intra_number 34, intra_number 50, DC, plane, intra_number 2}.

In some examples, if Max > =34 and Max < =50, the MPM list may be { intra_number 50, intra_number 34, intra_number 18, DC, planar, intra_number 66}.

In some examples, if Max > =50 and Max < =66, the MPM list may be { intra_number 50, intra_number 66, intra_number 18, DC, planar, intra_number 34}.

In some examples, one existing mode in the MPM list is replaced by a vertical mode and/or a horizontal mode.

In some examples, one existing pattern in the MPM list having a closest direction to the vertical pattern is replaced by the vertical pattern.

In some examples, one existing pattern in the MPM list having a closest direction to the horizontal pattern is replaced by the horizontal pattern.

In some examples, the location of the vertical mode and/or the horizontal mode in the MPM list is dependent on at least one of:

signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

xv. the location of the Prediction Unit (PU);

xvi block dimensions of the current block and/or its neighboring blocks;

xvii block shape of the current block and/or its neighboring blocks;

xviii. Prediction modes, including intra mode and inter mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

xx. motion vectors of neighboring blocks of the current block;

indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

xxii. an indication of QR-BDPCM mode of the transform type of the current block;

xxiii current quantization parameters of the current block and/or its neighboring blocks;

xxiv an indication of the color format of the current block;

xxv. codec tree structure;

xxvi. Slice, group and/or picture types.

In some examples, when the current block is a Residual Differential Pulse Code Modulation (RDPCM) codec block, an intra mode of the RDPCM codec block is set to a mode other than the first mode in the MPM list.

In some examples, the intra mode of the RDPCM codec block is set to the second MPM mode in the MPM list, with an indication of intra_luma_mpm_idx=0.

In some examples, the intra mode of the RDPCM codec block is set to any mode in the MPM list other than the first mode, with an indication of intra_luma_mpm_idx=0.

In some examples, whether and/or how to set the intra mode of the RDPCM codec block depends on at least one of:

signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

xv. the location of the Prediction Unit (PU);

xvi block dimensions of the current block and/or its neighboring blocks;

xvii block shape of the current block and/or its neighboring blocks;

xviii. Prediction modes, including intra mode and inter mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

xx. motion vectors of neighboring blocks of the current block;

indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

xxii. an indication of QR-BDPCM mode of the transform type of the current block;

xxiii current quantization parameters of the current block and/or its neighboring blocks;

xxiv an indication of the color format of the current block;

xxv. codec tree structure;

xxvi. Slice, group and/or picture types.

In some examples, the location of the planar mode is not always the first location in the MPM list.

In some examples, the flat pattern is placed in the last position in the MPM list.

In some examples, the flat pattern is placed in any position in the MPM list other than the first position.

In some examples, the flat mode is not allowed to be placed in the first position in the MPM list.

In some examples, whether and/or how the location of the planar mode is set depends on at least one of:

signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

xv. the location of the Prediction Unit (PU);

xvi block dimensions of the current block and/or its neighboring blocks;

xvii block shape of the current block and/or its neighboring blocks;

xviii. Prediction modes, including intra mode and inter mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

xx. motion vectors of neighboring blocks of the current block;

indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

xxii. an indication of QR-BDPCM mode of the transform type of the current block;

xxiii current quantization parameters of the current block and/or its neighboring blocks;

xxiv an indication of the color format of the current block;

xxv. codec tree structure;

xxvi. Slice, group and/or picture types.

In some examples, binarization of MPM indexes in the MPM list does not use truncated unary binarization.

In some examples, the MPM index is intra_luma_mpm_idx.

In some examples, the codeword length of the MPM index is fixed.

In some examples, signaling of the MPM index uses Exp-Golomb binarization with order K, K being an integer.

In some examples, K is 2.

In some examples, K depends on at least one of:

predicting a location of a unit (PU);

block dimensions of the current block and/or its neighboring blocks;

xv. block shapes of the current block and/or its neighboring blocks;

predictive modes including intra mode and inter mode of neighboring blocks of the current block;

xvii intra prediction modes of neighboring blocks of the current block;

xviii motion vectors of neighboring blocks of the current block;

indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

xx. an indication of the QR-BDPCM mode of the transform type of the current block;

Current quantization parameters of the current block and/or its neighboring blocks;

xxii an indication of the color format of the current block;

xxiii codec tree structure;

xxiv. Slice, group type and/or picture type.

In some examples, the indication of the MPM candidates in the MPM list is encoded and decoded with a first flag indicating whether the selected intra-prediction mode is the same as or different from the first MPM candidate in the MPM list, followed by the remaining MPM indexes of the MPM candidates.

In some examples, the first flag is an intra_luma_not_player_flag, where the semantics of the intra_luma_not_player_flag are interpreted as whether the selected intra prediction mode is not the first MPM candidate in the MPM list, rather than as whether it is not a planar mode.

In some examples, the first flag is conditionally encoded.

In some examples, the first flag is not encoded if the current block has screen content.

In some examples, if the selected intra-prediction mode is not the first MPM candidate in the MPM list, the remaining MPM indices are further signaled.

In some examples, the MPM list construction process includes vertical intra prediction and/or horizontal intra prediction, which correspond to directly copying reference samples during intra prediction.

In some examples, vertical intra prediction is generated by directly copying reference samples in the same column.

In some examples, horizontal intra prediction is generated by directly copying reference samples in the same row.

In some examples, applying vertical intra prediction and/or horizontal intra prediction depends on at least one of:

video content comprising at least one of screen content and natural content;

signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

xviii. Predicting the location of the unit (PU);

block dimensions of the current block and/or its neighboring blocks;

xx. block shapes of the current block and/or its neighboring blocks;

prediction modes including intra mode and inter mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

xxiii motion vectors of neighboring blocks of the current block;

xxiv an indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

xxv. an indication of QR-BDPCM mode of the transform type of the current block;

current quantization parameters of the current block and/or its neighboring blocks;

xxvii an indication of the color format of the current block;

xxviii codec tree structure;

xxix, slice group type and/or picture type;

xxx color components including at least one of a luma component and a chroma component;

in some examples, the first construction method and/or the second construction method further comprise at least one of an affine intra prediction method, a matrix-based intra prediction method, an intra sub-division prediction (ISP), and a multi-reference line (MRL).

In some examples, the color format of the current block is 4:2:0 or 4:4:4.

In some examples, the conversion is represented by a bitstream that generates a current block of video.

In some examples, the conversion is represented by a bitstream generated by a current block of video.

From the foregoing, it will be appreciated that specific embodiments of the technology disclosed herein have been described for purposes of illustration, but that various modifications may be made without deviating from the scope of the disclosed technology. Accordingly, the techniques disclosed herein are not limited except by the following claims.

Embodiments of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing unit" or "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus may include, in addition to hardware, code that creates an execution environment for a computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. The computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processing and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not necessarily have such a device. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and storage devices, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

It is intended that the specification, together with the drawings, be considered exemplary only, with the exemplary representation thereof. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, the use of "or" is intended to include "and/or" unless the context clearly dictates otherwise.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although certain features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Also, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Furthermore, the separation of various system components in the embodiments described herein should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and illustrated in this patent document.

Claims (78)

1. A method of processing video, comprising:

for a transition between a current block of video and a bitstream representation of the current block, constructing two or more Maximum Possible Mode (MPM) lists of intra-coding modes of the current block, wherein the two or more MPM lists include at least a first MPM list constructed using a first construction method and a second MPM list constructed using a second construction method different from the first construction method; and

performing the conversion using one MPM list selected from the constructed two or more MPM lists;

wherein one of the first MPM list and the second MPM list is used for natural content codec and the other is used for screen content codec.

2. The method of claim 1, wherein the first MPM list is a current MPM list of the current block, and the second MPM list is derived from the first MPM list.

3. The method of claim 2, wherein the first MPM list is generated first and the second MPM list is generated by reordering those patterns in the first MPM list.

4. A method according to claim 3, wherein a planar mode and/or a DC mode is put in different positions in the first MPM list.

5. The method of claim 2, wherein the first MPM list is first generated and the second MPM list is generated by replacing one or more modes in the first MPM list.

6. The method of claim 5, wherein the planar mode and/or the DC mode is replaced with an angular mode.

7. The method of claim 6, wherein the angular mode comprises at least one of a horizontal mode and a vertical mode.

8. The method of claim 5, wherein some modes other than the horizontal mode and the vertical mode in the first MPM list are replaced with a horizontal mode and/or a vertical mode.

9. The method of claim 1, wherein the second MPM list complies with logic of the first MPM list, wherein some non-angular modes are replaced by angular modes.

10. The method of any of claims 1-9, wherein signaling whether to allow two or more MPM lists is performed in at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a picture header, a slice group header, a Largest Codec Unit (LCU), a Codec Unit (CU), an LCU row, and an LCU group level.

11. The method according to any of claims 1-10, wherein the number of allowed MPM lists is predefined or signaled to a decoder.

12. The method of claim 11, wherein the number of MPM lists is a fixed value.

13. The method of claim 12, wherein the fixed value is 2.

14. The method of claim 11, wherein the number of MPM lists is signaled in at least one of VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, and LCU group level.

15. The method of any of claims 1-14, wherein which MPM list is selected depends on one or more of:

i. video content including at least one of screen content and natural content;

ii. at VPS, SPS, PPS, picture header, stripe header, slice group header, LCU,

Signaling messages in a CU, LCU row, LCU group, or other video unit;

predicting the location of the unit (PU);

block dimensions of the current block and/or its neighboring blocks;

v. block shapes of the current block and/or its neighboring blocks;

prediction modes including intra-mode and inter-mode of neighboring blocks of the current block;

Intra prediction modes of neighboring blocks of the current block;

motion vectors of neighboring blocks of the current block;

an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block of the current block;

x. an indication of QR-BDPCM mode of the transform type of the current block;

current quantization parameters of the current block and/or its neighboring blocks;

indication of the color format of the current block;

codec tree structure;

banding, slice group type, and/or picture type.

16. The method of claim 15, wherein a flag is signaled in a slice header to indicate a list to be selected for encoding and decoding a regular intra prediction mode.

17. The method of claim 15, wherein the MPM list is dependent on a message indicating video content information including at least one of screen content or camera capture content.

18. The method of claim 15, wherein the color format of the current block is 4:2:0 or 4:4:4.

19. The method of any of claims 1-18, wherein the MPM list is allowed to be switched at a slice, picture, CU, LCU, LCU group or block level.

20. The method of claim 19, wherein whether to switch the MPM list is dependent on at least one of:

i. video content including at least one of screen content and natural content;

signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

predicting the location of the unit (PU);

block dimensions of the current block and/or its neighboring blocks;

v. block shapes of the current block and/or its neighboring blocks;

prediction modes including intra-mode and inter-mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

motion vectors of neighboring blocks of the current block;

an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block of the current block;

x. an indication of QR-BDPCM mode of the transform type of the current block;

current quantization parameters of the current block and/or its neighboring blocks;

indication of the color format of the current block;

codec tree structure;

banding, slice group type, and/or picture type;

xv. includes a color component of at least one of a luminance component and a chrominance component;

21. The method of any of claims 1-20, wherein a particular angular pattern is always in the MPM list during construction of the MPM list.

22. The method of claim 21, wherein the particular angular mode is a vertical mode and/or a horizontal mode.

23. The method of claim 21 or 22, wherein the particular angle pattern is inserted at a given location in the MPM list.

24. The method of claim 23, wherein the particular angle pattern is inserted at a first location in the MPM list.

25. The method of claim 22, wherein the vertical mode and the horizontal mode are always in a first two positions in the MPM list in order.

26. The method of claim 22, wherein the horizontal mode and the vertical mode are always sequentially at first two positions in the MPM list.

27. The method of claim 21, wherein the MPM list is always { intra_anguar 50, intra_anguar 18, DC, plane, intra_anguar 34, intra_anguar 66}.

28. The method of claim 21, wherein the MPM list is always { intra_anguar 18, intra_anguar 50, plane, DC, intra_anguar 34, intra_anguar 66}.

29. The method of claim 21, wherein the MPM list is always { intra_anguar 50, intra_anguar 18, plane, DC, intra_anguar 34, intra_anguar 66}.

30. The method of claim 21, wherein the MPM list is always { plane, DC, intra_number 18, intra_number 50, intra_number 34, intra_number 66}.

31. The method of claim 21, wherein the MPM list is { intra_anguar 50, intra_anguar 18, DC, plane, intra_anguar 34, intra_anguar 66} if neither a left-side neighboring INTRA mode nor an upper neighboring INTRA mode of the current block is available.

32. The method of claim 21, wherein the MPM list is { intra_anguar 50, intra_anguar 18, DC, plane, intra_anguar 34, intra_anguar 2} if neither a left-side neighboring INTRA mode nor an upper neighboring INTRA mode of the current block is available.

33. The method of claim 21, wherein if the left-neighboring INTRA mode and the upper-neighboring INTRA mode of the current block are the same and such mode is a non-ANGULAR INTRA mode, the MPM list is { DC, planar, intra_anguar 50, intra_anguar 18, intra_anguar 34, intra_anguar 66}.

34. The method of claim 21, wherein the MPM list is constructed based on a direction of a left neighboring intra-mode and an upper neighboring intra-mode of the current block if the mode is the same and the mode a is an angular intra-mode.

35. The method of claim 34, wherein if a > = 2 and a < = 18, the MPM list is { intra_anguar 18, intra_anguar 2, intra_anguar 50, DC, plane, intra_anguar 66}.

36. The method of claim 34, wherein if a > = 18 and a < = 34, the MPM list is { intra_anguar 18, intra_anguar 34, intra_anguar 50, DC, plane, intra_anguar 2}.

37. The method of claim 34, wherein if a > = 34 and a < = 50, the MPM list is { intra_number 50, intra_number 34, intra_number 18, DC, plane, intra_number 66}.

38. The method of claim 34, wherein if a > = 50 and a < = 66, the MPM list is { intra_number 50, intra_number 66, intra_number 18, DC, plane, intra_number 34}.

39. The method of claim 21, wherein if the left-neighboring INTRA mode and the upper-neighboring INTRA mode of the current block are different modes and one of them is a non-angle mode, the MPM list is { DC, plane, intra_number 50, intra_number 18, intra_number 34, intra_number 66}.

40. The method of claim 21, wherein the left neighboring intra mode and the upper neighboring intra mode of the current block are different modes and both are angular modes, wherein Max represents a maximum value between the two intra modes.

41. The method of claim 40, wherein if Max > = 2 and Max < = 18, the MPM list is { intra_number 18, intra_number 2, intra_number 50, DC, plane, intra_number 66}.

42. The method of claim 40, wherein if Max > = 18 and Max < = 34, the MPM list may be { intra_angle 18, intra_angle 34, intra_angle 50, DC, plane, intra_angle 2}.

43. The method of claim 40, wherein if Max > = 34 and Max < = 50, the MPM list may be { intra_angle 50, intra_angle 34, intra_angle 18, DC, plane, intra_angle 66}.

44. The method of claim 40, wherein if Max > = 50 and Max < = 66, the MPM list may be { intra_angle 50, intra_angle 66, intra_angle 18, DC, planar, intra_angle 34}.

45. The method of claim 22, wherein one existing mode in the MPM list is replaced by a vertical mode and/or a horizontal mode.

46. The method of claim 45, wherein one of the existing modes in the MPM list having a closest direction to a vertical mode is replaced with the vertical mode.

47. The method of claim 45, wherein one of the existing patterns in the MPM list having a closest direction to a horizontal pattern may be replaced with the horizontal pattern.

48. The method of claim 22, wherein the position of the vertical mode and/or the horizontal mode in the MPM list is dependent on at least one of:

i. signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

predicting the location of the unit (PU);

block dimensions of the current block and/or its neighboring blocks;

block shapes of the current block and/or its neighboring blocks;

a prediction mode including an intra mode and an inter mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

motion vectors of neighboring blocks of the current block;

an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block of the current block;

An indication of QR-BDPCM mode of the transform type of the current block;

current quantization parameters of the current block and/or its neighboring blocks;

an indication of the color format of the current block;

the codec tree structure;

strip, group of tiles type and/or picture type.

49. The method of any one of claims 1-48, wherein when the current block is a Residual Differential Pulse Code Modulation (RDPCM) codec block, an intra mode of the RDPCM codec block is set to a mode other than the first mode in the MPM list.

50. The method of claim 49, wherein the intra mode of the RDPCM codec block is set to a second MPM mode in the MPM list, the second MPM mode having an indication of intra_luma_mpm_idx = 0.

51. The method of claim 49, wherein the intra mode of the RDPCM codec block is set to any mode in the MPM list other than the first mode, the any mode having an indication of intra_luma_mpm_idx = 0.

52. The method of any of claims 49-51, wherein whether and/or how to set the intra mode of the RDPCM codec block depends on at least one of:

i. Signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

predicting the location of the unit (PU);

block dimensions of the current block and/or its neighboring blocks;

block shapes of the current block and/or its neighboring blocks;

a prediction mode including an intra mode and an inter mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

motion vectors of neighboring blocks of the current block;

an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block of the current block;

an indication of QR-BDPCM mode of the transform type of the current block;

current quantization parameters of the current block and/or its neighboring blocks;

an indication of the color format of the current block;

the codec tree structure;

strip, group of tiles type and/or picture type.

53. The method of any one of claims 1-48, wherein the location of the planar mode is not always the first location in the MPM list.

54. The method of claim 53, wherein the flat pattern is placed in a last position in the MPM list.

55. The method of claim 53, wherein the flat pattern is placed in any position in the MPM list other than the first position.

56. The method of claim 53, wherein the flat mode is not allowed to be placed in the first position in the MPM list.

57. The method of any of claims 53-56, wherein whether and/or how the location of the planar mode is set depends on at least one of:

i. signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

predicting the location of the unit (PU);

block dimensions of the current block and/or its neighboring blocks;

block shapes of the current block and/or its neighboring blocks;

a prediction mode including an intra mode and an inter mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

motion vectors of neighboring blocks of the current block;

an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block of the current block;

an indication of QR-BDPCM mode of the transform type of the current block;

Current quantization parameters of the current block and/or its neighboring blocks;

an indication of the color format of the current block;

the codec tree structure;

strip, group of tiles type and/or picture type.

58. The method of any of claims 1-57, wherein binarization of MPM indexes in the MPM list does not use truncated unary binarization.

59. The method of claim 58, wherein the MPM index is intra_luma_mpm_idx.

60. The method of claim 58 or 59, wherein a codeword length of the MPM index is fixed.

61. The method of claim 58 or 59, wherein the signaling of the MPM index uses Exp-Golomb binarization with an order K, K being an integer.

62. The method of claim 61, wherein K is 2.

63. The method of claim 61, wherein K is dependent on at least one of:

i. a location of a Prediction Unit (PU);

block dimensions of the current block and/or its neighboring blocks;

block shapes of the current block and/or its neighboring blocks;

prediction modes including intra-mode and inter-mode of neighboring blocks of the current block;

v. intra prediction modes of neighboring blocks of the current block;

Motion vectors of neighboring blocks of the current block;

an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block of the current block;

an indication of QR-BDPCM mode of the transform type of the current block;

current quantization parameters of the current block and/or its neighboring blocks;

x. an indication of the color format of the current block;

coding and decoding tree structure;

strip, slice group type and/or picture type.

64. The method of any of claims 1-63, wherein the indication of MPM candidates in the MPM list is encoded, wherein a first flag is encoded, followed by remaining MPM indexes of the MPM candidates, the first flag being used to indicate whether the selected intra-prediction mode is the same as or different from the first MPM candidates in the MPM list.

65. The method of claim 64, wherein the first flag is an intra_luma_not_player_flag, wherein semantics of intra_luma_not_player_flag are interpreted as whether the selected intra prediction mode is not the first MPM candidate in the MPM list, but is not interpreted as whether it is a planar mode.

66. The method of claim 64 or 65, wherein the first flag is conditionally encoded.

67. The method of claim 66, wherein the first flag is not encoded if the current block has screen content.

68. The method of claim 66, wherein if the selected intra-prediction mode is not the first MPM candidate in the MPM list, further signaling a remaining MPM index.

69. The method of any one of claims 1-68, wherein the MPM list construction process comprises vertical intra prediction and/or horizontal intra prediction corresponding to directly copying reference samples during intra prediction.

70. A method as defined in claim 69, wherein the vertical intra prediction is generated by directly copying reference samples in the same column.

71. The method of claim 69, wherein the horizontal intra prediction is generated by directly copying reference samples in the same row.

72. The method of any of claims 69-71, wherein applying the vertical intra prediction and/or the horizontal intra prediction depends on at least one of:

i. video content including at least one of screen content and natural content;

signaling messages in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU lines, LCU group, or other video units;

Predicting the location of the unit (PU);

block dimensions of the current block and/or its neighboring blocks;

v. block shapes of the current block and/or its neighboring blocks;

prediction modes including intra-mode and inter-mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

motion vectors of neighboring blocks of the current block;

an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block of the current block;

x. an indication of QR-BDPCM mode of the transform type of the current block;

current quantization parameters of the current block and/or its neighboring blocks;

indication of the color format of the current block;

codec tree structure;

banding, slice group type, and/or picture type;

xv. includes a color component of at least one of a luminance component and a chrominance component;

73. the method of any one of claims 1-72, wherein the first and/or second build methods further comprise at least one of an affine intra-prediction method, a matrix-based intra-prediction method, an intra-sub-division prediction (ISP), and a multi-reference line (MRL).

74. The method of any of claims 20, 48, 52, 57, 63, 72, wherein the color format of the current block is 4:2:0 or 4:4:4.

75. The method of any of claims 1-74, wherein the converting generates the current block of video from the bitstream representation.

76. The method of any of claims 1-74, wherein the converting generates the bitstream representation from the current block of video.

77. An apparatus in a video system comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of claims 1-76.

78. A non-transitory computer readable medium having stored thereon a computer program for implementing the method of any one of claims 1 to 76.