WO2020224660A1

WO2020224660A1 - Most probable mode list construction for screen content coding

Info

Publication number: WO2020224660A1
Application number: PCT/CN2020/089376
Authority: WO
Inventors: Weijia Zhu; Li Zhang; Jizheng Xu; Yue Wang
Original assignee: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc.
Priority date: 2019-05-09
Filing date: 2020-05-09
Publication date: 2020-11-12
Also published as: CN113812160B; CN113812160A

Abstract

Most probable mode list construction for screen content coding is described. In an exemplary aspect, a method for video processing includes constructing, for a conversion between a current block of video and a bitstream representation of the current block, two or more Most Probable Modes (MPM) lists of intra coding modes for the current block, wherein the two or more MPM lists at least includes a first MPM list constructed with a first construction method and a second MPM list constructed with a second construction method different from the first construction method; and performing the conversion by using one MPM list selected from the constructed two or more MPM lists.

Description

MOST PROBABLE MODE LIST CONSTRUCTION FOR SCREEN CONTENT CODING

CROSS-REFERENCE TO RELATED APPLICATION

Under the applicable patent law and/or rules pursuant to the Paris Convention, this application is made to timely claim the priority to and benefits of International Patent Application No. PCT/CN2019/086114, filed on May 9, 2019. The entire disclosures of International Patent Application No. PCT/CN2019/086114 are incorporated by reference as part of the disclosure of this application.

TECHNICAL FIELD

This patent document relates to video coding techniques, devices and systems.

BACKGROUND

In spite of the advances in video compression, digital video still accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow.

SUMMARY

Devices, systems and methods related to digital video coding, and specifically, to adaptive loop filtering for video coding are described. The described methods may be applied to both the existing video coding standards (e.g., High Efficiency Video Coding (HEVC) ) and future video coding standards (e.g., Versatile Video Coding (VVC) ) or codecs.

Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards. The ITU-T produced H. 261 and H. 263, ISO/IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations jointly produced the H. 262/MPEG-2 Video and H. 264/MPEG-4 Advanced Video Coding (AVC) and H. 265/HEVC standards. Since H. 262, the video coding standards are based on the hybrid video coding structure wherein temporal prediction plus transform coding are utilized. To explore the future video coding technologies beyond HEVC, Joint Video Exploration Team (JVET) was founded by VCEG and MPEG jointly in 2015. Since then, many new methods have been adopted by JVET and put into the reference software named Joint Exploration Model (JEM) . In April 2018, the Joint Video Expert Team (JVET) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) was created to work on the VVC standard targeting at 50%bitrate reduction compared to HEVC.

In one representative aspect, the disclosed technology may be used to provide a method for video processing which includes constructing a first mode list of intra coding modes based on intra modes of neighboring blocks of a current video block; constructing a second mode list of intra coding modes based on the first mode list; and performing an intra mode coding for the current video block by using at least one of the first mode list or the second mode list.

In another representative aspect, the disclosed technology may be used to provide a method for video processing which includes constructing a mode list of intra coding modes to include candidate modes based on intra modes of neighboring blocks of a current video block, wherein the constructing the mode list includes locating the candidate modes at positions in the mode list based on types of the candidate modes.

In one representative aspect, the disclosed technology may be used to provide a method for video processing. The method includes constructing, for a conversion between a current block of video and a bitstream representation of the current block, two or more Most Probable Modes (MPM) lists of intra coding modes for the current block, wherein the two or more MPM lists at least includes a first MPM list constructed with a first construction method and a second MPM list constructed with a second construction method different from the first construction method; and performing the conversion by using one MPM list selected from the constructed two or more MPM lists.

In yet another representative aspect, the above-described method is embodied in the form of processor-executable code and stored in a computer-readable program medium.

In yet another representative aspect, a device that is configured or operable to perform the above-described method is disclosed. The device may include a processor that is programmed to implement this method.

In yet another representative aspect, a video decoder apparatus may implement a method as described herein.

The above and other aspects and features of the disclosed technology are described in greater detail in the drawings, the description and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of an intra block copy.

FIG. 2 shows an example of a block coded in palette mode.

FIG. 3 illustrates an example of a use of palette predictor to signal palette entries.

FIG. 4 shows examples of horizontal and vertical traverse scans that are used to code palette indices.

FIG. 5 shows an example of coding of palette indices.

FIG. 6 shows examples of multi-type tree splitting modes.

FIG. 7 shows examples of locations of samples used for derivation of α and β.

FIG. 8 shows an example of a luma mapping with chroma scaling architecture.

FIGS. 9 shows examples of 67 intra prediction modes.

FIG. 10 shows examples of left and above neighbors of a current block.

FIG. 11 shows an example of four reference lines neighboring to a prediction block.

FIG. 12a shows examples of sub-partitions for 4x8 and 8x4 CUs and FIG. 12b shows examples of sub-partitions for CUs other than 4x8, 8x4, and 4x4.

FIGS. 13a and 13b shows flowcharts of example methods for mode list construction in accordance with the disclosed technology.

FIG. 14 is a block diagram of an example of a hardware platform for implementing a visual media decoding or a visual media encoding technique described in the present document.

FIG. 15 shows a flowchart of an example method for video coding.

DETAILED DESCRIPTION

Due to the increasing demand of higher resolution video, video coding methods and techniques are ubiquitous in modern technology. Video codecs typically include an electronic circuit or software that compresses or decompresses digital video, and are continually being improved to provide higher coding efficiency. A video codec converts uncompressed video to a compressed format or vice versa. There are complex relationships between the video quality, the amount of data used to represent the video (determined by the bit rate) , the complexity of the encoding and decoding algorithms, sensitivity to data losses and errors, ease of editing, random access, and end-to-end delay (latency) . The compressed format usually conforms to a standard video compression specification, e.g., the High Efficiency Video Coding (HEVC) standard (also known as H. 265 or MPEG-H Part 2) , the Versatile Video Coding (VVC) standard to be finalized, or other current and/or future video coding standards.

Embodiments of the disclosed technology may be applied to existing video coding standards (e.g., HEVC, H. 265) and future standards to improve runtime performance. Section headings are used in the present document to improve readability of the description and do not in any way limit the discussion or the embodiments (and/or implementations) to the respective sections only.

1 Intra Block Copy

Intra block copy (IBC) , a.k.a. current picture referencing, has been adopted in HEVC Screen Content Coding extensions (HEVC-SCC) and the current VVC test model (VTM-4.0) . IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. As demonstrated in FIG. 1, the current block is predicted by a reference block in the same picture when IBC is applied. The samples in the reference block must have been already reconstructed before the current block is coded or decoded. Although IBC is not so efficient for most camera-captured sequences, it shows significant coding gains for screen content. The reason is that there are lots of repeating patterns, such as icons and text characters in a screen content picture. IBC can remove the redundancy between these repeating patterns effectively. In HEVC-SCC, an inter-coded coding unit (CU) can apply IBC if it chooses the current picture as its reference picture. The MV is renamed as block vector (BV) in this case, and a BV always has an integer-pixel precision. To be compatible with main profile HEVC, the current picture is marked as a “long-term” reference picture in the Decoded Picture Buffer (DPB) . It should be noted that similarly, in multiple view/3D video coding standards, the inter-view reference picture is also marked as a “long-term” reference picture.

Following a BV to find its reference block, the prediction can be generated by copying the reference block. The residual can be got by subtracting the reference pixels from the original signals. Then transform and quantization can be applied as in other coding modes.

However, when a reference block is outside of the picture, or overlaps with the current block, or outside of the reconstructed area, or outside of the valid area restricted by some constrains, part or all pixel values are not defined. Basically, there are two solutions to handle such a problem. One is to disallow such a situation, e.g. in bitstream conformance. The other is to apply padding for those undefined pixel values. The following sub-sessions describe the solutions in detail.

1.2 IBC in HEVC Screen Content Coding extensions

In the screen content coding extensions of HEVC, when a block uses current picture as reference, it should guarantee that the whole reference block is within the available reconstructed area, as indicated in the following spec text:

The variables offsetX and offsetY are derived as follows:

offsetX = (ChromaArrayType = = 0) ? 0: (mvCLX [0]&0x7 ? 2: 0) [Equation 1]

offsetY = (ChromaArrayType = = 0) ? 0: (mvCLX [1] &0x7 ? 2: 0) [Equation 2]

It is a requirement of bitstream conformance that when the reference picture is the current picture, the luma motion vector mvLX shall obey the following constraints:

-When the derivation process for z-scan order block availability as specified in clause 6.4.1 is invoked with (xCurr, yCurr) set equal to (xCb, yCb) and the neighbouring luma location (xNbY, yNbY) set equal to (xPb + (mvLX [0]>> 2) -offsetX, yPb + (mvLX [1]>> 2) -offsetY) as inputs, the output shall be equal to TRUE.

-When the derivation process for z-scan order block availability as specified in clause 6.4.1 is invoked with (xCurr, yCurr) set equal to (xCb, yCb) and the neighbouring luma location (xNbY, yNbY) set equal to (xPb + (mvLX [0]>> 2) + nPbW -1 + offsetX, yPb + (mvLX [1]>>2) + nPbH -1 + offsetY) as inputs, the output shall be equal to TRUE.

-One or both the following conditions shall be true:

The value of (mvLX [0]>> 2) + nPbW + xB1 + offsetX is less than or equal to 0.

The value of (mvLX [1]>> 2) + nPbH + yB1 + offsetY is less than or equal to 0.

-The following condition shall be true:

(xPb + (mvLX [0]>> 2) + nPbSw -1 + offsetX) /CtbSizeY -xCurr /CtbSizeY <= yCurr/CtbSizeY - (yPb + (mvLX [1] >> 2) + nPbSh -1 + offsetY) /CtbSizeY [Equation 3]

Thus, the case that the reference block overlaps with the current block or the reference block is outside of the picture will not happen. There is no need to pad the reference or prediction block.

1.3 IBC in VVC Test Model

In the current VVC test model, i.e. VTM-4.0 design, the whole reference block should be with the current coding tree unit (CTU) and does not overlap with the current block. Thus, there is no need to pad the reference or prediction block. The IBC flag is coded as a prediction mode of the current CU. Thus, there are totally three prediction modes, MODE_INTRA, MODE_INTER and MODE_IBC for each CU.

1.3.1. IBC Merge mode

In IBC merge mode, an index pointing to an entry in the IBC merge candidates list is parsed from the bitstream. The construction of the IBC merge list can be summarized according to the following sequence of steps:

Step 1: Derivation of spatial candidates

Step 2: Insertion of HMVP candidates

Step 3: Insertion of pairwise average candidates

In the derivation of spatial merge candidates, a maximum of four merge candidates are selected among candidates located in the positions depicted in A1, B1, B0, A0 and B2. The order of derivation is A1, B1, B0, A0 and B2. Position B2 is considered only when any PU of position A1, B1, B0, A0 is not available (e.g. because it belongs to another slice or tile) or is not coded with IBC mode. After candidate at position A1 is added, the insertion of the remaining candidates is subject to a redundancy check which ensures that candidates with same motion information are excluded from the list so that coding efficiency is improved.

After insertion of the spatial candidates, if the IBC merge list size is still smaller than the maximum IBC merge list size, IBC candidates from HMVP table may be inserted. Redundancy check are performed when inserting the HMVP candidates.

Finally, pairwise average candidates are inserted into the IBC merge list.

When a reference block identified by a merge candidate is outside of the picture, or overlaps with the current block, or outside of the reconstructed area, or outside of the valid area restricted by some constrains, the merge candidate is called invalid merge candidate.

It is noted that invalid merge candidates may be inserted into the IBC merge list.

1.3.2 IBC AMVP mode

In IBC AMVP mode, an AMVP index point to an entry in the IBC AMVP list is parsed from the bitstream. The construction of the IBC AMVP list can be summarized according to the following sequence of steps:

Step 1: Derivation of spatial candidates

Check A0, A1 until an available candidate is found.

Check B0, B1, B2 until an available candidate is found.

Step 2: Insertion of HMVP candidates

Step 3: Insertion of zero candidates

After insertion of the spatial candidates, if the IBC AMVP list size is still smaller than the maximum IBC AMVP list size, IBC candidates from HMVP table may be inserted.

Finally, zero candidates are inserted into the IBC AMVP list.

1.4 Adaptive motion vector resolution (AMVR)

In HEVC, motion vector differences (MVDs) (between the motion vector and predicted motion vector of a CU) are signalled in units of quarter-luma-sample when use_integer_mv_flag is equal to 0 in the slice header. In VVC, a CU-level adaptive motion vector resolution (AMVR) scheme is introduced. AMVR allows MVD of the CU to be coded in different precision. Dependent on the mode (normal AMVP mode or affine AVMP mode) for the current CU, the MVDs of the current CU can be adaptively selected as follows:

-Normal AMVP mode: quarter-luma-sample, integer-luma-sample or four-luma-sample.

-Affine AMVP mode: quarter-luma-sample, integer-luma-sample or 1/16 luma-sample.

The CU-level MVD resolution indication is conditionally signalled if the current CU has at least one non-zero MVD component. If all MVD components (that is, both horizontal and vertical MVDs for reference list L0 and reference list L1) are zero, quarter-luma-sample MVD resolution is inferred.

For a CU that has at least one non-zero MVD component, a first flag is signalled to indicate whether quarter-luma-sample MVD precision is used for the CU. If the first flag is 0, no further signaling is needed and quarter-luma-sample MVD precision is used for the current CU. Otherwise, a second flag is signalled to indicate whether integer-luma-sample or four-luma-sample MVD precision is used for normal AMVP CU. The same second flag is used to indicate whether integer-luma-sample or 1/16 luma-sample MVD precision is used for affine AMVP CU. In order to ensure the reconstructed MV has the intended precision (quarter-luma-sample, interger-luma- sample or four-luma-sample) , the motion vector predictors for the CU will be rounded to the same precision as that of the MVD before being added together with the MVD. The motion vector predictors are rounded toward zero (that is, a negative motion vector predictor is rounded toward positive infinity and a positive motion vector predictor is rounded toward negative infinity) .

The encoder determines the motion vector resolution for the current CU using RD check. To avoid always performing CU-level RD check three times for each MVD resolution, in VTM4, the RD check of MVD precisions other than quarter-luma-smaple is only invoked conditionally. For normal AVMP mode, the RD cost of quarter-luma-sample MVD precision and integer-luma sample MV precision is computed first. Then, the RD cost of integer-luma-sample MVD precision is compared to that of quarter-luma-sample MVD precision to decide whether it is necessary to further check the RD cost of four-luma-sample MVD precision. When the RD cost for quarter-luma-sample MVD precision is much smaller than that of the integer-luma-sample MVD precision, the RD check of four-luma-sample MVD precision is skipped. For affine AMVP mode, if affine inter mode is not selected after checking rate-distortion costs of affine merge/skip mode, merge/skip mode, quarter-luma samplel MVD precision normal AMVP mode and quarter-luma sample MVD precision affine AMVP mode, then 1/16 luma-sample MV precision and 1-pel MV precision affine inter modes are not checked. Furthermore affine parameters obtained in quarter-luma-sample MV precision affine inter mode is used as starting search point in 1/16 luma-sample and quarter-luma-sample MV precision affine inter modes.

1.5 Palette Mode in HEVC Screen Content Coding extensions (HEVC-SCC)

The basic idea behind a palette mode is that the samples in the CU are represented by a small set of representative colour values. This set is referred to as the palette. It is also possible to indicate a sample that is outside the palette by signalling an escape symbol followed by (possibly quantized) component values. This is illustrated in FIG. 2.

1.5.1 Coding of the palette entries

For coding of the palette entries, a palette predictor is maintained. The maximum size of the palette as well as the palette predictor is signaled in the SPS. In HEVC-SCC, a palette_predictor_initializer_present_flag is introduced in the PPS. When this flag is 1, entries for initializing the palette predictor are signaled in the bitstream. The palette predictor is initialized at the beginning of each CTU row, each slice and each tile. Depending on the value of the palette_predictor_initializer_present_flag, the palette predictor is reset to 0 or initialized using the palette predictor intializer entries signaled in the PPS. In HEVC-SCC, a palette predictor initializer of size 0 was enabled to allow explicit disabling of the palette predictor initialization at the PPS level.

For each entry in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette. This is illustrated in FIG. 3. The reuse flags are sent using run-length coding of zeros. After this, the number of new palette entries are signaled using exponential Golomb code of order 0. Finally, the component values for the new palette entries are signaled.

1.5.2 Coding of palette indices

The palette indices are coded using horizontal and vertical traverse scans as shown in Fig. 4. The scan order is explicitly signaled in the bitstream using the palette_transpose_flag. For the rest of the subsection it is assumed that the scan is horizontal.

The palette indices are coded using two main palette sample modes: 'INDEX' and 'COPY_ABOVE' . As explained previously, the escape symbol is also signaled as an 'INDEX' mode and assigned an index equal to the maximum palette size. The mode is signaled using a flag except for the top row or when the previous mode was 'COPY_ABOVE'. In the 'COPY_ABOVE'mode, the palette index of the sample in the row above is copied. In the 'INDEX'mode, the palette index is explicitly signaled. For both 'INDEX' and 'COPY_ABOVE'modes, a run value is signaled which specifies the number of subsequent samples that are also coded using the same mode. When escape symbol is part of the run in 'INDEX' or 'COPY_ABOVE' mode, the escape component values are signalled for each escape symbol. The coding of palette indices is illustrated in FIG. 5.

This syntax order is accomplished as follows. First the number of index values for the CU is signaled. This is followed by signaling of the actual index values for the entire CU using truncated binary coding. Both the number of indices as well as the the index values are coded in bypass mode. This groups the index-related bypass bins together. Then the palette sample mode (if necessary) and run are signaled in an interleaved manner. Finally, the component escape values corresponding to the escape samples for the entire CU are grouped together and coded in bypass mode.

An additional syntax element, last_run_type_flag, is signaled after signaling the index values. This syntax element, in conjunction with the number of indices, eliminates the need to signal the run value corresponding to the last run in the block.

In HEVC-SCC, the palette mode is also enabled for 4: 2: 2, 4: 2: 0, and monochrome chroma formats. The signaling of the palette entries and palette indices is almost identical for all the chroma formats. In case of non-monochrome formats, each palette entry consists of 3 components. For the monochrome format, each palette entry consists of a single component. For subsampled chroma directions, the chroma samples are associated with luma sample indices that are divisible by 2. After reconstructing the palette indices for the CU, if a sample has only a single component associated with it, only the first component of the palette entry is used. The only difference in signaling is for the escape component values. For each escape sample, the number of escape component values signaled may be different depending on the number of components associated with that sample.

1.6 Coefficients Coding in Transform Skip mode

Several modifications are proposed on the coefficients coding in transform skip (TS) mode in order to adapt the residual coding to the statistics and signal characteristics of the transform skip levels.

The proposed modifications are listed as follows.

No last significant scanning position: Since the residual signal reflects the spatial residual after the prediction and no energy compaction by transform is performed for TS, the higher probability for trailing zeros or insignificant levels at the bottom right corner of the transform block is not given anymore. Thus, last significant scanning position signalling is omitted in this case.

Subblock CBFs: The absence of the last significant scanning position signalling requires the subblock CBF signalling with coded_sub_block_flag for TS to be modified as follows:

Due to quantization, the aforementioned sequence of insignificance may still occur locally inside a transform block. Thus, the last significant scanning position is removed as described before and coded_sub_block_flag is coded for all sub-blocks except the case that all CGs except the last have zero coefficients, thus, no need to code the coded_sub_block_flag for the last CG.

The coded_sub_block_flag for the subblock covering the DC frequency position (top-left subblock) presents a special case. In VVC Draft 3, the coded_sub_block_flag for this subblock is never signaled and always inferred to be equal to 1. When the last significant scanning position is located in another subblock, it means that there is at least one significant level outside the DC subblock. Consequently, the DC subblock may contain only zero/non-significant levels although the coded_sub_block_flag for this subblock is inferred to be equal to 1. With the absence of the last scanning position information in TS, the coded_sub_block_flag for each subblock is signaled.

This also includes the coded_sub_block_flag for the DC subblock except when all other coded_sub_block_flag syntax elements are already equal to 0. In this case, the DC coded_sub_block_flag is inferred to be equal to 1 (inferDcSbCbf=1) . Since there has to be at least one significant level in this DC subblock, the sig_coeff_flag syntax element for the first position at (0, 0) is not signaled and derived to be equal to 1 (inferSbDcSigCoeffFlag=1) instead if all other sig_coeff_flag syntax elements in this DC subblock are equal to 0.

The context modeling for coded_sub_block_flag is changed. The context model index is calculated as the sum of the coded_sub_block_flag to the left and the coded_sub_block_flag abovess the current subblock instead of and a logical disjunction of both.

sig_coeff_flag context modelling: The local template in sig_coeff_flag context modeling is modified to only include the neighbor to the left (NB0) and the neighbor above (NB1) the current scanning position. The context model offset is just the number of significant neighboring positions sig_coeff_flag [NB0]+ sig_coeff_flag [NB1] . Hence, the selection of different context sets depending on the diagonal d within the current transform block is removed. This results in three context models and a single context model set for coding the sig_coeff_flag flag.

abs_level_gt1_flag and par_level_flag context modelling: a single context model is employed for abs_level_gt1_flag and par_level_flag.

abs_remainder coding: Although the empirical distribution of the transform skip residual absolute levels typically still fits a Laplacian or a Geometrical distribution, there exist larger instationarities than for transform coefficient absolute levels. Particularly, the variance within a window of consecutive realization is higher for the residual absolute levels. This motivates the following modifications of the abs_remainder syntax binarization and context modelling:

-Using a higher cutoff value in the binarization, i.e., the transition point from the coding with sig_coeff_flag, abs_level_gt1_flag, par_level_flag, and abs_level_gt3_flag to the Rice codes for abs_remainder, and dedicated context models for each bin position yields higher compression efficiency. Increasing the cutoff will result in more "greater than X" flags, e.g. introducing abs_level_gt5_flag, abs_level_gt7_flag, and so on until a cutoff is reached. The cutoff itself is fixed to 5 (numGtFlags=5) .

-The template for the rice parameter derivation is modified, i.e., only the neighbor to the left and the neighbor above the current scanning position are considered similar to the local template for sig_coeff_flag context modeling.

coeff_sign_flag context modelling: Due to the instationarities inside the sequence of signs and the fact that the prediction residual is often biased, the signs can be coded using context models, even when the global empirical distribution is almost uniformly distributed. A single dedicated context model is used for the coding of the signs and the sign is parsed after sig_coeff_flag to keep all context coded bins together.

1.7 Quantized residual Block Differential Pulse-code Modulation (QR-BDPCM)

A quantized residual block differential pulse-code modulation (QR-BDPCM) is proposed to code screen contents efficiently.

The prediction directions used in QR-BDPCM can be vertical and horizontal prediction modes. The intra prediction is done on the entire block by sample copying in prediction direction (horizontal or vertical prediction) similar to intra prediction. The residual is quantized and the delta between the quantized residual and its predictor (horizontal or vertical) quantized value is coded. This can be described by the following: For a block of size M (rows) × N (cols) , let r_ (i, j) , 0≤i≤M-1, 0≤j≤N-1 be the prediction residual after performing intra prediction horizontally (copying left neighbor pixel value across the the predicted block line by line) or vertically (copying top neighbor line to each line in the predicted block) using unfiltered samples from above or left block boundary samples. Let Q (r_ (i, j) ) , 0≤i≤M-1, 0≤j≤N-1 denote the quantized version of the residual r_ (i, j) , where residual is difference between original block and the predicted block values. Then the block DPCM is applied to the quantized residual samples, resulting in modified M × N array

with elements

When vertical BDPCM is signalled:

For horizontal prediction, similar rules apply, and the residual quantized samples are obtained by:

The residual quantized samples

are sent to the decoder.

On the decoder side, the above calculations are reversed to produce Q (r _i, j) , 0≤i≤M-1, 0≤j≤N-1. For vertical prediction case,

For horizontal case,

The inverse quantized residuals, Q ^-1 (Q (r _i, j) ) , are added to the intra block prediction values to produce the reconstructed sample values.

The main benefit of this scheme is that the inverse DPCM can be done on the fly during coefficient parsing simply adding the predictor as the coefficients are parsed or it can be performed after parsing.

The draft text changes of QR-BDPCM are shown as follows.

7.3.6.5 Coding Unit Syntax

bdpcm_flag [x0] [y0] equal to 1 specifies that a bdpcm_dir_flag is present in the coding unit including the luma coding block at the location (x0, y0)

bdpcm_dir_flag [x0] [y0] equal to 0 specifies that the prediction direction to be used in a bdpcm block is horizontal, otherwise it is vertical.

1.8 Partition Structure

In HEVC, a CTU is split into CUs by using a quaternary-tree structure denoted as coding tree to adapt to various local characteristics. The decision whether to code a picture area using inter-picture (temporal) or intra-picture (spatial) prediction is made at the leaf CU level. Each leaf CU can be further split into one, two or four PUs according to the PU splitting type. Inside one PU, the same prediction process is applied and the relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU splitting type, a leaf CU can be partitioned into transform units (TUs) according to another quaternary-tree structure similar to the coding tree for the CU. One of key feature of the HEVC structure is that it has the multiple partition conceptions including CU, PU, and TU.

In VVC, a quadtree with nested multi-type tree using binary and ternary splits segmentation structure replaces the concepts of multiple partition unit types, i.e. it removes the separation of the CU, PU and TU concepts except as needed for CUs that have a size too large for the maximum transform length, and supports more flexibility for CU partition shapes. In the coding tree structure, a CU can have either a square or rectangular shape. A coding tree unit (CTU) is first partitioned by a quaternary tree (a.k.a. quadtree) structure. Then the quaternary tree leaf nodes can be further partitioned by a multi-type tree structure. As shown in Fig. 6, there are four splitting types in multi-type tree structure, vertical binary splitting (SPLIT_BT_VER) , horizontal binary splitting (SPLIT_BT_HOR) , vertical ternary splitting (SPLIT_TT_VER) , and horizontal ternary splitting (SPLIT_TT_HOR) . The multi-type tree leaf nodes are called coding units (CUs) , and unless the CU is too large for the maximum transform length, this segmentation is used for prediction and transform processing without any further partitioning. This means that, in most cases, the CU, PU and TU have the same block size in the quadtree with nested multi-type tree coding block structure. The exception occurs when maximum supported transform length is smaller than the width or height of the colour component of the CU. In addition, luma and chroma components have separate partition structures on I tiles.

1.9 Cross-component linear model prediction

To reduce the cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in the VTM4, for which the chroma samples are predicted based on the reconstructed luma samples of the same CU by using a linear model as follows:

pred _C (i, j) =α·rec _L' (i, j) +β [Equation 8]

where pred _C (i, j) represents the predicted chroma samples in a CU and rec _L (i, j) represents the downsampled reconstructed luma samples of the same CU. Linear model parameter α and β are derived from the relation between luma values and chroma values from two samples, which are luma sample with minimum sample value and with maximum smample sample inside the set of downsampled neighboring luma samples, and their corresponding chroma samples. The linear model parameters α and β are obtained according to the following equations.

β=Y _b-α·X _b [Equation 10]

Where Y _a and X _a represent luma value and chroma value of the luma sample with maximum luam sample value. And X _b and Y _b represent luma value and chroma value of the luma sample with minimum luma sample, respectively. Fig. 8 shows an example of the location of the left and above samples and the sample of the current block involved in the CCLM mode.

2.10 Luma mapping with chroma scaling (LMCS)

In VTM4, a coding tool called the luma mapping with chroma scaling (LMCS) is added as a new processing block before the loop filters. LMCS has two main components: 1) in-loop mapping of the luma component based on adaptive piecewise linear models; 2) for the chroma components, luma-dependent chroma residual scaling is applied. FIG. 8 shows the LMCS architecture from decoder’s perspective. The shaded blocks in FIG. 8 indicate where the processing is applied in the mapped domain; and these include the inverse quantization, inverse transform, luma intra prediction and adding of the luma prediction together with the luma residual. The unshaded blocks in FIG. 8 indicate where the processing is applied in the original (i.e., non-mapped) domain; and these include loop filters such as deblocking, ALF, and SAO, motion compensated prediction, chroma intra prediction, adding of the chroma prediction together with the chroma residual, and storage of decoded pictures as reference pictures. The light-yellow shaded blocks in FIG. 8 are the new LMCS functional blocks, including forward and inverse mapping of the luma signal and a luma-dependent chroma scaling process. Like most other tools in VVC, LMCS can be enabled/disabled at the sequence level using an SPS flag.

2.11 Intra mode coding with 67 intra prediction modes

To capture the arbitrary edge directions presented in natural video, the number of directional intra modes in VTM4 is extended from 33, as used in HEVC, to 65. The new directional modes not in HEVC are depicted as red dotted arrows in FIG. 9, and the planar and DC modes remain the same. These denser directional intra prediction modes apply for all block sizes and for both luma and chroma intra predictions.

A unified 6-MPM list is proposed for intra blocks irrespective of whether MRL and ISP coding tools are applied or not. The MPM list is constructed based on intra modes of the left

The syntax elements intra_luma_mpm_flag [x0] [y0] , intra_luma_not_planar_flag [x0] [y0] , intra_luma_mpm_idx [x0] [y0] and intra_luma_mpm_remainder [x0] [y0] specify the intra prediction mode for luma samples. The array indices x0, y0 specify the location (x0 , y0) of the top-left luma sample of the considered

coding block relative to the top-left luma sample of the picture. When intra_luma_mpm_flag [x0] [y0] is equal to 1, the intra prediction mode is inferred from a neighbouring intra-predicted coding unit according to clause 8.4.2.

When intra_luma_mpm_flag [x0] [y0] is not present (e.g., ISP enabled, or MRL enabled (with reference index > 0) ) , it is inferred to be equal to 1.

When intra_luma_not_planar_flag [x0] [y0] is not present (e.g., MRL is enabled) , it is inferred to be equal to 1.

8.4.2 Derivation process for luma intra prediction mode

Input to this process are:

-a luma location (xCb , yCb) specifying the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture,

-a variable cbWidth specifying the width of the current coding block in luma samples,

-a variable cbHeight specifying the height of the current coding block in luma samples.

In this process, the luma intra prediction mode IntraPredModeY [xCb] [yCb] is derived.

Table 8-1 specifies the value for the intra prediction mode IntraPredModeY [xCb] [yCb] and the associated names.

Table 8-1 – Specification of intra prediction mode and associated names

Intra prediction mode	Associated name
0	INTRA_PLANAR
1	INTRA_DC
2.. 66	INTRA_ANGULAR2.. INTRA_ANGULAR66
81.. 83	INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

NOTE –: The intra prediction modes INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM are only applicable to chroma components.

IntraPredModeY [xCb] [yCb] is derived as follows:

-If intra_luma_not_planar_flag [xCb] [yCb] is equal to 1, the following ordered steps:

1. The neighbouring locations (xNbA, yNbA) and (xNbB, yNbB) are set equal to (xCb -1, yCb + cbHeight -1) and (xCb + cbWidth -1, yCb -1) , respectively.

2. For X being replaced by either A or B, the variables candIntraPredModeX are derived as follows:

-The availability derivation process for a block as specified in clause 6.4. X [Ed. (BB) : Neighbouring blocks availability checking process tbd] is invoked with the location (xCurr, yCurr) set equal to (xCb, yCb) and the neighbouring location (xNbY, yNbY) set equal to (xNbX, yNbX) as inputs, and the output is assigned to availableX.

-The candidate intra prediction mode candIntraPredModeX is derived as follows:

-If one or more of the following conditions are true, candIntraPredModeX is set equal to INTRA_PLANAR.

- The variable availableX is equal to FALSE.

- CuPredMode [xNbX] [yNbX] is not equal to MODE_INTRA and ciip_flag [xNbX] [yNbX] is not equal to 1.

- pcm_flag [xNbX] [yNbX] is equal to 1.

- X is equal to B and yCb -1 is less than ( (yCb >> CtbLog2SizeY) << CtbLog2SizeY) .

- Otherwise, candIntraPredModeX is set equal to IntraPredModeY [xNbX] [yNbX] .

3. The candModeList [x] with x = 0.. 4 is derived as follows:

- If candIntraPredModeB is equal to candIntraPredModeA and candIntraPredModeA is greater than INTRA_DC, candModeList [x] with x = 0.. 4 is derived as follows:

candModeList [0] = candIntraPredModeA [Equation 11]

candModeList [1] = 2 + ( (candIntraPredModeA + 61) %64) [Equation 12]

candModeList [2] = 2 + ( (candIntraPredModeA -1) %64) [Equation13]

candModeList [3] = INTRA_DC [Equation 14]

candModeList [4] = 2 + ( (candIntraPredModeA + 60) %64) [Equation 15]

-Otherwise if candIntraPredModeB is not equal to candIntraPredModeA and candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC, the following applies:

-The variables minAB and maxAB are derived as follows:

minAB = Min (candIntraPredModeA, candIntraPredModeB) [Equation 16]

maxAB= Max (candIntraPredModeA, candIntraPredModeB) [Equation 17]

-If candIntraPredModeA and candIntraPredModeB are both greater than INTRA_DC, candModeList [x] with x = 0.. 4 is derived as follows:

candModeList [0] = candIntraPredModeA [Equation 18]

candModeList [1] = candIntraPredModeB [Equation 19]

candModeList [2] = INTRA_DC [Equation 20]

-If maxAB -minAB is in the range of 2 to 62, inclusive, the following applies:

candModeList [3] = 2 + ( (maxAB + 61) %64) [Equation 21]

candModeList [4] = 2 + ( (maxAB -1) %64) [Equation 22]

-Otherwise, the following applies:

candModeList [3] = 2 + ( (maxAB + 60) %64) [Equation 23]

candModeList [4] = 2 + ( (maxAB) %64) [Equation 24]

-Otherwise (candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC) , candModeList [x] with x = 0.. 4 is derived as follows:

candModeList [0] = maxAB [Equation 25]

candModeList [1] = INTRA_DC [Equation 26]

candModeList [2] = 2 + ( (maxAB + 61) %64) [Equation 27]

candModeList [3] = 2 + ( (maxAB -1) %64) [Equation 28]

candModeList [4] = 2 + ( (maxAB + 60) %64) [Equation 29]

-Otherwise, the following applies:

candModeList [0] = INTRA_DC [Equation 30]

candModeList [1] = INTRA_ANGULAR50 [Equation 31]

candModeList [2] = INTRA_ANGULAR18 [Equation 32]

candModeList [3] = INTRA_ANGULAR46 [Equation 33]

candModeList [4] = INTRA_ANGULAR54 [Equation 34]

4. IntraPredModeY [xCb] [yCb] is derived by applying the following procedure:

- If intra_luma_mpm_flag [xCb] [yCb] is equal to 1, the IntraPredModeY [xCb] [yCb] is set equal to candModeList [intra_luma_mpm_idx [xCb] [yCb] ] .

- Otherwise, IntraPredModeY [xCb] [yCb] is derived by applying the following ordered steps:

1) When candModeList [i] is greater than candModeList [j] for i = 0.. 3 and for each i, j = (i + 1) .. 4, both values are swapped as follows:

(candModeList [i] , candModeList [j] ) = Swap (candModeList [i] , candModeList [j] ) [Equation 35]

2) IntraPredModeY [xCb] [yCb] is derived by the following ordered steps:

i. IntraPredModeY [xCb] [yCb] is set equal to intra_luma_mpm_remainder [xCb] [yCb] .

ii. The value of IntraPredModeY [xCb] [yCb] is incremented by one

iii. For i equal to 0 to 4, inclusive, when IntraPredModeY [xCb] [yCb] is greater than or equal to candModeList [i] , the value of IntraPredModeY [xCb] [yCb] is incremented by one.

- Otherwise (intra_luma_not_planar_flag [xCb] [yCb] is equal to 0) , IntraPredModeY [xCb] [yCb] is set equal to INTRA_PLANAR.

The variable IntraPredModeY [x] [y] with x = xCb.. xCb + cbWidth -1 and y = yCb.. yCb + cbHeight -1 is set to be equal to IntraPredModeY [xCb] [yCb] .

1.12 Multiple reference line (MRL) intra prediction

Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. In Fig. 11, an example of 4 reference lines is depicted, where the samples of segments A and F are not fetched from reconstructed neighboring samples but padded with the closest samples from Segment B and E, respectively. HEVC intra-picture prediction uses the nearest reference line (i.e., reference line 0) . In MRL, 2 additional lines (reference line 1 and reference line 3) are used.

The index of selected reference line (mrl_idx) is signaled and used to generate intra predictor. For reference line idx, which is greater than 0, only include additional reference line modes in MPM list and only signal mpm index without remaining mode. The reference line index is signaled before intra prediction modes, and Planar and DC modes are excluded from intra prediction modes in case a nonzero reference line index is signaled.

MRL is disabled for the first line of blocks inside a CTU to prevent using extended reference samples outside the current CTU line. Also, PDPC is disabled when additional line is used.

1.13 Intra Sub-Partitions (ISP)

The Intra Sub-Partitions (ISP) tool divides luma intra-predicted blocks vertically or horizontally into 2 or 4 sub-partitions depending on the block size. For example, minimum block size for ISP is 4x8 (or 8x4) . If block size is greater than 4x8 (or 8x4) then the corresponding block is divided by 4 sub-partitions. Fig. 12 shows examples of the two possibilities. All sub-partitions fulfill the condition of having at least 16 samples.

Table 2-1 –Entropy coding coefficient group size

Block Size	Coefficient group Size
1×N, N≥16	1×16
N×1, N≥16	16×1
2×N, N≥8	2×8
N×2, N≥8	8×2
All other possible M×N cases	4×4

For each sub-partition, reconstructed samples are obtained by adding the residual signal to the prediction signal. Here, a residual signal is generated by the processes such as entropy decoding, inverse quantization and inverse transform. Therefore, the reconstructed sample values of each sub-partition are available to generate the prediction of the next sub-partition, and each sub-partition is processed repeatedly. In addition, the first sub-partition to be processed is the one containing the top-left sample of the CU and then continuing downwards (horizontal split) or rightwards (vertical split) . As a result, reference samples used to generate the sub-partitions prediction signals are only located at the left and above sides of the lines. All sub-partitions share the same intra mode. The followings are summary of interaction of ISP with other coding tools.

Multiple Reference Line (MRL) : if a block has an MRL index other than 0, then the ISP coding mode will be inferred to be 0 and therefore ISP mode information will not be sent to the decoder.

Entropy coding coefficient group size: the sizes of the entropy coding sub-blocks have been modified so that they have 16 samples in all possible cases, as shown in Table 2-1. Note that the new sizes only affect blocks produced by ISP in which one of the dimensions is less than 4 samples. In all other cases coefficient groups keep the 4×4 dimensions.

CBF coding: it is assumed to have at least one of the sub-partitions has a non-zero CBF. Hence, if n is the number of sub-partitions and the first n-1 sub-partitions have produced a zero CBF, then the CBF of the n-th sub-partition is inferred to be 1.

MPM usage: the MPM flag will be inferred to be one in a block coded by ISP mode, and the MPM list is modified to exclude the DC mode and to prioritize horizontal intra modes for the ISP horizontal split and vertical intra modes for the vertical one.

Transform size restriction: all ISP transforms with a length larger than 16 points uses the DCT-II.

PDPC: when a CU uses the ISP coding mode, the PDPC filters will not be applied to the resulting sub-partitions.

MTS flag: if a CU uses the ISP coding mode, the MTS CU flag will be set to 0 and it will not be sent to the decoder. Therefore, the encoder will not perform RD tests for the different available transforms for each resulting sub-partition. The transform choice for the ISP mode will instead be fixed and selected according the intra mode, the processing order and the block size utilized. Hence, no signalling is required. For example, let t _H and t _V be the horizontal and the vertical transforms selected respectively for the w×h sub-partition, where w is the width and h is the height. Then the transform is selected according to the following rules:

If w=1 or h=1, then there is no horizontal or vertical transform respectively.

If w=2 or w>32, t _H = DCT-II

If h =2 or h >32, t _V = DCT-II

Otherwise, the transform is selected as in Table 2-2.

Table 2-2 –Transform selection depends on intra mode

Drawbacks of existing implementations

The MPM list is designed for the natural sequences. On screen content coding, the vertical and horizonal mode are two dominated modes, so a different MPM list construction may be needed in screen content coding. Also, on mixed contents, more than one MPM list may be needed, and the MPM list may be able to switch based on whether the current block is screen content block.

Exemplary methods for most probable mode list construction for screen content coding

The below should be considered as examples to explain general concepts. These examples are provided to facilitate the understanding of the disclosed technology only and should not be interpreted in a narrow way. Furthermore, the implementations can be combined in any manner.

1. Two or more than two methods for MPM list construction may be utilized for the conventional intra mode coding.

a. In one example, the first MPM list may be the current one, and the second MPM list may be derived from the first one.

i. In one example, to generate the 2nd MPM list, the first MPM list may be firstly generated, afterwards, reordering may be applied those modes in the first MPM list.

a) In one example, the Planar and/or DC modes may be put to a different position in the list.

ii. In one example, to generate the 2nd MPM list, the first MPM list may be firstly generated, afterwards, replacing one or more modes in the list may be applied.

a) In one example, the Planar and/or DC modes may be replaced by an angular mode (such as horizontal or vertical) .

b) In one example, some modes other than vertical and/or horizontal mode may be replaced by vertical and/or horizontal modes.

b. In one example, the 2nd MPM list may follow the logic of the first MPM list but replace some non-angular modes by angular modes.

c. In one example, whether to allow more than one MPM lists may be signaled in the SPS/VPS/PPS/picture header/slice header/tile group header/Largest coding unit (LCU) /Coding unit (CU) /LCU row/group of LCUs level.

d. In one example, the number of allowed MPM lists may be predefined or signaled to the decoder.

i. In one example, the number of MPM lists is a fixed value, e.g., 2.

ii. In one example, the number of MPM lists may be signaled to the decoder in the SPS/VPS/PPS/picture header/slice header/tile group header/Largest coding unit (LCU) /Coding unit (CU) /LCU row/group of LCUs level

e. In one example, one MPM list is for natural content coding and the other one is for screen content coding.

f. In one example, which MPM list is used may be based on

i. Video contents (e.g. screen contents or natural contents)

ii. A message signaled in the SPS/VPS/PPS/picture header/slice header/tile group header/Largest coding unit (LCU) /Coding unit (CU) /LCU row/group of LCUs or other video units.

a) In one example, a flag may be signaled in slice header to indicate the list to be selected for coding conventional intra prediction mode

b) In one example, the MPM list may depend on the message which tells the video content information, such as screen content or camera captured content.

iii. The position of a prediction unit (PU)

iv. Block dimension of current block and/or its neighboring blocks

v. Block shape of current block and/or its neighboring blocks

vi. Prediction modes (Intra/Inter) of the neighboring blocks of the current block

vii. Intra prediction modes of the neighboring blocks of the current block

viii. Motion vectors of the neighboring blocks of the current block

viiii. The indication of QR-BDPCM modes of the neighboring block of the current block

x. The indication of QR-BDPCM modes of transform type (e.g indicated by tu_mts_idx) of the current block

xi. Current quantization parameter of current block and/or that of its neighboring blocks

xii. Indication of the color format (such as 4: 2: 0, 4: 4: 4)

xiii. Coding tree structure

xiv. Slice/tile group type and/or picture type

2. The MPM list may be switched at slice/tile/picture/CU/LCU/Group of LCUs/block level. Whether to switch the MPM list may be based on

a. Video contents (e.g. screen contents or natural contents)

b. A message signaled in the SPS/VPS/PPS/picture header/slice header/tile group header/Largest coding unit (LCU) /Coding unit (CU) /LCU row/group of LCUs/

c. The position of a prediction unit (PU) .

d. Block dimension of current block and/or its neighboring blocks

e. Block shape of current block and/or its neighboring blocks

f. Prediction modes (Intra/Inter) of the neighboring blocks of the current block

g. Intra prediction modes of the neighboring blocks of the current block

h. Motion vectors of the neighboring blocks of the current block

i. The indication of QR-BDPCM modes of the neighboring block of the current block

j. The indication of QR-BDPCM modes of transform type (e.g indicated by tu_mts_idx) of the current block

k. Current quantization parameter of current block and/or that of its neighboring blocks

l. Indication of the color format (such as 4: 2: 0, 4: 4: 4)

m. Coding tree structure

n. Slice/tile group type and/or picture type

o. Color component (such as luma or chroma)

3. In the construction of an MPM list, a certain angular mode (e.g., the vertical and/or horizontal modes) may be always in the MPM list.

a. In one example, the certain angular mode may be inserted to the MPM list at a given position, such as the first one.

b. In one example, the vertical and horizontal modes may be always in the first two positions in the MPM list in order.

c. In one example, the horizontal and vertical modes may be always in the first two positions in the MPM list in order.

d. In one example, the MPM list may always be {INTRA_ANGULAR50, INTRA_ANGULAR18, DC, Planar, INTRA_ANGULAR34, INTRA_ANGULAR66}

e. In one example, the MPM list may always be {INTRA_ANGULAR18, INTRA_ANGULAR50, Planar, DC, INTRA_ANGULAR34, INTRA_ANGULAR66}

f. In one example, the MPM list may always be {INTRA_ANGULAR50, INTRA_ANGULAR18, Planar, DC, INTRA_ANGULAR34, INTRA_ANGULAR66}

g. In one example, the MPM list may always be {Planar, DC, INTRA_ANGULAR18, INTRA_ANGULAR50, INTRA_ANGULAR34, INTRA_ANGULAR66} .

h. In one example, if both the left neighboring and above neighboring intra modes are not available, the MPM list may be {INTRA_ANGULAR50, INTRA_ANGULAR18, DC, Planar, INTRA_ANGULAR34, INTRA_ANGULAR66}

i. In one example, if both the left neighboring and above neighboring intra modes are not available, the MPM list may be {INTRA_ANGULAR50, INTRA_ANGULAR18, DC, Planar, INTRA_ANGULAR34, INTRA_ANGULAR2}

j. In one example, if the left neighboring and above neighboring intra modes are same and this mode is a non-angular intra mode, the MPM list may be {DC, Planar, INTRA_ANGULAR50, INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR66}

k. In one example, if the left neighboring and above neighboring intra modes are same and this mode A is an angular intra mode, the MPM list may be constructed based on the direction of A.

i. In one example, if A >= 2 and A <= 18, the MPM list may be {INTRA_ANGULAR18, INTRA_ANGULAR2, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR66}

ii. In one example, if A >= 18 and A <= 34, the MPM list may be {INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR2}

iii. In one example, if A >= 34 and A <= 50, the MPM list may be {INTRA_ANGULAR50, INTRA_ANGULAR34, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR66}

iv. In one example, if A >= 50 and A <= 66, the MPM list may be {INTRA_ANGULAR50, INTRA_ANGULAR66, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR34}

l. In one example, the left neighboring and above neighboring intra modes are different modes and one of them are non-angular modes.

i. In one example, furthermore, the MPM list may be {DC, Planar, INTRA_ANGULAR50, INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR66}

m. In one example, the left neighboring and above neighboring intra modes are different modes and both of them are angular modes.

i. In one example, let Max denote the maximal value between these two intra modes.

ii. In one example, if Max >= 2 and Max <= 18, the MPM list may be {INTRA_ANGULAR18, INTRA_ANGULAR2, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR66}

iii. In one example, if Max >= 18 and Max <= 34, the MPM list may be {INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR2}

iv. In one example, if Max >= 34 and Max <= 50, the MPM list may be {INTRA_ANGULAR50, INTRA_ANGULAR34, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR66}

v. In one example, if Max >= 50 and Max <= 66, the MPM list may be {INTRA_ANGULAR50, INTRA_ANGULAR66, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR34}

n. In one example, on top of the current design on the MPM list construction, the vertical and/or horizontal mode may replace one existing mode in the MPM list.

i. In one example, in a MPM list, a mode which has a closest direction with vertical mode may be replaced by the vertical mode.

ii. In one example, in a MPM list, a mode which has a closest direction with horizontal mode may be replaced by the horizontal mode.

o. In one example, the position of vertical and/or horizontal modes in the MPM list may be based on

i. A message signaled in the SPS/VPS/PPS/picture header/slice header/tile group header/Largest coding unit (LCU) /Coding unit (CU) /LCU row/group of LCUs/

ii. The position of a prediction unit (PU) .

iii. Block dimension of current block and/or its neighboring blocks

iv. Block shape of current block and/or its neighboring blocks

v. Prediction modes (Intra/Inter) of the neighboring blocks of the current block

vi. Intra prediction modes of the neighboring blocks of the current block

vii. Motion vectors of the neighboring blocks of the current block

viii. The indication of QR-BDPCM modes of the neighboring block of the current block

ix. The indication of QR-BDPCM modes of transform type (e.g indicated by tu_mts_idx) of the current block

x. Current quantization parameter of current block and/or that of its neighboring blocks

xi. Indication of the color format (such as 4: 2: 0, 4: 4: 4)

xii. Coding tree structure

xiii. Slice/tile group type and/or picture type

4. The intra mode of RDPCM may be set to a mode other than the first one in an MPM list. In this case, the prediction signal may be generated from one intra mode and the stored intra mode may be corresponding to another one. The stored intra mode can be used to predict the intra modes of the successively coding blocks. In some implementations, the stored intra mode can be different from the intra mode used in the intra prediction process of the current block.

a. In one example, the intra mode of a RDPCM-coded blocks is set to the 2nd MPM candidate (e.g., corresponding to intra_luma_mpm_idx=0)

b. In one example, the intra mode of a RDPCM-coded blocks is set to any MPM candidate other than the first one on a MPM list (e.g., corresponding to intra_luma_mpm_idx=0)

c. In one example, whether and/or how to apply the above methods may be based on

ii. The position of a prediction unit (PU) .

iii. Block dimension of current block and/or its neighboring blocks

iv. Block shape of current block and/or its neighboring blocks

vi. Intra prediction modes of the neighboring blocks of the current block

vii. Motion vectors of the neighboring blocks of the current block

xi. Indication of the color format (such as 4: 2: 0, 4: 4: 4)

xii. Coding tree structure

xiii. Slice/tile group type and/or picture type

5. The position of Planar mode may not always be the first one in an MPM list.

a. In one example, the planar mode may be put to the last position in an MPM list.

b. In one example, the planar mode may be put to any position other than the first one in an MPM list.

c. In one example, the planar mode may be disallowed to put to first position in an MPM list.

d. In one example, whether and/or how to apply the above methods may be based on

ii. The position of a prediction unit (PU) .

iii. Block dimension of current block and/or its neighboring blocks

iv. Block shape of current block and/or its neighboring blocks

vi. Intra prediction modes of the neighboring blocks of the current block

vii. Motion vectors of the neighboring blocks of the current block

xi. Indication of the color format (such as 4: 2: 0, 4: 4: 4)

xii. Coding tree structure

xiii. Slice/tile group type and/or picture type

6. The binarization of the MPM index (e.g., intra_luma_mpm_idx) may not use the truncated unary binarization.

a. In one example, the codeword length of an MPM index (e.g. intra_luma_mpm_idx) may be fixed.

b. The signaling of an MPM index (e.g. intra_luma_mpm_idx) may use the Exp-Golomb binarization with order K.

i. In one example, K is an integer number (e.g. 2) and may be based on

a) The position of a prediction unit (PU) .

b) Block dimension of current block and/or its neighboring blocks

c) Block shape of current block and/or its neighboring blocks

d) Prediction modes (Intra/Inter) of the neighboring blocks of the current block

e) Intra prediction modes of the neighboring blocks of the current block

f) Motion vectors of the neighboring blocks of the current block

g) The indication of QR-BDPCM modes of the neighboring block of the current block

h) The indication of QR-BDPCM modes of transform type (e.g indicated by tu_mts_idx) of the current block

i) Current quantization parameter of current block and/or that of its neighboring blocks

j) Indication of the color format (such as 4: 2: 0, 4: 4: 4)

k) Coding tree structure

l) Slice/tile group type and/or picture type.

7. For coding the indication of an MPM candidate in the list, a first flag may be coded, followed by the remaining index. And the first flag is used to indicate whether the selected intra prediction mode is the same or different from the first MPM candidate in the list.

a. The semantics of intra_luma_not_planar_flag may be interpreted to be whether the intra prediction mode is not the first MPM candidate in the MPM list instead of whether it is not Planar mode.

b. Alternatively, the first flag may be conditionally coded.

i. In one example, the first flag may not be coded if the block is representing screen content.

c. Alternatively, furthermore, the signaling of the remaining MPM index (e.g., intra_luma_mpm_idx) may be further signaled if the selected mode is not the first MPM candidate in the list.

8. The vertical and horizontal intra prediction in the proposed MPM list construction process may correspond to directly copying reference samples in the intra prediction process.

a. In one example, the vertical intra prediction may be generated by directly copying the reference sample in the same column.

b. In one example, the horizontal intra prediction may be generated by directly copying the reference sample in the same row.

c. In one example, the indication of whether to apply the above prediction may be based on

i. Video contents (e.g. screen contents or natural contents)

ii. A message signaled in the SPS/VPS/PPS/picture header/slice header/tile group header/Largest coding unit (LCU) /Coding unit (CU) /LCU row/group of LCUs/

iii. The position of a prediction unit (PU) .

iv. Block dimension of current block and/or its neighboring blocks

v. Block shape of current block and/or its neighboring blocks

vii. Intra prediction modes of the neighboring blocks of the current block

viii. Motion vectors of the neighboring blocks of the current block

ix. The indication of QR-BDPCM modes of the neighboring block of the current block

xii. Indication of the color format (such as 4: 2: 0, 4: 4: 4)

xiii. Separate/dual coding tree structure

xiv. Slice/tile group type and/or picture type

xv. Color component (e.g. may be only applied on chroma components or luma component)

9. The above methods may be also applicable to other intra prediction methods, such as Affine intra prediction method (a.k.a. matrix-based intra prediction method) , intra sub-partition prediction (ISP) , multiple reference line (MRL) etc. al.

FIGS. 13a and 13b show a flowchart of an exemplary method for video processing. Referring to FIG. 13, the method 1300 includes, at step 1302, constructing a first mode list of intra coding modes based on intra modes of neighboring blocks of a current video block. The method 1300 further includes, at step 1304, constructing a second mode list of intra coding modes based on the first mode list. The mode 1300 further includes, at step 1306, performing an intra mode coding for the current video block by using at least one of the first mode list or the second mode list.

In some implementations, wherein the first mode list and the second mode list are MPM (Most Probable Modes) lists. In some implementations, constructing the second mode list includes reordering modes of the first mode list. In some implementations, constructing the second mode list includes replacing one or more modes of the first mode list. In some implementations, the second mode list has a same logic as that of the first mode list. In some implementations, the method further comprises: including an indication whether to allow both of the first mode list and the second mode list in a bitstream representation of the current video block. In some implementations, whether to use the first mode list or the second mode list is determined based on a message in a SPS, VPS, PPS, picture header, slice header, tile group header, largest coding unit (LCU) , coding unit (CU) , LCU row, group of LCUs, or other video units. In some implementations, the performing the intra mode coding includes switching the first mode list to the second mode list or from the second mode list to the first mode list. In some implementations, the switching occurs at a slice, tile, picture, CU, LCU, group of LCUs, or block level. In some implementations, the switching occurs based on at least one of: a. video contents, b. a message signaled in the a SPS, VPS, PPS, picture header, slice header, tile group header, largest coding unit (LCU) , coding unit (CU) , LCU row, group of LCUs, c. a position of a prediction unit (PU) , d. a block dimension of the current video block and/or the neighboring blocks, e. a block shape of the current video block and/or the neighboring blocks, f. prediction modes of the neighboring blocks of the current video block, g. intra prediction modes of the neighboring blocks of the current video block, h. motion vectors of the neighboring blocks of the current video block, i. an indication of QR-BDPCM modes of the neighboring block of the current video block, j. an indication of QR-BDPCM modes of transform type of the current video block, k. a current quantization parameter of the current video block and/or that of the neighboring blocks, l. an indication of a color format, m.a coding tree structure, n. a slice, tile group type, and/or picture type, or o. a color component. In some implementations, each of the first mode list and the second mode list includes a mode index and binarization of the mode index is performed not using a truncated unary binarization. In some implementations, at least one of the first mode list and the second mode list is applied to perform an intra prediction method, affine intra prediction method, intra sub-partition prediction (ISP) , or multiple reference line (MRL) .

Referring to FIG. 13b, the method 1320 includes constructing a mode list of intra coding modes to include candidate modes based on intra modes of neighboring blocks of a current video block, wherein the constructing the mode list includes locating the candidate modes at positions in the mode list based on types of the candidate modes. In some implementations, the locating the candidate modes includes locating a certain angular mode at a given position in the mode list. In some implementations, the locating the candidate modes includes locating vertical and horizontal modes or horizontal and vertical modes in first two positions in the mode list in order. In some implementations, the locating the candidate modes includes locating an intra mode of a RDPCM (Residual Differential Pulse-Code Modulation) to a position other than a first position in the mode list. In some implementations, a prediction signal is generated from the intra mode. In some implementations, another intra mode is used to predict intra modes of successive coding blocks. In some implementations, the locating the candidate modes includes locating a Plannar mode to a position other than a first position in the mode list. In some implementations, the mode list includes a mode index and binarization of the mode index is performed not using a truncated unary binarization. In some implementations, the method further comprises indicating whether a selected intra prediction mode is same or different from a first candidate mode in the mode list. In some implementations, the method further comprises generating an intra prediction sample based on the mode list by copying a reference sample in a same column or in a same row. In some implementations, the mode list is applied to perform an intra prediction method, affine intra prediction method, intra sub-partition prediction (ISP) , or multiple reference line (MRL) .

FIG. 14 is a block diagram of a video processing apparatus 1200. The apparatus 1200 may be used to implement one or more of the methods described herein. The apparatus 1200 may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, and so on. The apparatus 1200 may include one or more processors 1202, one or more memories 1204 and video processing hardware 1206. The processor (s) 1202 may be configured to implement one or more methods (including, but not limited to, methods 1100 and 1150) described in the present document. The memory (memories) 1204 may be used for storing data and code used for implementing the methods and techniques described herein. The video processing hardware 1206 may be used to implement, in hardware circuitry, some techniques described in the present document.

In some embodiments, the video coding methods may be implemented using an apparatus that is implemented on a hardware platform as described with respect to FIG. 14.

FIG. 15 is a flowchart for an example method 1500 of video processing. The method 1500 includes, at 1502, constructing, for a conversion between a current block of video and a bitstream representation of the current block, two or more Most Probable Modes (MPM) lists of intra coding modes for the current block, wherein the two or more MPM lists at least includes a first MPM list constructed with a first construction method and a second MPM list constructed with a second construction method different from the first construction method; and, at 1504, performing the conversion by using one MPM list selected from the constructed two or more MPM lists.

In some examples, the first MPM list is the current MPM list of the current block, and the second MPM list is derived from the first MPM list.

In some examples, the first MPM list is firstly generated, and the second MPM list is generated by reordering those modes in the first MPM list.

In some examples, a Planar mode and/or a DC mode are put to different positions in the first MPM list.

In some examples, the first MPM list is firstly generated, and the second MPM list is generated by replacing one or more modes in the first MPM list.

In some examples, a Planar mode and/or a DC mode are replaced by an angular mode.

In some examples, the angular mode includes at least one of horizontal mode and vertical mode.

In some examples, some modes other than horizontal mode and vertical mode in the first MPM list are replaced by the horizontal mode and/or vertical mode.

In some examples, the second MPM list follows the logic of the first MPM list with some non-angular modes are replaced by angular modes.

In some examples, whether to allow two or more MPM lists is signaled in at least one of video parameter set (VPS) , sequence parameter set (SPS) , picture parameter set (PPS) , picture header, slice header, tile group header, Largest coding unit (LCU) , Coding unit (CU) , LCU row and group of LCUs level.

In some examples, the number of allowed MPM lists is predefined or signaled to decoder.

In some examples, the number of MPM lists is a fixed value.

In some examples, the fixed value is 2.

In some examples, the number of MPM lists is signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row and group of LCUs level.

In some examples, one of the first and second MPM lists is for natural content coding and the other one is for screen content coding.

In some examples, which MPM list is selected depends on one or more of the following:

i. video contents including at least one of screen contents and natural contents;

ii. a message signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row, group of LCUs or other video units;

iii. position of a prediction unit (PU) ;

iv. block dimension of the current block and/or its neighboring blocks;

v. block shape of the current block and/or its neighboring blocks;

vi. prediction modes including Intra mode and Inter mode of the neighboring blocks of the current block;

vii. intra prediction modes of the neighboring blocks of the current block;

viii. motion vectors of the neighboring blocks of the current block;

ix. indication of quantized residual block differential pulse-code modulation (QR-BDPCM) modes of the neighboring block of the current block;

x. indication of QR-BDPCM modes of transform type of the current block;

xi. current quantization parameter of the current block and/or that of its neighboring blocks;

xii. indication of color format of the current block;

xiii. coding tree structure;

xiv. slice, tile group type and/or picture type.

In some examples, a flag is signaled in slice header to indicate a list to be selected for coding conventional intra prediction mode.

In some examples, the MPM list depends on the message which indicates video content information including at least one of screen content or camera captured content.

In some examples, the color format of the current block is 4: 2: 0 or 4: 4: 4.

In some examples, the MPM list is allowed to be switched at slice, tile, picture, CU, LCU, Group of LCUs, or block level.

In some examples, whether to switch the MPM list depends on at least one of the following:

iii. position of a prediction unit (PU) ;

iv. block dimension of the current block and/or its neighboring blocks;

v. block shape of the current block and/or its neighboring blocks;

vii. intra prediction modes of the neighboring blocks of the current block;

viii. motion vectors of the neighboring blocks of the current block;

x. indication of QR-BDPCM modes of transform type of the current block;

xii. indication of color format of the current block;

xiii. coding tree structure;

xiv. slice, tile group type and/or picture type;

xv. color component including at least one of luma component and chroma component.

In some examples, in the construction of an MPM list, a certain angular mode is always in the MPM list.

In some examples, the certain angular mode is the vertical and/or horizontal modes.

In some examples, the certain angular mode is inserted to the MPM list at a given position.

In some examples, the certain angular mode is inserted to the MPM list at a first position.

In some examples, the vertical and horizontal modes are always in the first two positions in the MPM list in order.

In some examples, the horizontal and vertical modes are always in the first two positions in the MPM list in order.

In some examples, the MPM list is always {INTRA_ANGULAR50, INTRA_ANGULAR18, DC, Planar, INTRA_ANGULAR34, INTRA_ANGULAR66} .

In some examples, the MPM list is always {INTRA_ANGULAR18, INTRA_ANGULAR50, Planar, DC, INTRA_ANGULAR34, INTRA_ANGULAR66} .

In some examples, the MPM list is always {INTRA_ANGULAR50, INTRA_ANGULAR18, Planar, DC, INTRA_ANGULAR34, INTRA_ANGULAR66} .

In some examples, the MPM list is always {Planar, DC, INTRA_ANGULAR18, INTRA_ANGULAR50, INTRA_ANGULAR34, INTRA_ANGULAR66} .

In some examples, if both the left neighboring and above neighboring intra modes of the current block are not available, the MPM list is {INTRA_ANGULAR50, INTRA_ANGULAR18, DC, Planar, INTRA_ANGULAR34, INTRA_ANGULAR66} .

In some examples, if both the left neighboring and above neighboring intra modes of the current block are not available, the MPM list is {INTRA_ANGULAR50, INTRA_ANGULAR18, DC, Planar, INTRA_ANGULAR34, INTRA_ANGULAR2} .

In some examples, if the left neighboring and above neighboring intra modes of the current block are same and this mode is a non-angular intra mode, the MPM list is {DC, Planar, INTRA_ANGULAR50, INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR66} .

In some examples, if the left neighboring and above neighboring intra modes of the current block are same and this mode A is an angular intra mode, the MPM list is constructed based on direction of this mode A.

In some examples, if A >= 2 and A <= 18, the MPM list is {INTRA_ANGULAR18, INTRA_ANGULAR2, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR66} .

In some examples, if A >= 18 and A <= 34, the MPM list is {INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR2} .

In some examples, if A >= 34 and A <= 50, the MPM list is {INTRA_ANGULAR50, INTRA_ANGULAR34, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR66} .

In some examples, if A >= 50 and A <= 66, the MPM list is {INTRA_ANGULAR50, INTRA_ANGULAR66, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR34} .

In some examples, if the left neighboring and above neighboring intra modes of the current block are different modes and one of them is non-angular mode, the MPM list is {DC, Planar, INTRA_ANGULAR50, INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR66} .

In some examples, the left neighboring and above neighboring intra modes of the current block are different modes and both of them are angular modes, where Max denotes the maximal value between these two intra modes.

In some examples, if Max >= 2 and Max <= 18, the MPM list is {INTRA_ANGULAR18, INTRA_ANGULAR2, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR66} .

In some examples, if Max >= 18 and Max <= 34, the MPM list may be {INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR2} .

In some examples, if Max >= 34 and Max <= 50, the MPM list may be {INTRA_ANGULAR50, INTRA_ANGULAR34, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR66} .

In some examples, if Max >= 50 and Max <= 66, the MPM list may be {INTRA_ANGULAR50, INTRA_ANGULAR66, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR34} ,

In some examples, one existing mode in the MPM list are replaced by the vertical mode and/or horizontal mode.

In some examples, one existing mode in the MPM list which has a closest direction with vertical mode is replaced by the vertical mode.

In some examples, one existing mode in the MPM list which has a closest direction with horizontal mode may be replaced by the horizontal mode.

In some examples, position of the vertical and/or horizontal modes in the MPM list depends on at least one of the following:

i. a message signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row, group of LCUs or other video units;

ii. position of a prediction unit (PU) ;

iii. block dimension of the current block and/or its neighboring blocks;

iv. block shape of the current block and/or its neighboring blocks;

v. prediction modes including Intra mode and Inter mode of the neighboring blocks of the current block;

vi. intra prediction modes of the neighboring blocks of the current block;

vii. motion vectors of the neighboring blocks of the current block;

viii. indication of quantized residual block differential pulse-code modulation (QR-BDPCM) modes of the neighboring block of the current block;

ix. indication of QR-BDPCM modes of transform type of the current block;

x. current quantization parameter of the current block and/or that of its neighboring blocks;

xi. indication of color format of the current block;

xii. coding tree structure;

xiii. slice, tile group type and/or picture type.

In some examples, when the current block is a residual differential pulse coded modulation (RDPCM) coded block, intra mode of the RDPCM coded block is set to a mode other than a first mode in the MPM list.

In some examples, the intra mode of the RDPCM-coded block is set to a second MPM mode in the MPM list with an indication that intra_luma_mpm_idx=0.

In some examples, the intra mode of the RDPCM-coded block is set to any mode other than the first mode in the MPM list with an indication that intra_luma_mpm_idx=0.

In some examples, whether and/or how to set the intra mode of the RDPCM-coded blocks depends on at least one of the following:

i. a message signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row, group of LCUs or other

video units;

ii. position of a prediction unit (PU) ;

iii. block dimension of the current block and/or its neighboring blocks;

iv. block shape of the current block and/or its neighboring blocks;

vi. intra prediction modes of the neighboring blocks of the current block;

vii. motion vectors of the neighboring blocks of the current block;

ix. indication of QR-BDPCM modes of transform type of the current block;

xi. indication of color format of the current block;

xii. coding tree structure;

xiii. slice, tile group type and/or picture type.

In some examples, position of a Planar mode is not always be the first position in the MPM list.

In some examples, the Planar mode is put to the last position in the MPM list.

In some examples, the Planar mode is put to any position other than the first position in the MPM list.

In some examples, the Planar mode is disallowed to put to the first position in the MPM list.

In some examples, whether and/or how to set the position of the Planar mode depends on at least one of the following:

ii. position of a prediction unit (PU) ;

iii. block dimension of the current block and/or its neighboring blocks;

iv. block shape of the current block and/or its neighboring blocks;

vi. intra prediction modes of the neighboring blocks of the current block;

vii. motion vectors of the neighboring blocks of the current block;

ix. indication of QR-BDPCM modes of transform type of the current block;

xi. indication of color format of the current block;

xii. coding tree structure;

xiii. slice, tile group type and/or picture type.

In some examples, binarization of MPM index in the MPM list does not use truncated unary binarization.

In some examples, the MPM index is intra_luma_mpm_idx.

In some examples, codeword length of the MPM index is fixed.

In some examples, signaling of the MPM index uses Exp-Golomb binarization with order K, K being an integer.

In some examples, K is 2.

In some examples, K depends on at least one of the following:

i. position of a prediction unit (PU) ;

ii. block dimension of the current block and/or its neighboring blocks;

iii. block shape of the current block and/or its neighboring blocks;

iv. prediction modes including Intra mode and Inter mode of the neighboring blocks of the current block;

v. intra prediction modes of the neighboring blocks of the current block;

vi. motion vectors of the neighboring blocks of the current block;

vii. indication of quantized residual block differential pulse-code modulation (QR-BDPCM) modes of the neighboring block of the current block;

viii. indication of QR-BDPCM modes of transform type of the current block;

ix. current quantization parameter of the current block and/or that of its neighboring blocks;

x. indication of color format of the current block;

xi. coding tree structure;

xii. slice, tile group type and/or picture type.

In some examples, indication of a MPM candidate in the MPM list is coded, where a first flag is coded, followed by remaining MPM index of the MPM candidate, the first flag being used to indicate whether a selected intra prediction mode is the same or different from a first MPM candidate in the MPM list.

In some examples, the first flag is intra_luma_not_planar_flag, where semantics of intra_luma_not_planar_flag is interpreted to be whether the selected intra prediction mode is not the first MPM candidate in the MPM list instead of whether it is not Planar mode.

In some examples, the first flag is conditionally coded.

In some examples, the first flag does not be coded if the current block is with screen content.

In some examples, the remaining MPM index is further signaled if the selected intra prediction mode is not the first MPM candidate in the MPM list.

In some examples, the MPM list construction process includes a vertical intra prediction and/or a horizontal intra prediction, which correspond to directly copying reference samples in an intra prediction process.

In some examples, the vertical intra prediction is generated by directly copying the reference sample in the same column.

In some examples, the horizontal intra prediction is generated by directly copying the reference sample in the same row.

In some examples, whether to apply the vertical intra prediction and/or the horizontal intra prediction depends on at least one of the following:

iii. position of a prediction unit (PU) ;

iv. block dimension of the current block and/or its neighboring blocks;

v. block shape of the current block and/or its neighboring blocks;

vii. intra prediction modes of the neighboring blocks of the current block;

viii. motion vectors of the neighboring blocks of the current block;

x. indication of QR-BDPCM modes of transform type of the current block;

xii. indication of color format of the current block;

xiii. coding tree structure;

xiv. slice, tile group type and/or picture type;

In some examples, the first construction method and/or the second construction method further include at least one of Affine intra prediction method, matrix-based intra prediction method, intra sub-partition prediction (ISP) and multiple reference line (MRL) .

In some examples, the color format of the current block is 4: 2: 0 or 4: 4: 4.

In some examples, the conversion generates the current block of video from the bitstream representation.

In some examples, the conversion generates the bitstream representation from the current block of video.

From the foregoing, it will be appreciated that specific embodiments of the presently disclosed technology have been described herein for purposes of illustration, but that various modifications may be made without deviating from the scope of the disclosed technology. Accordingly, the presently disclosed technology is not limited except as by the appended claims.

Implementations of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term “data processing unit” or “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document) , in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code) . A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit) .

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

It is intended that the specification, together with the drawings, be considered exemplary only, where exemplary means an example. As used herein, the singular forms “a” , “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Additionally, the use of “or” is intended to include “and/or” , unless the context clearly indicates otherwise.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

A method for processing video, comprising:

constructing, for a conversion between a current block of video and a bitstream representation of the current block, two or more Most Probable Modes (MPM) lists of intra coding modes for the current block, wherein the two or more MPM lists at least includes a first MPM list constructed with a first construction method and a second MPM list constructed with a second construction method different from the first construction method; and

performing the conversion by using one MPM list selected from the constructed two or more MPM lists.
The method of claim 1, wherein the first MPM list is the current MPM list of the current block, and the second MPM list is derived from the first MPM list.
The method of claim 2, wherein the first MPM list is firstly generated, and the second

MPM list is generated by reordering those modes in the first MPM list.
The method of claim 3, wherein a Planar mode and/or a DC mode are put to different positions in the first MPM list.
The method of claim 2, wherein the first MPM list is firstly generated, and the second MPM list is generated by replacing one or more modes in the first MPM list.
The method of claim 5, wherein a Planar mode and/or a DC mode are replaced by an angular mode.
The method of claim 6, wherein the angular mode includes at least one of horizontal mode and vertical mode.
The method of claim 5, wherein some modes other than horizontal mode and vertical mode in the first MPM list are replaced by the horizontal mode and/or vertical mode.
The method of claim 1, wherein the second MPM list follows the logic of the first MPM list with some non-angular modes are replaced by angular modes.
The method of any of claims 1-9, wherein whether to allow two or more MPM lists is signaled in at least one of video parameter set (VPS) , sequence parameter set (SPS) , picture parameter set (PPS) , picture header, slice header, tile group header, Largest coding unit (LCU) , Coding unit (CU) , LCU row and group of LCUs level.
The method of any of claims 1-10, wherein the number of allowed MPM lists is predefined or signaled to decoder.
The method of claim 11, wherein the number of MPM lists is a fixed value.
The method of claim 12, wherein the fixed value is 2.
The method of claim 11, wherein the number of MPM lists is signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row and group of LCUs level.
The method of any of claims 1-14, wherein one of the first and second MPM lists is for natural content coding and the other one is for screen content coding.
The method of any of claims 1-15, wherein which MPM list is selected depends on one or more of the following:

i. video contents including at least one of screen contents and natural contents;

ii. a message signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row, group of LCUs or other video units;

iii. position of a prediction unit (PU) ;

iv. block dimension of the current block and/or its neighboring blocks;

v. block shape of the current block and/or its neighboring blocks;

vi. prediction modes including Intra mode and Inter mode of the neighboring blocks of the current block;

vii. intra prediction modes of the neighboring blocks of the current block;

viii. motion vectors of the neighboring blocks of the current block;

ix. indication of quantized residual block differential pulse-code modulation (QR-BDPCM) modes of the neighboring block of the current block;

x. indication of QR-BDPCM modes of transform type of the current block;

xi. current quantization parameter of the current block and/or that of its neighboring blocks;

xii. indication of color format of the current block;

xiii. coding tree structure;

xiv. slice, tile group type and/or picture type.
The method of claim 16, wherein a flag is signaled in slice header to indicate a list to be selected for coding conventional intra prediction mode.
The method of claim 16, wherein the MPM list depends on the message which indicates video content information including at least one of screen content or camera captured content.
The method of claim 16, wherein the color format of the current block is 4: 2: 0 or 4: 4: 4.
The method of any of claims 1-19, wherein the MPM list is allowed to be switched at slice, tile, picture, CU, LCU, Group of LCUs, or block level.
The method of claim 20, wherein whether to switch the MPM list depends on at least one of the following:

xvi. video contents including at least one of screen contents and natural contents;

xvii. a message signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row, group of LCUs or other video units;

xviii. position of a prediction unit (PU) ;

xix. block dimension of the current block and/or its neighboring blocks;

xx. block shape of the current block and/or its neighboring blocks;

xxi. prediction modes including Intra mode and Inter mode of the neighboring blocks of the current block;

xxii. intra prediction modes of the neighboring blocks of the current block;

xxiii. motion vectors of the neighboring blocks of the current block;

xxiv. indication of quantized residual block differential pulse-code modulation (QR-BDPCM) modes of the neighboring block of the current block;

xxv. indication of QR-BDPCM modes of transform type of the current block;

xxvi. current quantization parameter of the current block and/or that of its neighboring blocks;

xxvii. indication of color format of the current block;

xxviii. coding tree structure;

xxix. slice, tile group type and/or picture type;

xxx. color component including at least one of luma component and chroma component.
The method of any of claims 1-21, wherein in the construction of an MPM list, a certain angular mode is always in the MPM list.
The method of claim 22, wherein the certain angular mode is the vertical and/or horizontal modes.
The method of claim 22 or 23, wherein the certain angular mode is inserted to the MPM list at a given position.
The method of claim 24, wherein the certain angular mode is inserted to the MPM list at a first position.
The method of claim 23, wherein the vertical and horizontal modes are always in the first two positions in the MPM list in order.
The method of claim 23, wherein the horizontal and vertical modes are always in the first two positions in the MPM list in order.
The method of claim 22, wherein the MPM list is always {INTRA_ANGULAR50, INTRA_ANGULAR18, DC, Planar, INTRA_ANGULAR34, INTRA_ANGULAR66} .
The method of claim 22, wherein the MPM list is always {INTRA_ANGULAR18, INTRA_ANGULAR50, Planar, DC, INTRA_ANGULAR34, INTRA_ANGULAR66} .
The method of claim 22, wherein the MPM list is always {INTRA_ANGULAR50, INTRA_ANGULAR18, Planar, DC, INTRA_ANGULAR34, INTRA_ANGULAR66} .
The method of claim 22, wherein the MPM list is always {Planar, DC, INTRA_ANGULAR18, INTRA_ANGULAR50, INTRA_ANGULAR34, INTRA_ANGULAR66} .
The method of claim 22, wherein if both the left neighboring and above neighboring intra modes of the current block are not available, the MPM list is {INTRA_ANGULAR50, INTRA_ANGULAR18, DC, Planar, INTRA_ANGULAR34, INTRA_ANGULAR66} .
The method of claim 22, wherein if both the left neighboring and above neighboring intra modes of the current block are not available, the MPM list is {INTRA_ANGULAR50, INTRA_ANGULAR18, DC, Planar, INTRA_ANGULAR34, INTRA_ANGULAR2} .
The method of claim 22, wherein if the left neighboring and above neighboring intra modes of the current block are same and this mode is a non-angular intra mode, the MPM list is {DC, Planar, INTRA_ANGULAR50, INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR66} .
The method of claim 22, wherein if the left neighboring and above neighboring intra modes of the current block are same and this mode A is an angular intra mode, the MPM list is constructed based on direction of this mode A.
The method of claim 35, wherein if A >= 2 and A <= 18, the MPM list is {INTRA_ANGULAR18, INTRA_ANGULAR2, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR66} .
The method of claim 35, wherein if A >= 18 and A <= 34, the MPM list is {INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR2} .
The method of claim 35, wherein if A >= 34 and A <= 50, the MPM list is {INTRA_ANGULAR50, INTRA_ANGULAR34, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR66} .
The method of claim 35, wherein if A >= 50 and A <= 66, the MPM list is {INTRA_ANGULAR50, INTRA_ANGULAR66, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR34} .
The method of claim 22, wherein if the left neighboring and above neighboring intra modes of the current block are different modes and one of them is non-angular mode, the MPM list is {DC, Planar, INTRA_ANGULAR50, INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR66} .
The method of claim 22, wherein the left neighboring and above neighboring intra modes of the current block are different modes and both of them are angular modes, where Max denotes the maximal value between these two intra modes.
The method of claim 41, wherein if Max >= 2 and Max <= 18, the MPM list is {INTRA_ANGULAR18, INTRA_ANGULAR2, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR66} .
The method of claim 41, wherein if Max >= 18 and Max <= 34, the MPM list may be {INTRA_ANGULAR18, INTRA_ANGULAR34, INTRA_ANGULAR50, DC, PLANAR, INTRA_ANGULAR2} .
The method of claim 41, wherein if Max >= 34 and Max <= 50, the MPM list may be {INTRA_ANGULAR50, INTRA_ANGULAR34, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR66} .
The method of claim 41, wherein if Max >= 50 and Max <= 66, the MPM list may be {INTRA_ANGULAR50, INTRA_ANGULAR66, INTRA_ANGULAR18, DC, PLANAR, INTRA_ANGULAR34} .
The method of claim 23, wherein one existing mode in the MPM list are replaced by the vertical mode and/or horizontal mode.
The method of claim 46, wherein one existing mode in the MPM list which has a closest direction with vertical mode is replaced by the vertical mode.
The method of claim 46, wherein one existing mode in the MPM list which has a closest direction with horizontal mode may be replaced by the horizontal mode.
The method of claim 23, wherein position of the vertical and/or horizontal modes in the MPM list depends on at least one of the following:

xiv. a message signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row, group of LCUs or other video units;

xv. position of a prediction unit (PU) ;

xvi. block dimension of the current block and/or its neighboring blocks;

xvii. block shape of the current block and/or its neighboring blocks;

xviii. prediction modes including Intra mode and Inter mode of the neighboring blocks of the current block;

xix. intra prediction modes of the neighboring blocks of the current block;

xx. motion vectors of the neighboring blocks of the current block;

xxi. indication of quantized residual block differential pulse-code modulation (QR-BDPCM) modes of the neighboring block of the current block;

xxii. indication of QR-BDPCM modes of transform type of the current block;

xxiii. current quantization parameter of the current block and/or that of its neighboring blocks;

xxiv. indication of color format of the current block;

xxv. coding tree structure;

xxvi. slice, tile group type and/or picture type.
The method of any of claims 1-49, wherein when the current block is a residual differential pulse coded modulation (RDPCM) coded block, intra mode of the RDPCM coded block is set to a mode other than a first mode in the MPM list.
The method of claim 50, wherein the intra mode of the RDPCM-coded block is set to a second MPM mode in the MPM list with an indication that intra_luma_mpm_idx=0.
The method of claim 50, wherein the intra mode of the RDPCM-coded block is set to any mode other than the first mode in the MPM list with an indication that intra_luma_mpm_idx=0.
The method of any of claims 50-52, wherein whether and/or how to set the intra mode of the RDPCM-coded blocks depends on at least one of the following:

xiv. a message signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row, group of LCUs or other video units;

xv. position of a prediction unit (PU) ;

xvi. block dimension of the current block and/or its neighboring blocks;

xvii. block shape of the current block and/or its neighboring blocks;

xviii. prediction modes including Intra mode and Inter mode of the neighboring blocks of the current block;

xix. intra prediction modes of the neighboring blocks of the current block;

xx. motion vectors of the neighboring blocks of the current block;

xxi. indication of quantized residual block differential pulse-code modulation (QR-BDPCM) modes of the neighboring block of the current block;

xxii. indication of QR-BDPCM modes of transform type of the current block;

xxiii. current quantization parameter of the current block and/or that of its neighboring blocks;

xxiv. indication of color format of the current block;

xxv. coding tree structure;

xxvi. slice, tile group type and/or picture type.
The method of any of claims 1-49, wherein position of a Planar mode is not always be the first position in the MPM list.
The method of claim 54, wherein the Planar mode is put to the last position in the MPM list.
The method of claim 54, wherein the Planar mode is put to any position other than the first position in the MPM list.
The method of claim 54, wherein the Planar mode is disallowed to put to the first position in the MPM list.
The method of any of claims 54-57, wherein whether and/or how to set the position of the Planar mode depends on at least one of the following:

xiv. a message signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row, group of LCUs or other video units;

xv. position of a prediction unit (PU) ;

xvi. block dimension of the current block and/or its neighboring blocks;

xvii. block shape of the current block and/or its neighboring blocks;

xviii. prediction modes including Intra mode and Inter mode of the neighboring blocks of the current block;

xix. intra prediction modes of the neighboring blocks of the current block;

xx. motion vectors of the neighboring blocks of the current block;

xxi. indication of quantized residual block differential pulse-code modulation (QR-BDPCM) modes of the neighboring block of the current block;

xxii. indication of QR-BDPCM modes of transform type of the current block;

xxiii. current quantization parameter of the current block and/or that of its neighboring blocks;

xxiv. indication of color format of the current block;

xxv. coding tree structure;

xxvi. slice, tile group type and/or picture type.
The method of any of claims 1-58, wherein binarization of MPM index in the MPM list does not use truncated unary binarization.
The method of claim 59, wherein the MPM index is intra_luma_mpm_idx.
The method of claim 59 or 60, wherein codeword length of the MPM index is fixed.
The method of claim 59 or 60, wherein signaling of the MPM index uses Exp-Golomb binarization with order K, K being an integer.
The method of claim 62, wherein K is 2.
The method of claim 62, wherein K depends on at least one of the following:

xiii. position of a prediction unit (PU) ;

xiv. block dimension of the current block and/or its neighboring blocks;

xv. block shape of the current block and/or its neighboring blocks;

xvi. prediction modes including Intra mode and Inter mode of the neighboring blocks of the current block;

xvii. intra prediction modes of the neighboring blocks of the current block;

xviii. motion vectors of the neighboring blocks of the current block;

xix. indication of quantized residual block differential pulse-code modulation (QR-BDPCM) modes of the neighboring block of the current block;

xx. indication of QR-BDPCM modes of transform type of the current block;

xxi. current quantization parameter of the current block and/or that of its neighboring blocks;

xxii. indication of color format of the current block;

xxiii. coding tree structure;

xxiv. slice, tile group type and/or picture type.
The method of any of claims 1-64, wherein indication of a MPM candidate in the MPM list is coded, where a first flag is coded, followed by remaining MPM index of the MPM candidate, the first flag being used to indicate whether a selected intra prediction mode is the same or different from a first MPM candidate in the MPM list.
The method of claim 65, wherein the first flag is intra_luma_not_planar_flag, where semantics of intra_luma_not_planar_flag is interpreted to be whether the selected intra prediction mode is not the first MPM candidate in the MPM list instead of whether it is not Planar mode.
The method of claim 65 or 66, wherein the first flag is conditionally coded.
The method of claim 67, wherein the first flag does not be coded if the current block is with screen content.
The method of claim 67, wherein the remaining MPM index is further signaled if the selected intra prediction mode is not the first MPM candidate in the MPM list.
The method of any of claims 1-69, wherein the MPM list construction process includes a vertical intra prediction and/or a horizontal intra prediction, which correspond to directly copying reference samples in an intra prediction process.
The method of claim 70, wherein the vertical intra prediction is generated by directly copying the reference sample in the same column.
The method of claim 70, wherein the horizontal intra prediction is generated by directly copying the reference sample in the same row.
The method of any of claims 70-72, wherein whether to apply the vertical intra prediction and/or the horizontal intra prediction depends on at least one of the following:

xvi. video contents including at least one of screen contents and natural contents;

xvii. a message signaled in at least one of VPS, SPS, PPS, picture header, slice header, tile group header, LCU, CU, LCU row, group of LCUs or other video units;

xviii. position of a prediction unit (PU) ;

xix. block dimension of the current block and/or its neighboring blocks;

xx. block shape of the current block and/or its neighboring blocks;

xxi. prediction modes including Intra mode and Inter mode of the neighboring blocks of the current block;

xxii. intra prediction modes of the neighboring blocks of the current block;

xxiii. motion vectors of the neighboring blocks of the current block;

xxiv. indication of quantized residual block differential pulse-code modulation (QR-BDPCM) modes of the neighboring block of the current block;

xxv. indication of QR-BDPCM modes of transform type of the current block;

xxvi. current quantization parameter of the current block and/or that of its neighboring blocks;

xxvii. indication of color format of the current block;

xxviii. coding tree structure;

xxix. slice, tile group type and/or picture type;

xxx. color component including at least one of luma component and chroma component.
The method of any of claims 1-73, wherein the first construction method and/or the second construction method further include at least one of Affine intra prediction method, matrix-based intra prediction method, intra sub-partition prediction (ISP) and multiple reference line (MRL) .
The method of any of claims 21, 49, 53, 58, 64, 73, wherein the color format of the current block is 4: 2: 0 or 4: 4: 4.
The method of any of claims 1-75, wherein the conversion generates the current block of video from the bitstream representation.
The method of anyone of claims 1-75, wherein the conversion generates the bitstream representation from the current block of video.
An apparatus in a video system comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to implement the method in any one of claims 1 to 77.
A computer program product stored on a non-transitory computer readable media, the computer program product including program code for carrying out the method in any one of claims 1 to 77.