WO2025218784A1

WO2025218784A1 - Intra merge mode with multiple lists

Info

Publication number: WO2025218784A1
Application number: PCT/CN2025/089811
Authority: WO
Inventors: Man-Shu CHIANG; Hsin-Yi Tseng; Cheng-Yen Chuang; Chih-Wei Hsu; Ching-Yeh Chen; Tzu-Der Chuang
Original assignee: MediaTek Inc
Current assignee: MediaTek Inc
Priority date: 2024-04-19
Filing date: 2025-04-18
Publication date: 2025-10-23
Anticipated expiration: 2026-10-19

Abstract

A method for coding a pixel block using intra merge mode with one or more merge candidate lists or sets is provided. A video coder constructs one or more merge candidate lists for the current block. Each merge candidate corresponds to a set of coding information that is inherited from a previously coded block. Each merge candidate list is associated with only one mode category. Each merge candidate in a merge candidate list or set associated with a mode category corresponds to a set of coding information for a coding mode belonging to the mode category. The video coder selects a merge candidate from the constructed merge candidate lists. The video coder generates a prediction of the current block according to a target mode using the set of coding information corresponding to the selected merge candidate. The video coder encodes or decodes the current blockby using the generated prediction.

Description

INTRA MERGE MODE WITH MULTIPLE LISTS

CROSS REFERENCE TO RELATED PATENT APPLICATION (S)

The present disclosure is part of a non-provisional application that claims the priority benefit of U.S. Provisional Patent Application No. 63/636,170, filed on 19 April 2024. Content of above-listed application is herein incorporated by reference.

TECHNICAL FIELD

The present disclosure relates generally to video coding. In particular, the present disclosure relates to methods of coding pixel blocks by intra coding and merge mode.

BACKGROUND

Unless otherwise indicated herein, approaches described in this section are not prior art to the claims listed below and are not admitted as prior art by inclusion in this section.

High-Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC) . HEVC is based on the hybrid block-based motion-compensated DCT-like transform coding architecture. The basic unit for compression, termed coding unit (CU) , is a 2Nx2N square block of pixels, and each CU can be recursively split into four smaller CUs until the predefined minimum size is reached. Each CU contains one or multiple prediction units (PUs) .

Versatile video coding (VVC) is the latest international video coding standard developed by the Joint Video Expert Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11. The input video signal is predicted from the reconstructed signal, which is derived from the coded picture regions. The prediction residual signal is processed by a block transform. The transform coefficients are quantized and entropy coded together with other side information in the bitstream. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal after inverse transform on the de-quantized transform coefficients. The reconstructed signal is further processed by in-loop filtering for removing coding artifacts. The decoded pictures are stored in the frame buffer for predicting the future pictures in the input video signal.

In VVC, a coded picture is partitioned into non-overlapped square block regions represented by the associated coding tree units (CTUs) . The leaf nodes of a coding tree correspond to the coding units (CUs) . A coded picture can be represented by a collection of slices, each comprising an integer number of CTUs. The individual CTUs in a slice are processed in raster-scan order. A bi-predictive (B) slice may be decoded using intra prediction or inter prediction with at most two motion vectors (MVs) and reference indices to predict the sample values of each block. A predictive (P) slice is decoded using intra prediction or inter prediction with at most one motion vector and reference index to predict the sample values of each block. An intra (I) slice is decoded using intra prediction only.

Each CU contains one or more prediction units (PUs) . The prediction unit, together with the associated CU syntax, works as a basic unit for signaling the predictor information. The specified prediction process is employed to predict the values of the associated pixel samples inside the PU.

For each inter-predicted CU, motion parameters consisting of motion vectors, reference picture indices and reference picture list usage index, and additional information are used for inter-predicted sample generation. The motion parameter can be signaled in an explicit or implicit manner. When a CU is coded with skip mode, the CU is associated with one PU and has no significant residual coefficients, no coded motion vector delta or reference picture index. A merge mode is specified whereby the motion parameters for the current CU are obtained from neighbouring CUs, including spatial and temporal candidates, and additional schedules introduced in VVC. The merge mode can be applied to any inter-predicted CU. The alternative to merge mode is the explicit transmission of motion parameters, where motion vector, corresponding reference picture index for each reference picture list and reference picture list usage flag and other needed information are signaled explicitly per each CU.

SUMMARY

The following summary is illustrative only and is not intended to be limiting in any way. That is, the following summary is provided to introduce concepts, highlights, benefits and advantages of the novel and non-obvious techniques described herein. Select and not all implementations are further described below in the detailed description. Thus, the following summary is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.

Some embodiments, of the disclosure provide a method for coding a pixel block using intra merge mode with one or more merge candidate lists or sets. A video coder constructs at least one or more merge candidate lists or sets for the current block. Each merge candidate corresponds to a set of coding information that is inherited from a previously coded block. Each merge candidate list or set is associated with only one mode category. Each merge candidate in a merge candidate list associated with a mode category corresponds to a set of coding information for a coding mode belonging to the mode category. The video coder selects a merge candidate from the constructed merge candidate lists. The video coder generates a prediction of the current block according to a target mode using the set of coding information corresponding to the selected merge candidate. The video coder encodes or decodes the current block by using the generated prediction.

In some embodiments, different merge candidate lists or sets may be associated with different mode categories.

In some embodiments, a merge candidate list or set may be associated with a mode category for only modes related to Decoder-Side Intra Mode Derivation (DIMD) . The modes related to DIMD may include DIMD, Merged Intra Mode Derivation (MIMD) , occurrence-based intra coding (OBIC) , decoder-derived intra prediction (DIP) merge mode with a mode type being DIMD, any mode using histograms to determine the intra prediction modes for the current block, any mode enabled by DIMD enabling flag, or any subset or extension of above-mentioned modes.

In some embodiments, a merge candidate list or set may be associated with a mode category for only modes related to Template-Based Intra Mode Derivation (TIMD) . The modes related to TIMD may include TIMD, Template-based multiple reference line intra prediction (TMRL) , DIP merge with a mode type being TIMD or TMRL, any mode using distortion between reconstruction and prediction on the template to determine the intra prediction modes for the current block, any mode enabled by TIMD enabling flag, any mode enabled by TMRL enabling flag, or any subset or extension of above-mentioned modes.

In some embodiments, a merge candidate list or set may be associated with a mode category for only multiple reference line (MRL) related modes. The MRL-related modes may include any of MRL, TMRL, DIP merge with a mode type being MRL or TMRL, any mode enabled by MRL enabling flag, any mode enabled by TMRL enabling flag, any mode using the reference line non-adjacent to the current block, or any subset or extension of above-mentioned modes.

In some embodiments, a merge candidate list or set may be associated with a mode category for only model-related modes. The model-related modes may include Extrapolation filter-based intra prediction (EIP) , DIP merge with a mode type being EIP, matrix-related mode, parameter-related mode, any mode using model coefficients with input reference samples to derive the prediction for the current block, any mode enabled by an EIP enabling flag, or any subset or extension of above-mentioned modes.

In some embodiments, a merge candidate list or set may be associated with a mode category for only matrix-related modes. The matrix-related modes may include matrix-based or matrix-weighted intra prediction (MIP) , DIP merge with a mode type being MIP, matrix-related mode, parameter-related mode, any mode enabled by a MIP enabling flag, any mode using a pre-defined or selected matrix with input reference samples to derive the prediction for the current block, or any subset or extension of above-mentioned modes.

In some embodiments, a merge candidate list or set may be associated with a mode category for only decoder-analysis-related modes. The decoder-analysis-related modes may include DIMD-related modes, TIMD-related modes, any mode using decoder-side analysis to determine the intra prediction mode for the current block, or any subset or extension of above-mentioned modes. In some embodiments, the decoder-side analysis may include a texture gradient analysis on a template or a distortion measurement on the template.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide a further understanding of the present disclosure, and are incorporated in and constitute a part of the present disclosure. The drawings illustrate implementations of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. It is appreciable that the drawings are not necessarily in scale as some components may be shown to be out of proportion than the size in actual implementation in order to clearly illustrate the concept of the present disclosure.

FIG. 1 illustrates 67 intra prediction modes, including 65 directional or angular intra prediction modes.

FIG. 2 conceptually illustrates Histogram of Gradient computation from a template from which two intra prediction modes are derived.

FIGS. 3A-C illustrate three Extrapolation filter-based intra prediction (EIP) filter shapes.

FIGS. 4A-C illustrate three types of reconstructed area for EIP.

FIG. 5 illustrates a template area and reference lines for computing Template-based multiple reference line intra prediction (TMRL) costs.

FIG. 6 shows an example of the adjacent neighboring blocks of the current block.

FIG. 7 illustrates an example video encoder that may implement intra merge mode.

FIG. 8 illustrates portions of the video encoder that implement intra merge mode with at least one or multiple merge candidate lists or sets.

FIG. 9 conceptually illustrates a process for encoding a block of pixels using intra merge mode with one or multiple merge candidate lists.

FIG. 10 illustrates an example video decoder that may implement intra merge mode.

FIG. 11 illustrates portions of the video decoder that implement intra merge mode with at least one or multiple merge candidate lists or sets.

FIG. 12 conceptually illustrates a process for decoding a block of pixels using intra merge mode with one or multiple merge candidate lists.

FIG. 13 conceptually illustrates an electronic system with which some embodiments of the present disclosure are implemented.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant teachings. Any variations, derivatives and/or extensions based on teachings described herein are within the protective scope of the present disclosure. In some instances, well-known methods, procedures, components, and/or circuitry pertaining to one or more example implementations disclosed herein may be described at a relatively high level without detail, in order to avoid unnecessarily obscuring aspects of teachings of the present disclosure.
I. Intra Prediction

A. Intra Prediction Modes and MPMs

Intra-prediction method exploits one or more reference lines adjacent to the current prediction unit (PU) and one of the intra-prediction modes to generate the predictors for the current PU.The Intra-prediction mode can be chosen among a mode set containing multiple prediction directions, DC mode, and Planar mode. The number of directional intra modes may be 33 or extended to 65 direction modes. By including DC and Planar modes, the number of intra-prediction modes is 35 or extended to 67. FIG. 1 illustrates 67 intra prediction modes, including 65 directional or angular intra prediction modes (from 2 to 66) .

For some embodiments, several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for the non-square blocks.

Some intra-prediction modes may be identified as a set of most probable modes (MPM) for intra-prediction in current prediction block so an index may be signaled to select one of the MPMs. In some embodiments, a MPM list is generated by considering two available neighboring intra modes, intra modes used by neighboring blocks.

B. Decoder-Side Intra Mode Derivation (DIMD)

Decoder-Side Intra Mode Derivation (DIMD) is a technique in which one or more (e.g. 2) intra prediction modes such as angles or directions are derived from the reconstructed neighbor samples (template) of a block, and those two predictors are combined with the non-angular predictor such as planar mode predictor with the weights derived from the gradients. To implicitly derive the intra prediction modes of a block, a texture gradient analysis is performed at both encoder and decoder sides. This process starts with an empty Histogram of Gradient (HoG) having 65 entries, corresponding to the 65 angular/directional intra prediction modes. Amplitudes of these entries are determined during the texture gradient analysis. FIG. 2 conceptually illustrates Histogram of Gradient computation from a template with 3 pixels, from which two intra prediction modes (IPMs) M₁ and M₂ are derived.

In some embodiments, when DIMD is applied, up to five intra modes are derived from the reconstructed neighbor samples, and those five predictors are combined with the planar mode predictor with the weights derived from the histogram of gradients.

C. Template-based Intra mode Derivation (TIMD)

For mode selection, template matching method can be applied by computing the cost between reconstructed samples and predicting samples on the template. One of the examples is template-based intra mode derivation (TIMD) . TIMD is a coding method in which the intra prediction mode of a CU is implicitly derived by using a neighboring template at both encoder and decoder, instead of the encoder signaling the exact intra prediction mode to the decoder.

The prediction samples of the template are generated using the reference samples of the template for each candidate mode. SATD between the prediction and reconstruction samples of the template is calculated as cost. First two intra prediction modes with the minimum SATD are selected as the TIMD modes. These two TIMD modes are fused with the weights to generate prediction for the current CU.

D. Extrapolation filter-based intra prediction (EIP) mode

In the Extrapolation filter-based intra prediction (EIP) mode, the samples in a CU are predicted from the top-left position to the bottom-right position by applying an extrapolation filter to neighboring reconstructed samples or predicted samples. The EIP mode uses a 15-tap filter for prediction as below:

where pred _(x, y) is the predicted value at position (x, y) in the CU, c_i is the filter coefficient, and the t _{(x-offsetXi, y-offsetYi)} is the reconstructed samples or predicted samples. The EIP filter can be derived from the neighboring reconstructed samples or be inherited from the previous EIP coded blocks. FIGS. 3A-C illustrate three EIP filter shapes. FIGS. 4A-C illustrate three types of reconstructed area for EIP.

For a CU coded in the EIP mode, an EIP merge flag is signaled to indicate whether the EIP filter is inherited from previous blocks coded in EIP mode. When the EIP merge flag is true, an EIP merge list is constructed from the spatial adjacent, spatial non-adjacent, temporal and history candidates. An EIP merge index is further signaled to indicate which EIP merge candidate is selected. The filter shape and the filter coefficients of the selected candidate are then inherited to code the CU.

When the EIP merge flag is false, the EIP filter is derived from the neighboring reconstructed samples and the relevant syntax element is signaled to indicate which one of the three types of reconstructed area and which one of the three filter shapes are used for the CU. The selected filter moves in the selected reconstructed area either horizontally or vertically with a one-pixel step to construct the auto-correlation matrix and the cross-correlation vector.

After generating the prediction samples of the CU using the EIP filter, an intra prediction mode is derived by applying the DIMD process to the prediction samples. Specifically, a horizontal gradient and a vertical gradient are calculated for each predicted sample to build a histogram of gradient. Then the intra prediction mode corresponding to the largest histogram count is used to determine the low-frequency non-separable transform (LFNST) , non-separable primary transform (NSPT) or multiple transform selection (MTS) transform set.

E. Template-based multiple reference line intra prediction (TMRL)

Template-based multiple reference line intra prediction (TMRL) mode combines reference line and prediction mode together and uses a template matching method to construct a list of candidate combinations. FIG. 5 illustrates a template area and reference lines for computing TMRL costs. An index to the candidate combination list is signaled. The extended reference line starts from reference line 1. Reference line 0 is used for template matching. The SAD costs (TMRL costs) over the template area are calculated between the predictions (generated by 50 combinations) and the reconstructions. The 20 combinations with the least SAD cost are selected in an ascending order to form the TMRL candidate list.

F. Intra Prediction Using Merged Histogram of Gradients

Merged Intra Mode Derivation (MIMD) is an intra prediction mode based on the computation of a Merged Histogram of Gradients (MHoG) . Similar to DIMD, up to five MIMD modes are derived from the MHoG and are then blended together. The derivation of the modes and blending weights follows the same process to derive DIMD modes and blending weights from the HoG. But differently than DIMD, the MHoG is not computed directly analysing the template samples, but rather is computed based on information extracted from neighbouring blocks.

In particular, a number of N neighbouring blocks is considered. A neighbouring block is considered if it is encoded with at least one directional intra-prediction mode. In case the neighbouring block i is encoded using DIMD or MIMD, then its HoG or MHoG is directly considered as H_i, where H_i (m) refers to the amplitude of directional mode m in the HoG, where m can take values from 0 to M where M is the maximum number of intra-prediction modes. A normalization process can be used when considering H_i.

In case the neighbouring block i is instead encoded using a non-DIMD intra-prediction directional mode m, then an HoG H_i is derived for that neighbouring block, where H_i (k) = 0 for k = 0, 1, …M, k ≠ m and H_i (m) = A, where the value A depends on the size of the current block. For neighbouring blocks encoded using SGPM or TIMD where more than one directional intra prediction modes may be available, both directional modes can be considered in the derivation of H_i. Then, the MHoG can be computed using all the HoGs extracted from available neighbouring blocks as:

MHoG may be used to compute MIMD modes and weights. The directional modes and their weights corresponding to the five highest amplitudes in the MHoG are selected as directional modes and weights for MIMD. In some embodiments, MIMD may be signaled as a sub-mode of DIMD.

G. Occurrence-based Intra Coding (OBIC)

The occurrence-based intra coding (OBIC) derives the intra prediction modes of the current block based on the sample-wise occurrence of the intra modes in the spatial neighborhood of the block. For this, adjacent and non-adjacent spatial neighboring blocks are checked and the intra prediction modes of the blocks are collected into an occurrence histogram. Instead of Histogram of Gradient (HoG) as in DIMD, the OBIC method uses the Histogram of oCcurrence (HoC) , which consists of the intra modes and their sample-wise occurrences. The occurrence values are calculated based on the number of samples that are coded in a certain intra prediction mode in that neighborhood. For example, if a uiWidth × uiHeight block is coded with an IPM mode, the occurrence of the mode in that particular block is calculated as:
HoC [IPM] += uiWidth *uiHeight;
Where uiWidth and uiHeight are the width and height of a spatial neighboring block.

The occurrences of the existing modes from the spatial neighborhood blocks are accumulated into the histogram. Up to five angular modes with the highest occurrence along with the planar mode or block vector based prediction (same as in DIMD) are selected from the HoC and used for final prediction by blending the prediction of the selected modes. Some blocks use more than one intra mode for prediction. In such cases, all the intra modes of such blocks are selected and used when creating the OBIC histogram, for example, up to 5 angular modes from DIMD, up to 2 modes from TIMD, 2 modes from SGPM, and up to 5 angular modes from OBIC.

Moreover, the virtual intra prediction modes (VIPMs) of MIP block, IntraTMP block, IBC block, and EIP block are considered only in inter slices when creating the histogram of OBIC mode. The blending weights are calculated similar to the DIMD mode, but instead of using gradient values from the template, the occurrence values are used for OBIC. Moreover, the planar mode’s weight is also decided similar to DIMD mode.
II. Intra Merge Mode

In some embodiments, an intra merge scheme is used to improve intra prediction using a merge scheme. This intra merge mode can also be referred to as decoder-derived intra prediction (DIP) merge mode. The merge scheme is used to inherit (or reference) the information from previous coded blocks and use the inherited coding information (or inheritance information) to predict the current block.

For an intra block, the related intra MPM inherit only the intra prediction mode signaled in the previous coded block (e.g., the intra prediction mode with the mode index referring to angular prediction modes, DC, or planar from the above neighboring block and/or the left neighboring block. ) On the other hand, the intra merge scheme brings more flexible inheritance flows and/or more efficient inheritance information. For example, in some embodiments under the intra merge scheme, the inheritance information may be any mode information, any sample information, any block information, any model information, and/or any information associated with prediction generation. Examples of the inheritance information (or coding information) are provided in Section II. B below.

In some embodiments, the current block can find the previous coded blocks containing the inheritance information through several merge methods described in Section II. C below. In some embodiments, the intra merge scheme is used to obtain the information associated with prediction generation of the current block using a target mode. For some embodiments, the intra merge scheme is enabled or disabled depending on a shortcut syntax setting. Target mode and shortcut syntax setting are described in Section II. D below.

A. Merge Candidate Lists and Mode Categories

In some embodiments, when generating the prediction of the current block coded by the target mode, one or more inheritance blocks are selected from a merge candidate list or among several merge candidates (for example, a set of merge candidates) . Then, the inheritance information from the inheritance blocks is used by the target mode to generate a prediction.

In some embodiments, the inheritance information of different merging candidates in one merge candidate list or set may belong to different target modes, that is, merge candidates may be mixed modes. For example, the inheritance information of merge candidates may be related to DIMD, and/or TIMD, and/or MRL, and/or MIP, and/or EIP. One merge list or set may contain the merge candidates for different target modes (mixed modes) .

In some embodiments, the current block may be predicted following DIMD, TIMD, MRL, MIP, or EIP depending on which merge candidate is used. In some embodiments, only one merge candidate list or set is built for the current block. In some embodiments, the merge candidate list or set may be a unified list or set that is used for the current block, for example, to generate the intra luma prediction. In some embodiments, which target mode is applied to the current block depends on the selected candidate from the one merge candidate list or set.

In some embodiments, at least one or at least two merging candidate lists or sets are built (or constructed) for the current block. For example, in some embodiments, for each mode category, a candidate list or set, which contains one or more candidates belonging to (or associated with) the same mode category, is built. For example, in some embodiments, there may be a mode category for DIMD-related modes only, a mode category for TIMD-related modes only, a mode category for model-related modes only, a mode category for MRL-related modes only, a mode category for matrix-related modes only, and/or other mode categories, etc.

The DIMD-related modes may contain DIMD, MIMD, OBIC, DIP merge with the mode type as DIMD, any mode using histograms to determine the intra prediction modes for the block, any mode with the enabling flag or enabling under DIMD enabling flag (for example, the mode signaled or used only when DIMD enabling flag indicates enabled) , and/or any subset or extension of above-mentioned modes.

The TIMD-related modes may contain TIMD, TMRL, DIP merge with the mode type as TIMD or TMRL, any mode using distortion between reconstruction and prediction (from each available candidate intra prediction mode/combination) on the template to determine the intra prediction modes for the block, any mode with the enabling flag or enabling under TIMD enabling flag (for example, the mode signaled or used only when TIMD enabling flag indicates enabled) , any mode with the enabling flag or enabling under TMRL enabling flag (for example, the mode signaled or used only when TMRL enabling flag indicates enabled) , and/or any subset or extension of above-mentioned modes.

The MRL-related modes may contain MRL, TMRL, DIP merge with the mode type as MRL or TMRL, any mode with the enabling flag or enabling under MRL enabling flag (for example, the mode signaled or used only when MRL enabling flag indicates enabled) , any mode with the enabling flag or enabling under TMRL enabling flag (for example, the mode signaled or used only when TMRL enabling flag indicates enabled) , and/or any mode using the reference line non-adjacent to the block.

The model-related modes may refer to EIP, DIP merge with the mode type as EIP or matrix/parameter-related, any mode using model coefficients (parameters) with input reference samples to derive the prediction for the block, any mode with the enabling flag or enabling under EIP enabling flag (for example, the mode signaled or used only when EIP enabling flag indicates enabled) , and/or any subset or extension of above-mentioned modes.

The matrix-related modes may contain MIP (matrix-based or matrix-weighted intra prediction) , DIP merge with the mode type as MIP or matrix/parameter-related, any mode with the enabling flag or enabling under MIP enabling flag (for example, the mode signaled or used only when MIP enabling flag indicates enabled) , any mode using a pre-defined or selected matrix with input reference samples to derive the prediction for the block, and/or any subset or extension of above-mentioned modes.

In some embodiments, one mode category is for decoder-analysis-related modes only, another mode category is for model-related modes only, another mode category is for MRL-related modes only, and/or another mode category is for matrix-related modes only. The decoder-analysis-related modes may contain DIMD-related modes, TIMD-related modes, any mode using decoder-side analysis to determine the intra prediction mode for the block, and/or any subset or extension of above-mentioned modes. (For example, instead of a signaled index, decoder-analysis-related mode may be a texture gradient analysis on the template and/or a distortion measurement on the template. )

For some embodiments, the DIMD-related modes may include DIMD, MIMD, OBIC, DIP merge with the mode type as DIMD, any mode using histograms to determine the intra prediction modes for the block, any mode with the enabling flag or enabling under DIMD enabling flag (for example, the mode signaled or used only when DIMD enabling flag indicates enabled) , and/or any subset or extension of above-mentioned modes. The TIMD-related modes may contain TIMD, TMRL, DIP merge with the mode type as TIMD or TMRL, any mode using distortion between reconstruction and prediction (from each available candidate intra prediction mode/combination) on the template to determine the intra prediction modes for the block, any mode with the enabling flag or enabling under TIMD enabling flag (for example, the mode signaled or used only when TIMD enabling flag indicates enabled) , any mode with the enabling flag or enabling under TMRL enabling flag (for example, the mode signaled or used only when TMRL enabling flag indicates enabled) , and/or any subset or extension of above-mentioned modes.

In some embodiments, one mode category is for parameter-related modes only, another mode category is for DIMD-related modes only, another mode category is for MRL-related modes only, and/or another mode category is for TIMD-related modes only. The parameter-related modes may contain EIP-related modes, MIP-related modes, any mode using a model or matrix (for example, a pre-defined model or matrix in the standard and/or a model or matrix derived based on a regression or derivation or on-the-fly-deciding technique, which may be (but not limited to be) performed on the template, instead of using an intra prediction direction) to determine the intra predictor for the block, and/or any subset or extension of above-mentioned modes. For example, the model-related modes may include EIP, DIP merge with the mode type as EIP or matrix/parameter-related, any mode using model coefficients (parameters) with input reference samples to derive the prediction for the block, any mode with the enabling flag or enabling under EIP enabling flag (for example, the mode signaled or used only when EIP enabling flag indicates enabled) , and/or any subset or extension of above-mentioned modes. The matrix-related modes contain MIP (matrix-based or matrix-weighted intra prediction) , DIP merge with the mode type as MIP or matrix/parameter-related, any mode with the enabling flag or enabling under MIP enabling flag (for example, the mode signaled or used only when MIP enabling flag indicates enabled) , any mode using a pre-defined or selected matrix with input reference samples to derive the prediction for the block, and/or any subset or extension of above-mentioned modes.

In some embodiments, at least one or at least two merging candidate lists or sets are built for the current block. To construct the merge candidate list (s) or set (s) , different thresholds (for example, the thresholds for similarity check of inserting candidates into the list or set) and/or designs and/or methods and/or modes may be used according to the block width, block height, and/or block area of the current block, the block width, block height, block area of the inheritance block (e.g., the block providing the inheritance information) , and/or the explicit signaling at the block (e.g., the flag/index signaled for the current block and/or a refinement flag for the current block which indicates whether to perform refinement for the modes wherein the refinement can be but not limited to be a template refinement for the modes) , sequence parameter set (SPS) , picture parameter set (PPS) , sequence, slice, and/or tile. In some embodiments, which one of the merge candidate list (s) or set (s) is constructed for the current block according to the block width, block height, and/or block area of the current block. In some embodiments, which one of the merge candidate list (s) or set (s) is constructed for the current block according to the explicit signaling at the block (e.g., the flag/index signaled for the current block. ) For example, in response to the explicit signaling at the current block indicating to apply intra merge mode (with the mode type associated with decoder-analysis-related modes) to the current block, the merge candidate list or set (which contains one or more candidates belonging to decoder-analysis-related mode category) is constructed for the current block. For example, when the explicit signaling at the current block indicating to apply intra merge mode (with the mode type associated with parameter-related modes, for example, EIP) to the current block, the merge candidate list or set (which contains one or more candidates belonging to parameter-related mode category) is constructed for the current block. In some embodiments, there is a maximum number (e.g., 12) of allowed candidates in the merge candidate list.

B. Inheritance information

Inheritance information settings are defined for generating prediction of the current block using a target mode. When the target mode is DIMD mode, the inheritance information may include one or more of the following information (1a) through (1d) :
(1a) one or more histogram (bar) values for the available DIMD intra prediction modes (such as
DC, planar, and/or directional prediction modes) ;
(1b) the N intra prediction modes (with the highest N histogram bars) suggested by the histogram
values;
(1c) DIMD weighting information and/or fusion or not;
(1d) reference line information and/or wide-angle conditions.

When the target mode is TIMD mode, the inheritance information may include one or more of the following inheritance information (2a) through (2d) :
(2a) one or more TIMD cost values for the available TIMD intra prediction modes (such as DC,
planar, and/or directional prediction modes) ;
(2b) the N intra prediction modes (with the smallest N TIMD costs) suggested by the TIMD costs;
(2c) TIMD weighting information and/or fusion or not;
(2d) reference line information and/or wide-angle conditions.

When the target mode is MRL mode, the inheritance information may include one or more of the following inheritance information (3a) through (3d) :
(3a) one or more reference lines jointly with intra prediction modes (such as DC, planar, and/or
directional prediction modes) ;
(3b) the N intra prediction modes (with the smallest N TIMD costs or TMRL costs) suggested
by the TIMD costs or TMRL costs;
(3c) weighting information and/or fusion or not;
(3d) reference line information and/or wide-angle conditions.

When the target mode is EIP mode, the inheritance information may include one or more of the following inheritance information (4a) through (4c) :
(4a) the filter shape;
(4b) all or parts of the filter coefficients;
(4c) the template used to derive the filter coefficients.

When the target mode is intra merge mode or decoder-derived intra prediction (DIP) merge mode, the inheritance information may include one or more of the above inheritance information (1a) through (4c) (inheritance information when target mode is DIMD, TIMD, MRL, or EIP) , and/or coding mode inheritance information. Inheritance information about the coding mode (or called coding mode inheritance information) may include the coding mode (for example, DIMD-related mode or not, TIMD-related mode or not, MRL-related mode or not, SGPM-related mode or not, ISP-related mode or not, intraTMP-related mode or not, MPM-related mode or not, MIP-related mode or not, EIP-related mode or not, and/or any mode related to a pre-defined intra mode) and/or corresponding information of the coding mode.

In some embodiments, one or more of inheritance information (1a) through (4c) may be used to obtain the inheritance information when the target mode is DIMD, TIMD, MRL, or EIP.

In some embodiments, the coding mode inheritance information is included in the inheritance information and/or the coding mode in the coding mode inheritance information is used to determine the corresponding information to be further included in the DIP merge inheritance information. For example, when the coding mode in the coding mode inheritance information is MRL-related mode, only the corresponding information from inheritance information (3a) through (3d) is included in the DIP merge inheritance information.

In some embodiments, the coding mode in the coding mode inheritance information is from a previous coded block. The previous coded block may store the mixed-mode inheritance information. The stored mixed-mode inheritance information of the previous coded block can be referenced by subsequent coding blocks (e.g., the current block) . For example, for each block containing the mixed-mode information, the mixed-mode information may be stored and/or referenced by subsequent coding blocks. For each pre-defined unit containing the mixed-mode information, the mixed-mode information may be stored and/or referenced by subsequent coding blocks. The unit can be any pre-defined region or kxk grids, where k can be 2, 4, 8, 16, or any pre-defined positive integer.

In some embodiments, the coding mode in the coding mode inheritance information may be EIP. The previous coded block generates the corresponding EIP inheritance information (e.g., filter shape and/or filter coefficients) and/or stores the DIP merge inheritance information (coding mode and/or the corresponding information) .

In some embodiments, the coding mode in the coding mode inheritance information may be SGPM (spatial geometric partitioning mode) . This coding mode generates multiple hypotheses of predictions from different intra prediction modes and combines the hypotheses of predictions to form the final prediction using the weights based on a geometric partitioning line as inter GPM. The previous coded block generates the corresponding SGPM inheritance information (e.g., intra prediction modes and/or partitioning line and/or weights) and/or stores the DIP merge inheritance information (coding mode and/or the corresponding information) .

In some embodiments, the coding mode in the coding mode inheritance information may be ISP (intra sub-partition) . This coding mode splits the current block into several sub-partitions and generates prediction of each sub-partition using the reconstructed samples (possibly from the previous sub-partition) adjacent to the current sub-partition as the reference samples. The previous coded block generates the corresponding ISP inheritance information (e.g., one or more intra prediction modes and/or splitting method) and/or stores the DIP merge inheritance information (coding mode and/or the corresponding information) .

In some embodiments, the coding mode in the coding mode inheritance information may be IntraTMP (intra template matching prediction) . This coding mode uses template matching to find a displacement (i.e., block vector) to refer a reference block in a pre-defined range of the current picture and generates prediction using the reconstructed samples of the reference block. The previous coded block generates the corresponding intraTMP inheritance information (for example, one or more block vectors) and/or stores the DIP merge inheritance information (coding mode and/or the corresponding information) .

In some embodiments, the coding mode in the coding mode inheritance information may be MIP (matrix-based intra prediction) . This mode uses a pre-defined matrix and pre-defined adjacent or non-adjacent reference samples to generate the prediction. The previous coded block generates the corresponding MIP inheritance information (for example, one or more matrix indications and/or one or more matrix coefficients of the indicated matrixes and/or block width and/or block height and/or block area and/or block position) and/or stores the DIP merge inheritance information (coding mode and/or the corresponding information) .

In some embodiments, instead of storing all DIP merge information (which may be used for reference) , to reduce the storage, only a subset of DIP merge information is stored. For example, only 3 or any pre-defined positive number of DIP merge information from all generated prediction information are stored. For example, the subset may be the first 3 from all generated prediction information such as the first 3 generated intra prediction modes or histogram (bar) values when the inheritance information is associated with DIMD. In some embodiments, the selected subset of DIP merge information always includes the coding mode inheritance information.

In some embodiments, the inheritance information of the target mode depends on an inherited coding mode value of the selected merge candidate. In some embodiments, if the coding mode of the selected merge candidate is DIMD, TIMD, EIP or MRL, the inherited coding mode value is the same as the coding mode (i.e., is DIMD, TIMD, EIP or MRL respectively) .

In some embodiments, for a block coded in DIP merge mode, the inherited mode type value is set as following: When the current block is coded in DIP merge mode, the block will select a merge candidate to generate prediction. If the selected merging candidate for the current block is coded in DIMD, TIMD, MRL or EIP, the inherited mode type value of the current block is set to DIMD, TIMD, MRL or EIP respectively. If the selected merging candidate for the current block is coded in DIP merge mode, the inherited mode type value of the current block is set to the inherited mode type value of the selected merging candidate. In some embodiments, if the inherited mode type is DIMD, the inheritance information setting may include one or more of inheritance information (1a) - (1d) . In some embodiments, if the inherited mode type is TIMD, the inheritance information setting may include one or more of inheritance information (2e) - (2d) . In some embodiments, if the inherited mode type is MRL, the inheritance information setting may include one or more of inheritance information (3a) - (3d) . In some embodiments, if the inherited mode type is EIP, the inheritance information setting may include one or more of inheritance information (4a) - (4c) .

C. Merge Candidates

The inheritance information is obtained from the previous coded blocks according to inheritance block setting. In some embodiments, one or more candidates of a merge candidate list may include spatial adjacent candidates and/or non-adjacent candidates, history candidates, temporal candidates, default candidates, or any subset of above-mentioned candidates. These candidates may provide the inheritance information from previously coded blocks (also referred to as inheritance blocks) .

In some embodiments, after building the merge candidates list, one or more candidates are selected from the list for the current block to use. The selection may depend on explicitly signalling an index or implicitly select the one or more (promising) candidates. For example, in some embodiments, the first one or more candidates with the smallest costs are selected. For example, the explicit index indicates one or more candidates in the reordered list as the selected candidates. The cost calculation and/or list reordering may depend on the template matching process which calculates the cost based on the distortion between the prediction (using a certain candidate) on the template and the reconstruction on the template. The candidates in the list are reordered based on costs in ascending order.

The spatial adjacent candidates are from the adjacent neighboring blocks of the current block. The adjacent neighboring blocks may be the same as the 5 spatial neighboring blocks for inter merge mode or any subset of the adjacent neighboring blocks of the current block. For example, for adding the spatial adjacent candidates into the merge list, as illustrated in FIG. 6, which shows an example of the adjacent neighboring blocks of the current block, the inclusion order can be A1 → B1 → A0 → B0 → B2 or B1 → A1 → B0 → A0 → B2. The non-adjacent candidates are from a search range around (but not adjacent to) the current block. The search range can be the same as the search range of non-adjacent candidates for inter merge mode. The non-adjacent candidates can be from pre-defined positions and are added into the merge list in a pre-defined inclusion order. For example, the pre-defined positions and the inclusion order are the same as those of the non-adjacent candidates of inter merge mode.

The history candidates are selected from a history-based buffer array. In the history-based buffer array, the inheritance information of each valid previous coded block is stored where the valid previous coded block refers to any block containing inheritance information.

The temporal candidates are obtained from the inheritance information stored in one or more previous coded picture. The temporal candidates are obtainable when the current slice/picture is a non-intra slice/picture. In some embodiments, the temporal candidates can be from the block at some pre-defined positions (x′, y′) of the previous coded slice/picture. In some embodiments, the predefined positions are inside the corresponding area of the current encoding/decoding block. In some embodiments, the pre-defined positions are outside of the corresponding area of the current encoding/decoding block. In some embodiments, the pre-defined positions can be determined based on the position, width and height of the current block. In some embodiments, the pre-defined positions can be determined based on the position of the current block, and some pre-defined fixed x-y distances.

In some embodiments, the previous coded pictures are among the pictures in the reference lists. In some embodiments, the previous coded pictures are the same pictures as the collocated picture of the regular inter merge mode. In some embodiments, there can be more than one selected/pre-defined previous coded pictures. In some embodiments, the previous coded pictures can be signaled in the picture/slice header. The reference list and the reference index are signaled in the picture/slice header. For example, L0 [0] is signaled. For another example, L1 [0] is signaled. In some embodiments, the previous coded pictures are selected from a picture set with some pre-defined rules. In some embodiments, the previous coded pictures are selected from pictures in the reference lists. The selection can be determined based on picture order count (POC) , and/or POC distance, and/or QP, and/or QP difference. For another example, pictures with the smaller POC are selected. For another example, the picture with the larger POC is selected. For example, the picture whose POC distance between it and the current picture is the smallest is selected. For another example, the picture with the smaller QP is selected. For another example, the picture with the larger QP is selected. For another example, the picture with smaller QP difference between it and the current picture is selected. In some embodiments, the previous coded picture is the most recently coded I-picture. In some embodiments, the rules to select/not select the previous coded pictures described in the paragraphs above can be combined. For example, in some embodiments, the picture whose QP is the smallest among the un-scaled pictures in the reference lists is selected.

The default candidates are the candidates containing pre-defined default information and/or the default information is derived according to the candidates already put in the merge candidate list. In some embodiments, the default modes for the default candidates can be derived using MIMD scheme and/or OBIC scheme and/or EIP derived mode scheme and/or any existing or mentioned intra scheme (for example, TIMD, DIMD, EIP, and/or MRL) and/or any variations of the existing or mentioned intra scheme.

D. Target Mode Setting and Shortcut Syntax Setting

In some embodiments, when generating a prediction for the current block, the target mode of the current block determines how the inheritance information obtained from a merge candidate is used to generate the prediction. In other words, each target mode has its own specific settings for prediction generation. For example, in some embodiments, when DIMD merge mode is used (e.g., for when the target mode is a DIMD-related mode) , the DIMD inheritance information, from one or more pre-defined candidates according to the inheritance block setting, is used to decide the prediction information, required for generating the prediction of the current block. For example, the inheritance information refers to the histogram values from the previous coded block and is used to decide one or more intra prediction modes (one kind of prediction information) and/or corresponding weights for the current block as regular DIMD. Then, unified with regular DIMD, the hypothesis of prediction from each derived intra prediction mode is combined using blending process to form the final prediction of the current block.

In some embodiments, when the TIMD merge mode is used (e.g., for when the target mode is a TIMD-related mode) , the TIMD inheritance information from one or more pre-defined candidates (for example, each candidate providing N intra prediction modes for the available TIMD intra prediction modes and TIMD weighting information, such as {IPM1, IPM2, W1, W2} , and/or fusion or not, and/or wide-angle conditions) according to the inheritance block setting, is used to decide the prediction information, required for generating the prediction of the current block. For example, the merge candidate list is built and reordered according to the costs in the template matching process. The template size and the template cost calculation may be unified with regular TIMD. After reordering the TIMD merge candidates, the first 2 or any pre-defined number candidates from the list are kept for signaling. Then, unified with regular TIMD, the hypothesis of prediction from each inherited intra prediction mode is combined using blending process to form the final prediction of the current block.

In some embodiments, when MRL merge mode is used (e.g., for when the target mode is a MRL-related mode) , the inheritance information from one or more pre-defined candidates (for example, each candidate providing one or more reference lines jointly with intra prediction modes (IPM1 and MRL1) and/or fusion or not, and/or wide-angle conditions) according to the inheritance block setting, is used to decide the prediction information, required for generating the prediction of the current block. For example, the merge candidate list is built and reordered according to the costs in the template matching process. The template size and the template cost calculation may be unified with regular TMRL. After reordering the MRL merge candidates, the first 2 or any pre-defined number candidates from the list are kept for signaling. Then, unified with regular intra prediction, for the example of inherited IPM1 and MRL1, the prediction from the inherited intra prediction mode and the inherited reference line is to form the final prediction of the current block; for the example of inherited IPM1 with MRL1 and IPM2 with MRL2, the hypothesis of prediction from each inherited intra prediction mode with the corresponding inherited reference line is combined using blending process to form the final prediction of the current block; for the example of inherited IPM1 with MRL1/W1 and IPM2 with MRL2/W2, the hypothesis of prediction from each inherited intra prediction mode with the corresponding inherited reference line is combined using blending process with W1 and W2 to form the final prediction of the current block.

In some embodiments, when the intra merge mode (DIP merge mode) is used, the inherited mode information is used to generate the prediction of the current block. The inheritance information of a candidate may refer to the coding mode and/or the corresponding information. The current block obtains its own prediction information from the inheritance information.

An example of the coding mode from the inherited mode information being DIMD-related is that DIMD merge mode or regular DIMD is applied to the current block.

An example of the coding mode from the inherited mode information being TIMD-related is that TIMD merge mode or regular TIMD is applied to the current block.

An example of the coding mode from the inherited mode information being MRL-related is that MRL merge mode or regular intra prediction using MRL or TMRL is applied to the current block.

An example of the coding mode from the inherited mode information being EIP-related is that EIP merge mode or EIP derived mode is applied to the current block.

In some embodiments, only one candidate (e.g., the first available candidate) according to the inheritance block setting, is used to decide the prediction information for the current block. In some embodiments, one or more candidates are used to decide the prediction information for the current block. For example, all available candidates are used to decide the prediction information for the current block. For example, assume a pre-defined maximum number is N. The first N available candidates according to the checking order are used to decide the prediction information for the current block. If the number of total available candidates is smaller than N, all the available candidates are used to decide the prediction information for the current block.

In some embodiments, an additional flag is signaled to indicate whether DIP merge mode is used or not before the DIMD flag. If DIP merge mode is used, a candidate index is additionally signaled. In some embodiments, the candidate index is coded using truncated unary coding. In some embodiments, each bin of candidate index is context coded with a separate context.

Any combination of the proposed methods in this invention can be applied. The methods described in this disclosure can be enabled and/or disabled according to implicit rules (e.g. block width, height, or area) or according to explicit rules (e.g., syntax on block, tile, slice, picture, SPS, or PPS level) . For example, the proposed method is applied when the block area is smaller/larger than a threshold. The term “block” in this invention can refer to TU/TB, CU/CB, PU/PB, pre-defined region, or CTU/CTB.

Any of the foregoing proposed methods can be implemented in encoders and/or decoders. For example, any of the proposed methods can be implemented in an inter/intra/IBC/prediction/transform module of an encoder, and/or an inter/intra/IBC/prediction/transform module of a decoder. Alternatively, any of the proposed methods can be implemented as a circuit coupled to the inter/intra/IBC/prediction/transform module of the encoder and/or the inter/intra/IBC/prediction/transform module of the decoder, so as to provide the information needed by the inter/intra/IBC/prediction/transform module.
III. Example Video Encoder

FIG. 7 illustrates an example video encoder 700 that may implement intra merge mode. As illustrated, the video encoder 700 receives input video signal from a video source 705 and encodes the signal into bitstream 795. The video encoder 700 has several components or modules for encoding the signal from the video source 705, at least including some components selected from a transform module 710, a quantization module 711, an inverse quantization module 714, an inverse transform module 715, an intra-picture estimation module 724, an intra-prediction module 725, a motion compensation module 730, a motion estimation module 735, an in-loop filter 745, a reconstructed picture buffer 750, a MV buffer 765, and a MV prediction module 775, and an entropy encoder 790. The motion compensation module 730 and the motion estimation module 735 are part of an inter-prediction module 740. The intra-prediction module 725 and the intra-prediction estimation module 724 are part of a current picture prediction module 720, which uses current picture reconstructed samples as reference samples for prediction of the current block.

In some embodiments, the modules 710 –790 are modules of software instructions being executed by one or more processing units (e.g., a processor) of a computing device or electronic apparatus. In some embodiments, the modules 710 –790 are modules of hardware circuits implemented by one or more integrated circuits (ICs) of an electronic apparatus. Though the modules 710 –790 are illustrated as being separate modules, some of the modules can be combined into a single module.

The video source 705 provides a raw video signal that presents pixel data of each video frame without compression. A subtractor 708 computes the difference between the raw video pixel data of the video source 705 and the predicted pixel data 713 from the motion compensation module 730 or intra-prediction module 725 as prediction residual 709. The transform module 710 converts the difference (or the residual pixel data or residual signal 708) into transform coefficients (e.g., by performing Discrete Cosine Transform, or DCT) . The quantization module 711 quantizes the transform coefficients into quantized data (or quantized coefficients) 712, which is encoded into the bitstream 795 by the entropy encoder 790.

The inverse quantization module 714 de-quantizes the quantized data (or quantized coefficients) 712 to obtain transform coefficients 718, and the inverse transform module 715 performs inverse transform on the transform coefficients 718 to produce reconstructed residual 719. The reconstructed residual 719 is added with the predicted pixel data 713 to produce reconstructed pixel data 717. In some embodiments, the reconstructed pixel data 717 is temporarily stored in a line buffer 727 (or intra prediction buffer) for intra-picture prediction and spatial MV prediction. The reconstructed pixels are filtered by the in-loop filter 745 and stored in the reconstructed picture buffer 750. In some embodiments, the reconstructed picture buffer 750 is a storage external to the video encoder 700. In some embodiments, the reconstructed picture buffer 750 is a storage internal to the video encoder 700.

The intra-picture estimation module 724 performs intra-prediction based on the reconstructed pixel data 717 to produce intra prediction data. The intra-prediction data is provided to the entropy encoder 790 to be encoded into bitstream 795. The intra-prediction data is also used by the intra-prediction module 725 to produce the predicted pixel data 713.

The motion estimation module 735 performs inter-prediction by producing MVs to reference pixel data of previously decoded frames stored in the reconstructed picture buffer 750. These MVs are provided to the motion compensation module 730 to produce predicted pixel data.

Instead of encoding the complete actual MVs in the bitstream, the video encoder 700 uses MV prediction to generate predicted MVs, and the difference between the MVs used for motion compensation and the predicted MVs is encoded as residual motion data and stored in the bitstream 795.

The MV prediction module 775 generates the predicted MVs based on reference MVs that were generated for encoding previously video frames, i.e., the motion compensation MVs that were used to perform motion compensation. The MV prediction module 775 retrieves reference MVs from previous video frames from the MV buffer 765. The video encoder 700 stores the MVs generated for the current video frame in the MV buffer 765 as reference MVs for generating predicted MVs.

The MV prediction module 775 uses the reference MVs to create the predicted MVs. The predicted MVs can be computed by spatial MV prediction or temporal MV prediction. The difference between the predicted MVs and the motion compensation MVs (MC MVs) of the current frame (residual motion data) are encoded into the bitstream 795 by the entropy encoder 790.

The entropy encoder 790 encodes various parameters and data into the bitstream 795 by using entropy-coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman encoding. The entropy encoder 790 encodes various header elements, flags, along with the quantized transform coefficients 712, and the residual motion data as syntax elements into the bitstream 795. The bitstream 795 is in turn stored in a storage device or transmitted to a decoder over a communications medium such as a network.

The in-loop filter 745 performs filtering or smoothing operations on the reconstructed pixel data 717 to reduce the artifacts of coding, particularly at boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the in-loop filter 745 include deblock filter (DBF) , sample adaptive offset (SAO) , and/or adaptive loop filter (ALF) . In some embodiments, luma mapping chroma scaling (LMCS) is performed before the loop filters.

FIG. 8 illustrates portions of the video encoder 700 that implement intra merge mode with at least one or multiple merge candidate lists or sets. Specifically, the figure illustrates the components that are involved in generating a prediction of the current block using intra merge mode.

As illustrated, the intra-prediction module 720 uses reference samples stored in the line buffer 727 to generate a prediction as the predicted pixel data 713 according to a set of coding information 835 provided by a candidate fetch module 830. The reference samples are reconstructed samples of the current picture that are in neighboring regions of the current block. The set of coding information may include inheritance information described in Section II. B above. Based on the target mode according to which the current block is to be predicted, the intra-prediction module 720 may perform the prediction of the current block according to a set of target mode specific settings as described in Section II. D. The target mode may be indicated within the coding information 835 or provided by the entropy encoder 790.

The candidate fetch module 830 provides the set of coding information 835 based on inheritance information stored in an inheritance information storage 820 and a selected merge candidate 815. The candidate fetch module 830 selects the merge candidate 815 from a merge candidate list according to a merge index provided by the entropy encoder 790. The candidates in the merge candidate list may include spatial adjacent candidates, non-adjacent candidates, history candidates, temporal candidates, or default candidates. Each merge candidate may correspond to a position associated with a previously coded block to provide a set of inheritance information that can be inherited by the current block. Merge candidates are described in greater detail at Section II. C above. The inheritance information of the selected merge candidate 815 is used as the coding information 835 for generating the prediction of the current block.

The merge candidate list may be one of several merge candidate lists 811-816 constructed by an intra merge candidate list constructor 810. In some embodiments, each merge candidate list may be associated with a different mode category, such that the merge candidates of the merge candidate list are only for that mode category. Examples of such mode categories include DIMD-related, TIMD-related, MRL-related, model-related, matrix-related, and/or decoder-side-analysis related. Descriptions of the different mode categories are provided in Section II. A above.

FIG. 9 conceptually illustrates a process 900 for encoding a block of pixels using intra merge mode with one or multiple merge candidate lists. In some embodiments, one or more processing units (e.g., a processor) of a computing device implementing the encoder 700 performs the process 900 by executing instructions stored in a computer readable medium. In some embodiments, an electronic apparatus implementing the encoder 700 performs the process 900.

The encoder receives (at block 910) data to be encoded as a current block of pixels in a current picture.

The encoder constructs (at block 920) one or more merge candidate lists or sets for the current block, each merge candidate corresponding to a set of coding information that is inherited from a previously coded block. In some embodiments, the encoder constructs at least one merge candidate list or set for the current block.

In some embodiments, each merge candidate list or set is associated with a mode category. A first merge candidate list or set may be associated with a first mode category and a second merge candidate list or set may be associated with a second, different mode category. A merge candidate list or set may be associated with a mode category for only modes related to DIMD, or a mode category for only modes related to TIMD, or a mode category for only modes related to MRL, or a mode category for only model-related modes, or a mode category for only matrix-related modes, or a mode category for only decoder-analysis-related modes. The various mode categories are described in detail in Section II. A above.

Each merge candidate in a merge candidate list or set associated with a mode category corresponds to a set of coding information for a coding mode belonging to (or associated with) that mode category. The set of coding information may include inheritance information described in Section II. B above. Merge candidates are described in greater detail at Section II. C above.

The encoder selects (at block 930) a merge candidate from the constructed merge candidate lists or sets. In some embodiments, the merge candidate list or set is selected based on the mode categories associated with the at least one or two merge candidate lists or sets. The selected merge candidate may be selected from the selected merge candidate list or set based on a merge index that is to be encoded into the bitstream.

The encoder generates (at block 940) a prediction of the current block according to a target mode by using the set of coding information corresponding to the selected merge candidate. In some embodiments, the encoder may perform the prediction of the current block according to a set of target mode specific settings as described in Section II. D. The encoder encodes (at block 950) the current block by using the generated prediction to generate prediction residuals.
IV. Example Video Decoder

In some embodiments, an encoder may signal (or generate) one or more syntax element in a bitstream, such that a decoder may parse said one or more syntax element from the bitstream.

FIG. 10 illustrates an example video decoder 1000 that may implement intra merge mode. As illustrated, the video decoder 1000 is an image-decoding or video-decoding circuit that receives a bitstream 1095 and decodes the content of the bitstream into pixel data of video frames for display. The video decoder 1000 has several components or modules for decoding the bitstream 1095, including some components selected from an inverse quantization module 1014, an inverse transform module 1015, an intra-prediction module 1025, a motion compensation module 1030, an in-loop filter 1045, a decoded picture buffer 1050, a MV buffer 1065, a MV prediction module 1075, and a parser 1090. The motion compensation module 1030 is part of an inter-prediction module 1040. The intra-prediction module 1025 is part of a current picture prediction module 1020, which uses current picture reconstructed samples as reference samples for prediction of the current block.

In some embodiments, the modules 1014 –1090 are modules of software instructions being executed by one or more processing units (e.g., a processor) of a computing device. In some embodiments, the modules 1014 –1090 are modules of hardware circuits implemented by one or more ICs of an electronic apparatus. Though the modules 1014 –1090 are illustrated as being separate modules, some of the modules can be combined into a single module.

The parser 1090 (or entropy decoder) receives the bitstream 1095 and performs initial parsing according to the syntax defined by a video-coding or image-coding standard. The parsed syntax element includes various header elements, flags, as well as quantized data (or quantized coefficients) 1012. The parser 1090 parses out the various syntax elements by using entropy-coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman encoding.

The inverse quantization module 1014 de-quantizes the quantized data (or quantized coefficients) 1012 to obtain transform coefficients, and the inverse transform module 1015 performs inverse transform on the transform coefficients 1018 to produce reconstructed residual signal 1019. The reconstructed residual signal 1019 is added with predicted pixel data 1013 from the intra-prediction module 1025 or the motion compensation module 1030 to produce decoded pixel data 1017. The decoded pixels data are filtered by the in-loop filter 1045 and stored in the decoded picture buffer 1050. In some embodiments, the decoded picture buffer 1050 is a storage external to the video decoder 1000. In some embodiments, the decoded picture buffer 1050 is a storage internal to the video decoder 1000.

The intra-prediction module 1025 receives intra-prediction data from bitstream 1095 and according to which, produces the predicted pixel data 1013 from the decoded pixel data 1017 stored in the decoded picture buffer 1050. In some embodiments, the decoded pixel data 1017 is also stored in a line buffer 1027 (or intra prediction buffer) for intra-picture prediction and spatial MV prediction.

In some embodiments, the content of the decoded picture buffer 1050 is used for display. A display device 1005 either retrieves the content of the decoded picture buffer 1050 for display directly, or retrieves the content of the decoded picture buffer to a display buffer. In some embodiments, the display device receives pixel values from the decoded picture buffer 1050 through a pixel transport.

The motion compensation module 1030 produces predicted pixel data 1013 from the decoded pixel data 1017 stored in the decoded picture buffer 1050 according to motion compensation MVs (MC MVs) . These motion compensation MVs are decoded by adding the residual motion data received from the bitstream 1095 with predicted MVs received from the MV prediction module 1075.

The MV prediction module 1075 generates the predicted MVs based on reference MVs that were generated for decoding previous video frames, e.g., the motion compensation MVs that were used to perform motion compensation. The MV prediction module 1075 retrieves the reference MVs of previous video frames from the MV buffer 1065. The video decoder 1000 stores the motion compensation MVs generated for decoding the current video frame in the MV buffer 1065 as reference MVs for producing predicted MVs.

The in-loop filter 1045 performs filtering or smoothing operations on the decoded pixel data 1017 to reduce the artifacts of coding, particularly at boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the in-loop filter 1045 include deblock filter (DBF) , sample adaptive offset (SAO) , and/or adaptive loop filter (ALF) . In some embodiments, luma mapping chroma scaling (LMCS) is performed before the loop filters.

FIG. 11 illustrates portions of the video decoder 1000 that implement intra merge mode with at least one or multiple merge candidate lists or sets. Specifically, the figure illustrates the components that are involved in generating a prediction of the current block using intra merge mode.

As illustrated, the intra-prediction module 1020 uses reference samples stored in the line buffer 1027 to generate a prediction as the predicted pixel data 1013 according to a set of coding information 1135 provided by a candidate fetch module 1130. The reference samples are reconstructed samples of the current picture that are in neighboring regions of the current block. The set of coding information may include inheritance information described in Section II. B above. Based on the target mode according to which the current block is to be predicted, the intra-prediction module 1020 may perform the prediction of the current block according to a set of target mode specific settings as described in Section II. D. The target mode may be indicated within the coding information 1135 or provided by the entropy decoder 1090.

The candidate fetch module 1130 provides the set of coding information 1135 based on inheritance information stored in an inheritance information storage 1120 and a selected merge candidate 1115. The candidate fetch module 1130 selects the merge candidate 1115 from a merge candidate list or set according to a merge index provided by the entropy decoder 1090. The candidates in the merge candidate list or set may include spatial adjacent candidates, non-adjacent candidates, history candidates, temporal candidates, or default candidates. Each merge candidate may correspond to a position associated with a previously coded block to provide a set of inheritance information that can be inherited by the current block. Merge candidates are described in greater detail at Section II. C above. The inheritance information of the selected merge candidate 1115 is used as the coding information 1135 for generating the prediction of the current block.

The merge candidate list may be one of several merge candidate lists 1111-1116 constructed by an intra merge candidate list constructor 1110. In some embodiments, each merge candidate list may be associated with a different mode category, such that the merge candidates of the merge candidate list are only for that mode category. Examples of such mode categories include DIMD-related, TIMD-related, MRL-related, model-related, matrix-related, and/or decoder-side-analysis related. Descriptions of the different mode categories are provided in Section II. A above.

FIG. 12 conceptually illustrates a process 1200 for decoding a block of pixels using intra merge mode with one or multiple merge candidate lists or sets. In some embodiments, one or more processing units (e.g., a processor) of a computing device implementing the decoder 1000 performs the process 1200 by executing instructions stored in a computer readable medium. In some embodiments, an electronic apparatus implementing the decoder 1000 performs the process 1200.

The decoder receives (at block 1210) data to be decoded as a current block of pixels in a current picture.

The decoder constructs (at block 1220) one or more merge candidate lists or sets for the current block, each merge candidate corresponding to a set of coding information that is inherited from a previously coded block. In some embodiments, the decoder constructs at least one merge candidate list or set for the current block.

Each merge candidate in a merge candidate list or set associated with a mode category corresponds to a set of coding information for a coding mode belonging to that mode category. The set of coding information may include inheritance information described in Section II. B above. Merge candidates are described in greater detail at Section II. C above.

The decoder selects (at block 1230) a merge candidate from the constructed merge candidate lists or sets. In some embodiments, the merge candidate list or set is selected based on the mode categories associated with the at least one or two merge candidate lists or sets. The selected merge candidate may be selected from the selected merge candidate list or set based on a merge index that is decoded from the bitstream.

The decoder generates (at block 1240) a prediction of the current block according to a target mode by using the set of coding information provided by the selected merge candidate. In some embodiments, the decoder may perform the prediction of the current block according to a set of target mode specific settings as described in Section II. D.

The decoder reconstructs (at block 1250) the current block by using the generated prediction to combine with prediction residuals. The decoder may then provide the reconstructed current block for display as part of the reconstructed current picture.
V. Example Electronic System

Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer readable storage medium (also referred to as computer readable medium) . When these instructions are executed by one or more computational or processing unit (s) (e.g., one or more processors, cores of processors, or other processing units) , they cause the processing unit (s) to perform the actions indicated in the instructions. Examples of computer readable media include, but are not limited to, CD-ROMs, flash drives, random-access memory (RAM) chips, hard drives, erasable programmable read only memories (EPROMs) , electrically erasable programmable read-only memories (EEPROMs) , etc. The computer readable media does not include carrier waves and electronic signals passing wirelessly or over wired connections.

In this specification, the term “software” is meant to include firmware residing in read-only memory or applications stored in magnetic storage which can be read into memory for processing by a processor. Also, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while remaining distinct software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement a software invention described here is within the scope of the present disclosure. In some embodiments, the software programs, when installed to operate on one or more electronic systems, define one or more specific machine implementations that execute and perform the operations of the software programs.

FIG. 13 conceptually illustrates an electronic system 1300 with which some embodiments of the present disclosure are implemented. The electronic system 1300 may be a computer (e.g., a desktop computer, personal computer, tablet computer, etc. ) , phone, PDA, or any other sort of electronic device. Such an electronic system includes various types of computer readable media and interfaces for various other types of computer readable media. Electronic system 1300 includes a bus 1305, processing unit (s) 1310, a graphics-processing unit (GPU) 1315, a system memory 1320, a network 1325, a read-only memory 1330, a permanent storage device 1335, input devices 1340, and output devices 1345.

The bus 1305 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 1300. For instance, the bus 1305 communicatively connects the processing unit (s) 1310 with the GPU 1315, the read-only memory 1330, the system memory 1320, and the permanent storage device 1335.

From these various memory units, the processing unit (s) 1310 retrieves instructions to execute and data to process in order to execute the processes of the present disclosure. The processing unit (s) may be a single processor or a multi-core processor in different embodiments. Some instructions are passed to and executed by the GPU 1315. The GPU 1315 can offload various computations or complement the image processing provided by the processing unit (s) 1310.

The read-only-memory (ROM) 1330 stores static data and instructions that are used by the processing unit (s) 1310 and other modules of the electronic system. The permanent storage device 1335, on the other hand, is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 1300 is off. Some embodiments of the present disclosure use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 1335.

Other embodiments use a removable storage device (such as a floppy disk, flash memory device, etc., and its corresponding disk drive) as the permanent storage device. Like the permanent storage device 1335, the system memory 1320 is a read-and-write memory device. However, unlike storage device 1335, the system memory 1320 is a volatile read-and-write memory, such a random access memory. The system memory 1320 stores some of the instructions and data that the processor uses at runtime. In some embodiments, processes in accordance with the present disclosure are stored in the system memory 1320, the permanent storage device 1335, and/or the read-only memory 1330. For example, the various memory units include instructions for processing multimedia clips in accordance with some embodiments. From these various memory units, the processing unit (s) 1310 retrieves instructions to execute and data to process in order to execute the processes of some embodiments.

The bus 1305 also connects to the input and output devices 1340 and 1345. The input devices 1340 enable the user to communicate information and select commands to the electronic system. The input devices 1340 include alphanumeric keyboards and pointing devices (also called “cursor control devices” ) , cameras (e.g., webcams) , microphones or similar devices for receiving voice commands, etc. The output devices 1345 display images generated by the electronic system or otherwise output data. The output devices 1345 include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD) , as well as speakers or similar audio output devices. Some embodiments include devices such as a touchscreen that function as both input and output devices.

Finally, as shown in FIG. 13, bus 1305 also couples electronic system 1300 to a network 1325 through a network adapter (not shown) . In this manner, the computer can be a part of a network of computers (such as a local area network ( “LAN” ) , a wide area network ( “WAN” ) , or an Intranet, or a network of networks, such as the Internet. Any or all components of electronic system 1300 may be used in conjunction with the present disclosure.

Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media) . Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM) , recordable compact discs (CD-R) , rewritable compact discs (CD-RW) , read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM) , a variety of recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc. ) , flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc. ) , magnetic and/or solid state hard drives, read-only and recordablediscs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media may store a computer program that is executable by at least one processing unit and includes sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.

While the above discussion primarily refers to microprocessor or multi-core processors that execute software, many of the above-described features and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) . In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself. In addition, some embodiments execute software stored in programmable logic devices (PLDs) , ROM, or RAM devices.

As used in this specification and any claims of this application, the terms “computer” , “server” , “processor” , and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people. For the purposes of the specification, the terms display or displaying means displaying on an electronic device. As used in this specification and any claims of this application, the terms “computer readable medium, ” “computer readable media, ” and “machine readable medium” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals.

While the present disclosure has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the present disclosure can be embodied in other specific forms without departing from the spirit of the present disclosure. In addition, a number of the figures (including FIG. 9 and FIG. 12) conceptually illustrate processes. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed in one continuous series of operations, and different specific operations may be performed in different embodiments. Furthermore, the process could be implemented using several sub-processes, or as part of a larger macro process. Thus, one of ordinary skill in the art would understand that the present disclosure is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.
Additional Notes

The herein-described subject matter sometimes illustrates different components contained within, or connected with, different other components. It is to be understood that such depicted architectures are merely examples, and that in fact many other architectures can be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as "associated with" each other such that the desired functionality is achieved, irrespective of architectures or intermediate components. Likewise, any two components so associated can also be viewed as being "operably connected" , or "operably coupled" , to each other to achieve the desired functionality, and any two components capable of being so associated can also be viewed as being "operably couplable" , to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically mateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components.

Further, with respect to the use of substantially any plural and/or singular terms herein, those having skill in the art can translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity.

Moreover, it will be understood by those skilled in the art that, in general, terms used herein, and especially in the appended claims, e.g., bodies of the appended claims, are generally intended as “open” terms, e.g., the term “including” should be interpreted as “including but not limited to, ” the term “having” should be interpreted as “having at least, ” the term “includes” should be interpreted as “includes but is not limited to, ” etc. It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim recitation to implementations containing only one such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an, " e.g., “a” and/or “an” should be interpreted to mean “at least one” or “one or more; ” the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number, e.g., the bare recitation of "two recitations, " without other modifiers, means at least two recitations, or two or more recitations. Furthermore, in those instances where a convention analogous to “at least one of A, B, and C, etc. ” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention, e.g., “a system having at least one of A, B, and C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc. In those instances where a convention analogous to “at least one of A, B, or C, etc. ” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention, e.g., “a system having at least one of A, B, or C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc. It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” will be understood to include the possibilities of “A” or “B” or “A and B. ”

From the foregoing, it will be appreciated that various implementations of the present disclosure have been described herein for purposes of illustration, and that various modifications may be made without departing from the scope and spirit of the present disclosure. Accordingly, the various implementations disclosed herein are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

A video coding method comprising:

receiving data to be encoded or decoded as a current block of pixels of a current picture of a video;

constructing one or more merge candidate lists or sets for the current block, each merge candidate corresponding to a set of coding information that is inherited from a previously coded block,

wherein each merge candidate list or set is associated with only one mode category, wherein each merge candidate in a merge candidate list or set associated with a mode category corresponds to a set of coding information for a coding mode belonging to the mode category;

selecting a merge candidate from the constructed merge candidate lists or sets;

generating a prediction of the current block according to a target mode using the set of coding information corresponding to the selected merge candidate; and

encoding or decoding the current block by using the generated prediction.
The video coding method of claim 1, further comprising selecting one merge candidate list from the constructed merge candidate lists or sets based on the mode categories associated with the constructed merge candidate lists or sets, wherein the selected merge candidate is selected from the selected merge candidate list or set.
The video coding method of claim 1, wherein the constructed merge candidate lists or sets comprise at least first and second merge candidate lists or sets, wherein the first merge candidate list or set is associated with a first mode category and the second merge candidate list or set is associated with a second different mode category.
The video coding method of claim 1, wherein at least one of the constructed merge candidate lists or sets is associated with a mode category for only modes related to Decoder-Side Intra Mode Derivation (DIMD) .
The video coding method of claim 4, wherein the modes related to DIMD comprise any of DIMD, Merged Intra Mode Derivation (MIMD) , occurrence-based intra coding (OBIC) , decoder-derived intra prediction (DIP) merge mode with a mode type being DIMD, any mode using histograms to determine the intra prediction modes for the current block, any mode enabled by DIMD enabling flag, or any subset or extension of above-mentioned modes.
The video coding method of claim 1, wherein at least one of the constructed merge candidate lists or sets is associated with a mode category for only modes related to Template-Based Intra Mode Derivation (TIMD) .
The video coding method of claim 6, wherein the modes related to TIMD comprise any of TIMD, Template-based multiple reference line intra prediction (TMRL) , DIP merge with a mode type being TIMD or TMRL, any mode using distortion between reconstruction and prediction on the template to determine the intra prediction modes for the current block, any mode enabled by TIMD enabling flag, any mode enabled by TMRL enabling flag, or any subset or extension of above-mentioned modes.
The video coding method of claim 1, wherein at least one of the constructed merge candidate lists or sets is associated with a mode category for only multiple reference line (MRL) related modes.
The video coding method of claim 8, wherein the MRL-related modes comprise any of MRL, TMRL, DIP merge with a mode type being MRL or TMRL, any mode enabled by MRL enabling flag, any mode enabled by TMRL enabling flag, any mode using the reference line non-adjacent to the current block, or any subset or extension of above-mentioned modes.
The video coding method of claim 1, wherein at least one of the constructed merge candidate lists or sets is associated with a mode category for only model-related modes.
The video coding method of claim 10, wherein the model-related modes comprise Extrapolation filter-based intra prediction (EIP) , DIP merge with a mode type being EIP, matrix-related mode, parameter-related mode, any mode using model coefficients with input reference samples to derive the prediction for the current block, any mode enabled by an EIP enabling flag, or any subset or extension of above-mentioned modes.
The video coding method of claim 1, wherein at least one of the constructed merge candidate lists or sets is associated with a mode category for only matrix-related modes.
The video coding method of claim 12, wherein the matrix-related modes comprise any of matrix-based or matrix-weighted intra prediction (MIP) , DIP merge with a mode type being MIP, matrix-related mode, parameter-related mode, any mode enabled by a MIP enabling flag, any mode using a pre-defined or selected matrix with input reference samples to derive the prediction for the current block, or any subset or extension of above-mentioned modes.
The video coding method of claim 1, wherein at least one of the constructed merge candidate lists or sets is associated with a mode category for only decoder-analysis-related modes.
The video coding method of claim 14, wherein the decoder-analysis-related modes comprise any of DIMD-related modes, TIMD-related modes, any mode using decoder-side analysis to determine the intra prediction mode for the current block, or any subset or extension of above-mentioned modes.
The video coding method of claim 15, wherein the decoder-side analysis comprises a texture gradient analysis on a template or a distortion measurement on the template.
The video coding method of claim 1, wherein at least two merge candidate lists or sets are constructed for the current block.
An electronic apparatus comprising:

a video coder circuit configured to perform operations comprising:

receiving data to be encoded or decoded as a current block of pixels of a current picture of a video;

constructing one or more merge candidate lists or sets for the current block, each merge candidate corresponding to a set of coding information that is inherited from a previously coded block,

wherein each merge candidate list or set is associated with only one mode category, wherein each merge candidate in a merge candidate list or set associated with a mode category corresponds to a set of coding information for a coding mode belonging to the mode category;

selecting a merge candidate from the constructed merge candidate lists or sets;

generating a prediction of the current block according to a target mode using the set of coding information corresponding to the selected merge candidate; and

encoding or decoding the current block by using the generated prediction.
A video decoding method comprising:

receiving data to be decoded as a current block of pixels of a current picture of a video;

constructing one or more merge candidate lists or sets for the current block, each merge candidate providing a set of coding information that is inherited from a previously coded block,

wherein each merge candidate list or set is associated with only one mode category, wherein each merge candidate in a merge candidate list or set associated with a mode category provides a set of coding information for a coding mode belonging to the mode category;

selecting a merge candidate from the constructed merge candidate lists or sets;

generating a prediction of the current block according to a target mode using the set of coding information provided by the selected merge candidate; and

reconstructing the current block by using the generated prediction.