EP4690782A1 - High granularity decoder-side cross-component loop filter - Google Patents

Abstract

In accordance with example embodiments of the invention there is at least a method and an apparatus to perform obtaining for a set of samples at least one convolutional cross-component model filter, wherein the set of samples are reconstructed samples of two channels for an image; applying at least one filter to the set of reconstructed samples of the first channel; applying the at least one convolutional cross-component model filter using as input an output of the at least one filter to the set of reconstructed samples of the first channel; applying at least one filter to the output of the at least one convolutional cross-component model filter; and applying a cross component filter to the set of reconstructed samples of the first channel or to the output of the at least one filter to the set of reconstructed samples of the first channel, to obtain a correction for the output of the at least one filter using as input the output of the at least one convolutional cross-component model filter, as may be seen in FIG. 13.

Description

HIGH GRANULARITY DECODER-SIDE CROSS-COMPONENT LOOP FILTER TECHNICAL FIELD: [0001] The teachings in accordance with the exemplary embodiments of this invention relate generally to video coding and decoding and, more particularly, to improving chroma construction. BACKGROUND: [0002] This section is intended to provide a background or context to the invention that is recited in the claims. The description herein may include concepts that could be pursued, but are not necessarily ones that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, what is described in this section is not prior art to the description and claims in this application and is not admitted to be prior art by inclusion in this section. [0003] Certain abbreviations that may be found in the description and/or in the Figures are herewith defined as follows: ALF adaptive loop filter AMVR adaptive motion vector resolution CC cross component CC-ALF cross component adaptive loop filter CCLM cross component linear model intra prediction CTU coding tree unit CU central unit ISM industrial, scientific, medical ISP intra sub-partitions LM linear model LMS least mean square MRL multiple reference line MMLM multi-model LM MVD motion vector difference VVC versatile video codec WP weighted prediction [0004] Brief Description of Prior Developments [0005] Block-based processing is widely used in video coding, as it provides a good tradeoff between coding efficiency and computational complexity. Intra block copy tools are known to be able to generate a prediction for a current block. [0006] [0007] Example embodiments of this invention proposes improved operations for model monitoring procedures such as for beam prediction. SUMMARY: [0008] This section contains examples of possible implementations and is not meant to be limiting. [0009] In another example aspect of the invention, there is an apparatus, such as a user equipment side apparatus, comprising: at least one processor; and at least one non-transitory memory storing instructions, that when executed by the at least one processor, cause the apparatus at least to: obtain for a set of samples at least one convolutional cross-component model filter, wherein the set of samples are reconstructed samples of two channels for an image; apply at least one filter to the set of reconstructed samples of the first channel; apply the at least one convolutional cross- component model filter using as input an output of the at least one filter to the set of reconstructed samples of the first channel; apply at least one filter to the output of the at least one convolutional cross-component model filter; and apply a cross component filter to the set of reconstructed samples of the first channel or to the output of the at least one filter to the set of reconstructed samples of the first channel, to obtain a correction for the output of the at least one filter using as input the output of the at least one convolutional cross-component model filter. [0010] In still another example aspect of the invention, there is a method, comprising: obtaining for a set of samples at least one convolutional cross-component model filter, wherein the set of samples are reconstructed samples of two channels for an image; apply at least one filter to the set of reconstructed samples of the first channel; applying the at least one convolutional cross-component model filter using as input an output of the at least one filter to the set of reconstructed samples of the first channel; applying at least one filter to the output of the at least one convolutional cross- component model filter; and applying a cross component filter to the set of reconstructed samples of the first channel or to the output of the at least one filter to the set of reconstructed samples of the first channel, to obtain a correction for the output of the at least one filter using as input the output of the at least one convolutional cross- component model filter. [0011] A further example embodiment is an apparatus and a method comprising the apparatus and the method of the previous paragraphs, wherein there is adding an output of the the cross-component filter to the output of the at least one filter using as input the output of the at least one convolutional cross-component model filter, wherein the convolutional cross-component model filter is derived and applied for blocks that are not determined by any coding or prediction partitions, wherein said blocks have a square or rectangular shape, and wherein said blocks can be overlapped or distinct, wherein the usage of the convolutional cross-component model for each block is determined without additional signalling. [0012] A non-transitory computer-readable medium storing program code, the program code executed by at least one processor to perform at least the method as described in the paragraphs above. [0013] In yet another example aspect of the invention, there is an apparatus comprising: means for adding an output of the the cross-component filter to the output of the at least one filter using as input the output of the at least one convolutional cross- component model filter, wherein the convolutional cross-component model filter is derived and applied for blocks that are not determined by any coding or prediction partitions, wherein said blocks have a square or rectangular shape, and wherein said blocks can be overlapped or distinct, wherein the usage of the convolutional cross- component model for each block is determined without additional signalling. [0014] In accordance with the example embodiments as described in the paragraph above, at least the means for adding, deriving, applying, and determining comprises a network interface, and computer program code stored on a computer- readable medium and executed by at least one processor. [0015] A communication system comprising the network side apparatus and the user equipment side apparatus performing operations as described above. BRIEF DESCRIPTION OF THE DRAWINGS: [0016] The above and other aspects, features, and benefits of various embodiments of the present disclosure will become more fully apparent from the following detailed description with reference to the accompanying drawings, in which like reference signs are used to designate like or equivalent elements. The drawings are illustrated for facilitating better understanding of the embodiments of the disclosure and are not necessarily drawn to scale, in which: [0017] FIG.1A shows locations of the samples used for the derivation of α and β; [0018] FIG. 1B shows a Table 1 derivation of chroma prediction mode from luma mode when cclm_is enabled; [0019] FIG.1C shows a Table 2 unified binarization table for chroma prediction mode; [0020] FIG.2 shows Classification of luma samples into two classes used in the derivation of two sets of α and β Top and the sample domain bottom) the spatial domain [0021] FIG.3 shows locations of the samples used for the derivation of CCCM filter when six reference lines are used; and [0022] FIG.4 shows from left: 3-tap vertical, 3-tap horizontal, 5-tap cross, 25- tap diamond; [0023] FIG. 5 shows an example of four reference lines neighboring to a prediction block; [0024] FIG.6 shows a matrix weighted intra prediction process; [0025] FIG. 7 shows top and left neighboring blocks used in CIIP weight derivation; [0026] FIG. 8 shows ALF filter shapes (chroma: 5×5 diamond, luma: 7×7 diamond); [0027] FIG.9A shows a Subsampled Laplacian calculation; [0028] FIG.9B shows a table with a mapping of the gradient calculated for one block and the transformations; [0029] FIG. 10 shows (a) Placement of CC-ALF with respect to other loop filters (b) Diamond shaped filter; [0030] FIG. 11 and improved ALF pipeline with the CCCM based chroma update; [0031] FIG.12 shows FIG.12 shows a block diagram of one possible and non- limiting exemplary system in which the example embodiments may be practiced; and [0032] FIG. 13 shows a method in accordance with example embodiments of the invention which may be performed by an apparatus. DETAILED DESCRIPTION: [0033] In example embodiments of this invention there is proposed at least a method and apparatus for at least improving chroma construction for video coding and decoding. [0034] Hybrid video codecs, for example ITU-T H.263, H.264/AVC and HEVC, may encode the video information in two phases. At first, pixel values in a certain picture are (or “block”) are predicted for example by motion compensation means (finding and indicating an area in one of the previously coded video frames that corresponds closely to the block being coded) or by spatial means (using the pixel values around the block to be coded in a specified manner). In the first phase, predictive coding may be applied, for example, as so-called sample prediction and/or so-called syntax prediction. [0035] In the sample prediction, pixel or sample values in a certain picture area or "block" are predicted. These pixel or sample values can be predicted, for example, using one or more of motion compensation or intra prediction mechanisms. [0036] Motion compensation mechanisms (which may also be referred to as inter prediction, temporal prediction or motion-compensated temporal prediction or motion-compensated prediction or MCP) involve finding and indicating an area in one of the previously encoded video frames that corresponds closely to the block being coded. Inter prediction may reduce temporal redundancy. [0037] Intra prediction, where pixel or sample values can be predicted by spatial mechanisms, involve finding and indicating a spatial region relationship. Intra prediction utilizes the fact that adjacent pixels within the same picture are likely to be correlated. Intra prediction can be performed in spatial or transform domain, i.e., either sample values or transform coefficients can be predicted. Intra prediction is typically exploited in intra coding, where no inter prediction is applied. [0038] In the syntax prediction, which may also be referred to as parameter prediction, syntax elements and/or syntax element values and/or variables derived from syntax elements are predicted from syntax elements (de)coded earlier and/or variables derived earlier. Non-limiting examples of syntax prediction are provided below. [0039] In motion vector prediction, motion vectors e.g. for inter and/or inter- view prediction may be coded differentially with respect to a block-specific predicted motion vector. In many video codecs, the predicted motion vectors are created in a predefined way, for example by calculating the median of the encoded or decoded motion vectors of the adjacent blocks. Another way to create motion vector predictions, sometimes referred to as advanced motion vector prediction (AMVP), is to generate a list of candidate predictions from adjacent blocks and/or co-located blocks in temporal reference pictures and signalling the chosen candidate as the motion vector predictor. In addition to predicting the motion vector values, the reference index of previously coded/decoded picture can be predicted. The reference index is typically predicted from adjacent blocks and/or co-located blocks in temporal reference picture. Differential coding of motion vectors is typically disabled across slice boundaries. [0040] The block partitioning, e.g. from CTU to CUs and down to PUs, may be predicted. [0041] In filter parameter prediction, the filtering parameters e.g. for sample adaptive offset may be predicted. [0042] Prediction approaches using image information from a previously coded image can also be called as inter prediction methods which may also be referred to as temporal prediction and motion compensation. Prediction approaches using image information within the same image can also be called as intra prediction methods. [0043] Secondly, the prediction error, i.e. the difference between the predicted block of pixels and the original block of pixels, is coded. This may be done by transforming the difference in pixel values using a specified transform (e.g. Discrete Cosine Transform (DCT) or a variant of it), quantizing the coefficients and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, encoder can control the balance between the accuracy of the pixel representation (picture quality) and size of the resulting coded video representation (file size of transmission bitrate). [0044] In many video codecs, including H.264/AVC and HEVC, motion information is indicated by motion vectors associated with each motion compensated image block. Each of these motion vectors represents the displacement of the image block in the picture to be coded (in the encoder) or decoded (at the decoder) and the prediction source block in one of the previously coded or decoded images (or pictures). H.264/AVC and HEVC, as many other video compression standards, a picture is divided into a mesh of rectangles, for each of which a similar block in one of the reference pictures is indicated for inter prediction. The location of the prediction block is coded as a motion vector that indicates the position of the prediction block relative to the block being coded. [0045] In under developing Versatile Video Codec (VVC), there are the following new coding tools. (More description will be added later in the final patent draft if needed): ^ Intra prediction: – 67 intra mode with wide angles mode extension; – Block size and mode dependent 4 tap interpolation filter; – Position dependent intra prediction combination (PDPC); – Cross component linear model intra prediction (CCLM); – Multi-reference line intra prediction; – Intra sub-partitions; – Weighted intra prediction with matrix multiplication; ^ Inter-picture prediction: – Block motion copy with spatial, temporal, history-based, and pairwise average merging candidates; – Affine motion inter prediction; – sub-block based temporal motion vector prediction; – Adaptive motion vector resolution; – 8x8 block-based motion compression for temporal motion prediction; – High precision (1/16 pel) motion vector storage and motion compensation with 8-tap interpolation filter for luma component and 4- tap interpolation filter for chroma component; – Triangular partitions; – Combined intra and inter prediction; – Merge with MVD (MMVD); – Symmetrical MVD coding; – Bi-directional optical flow; – Decoder side motion vector refinement; – Bi-prediction with CU-level weight; ^ Transform, quantization and coefficients coding: – Multiple primary transform selection with DCT2, DST7 and DCT8; – Secondary transform for low frequency zone; – Sub-block transform for inter predicted residual; – Dependent quantization with max QP increased from 51 to 63; – Transform coefficient coding with sign data hiding; – Transform skip residual coding; – ^ Entropy Coding: – Arithmetic coding engine with adaptive double windows probability update; ^ In loop filter: – In-loop reshaping; – Deblocking filter with strong longer filter; – Sample adaptive offset; – Adaptive Loop Filter; ^ Screen content coding: – Current picture referencing with reference region restriction; ^ 360-degree video coding: – Horizontal wrap-around motion compensation; ^ High-level syntax and parallel processing: – Reference picture management with direct reference picture list signalling; – Tile groups with rectangular shape tile groups. [0046] Partitioning in VVC [0047] In VVC, each picture is divided into coding tree units (CTUs) similar to HEVC. A picture may also be divided into slices, tiles, bricks and sub-pictures. CTU may be split into smaller CUs using quaternary tree structure. Each CU may be divided using quad-tree and nested multi-type tree including ternary and binary split. [0048] There are specific rules to infer partitioning in in picture boundaries. [0049] The redundant split patterns are disallowed in nested multi-type partitioning. [0050] Cross-component linear model prediction (CCLM) [0051] To reduce the cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in the VVC, for which the chroma samples are predicted based on the reconstructed luma samples of the same CU by using a linear model as follows: pred_C (i,j)=α·rec_L'(i,j)+ β (3-1) where pred_^ ⁽i, j⁾ represents the predicted chroma samples in a CU and rec_^'⁽i, j⁾ represents the downsampled reconstructed luma samples of the same CU. [0052] The CCLM parameters (α and β) are derived with at most four neighbouring chroma samples and their corresponding down-sampled luma samples. Suppose the current chroma block dimensions are W×H, then W’ and H’ are set as: – W’ = W, H’ = H when LM mode is applied; – W’ =W + H when LM-A mode is applied; – H’ = H + W when LM-L mode is applied; [0053] The above neighbouring positions are denoted as S[ 0, −1 ]…S[ W’ − 1, −1 ] and the left neighbouring positions are denoted as S[ −1, 0 ]…S[ −1, H’ − 1 ]. Then the four samples are selected as: – S[W’ / 4, −1 ], S[ 3 * W’ / 4, −1 ], S[ −1, H’ / 4 ], S[ −1, 3 * H’ / 4 ] when LM mode is applied and both above and left neighbouring samples are available; – S[ W’ / 8, −1 ], S[ 3 * W’ / 8, −1 ], S[ 5 * W’ / 8, −1 ], S[ 7 * W’ / 8, −1 ] when LM-A mode is applied or only the above neighbouring samples are available; – S[ −1, H’ / 8 ], S[ −1, 3 * H’ / 8 ], S[ −1, 5 * H’ / 8 ], S[ −1, 7 * H’ / 8 ] when LM-L mode is applied or only the left neighbouring samples are available. [0054] The four neighbouring luma samples at the selected positions are down- sampled and compared four times to find two smaller values: x0A and x1A, and two larger values: x0B and x1B. Their corresponding chroma sample values are denoted as y0A, y1A, y0B and y1B. Then xA, xB, yA and yB are derived as: +1)>>1 (3-1) [0055] Finally, the linear model parameters ^ and ^ are obtained according to the following equations: ^ ^^ ^ = _{^ ^} ^_^^^_^ (3-2) β = ^_^ − α · ^_^ (3-3) [0056] FIG.1 shows an example of the location of the left and above samples and the sample of the current block involved in the CCLM mode. [0057] The division operation to calculate parameter α is implemented with a look-up table. To reduce the memory required for storing the table, the diff value (difference between maximum and minimum values) and the parameter α are expressed by an exponential notation. For example, diff is approximated with a 4-bit significant part and an exponent. Consequently, the table for 1/diff is reduced into 16 elements for 16 values of the significand as follows: DivTable [ ] = { 0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0 } (3-4) [0058] This would have a benefit of both reducing the complexity of the calculation as well as the memory size required for storing the needed tables. [0059] Besides the above template and left template can be used to calculate the linear model coefficients together, they also can be used alternatively in the other 2 LM modes, called LM_A, and LM_L modes. [0060] In LM_A mode, only the above template is used to calculate the linear model coefficients. To get more samples, the above template is extended to (W+H). In LM_L mode, only left template is used to calculate the linear model coefficients. To get more samples, the left template is extended to (H+W). [0061] For a non-square block, the above template is extended to W+W, the left template is extended to H+H. [0062] To match the chroma sample locations for 4:2:0 video sequences, two types of down-sampling filter are applied to luma samples to achieve 2 to 1 down- sampling ratio in both horizontal and vertical directions. The selection of down- sampling filter is specified by a SPS level flag. The two down-sampling filters are as follows, which are corresponding to “type-0” and “type-2” content, respectively. Rec_^′(^, ^) = [0063] Note that only one luma line (general line buffer in intra prediction) is used to make the down-sampled luma samples when the upper reference line is at the CTU boundary. [0064] This parameter computation is performed as part of the decoding process and is not just as an encoder search operation. As a result, no syntax is used to convey the α and β values to the decoder. [0065] For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. Those modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). Chroma mode signalling and derivation process are shown in Table 1 of FIG. 1B. Chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partitioning structure for luma and chroma components is enabled in I slices, one chroma block may correspond to multiple luma blocks. Therefore, for Chroma DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited. [0066] In Table 2 of FIG.1C, the first bin indicates whether it is regular (0) or LM modes (1). If it is LM mode, then the next bin indicates whether it is LM_CHROMA (0) or not. If it is not LM_CHROMA, next 1 bin indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first bin of the binarization table for the corresponding intra_chroma_pred_mode can be discarded prior to the entropy coding. Or, in other words, the first bin is inferred to be 0 and hence not coded. This single binarization table is used for both sps_cclm_enabled_flag equal to 0 and 1 cases. The first two bins in Error! Reference source not found.2 are context coded with its own context model, and the rest bins are bypass coded. [0067] In addition, in order to reduce luma-chroma latency in dual tree, when the 64x64 luma coding tree node is partitioned with Not Split (and ISP is not used for the 64x64 CU) or QT, the chroma CUs in 32x32 / 32x16 chroma coding tree node are allowed to use CCLM in the following way: – If the 32x32 chroma node is not split or partitioned QT split, all chroma CUs in the 32x32 node can use CCLM; – If the 32x32 chroma node is partitioned with Horizontal BT, and the 32x16 child node does not split or uses Vertical BT split, all chroma CUs in the 32x16 chroma node can use CCLM. [0068] In all the other luma and chroma coding tree split conditions, CCLM is not allowed for chroma CU. [0069] Multi-model LM (MMLM) [0070] The CCLM included in VVC is extended by adding three Multi-model LM (MMLM) modes. In each MMLM mode, the reconstructed neighbouring samples are classified into two classes using a threshold which is the average of the luma reconstructed neighbouring samples. The linear model of each class is derived using the Least-Mean-Square (LMS) method. For the CCLM mode, the LMS method is also used to derive the linear model. FIG.2 illustrates two luma-to-chroma models obtained for luma (Y) threshold of 17. Each luma-to-chroma model has its own linear model parameters α and β. As can be seen from the bottom figure, each luma-to-chroma model corresponds to a spatial segmentation of the content (i.e., they correspond to different objects or textures in the scene). [0071] Convolutional cross-component model (CCCM) [0072] An improved version of cross-component prediction, known as CCCM, uses 2D filter kernel to derive the luma-to-chroma model. The filter coefficients are derived decoder-side using reconstructed set of input data and chroma samples. For the filter coefficient derivation, co-located reference sample areas (consisting of reconstructed luma and chroma samples) are defined for both luma and chroma as shown in FIG. 3 where the typically used 4:2:0 chroma down-sampling has been applied. The reference sample area for a given block can be, for example, six lines above and left as shown in FIG. 3, yet any number of reference lines (that can be realized by both the encoder and decoder) can be used. Generally, reference samples can contain any chroma and luma samples that have been reconstructed by both the encoder and decoder. Once the reference samples are determined the filter coefficients can be derived, for example, using different types of linear regression tools such as ordinary least-squares estimation, orthogonal matching pursuit, optimized orthogonal matching pursuit, ridge regression, or least absolute shrinkage and selection operator. [0073] The dimensions of the filter kernel can be for example 1^3 (1D vertical), 3^1 (1D horizontal), 3^3, 7^7 or any dimensions, and can be shaped (by selecting only a subset of all possible kernel locations) as a cross or a diamond (as shown in FIG.4) or as any given shape. When referring to the samples within the filter kernel the following notation is used: north (above), east (right), south (below), west (left) and center, as illustrated in FIG.4 using the letters N, E, S, W, C. [0074] The overall method of reconstructing chroma samples using convolution between a decoder-side obtained filter kernel and a set of input data is referred to as convolutional cross-component model (CCCM) here. The following steps can be applied to perform a CCCM operation: 1. Define co-located reference areas over the luma and chroma components; 2. Down-sample the luma samples to match the chroma grid (optional); 3. Scan the luma and chroma samples of the reference area and collect available statistics (such as auto-correlation matrix and cross-correlation vector) based on the filter shape; 4. Solve the filter coefficients by minimizing squared-error (or any other metric) based on the available statistics (such as the auto-correlation matrix and cross-correlation vector); 5. Calculate a predicted chroma block by convolving the down-sampled luma samples with the filter kernel. [0075] Let us define the (possibly down-sampled) luma samples as a 2D array ^(^, ^) indexed using horizontal ^-coordinate and vertical ^-coordinate. Let us also define the co-located chroma samples as a 2D array ^(^, ^) and the filter kernel (i.e., coefficients) as 3^3 array ^(^, ^). On a sample level there is defined the convolution between ^ and ^ as: [0076] When using other data terms, such as the non-linear square-root term, the appended convolution becomes, where ^^{^} are filter coefficients that reside outside of the 2D filter kernel yet have been obtained as a part of the system of linear equations that were used to solve the 2D filter coefficients in Step 4 above. Similarly, the bias term can be added to the convolution with, [0077] Multiple reference line (MRL) intra prediction [0078] Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. In FIG. 5, an example of 4 reference lines is depicted, where the samples of segments A and F are not fetched from reconstructed neighbouring samples but padded with the closest samples from Segment B and E, respectively. HEVC intra- picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, 2 additional lines (reference line 1 and reference line 3) are used. [0079] The index of selected reference line (mrl_idx) is signalled and used to generate intra predictor. For reference line idx, which is greater than 0, only include additional reference line modes in MPM list and only signal mpm index without remaining mode. The reference line index is signalled before intra prediction modes, and Planar mode is excluded from intra prediction modes in case a nonzero reference line index is signalled. [0080] MRL is disabled for the first line of blocks inside a CTU to prevent using extended reference samples outside the current CTU line. Also, PDPC is disabled when additional line is used. For MRL mode, the derivation of DC value in DC intra prediction mode for non-zero reference line indices are aligned with that of reference line index 0. MRL requires the storage of 3 neighbouring luma reference lines with a CTU to generate predictions. The Cross-Component Linear Model (CCLM) tool also requires 3 neighbouring luma reference lines for its down-sampling filters. The definition of MLR to use the same 3 lines is aligned as CCLM to reduce the storage requirements for decoders. [0081] Intra sub-partitions (ISP) [0082] The intra sub-partitions (ISP) divides luma intra-predicted blocks vertically or horizontally into 2 or 4 sub-partitions depending on the block size. For example, minimum block size for ISP is 4x8 (or 8x4). If block size is greater than 4x8 (or 8x4) then the corresponding block is divided by 4 sub-partitions. It has been noted that the ^ × 128 (with ^ ≤ 64) and 128 × ^ (with ^ ≤ 64) ISP blocks could generate a potential issue with the 64 × 64 VDPU. For example, an ^ × 128 CU in ^ the single tree case has an ^ × 128 luma TB and two corresponding ^ × 64 chroma TBs. If the CU uses ISP, then the luma TB will be divided into four ^ × 32 TBs (only the horizontal split is possible), each of them smaller than a 64 × 64 block. However, in the current design of ISP chroma blocks are not divided. Therefore, both chroma components will have a size greater than a 32 × 32 block. Analogously, a similar situation could be created with a 128 × ^ CU using ISP. Hence, these two cases are an issue for the 64 × 64 decoder pipeline. For this reason, the CU sizes that can use ISP is restricted to a maximum of 64 × 64. All sub-partitions fulfil the condition of having at least 16 samples. [0083] Matrix weighted Intra Prediction (MIP) [0084] Matrix weighted intra prediction (MIP) method is a newly added intra prediction technique into VVC. For predicting the samples of a rectangular block of width ^ and height ^, matrix weighted intra prediction (MIP) takes one line of H reconstructed neighbouring boundary samples left of the block and one line of ^ reconstructed neighbouring boundary samples above the block as input. If the reconstructed samples are unavailable, they are generated as it is done in the conventional intra prediction. The generation of the prediction signal is based on the following three steps, which are averaging, matrix vector multiplication and linear interpolation as shown in FIG.6. [0085] Inter prediction in VVC [0086] Merge list may include the following candidates: 1) Spatial MVP from spatial neighbour CUs; 2) Temporal MVP from collocated CUs; 3) History-based MVP from a FIFO table; 4) Pairwise average MVP (using the candidates already in the list); 5) Zero MVs. [0087] Merged mode width motion vector difference (MMVD) is to signal MVDs and a resolution index after signaling merge candidate. [0088] In Symmetric MVD, motion information of list-1 are derived from motion information of list-0 in bi-prediction case. [0089] In Affine prediction, several motion vectors are indicated/signaled for different corners of a block, which are used to derive the motion vectors of sub-block. In affine merge, affine motion information of a block is generated based on the normal or affine motion information of the neighboring blocks. [0090] In Sub-block-based temporal motion vector prediction, motion vectors of sub-blocks of the current block are predicted from a proper subblocks in the reference frame which are indicated by the motion vector of a spatial neighboring block (if available). [0091] In Adaptive motion vector resolution (AMVR), precision of MVD is signaled for each CU. [0092] In Bi-prediction with CU-level weight, an index indicated the weight values for weighted average of two prediction block. [0093] Bi-directional optical flow (BDOF) refines the motion vectors in bi- prediction case. BDOF generates two prediction blocks using the signaled motion vectors. Then a motion refinement is calculated two minimize the error between two prediction blocks using their gradient values. The final prediction blocks are refined using the motion refinement and gradient values. [0094] Bi-prediction with CU-level weight (BCW) and weighted prediction (WP) [0095] In HEVC, the bi-prediction signal is generated by averaging two prediction signals obtained from two different reference pictures and/or using two different motion vectors. In VVC, the bi-prediction mode is extended beyond simple averaging to allow weighted averaging of the two prediction signals. [0096] Five weights are allowed in the weighted averaging bi-prediction, ^ ∈ {−2, 3, 4, 5, 10}. For each bi-predicted CU, the weight w is determined in one of two ways: 1) for a non-merge CU, the weight index is signalled after the motion vector difference; 2) for a merge CU, the weight index is inferred from neighbouring blocks based on the merge candidate index. BCW is only applied to CUs with 256 or more luma samples (i.e., CU width times CU height is greater than or equal to 256). For low- delay pictures, all 5 weights are used. For non-low-delay pictures, only 3 weights (w∈{3,4,5}) are used. – At the encoder, fast search algorithms are applied to find the weight index without significantly increasing the encoder complexity. These algorithms are summarized as follows. For further details readers are referred to the VTM software and document JVET-L0646. When combined with AMVR, unequal weights are only conditionally checked for 1-pel and 4-pel motion vector precisions if the current picture is a low-delay picture; – When combined with affine, affine ME will be performed for unequal weights if and only if the affine mode is selected as the current best mode; – When the two reference pictures in bi-prediction are the same, unequal weights are only conditionally checked; – Unequal weights are not searched when certain conditions are met, depending on the POC distance between current picture and its reference pictures, the coding QP, and the temporal level. [0097] The BCW weight index is coded using one context coded bin followed by bypass coded bins. The first context coded bin indicates if equal weight is used; and if unequal weight is used, additional bins are signalled using bypass coding to indicate which unequal weight is used. [0098] Weighted prediction (WP) is a coding tool supported by the H.264/AVC and HEVC standards to efficiently code video content with fading. Support for WP was also added into the VVC standard. WP allows weighting parameters (weight and offset) to be signalled for each reference picture in each of the reference picture lists L0 and L1. Then, during motion compensation, the weight(s) and offset(s) of the corresponding reference picture(s) are applied. WP and BCW are designed for different types of video content. In order to avoid interactions between WP and BCW, which will complicate VVC decoder design, if a CU uses WP, then the BCW weight index is not signalled, and w is inferred to be 4 (i.e. equal weight is applied). For a merge CU, the weight index is inferred from neighbouring blocks based on the merge candidate index. This can be applied to both normal merge mode and inherited affine merge mode. For constructed affine merge mode, the affine motion information is constructed based on the motion information of up to 3 blocks. The BCW index for a CU using the constructed affine merge mode is simply set equal to the BCW index of the first control point MV. [0099] In VVC, CIIP and BCW cannot be jointly applied for a CU. When a CU is coded with CIIP mode, the BCW index of the current CU is set to 2, e.g. equal weight. [00100] Combined inter and intra prediction (CIIP) [00101] In VVC, when a CU is coded in merge mode, if the CU contains at least 64 luma samples (that is, CU width times CU height is equal to or larger than 64), and if both CU width and CU height are less than 128 luma samples, an additional flag is signalled to indicate if the combined inter/intra prediction (CIIP) mode is applied to the current CU. As its name indicates, the CIIP prediction combines an inter prediction signal with an intra prediction signal. The inter prediction signal in the CIIP mode ^_^^^^^ is derived using the same inter prediction process applied to regular merge mode; and the intra prediction signal ^_^^^^^ is derived following the regular intra prediction process with the planar mode. Then, the intra and inter prediction signals are combined using weighted averaging, where the weight value is calculated depending on the coding modes of the top and left neighbouring blocks (depicted in Error! Reference source not found.) as follows: – If the top neighbor is available and intra coded, then set isIntraTop to 1, otherwise set isIntraTop to 0; – If the left neighbor is available and intra coded, then set isIntraLeft to 1, otherwise set isIntraLeft to 0; – If (isIntraLeft + isIntraTop) is equal to 2, then wt is set to 3; – Otherwise, if (isIntraLeft + isIntraTop) is equal to 1, then wt is set to 2; – Otherwise, set wt to 1. [00102] LIC is an inter prediction technique to model local illumination variation between current block and its prediction block as a function of that between current block template and reference block template. The parameters of the function can be denoted by a scale α and an offset β, which forms a linear equation, that is, α*p[x]+β to compensate illumination changes, where p[x] is a reference sample pointed to by MV at a location x on reference picture. Since α and β can be derived based on current block template and reference block template, no signaling overhead is required for them, except that an LIC flag is signaled for AMVP mode to indicate the use of LIC. [00103] The local illumination compensation proposed in JVET-O0066 is used in ECM for uni-prediction inter CUs with the following modifications. • Intra neighbor samples can be used in LIC parameter derivation; • LIC is disabled for blocks with less than 32 luma samples; • For both non-subblock and affine modes, LIC parameter derivation is performed based on the template block samples corresponding to the current CU, instead of partial template block samples corresponding to first top-left 16x16 unit; • Samples of the reference block template are generated by using MC with the block MV without rounding it to integer-pel precision. [00104] Handling of out-of-boundary samples (OOB) [00105] In bi-directional motion compensation the out of boundary (OOB) prediction samples are discarded and only the non-OOB predictors, when available, are used to generate the final predictor. Specifically, let ^^^_^_^,^ and ^^^_^_^,^ denote the position of one prediction sample in one current block, ^^_^^^ ^^ ^_,^ and ^^_^_^,^ (x = 0,1) denote the MV of the current block; ^^^_^^^^^^^^ , ^^^_^^^^^^^^^ , ^^^_^^^^^^^ and ^^^_^^^^^^^^^^ are the positions of four boundaries of the picture. One prediction sample is regarded as OOB when at least one of the following conditions is satisfied: (^^^_^_^,^ + ^^_^^^ ^_,^) > (^^^_^^^^^^^^^+half_pixel), (^^^_^_^,^ + ^^_^^^ ^_,^) < (^^^_^^^^^^^^- half_pixel), where half_pixel is equal to 8 that represents the half-pel sample distance in the 1/16- pel sample precision. After examining the OOB condition for each sample, the final prediction samples of one bi-directional block are generated as follows: If ^^^ ^_,^ is OOB and ^^^ ^_,^ is non-OOB ^ ^^^^^ = ^^^ ^,^ _^,^ else if ^^^ ^_,^ is non-OOB and ^^^ ^_,^ is OOB ^ ^^^^^ ^^ ^,^ = ^_^,^ else OOB checking process is also applicable when BCW is enabled. [00106] Finally, note this sample-adaptive bi-prediction process only applies to prediction units for which at least a reference bock is first detected as partially or entirely out-of-bounds. Thus, a block-level OOB criteria is first checked. If both prediction blocks are non-OOB, then the usual bi-prediction takes place. [00107] In-loop filters [00108] There are totally three in-loop filters in VVC. Besides deblocking filter and SAO (the two loop filters in HEVC), adaptive loop filter (ALF) are applied. The ALF comprises of luma ALF, chroma ALF and cross-component ALF (CC-ALF). The ALF filtering process is designed so that luma ALF, chroma ALF and CC-ALF can be executed in parallel. The order of the filtering process in the VVC is the deblocking filter, SAO and ALF. The SAO in VVC is the same as that in HEVC. [00109] In VVC, an Adaptive Loop Filter (ALF) with block-based filter adaption is applied. For the luma component, one among 25 filters is selected for each 4×4 block, based on the direction and activity of local gradients. [00110] Two diamond filter shapes (as shown in FIG. 8) are used. The 7×7 diamond shape is applied for luma component and the 5×5 diamond shape is applied for chroma components. [00111] For luma component, each 4 × 4 block is categorized into one out of 25 classes. The classification index C is derived based on its directionality ^ and a quantized value of activity ^^{^}, as follows: ^ = 5^ + ^^{^} To calculate ^ and ^^{^}, gradients of the horizontal, vertical and two diagonal directions are first calculated using 1-D Laplacian: Where indices ^ and ^ refer to the coordinates of the upper left sample within the 4 × 4 block and ^(^, ^) indicates a reconstructed sample at coordinate (^, ^). [00112] To reduce the complexity of block classification, the subsampled 1-D Laplacian calculation is applied. As shown in FIG. 9, the same subsampled positions are used for gradient calculation of all directions. [00113] Then ^ maximum and minimum values of the gradients of horizontal and vertical directions are set as: ^^^^ ^_,^ = ^^^⁽^_^ , ^_^ ⁾, ^^^^ ^_,^ = ^^^⁽^_^ , ^_^ ⁾ The maximum and minimum values of the gradient of two diagonal directions are set as: To derive the value of the directionality ^, these values are compared against each other and with two thresholds ^_^ and ^_^: Step 1. If both are true, ^ is set to 0; Step 2. If ^^^^ ^_,^ ^⁄ ^_^ ^{^} ,_^ ^{^^} > ^ ^^^ ^_^,^^ ^⁄ ^_^ ^{^} ^^{^} ,^{^} ^_^ , continue from Step 3, otherwise continue from Step 4; Step 3. set to 2; otherwise ^ is set to 1; Step 4. set to 4; otherwise ^ is set to 3. [00114] The activity value ^ is calculated as: ^ is further quantized to the range of 0 to 4, inclusively, and the quantized value is denoted as ^^{^}. [00115] For chroma components in a picture, no classification method is applied. [00116] Before filtering each 4×4 luma block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients ^(^, ^) and to the corresponding filter clipping values ^(^, ^) depending on gradient values calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks to which ALF is applied more similar by aligning their directionality. [00117] Three geometric transformations, including diagonal, vertical flip and rotation are introduced: Diagonal: ^_^ ⁽^, ^⁾ = ^⁽^, ^⁾, ^_^ ⁽^, ^⁾ = ^⁽^, ^⁾, Vertical flip: ^_^ ⁽^, ^⁾ = ^⁽^, ^ − ^ − 1⁾, ^_^ ⁽^, ^⁾ = ^⁽^, ^ − ^ − 1⁾, Rotation: ^_^(^, ^) = ^(^ − ^ − 1, ^), ^_^(^, ^) = ^(^ − ^ − 1, ^), where ^ is the size of the filter and 0 ≤ ^, ^ ≤ ^ − 1 are coefficients coordinates, such that location (0,0) is at the upper left corner and location (^ − 1, ^ − 1) is at the lower right corner. The transformations are applied to the filter coefficients f (k, l) and to the clipping values ^(^, ^) depending on gradient values calculated for that block. The relationship between the transformation and the four gradients of the four directions are summarized in the following table. [00118] At decoder side, when ALF is enabled for a CTB, each sample ^(^, ^) within the CU is filtered, resulting in sample value ^′(^, ^) as shown below: ^^{^}(^, ^) = ^(^, ^) where ^(^, ^) denotes the decoded filter coefficients, is the clipping function and ^(^, ^) denotes the decoded clipping parameters. The variable k and l varies between − where L denotes the filter length. [00119] The clipping function ^⁽^, ^⁾ = min (^, max⁽−^, ^⁾) which corresponds to the function ^^^^3 ⁽−^, ^, ^⁾. The clipping operation introduces non- linearity to make ALF more efficient by reducing the impact of neighbor sample values that are too different with the current sample value. [00120] CC-ALF uses luma sample values to refine each chroma component by applying an adaptive, linear filter to the luma channel and then using the output of this filtering operation for chroma refinement. FIG.10 (a) provides a system level diagram of the CC-ALF process with respect to the SAO, luma ALF and chroma ALF processes. [00121] Filtering in CC-ALF is accomplished by applying a linear, diamond shaped filter (FIG. 10 (b)) to the luma channel. One filter is used for each chroma channel, and the operation is expressed as where ⁽^, ^⁾ is chroma component i location being refined ⁽^_^ , ^_^ ⁾ is the luma location based on (^, ^), ^_^ is filter support area in luma component, ^_^ ⁽^_^, ^_^ ⁾ represents the filter coefficients [00122] As shown in FIG. 10, the luma filter support is the region collocated with the current chroma sample after accounting for the spatial scaling factor between the luma and chroma planes. [00123] In the VVC reference software, CC-ALF filter coefficients are computed by minimizing the mean square error of each chroma channels with respect to the original chroma content. To achieve this, the VTM algorithm uses a coefficient derivation process similar to the one used for chroma ALF. Specifically, a correlation matrix is derived, and the coefficients are computed using a Cholesky decomposition solver in an attempt to minimize a mean square error metric. In designing the filters, a maximum of 8 CC-ALF filters can be designed and transmitted per picture. The resulting filters are then indicated for each of the two chroma channels on a CTU basis. [00124] Adaptive in-loop filters (such as ALF in VVC) perform luma and chroma filtering in parallel fashion to lower the mean square error (MSE) between the reconstruction and the original samples. More specifically, in VVC the ALF first filters both luma and chroma in parallel, and subsequently also applies cross-component filtering (CC-ALF) to further improve the chroma, see FIG. 10a. The filters are signalled at a coarse spatial granularity meaning that filters change at most at CTU level. [00125] The CC-ALF in VVC uses as input the luma samples before ALF and therefore neglects any improvement obtained from luma ALF. Even if CC-ALF considered the ALF filtered luma as input, there would be little benefit since CC-ALF uses Wiener filters (that are signalled at CTU level) to improve the chroma and at this scale small changes to the input of the Wiener filter derivation does not change the output a lot (i.e., the filter coefficients are slightly different, but the output is pretty much the same). [00126] Instead of CTU level filters more localized (and independent of coding or prediction partitioning) luma-to-chroma filters are required to fully convert luma improvements into chroma improvements but signalling of such filters is prohibited by the significant signalling cost. [00127] In order to improve the usage of luma output of ALF for improving the chroma reconstruction example embodiments of the invention apply filters that can directly convert the luma improvements into chroma improvements before chroma ALF or CC-ALF are applied. These filters need to be derived and applied locally at fine spatial granularity (i.e., distinct filters for each e.g., 4x4 block). In accordance with example embodiments of the invention the convolutional cross-component model (CCCM) is used to map the improved luma into an improved chroma without additional signalling. A variant of such a method is already described in [1] where the luma residual is mapped into chroma correction using CCCM. [00128] Before describing the example embodiments as disclosed herein in detail, reference is made to FIG.12 for illustrating a simplified block diagram of various electronic devices that are suitable for use in practicing the example embodiments of this invention. [00129] FIG. 12 shows a block diagram of one possible and non-limiting exemplary system in which the example embodiments may be practiced. In FIG.12, a user equipment (UE) 10 is in wireless communication with a wireless network 1 or network, 1 as in FIG. 12. The wireless network 1 or network 1 as in FIG. 12 can comprise a communication network such as a mobile network e.g., the mobile network 1 or first mobile network as disclosed herein. Any reference herein to a wireless network 1 as in FIG.12 can be seen as a reference to any wireless network as disclosed herein. Further, the wireless network 1 as in FIG. 12 can also comprises hardwired features as may be required by a communication network. A UE is a wireless, typically mobile device that can access a wireless network. The UE, for example, may be a mobile phone (or called a "cellular" phone) and/or a computer with a mobile terminal function. For example, the UE or mobile terminal may also be a portable, pocket, handheld, computer-embedded or vehicle-mounted mobile device and performs a language signaling and/or data exchange with the RAN. [00130] The UE 10 includes one or more processors DP 10A, one or more memories MEM 10B, and one or more transceivers TRANS 10D interconnected through one or more buses. Each of the one or more transceivers TRANS 10D includes a receiver and a transmitter. The one or more buses may be address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, and the like. The one or more transceivers TRANS 10D which can be optionally connected to one or more antennas for communication to NN 12 and ISM 13, respectively. The one or more memories MEM 10B include computer program code PROG 10C. The UE 10 communicates with NN 12 and/or ISM 13 via a wireless link 11 or 16. [00131] The NN 12 (NR/5G Node B, an evolved NB, or LTE device) is a network node such as a master or secondary node base station (e.g., for NR or LTE long term evolution) that communicates with devices such as ISM 13 and UE 10 of FIG.12. The NN 12 provides access to wireless devices such as the UE 10 to the wireless network 1. The NN 12 includes one or more processors DP 12A, one or more memories MEM 12B, and one or more transceivers TRANS 12D interconnected through one or more buses. In accordance with the example embodiments these TRANS 12D can include X2 and/or Xn interfaces for use to perform the example embodiments. Each of the one or more transceivers TRANS 12D includes a receiver and a transmitter. The one or more transceivers TRANS 12D can be optionally connected to one or more antennas for communication over at least link 11 with the UE 10. The one or more memories MEM 12B and the computer program code PROG 12C are configured to cause, with the one or more processors DP 12A, the NN 12 to perform one or more of the operations as described herein. The NN 12 may communicate with another gNB or eNB, or a device such as the ISM 13 such as via link 16. Further, the link 11, link 16 and/or any other link may be wired or wireless or both and may implement, e.g., an X2 or Xn interface. Further the link 11 and/or link 16 may be through other network devices such as, but not limited to an NCE/MME/SGW/UDM/PCF/AMF/SMF/LMF 14 device as in FIG.12. The NN 12 may perform functionalities of an MME (Mobility Management Entity) or SGW (Serving Gateway), such as a User Plane Functionality, and/or an Access Management functionality for LTE and similar functionality for 5G. [00132] The ISM 13 can be for WiFi or Bluetooth or other wireless device associated with a mobility function device such as an AMF or SMF, further the ISM 13 may comprise a NR/5G Node B or possibly an evolved NB a base station such as a master or secondary node base station (e.g., for NR or LTE long term evolution) that communicates with devices such as the NN 12 and/or UE 10 and/or the wireless network 1. The ISM 13 includes one or more processors DP 13A, one or more memories MEM 13B, one or more network interfaces, and one or more transceivers TRANS 13D interconnected through one or more buses. In accordance with the example embodiments these network interfaces of ISM 13 can include X2 and/or Xn interfaces for use to perform the example embodiments. Each of the one or more transceivers TRANS 13D includes a receiver and a transmitter that can optionally be connected to one or more antennas. The one or more memories MEM 13B include computer program code PROG 13C. For instance, the one or more memories MEM 13B and the computer program code PROG 13C are configured to cause, with the one or more processors DP 13A, the ISM 13 to perform one or more of the operations as described herein. The ISM 13 may communicate with another mobility function device and/or eNB such as the NN 12 and the UE 10 or any other device using, e.g., link 11 or link 16 or another link. The Link 16 as shown in FIG.12 can be used for communication with the NN12. These links maybe wired or wireless or both and may implement, e.g., an X2 or Xn interface. Further, as stated above the link 11 and/or link 16 may be through other network devices such as, but not limited to an NCE/MME/SGW device such as the NCE/MME/SGW/UDM/PCF/AMF/SMF/LMF 14 of FIG.12. [00133] The one or more buses of the device of FIG.12 may be address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, wireless channels, and the like. For example, the one or more transceivers TRANS 12D, TRANS 13D and/or TRANS 10D may be implemented as a remote radio head (RRH), with the other elements of the NN 12 being physically in a different location from the RRH, and these devices can include one or more buses that could be implemented in part as fiber optic cable to connect the other elements of the NN 12 to a RRH. [00134] It is noted that although FIG.12 shows a network nodes such as NN 12 and ISM 13, any of these nodes may can incorporate or be incorporated into an eNodeB or eNB or gNB such as for LTE and NR, and would still be configurable to perform example embodiments. [00135] Also it is noted that description herein indicates that “cells” perform functions, but it should be clear that the gNB that forms the cell and/or a user equipment and/or mobility management function device that will perform the functions. In addition, the cell makes up part of a gNB, and there can be multiple cells per gNB. [00136] The wireless network 1 or any network it can represent may or may not include a NCE/MME/SGW/UDM/PCF/AMF/SMF/LMF 14 that may include (NCE) network control element functionality, MME (Mobility Management Entity)/SGW (Serving Gateway) functionality, and/or serving gateway (SGW), and/or MME (Mobility Management Entity) and/or SGW (Serving Gateway) functionality, and/or user data management functionality (UDM), and/or PCF (Policy Control) functionality, and/or Access and Mobility Management Function (AMF) functionality, and/or Session Management (SMF) functionality, and/or Location Management Function (LMF), and/or Authentication Server (AUSF) functionality and which provides connectivity with a further network, such as a telephone network and/or a data communications network (e.g., the Internet), and which is configured to perform any 5G and/or NR operations in addition to or instead of other standard operations at the time of this application. The NCE/MME/SGW/UDM/PCF/AMF/SMF/LMF 14 is configurable to perform operations in accordance with example embodiments in any of an LTE, NR, 5G and/or any standards based communication technologies being performed or discussed at the time of this application. In addition, it is noted that the operations in accordance with example embodiments, as performed by the NN 12 and/or ISM 13, may also be performed at the NCE/MME/SGW/UDM/PCF/AMF/SMF/LMF 14. [00137] The NCE/MME/SGW/UDM/PCF/AMF/SMF/LMF 14 includes one or more processors DP 14A, one or more memories MEM 14B, and one or more network interfaces (N/W I/F(s)), interconnected through one or more buses coupled with the link 13 and/or link 16. In accordance with the example embodiments these network interfaces can include X2 and/or Xn interfaces for use to perform the example embodiments. The one or more memories MEM 14B include computer program code PROG 14C. The one or more memories MEM14B and the computer program code PROG 14C are configured to, with the one or more processors DP 14A, cause the NCE/MME/SGW/UDM/PCF/AMF/SMF/LMF 14 to perform one or more operations which may be needed to support the operations in accordance with the example embodiments. [00138] It is noted that that the NN 12 and/or ISM 13 and/or UE 10 can be configured (e.g. based on standards implementations etc.) to perform functionality of a Location Management Function (LMF). The LMF functionality may be embodied in any of these network devices or other devices associated with these devices. In addition, an LMF such as the LMF of the MME/SGW/UDM/PCF/AMF/SMF/LMF 14 of FIG. 12, as at least described below, can be co-located with UE 10 such as to be separate from the NN 12 and/or ISM 13 of FIG.12 for performing operations in accordance with example embodiments as disclosed herein. [00139] The wireless Network 1 may implement network virtualization, which is the process of combining hardware and software network resources and network functionality into a single, software-based administrative entity, a virtual network. Network virtualization involves platform virtualization, often combined with resource virtualization. Network virtualization is categorized as either external, combining many networks, or parts of networks, into a virtual unit, or internal, providing network-like functionality to software containers on a single system. Note that the virtualized entities that result from the network virtualization are still implemented, at some level, using hardware such as processors DP10, DP12A, DP13A, and/or DP14A and memories MEM 10B, MEM 12B, MEM 13B, and/or MEM 14B, and also such virtualized entities create technical effects. [00140] The computer readable memories MEM 12B, MEM 13B, and MEM 14B may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor based memory devices, flash memory, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The computer readable memories MEM 12B, MEM 13B, and MEM 14B may be means for performing storage functions. The processors DP10, DP12A, DP13A, and DP14A may be of any type suitable to the local technical environment, and may include one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on a multi-core processor architecture, as non-limiting examples. The processors DP10, DP12A, DP13A, and DP14A may be means for performing functions, such as controlling the UE 10, NN 12, ISM 13, and other functions as described herein. [00141] In general, various embodiments of any of these devices can include, but are not limited to, cellular telephones such as smart phones, tablets, personal digital assistants (PDAs) having wireless communication capabilities, portable computers having wireless communication capabilities, image capture devices such as digital cameras having wireless communication capabilities, gaming devices having wireless communication capabilities, music storage and playback appliances having wireless communication capabilities, Internet appliances permitting wireless Internet access and browsing, tablets with wireless communication capabilities, as well as portable units or terminals that incorporate combinations of such functions. [00142] Further, the various embodiments of any of these devices can be used with a UE vehicle, a High Altitude Platform Station, or any other such type node associated with a terrestrial network or any drone type radio or a radio in aircraft or other airborne vehicle or a vessel that travels on water such as a boat. [00143] Example embodiments of the invention at least provide an improved ALF pipeline that uses CCCM to locally map ALF improved luma into improved chroma. The output of the CCCM filtering stage is then used as input to both ALF chroma and CC-ALF. Also example embodiments of the invention provide several embodiments to address more specific cases. [00144] In VVC, ALF uses Wiener filters to lower the MSE of luma and chroma reconstructions. The filter coefficients are either selected from a fixed set of coefficients using a signalled index or explicitly signalled from the encoder to the decoder. In both cases signalling is involved. A cross-component variant called CC-ALF improves chroma by applying a Wiener filter to the luma component in order to obtain a correction term for the chroma component, see FIG. 10. In all ALF related filter derivations, the goal is (at the encoder) to minimize the error against the original luma and chroma. [00145] The ALF pipeline in VVC at the decoder is summarized in the following steps, 1a. Apply luma ALF on the reconstructed luma; 2a. Apply chroma ALF on the reconstructed chroma; 3a. Apply CC-ALF on reconstructed luma (i.e., the input of Step 1a) to obtain a chroma correction; 4a. Add the output of Step 3a to the output of Step 2a. [00146] In example embodiments of this invention an improved ALF pipeline is introduced as shown in FIG.11. [00147] FIG.11 shows an improved ALF pipeline with the CCCM based chroma update. [00148] In the illustrated embodiment the luma input to CC-ALF is the output of ALF luma stage but it can be configured to be the output of SAO luma stage as well. Additionally, depending on the embodiment, the CCCM model derivation can be placed before the SAO stages. The optional input paths to CCCM filtering stage are used when some of the output samples of CCCM filtering stage are to be blended with the SAO output samples or some of the CCCM filtering output samples are to be exactly the SAO output samples. It should be understood, that all illustrated CCCM stages can be, if necessary, replaced with any cross-component prediction tool such as CCLM, GL-CCCM or any other cross-component prediction tool [00149] Most importantly an additional filtering stage is inserted between the ALF luma and ALF chroma stages. [00150] This filtering stage is based on the CCCM method and therefore requires no signalling and uses a different minimization goal than ALF. [00151] In CCCM the minimization goal is not the original chrominance, but the reconstructed chrominance and the model derivation is performed at the decoder as well. The following steps describe the improved pipeline at the decoder, 1b. Obtain CCCM filters at high spatial granularity using the reconstructed luma and chroma as reference samples; 2b. Apply luma ALF on the reconstructed luma; 3b. Apply the CCCM filters of Step 1b using the output of Step 2b as input; 4b. Apply chroma ALF on the output of Step 3b; 5b. Apply CC-ALF on either a) reconstructed luma OR b) output of Step 2b, to obtain a chroma correction; 6b. Add the output of Step 5b to the output of Step 4b. [00152] The output of Step 2b is luma that has higher quality than that used as input in Step 1b. When the higher quality luma is used as input in Step 3b there is essentially in accordance example embodiments of the invention obtaining a higher quality version of the chroma as output. CCCM filtering retains the color space characteristics (obtained by the model in Step 1b) but converts luminance corrections (such as corrected edges/gradients or smoothened/sharpened textures) into a higher quality version of the chroma. [00153] One of the main benefits of the improved ALF pipeline is the granularity at which CCCM model derivation and filtering happens. The CCCM models are derived and applied for small blocks that can be independent of coding and prediction partitions. For example, the said blocks can be 1x1, 2x2, 4x4, etc., and can have square or rectangular shapes. The blocks can also be overlapped or distinct. Compared to the CTU-level CC-ALF filters the proposed method can track cross-component model at high spatial precision and therefore also maps finely detailed luma improvements into chroma improvements. This is contrary to the CC-ALF in which the filter coefficients are changed only at CTU level and only 8 different filters are available per picture. With CCCM there can be thousands of filters, for example one for each 8x8 block, without any additional signalling cost. [00154] In the proposed improved pipeline, the CC-ALF can still be applied and still provides a benefit since the CC-ALF filters are derived at the encoder by minimizing squared error against the original samples. [00155] In Step 5b above, depending on the embodiment, the input to CC-ALF can be either the un-filtered reconstructed luma (i.e., input of Step 1b) or the ALF filtered luma (i.e., output of Step 2b). [00156] In the filtering stage in Step 3b the CCCM model performance can be considered. During model derivation the MSE of the model is obtained and if the said MSE is considered high the CCCM stage can be skipped for the given block. [00157] It should be understood, that all mentioned CCCM stages can be, if necessary, replaced with any cross-component prediction tool such as CCLM, GL- CCCM or any other cross-component prediction tool. For clarity, in the following embodiments the state-of-the-art cross-component prediction tool CCCM is used an example. [00158] In an embodiment the CCCM model derivation and filtering can be performed at any given granularity for example at 1x1, 2x2, 4x4, 8x8, 16x16, etc., blocks or using rectangular blocks such as 4x8 or 8x4. [00159] In an embodiment the blocks at which CCCM models are derived and applied can be distinct or overlapping. [00160] In an embodiment the CCCM blocks can be independent or dependent on the coding or prediction partitioning. For example, instead of using very fine granularity the blocks can follow the chroma or luma partitioning at some precision to accelerate the model derivation and filtering. [00161] In an embodiment during the CCCM model derivation with the MSE, the model can be examined at every block and if the MSE exceeds a given threshold the CCCM stage can be skipped for the given block. [00162] In an embodiment based on the previous embodiment the MSE threshold can be fixed or signalled from encoder to decoder. [00163] In an embodiment based on the previous embodiment the MSE threshold can be inferred based on underlying coding partitioning, luma sample values or chroma sample values. [00164] In an embodiment the CCCM model can have any number of filter coefficients. [00165] In an embodiment the CCCM model derivation and filtering can be replaced with simplified variants such as CCLM. [00166] In an embodiment the CCCM model derivation and filtering can be replaced with more advanced variants such as those considering the gradient and location information (such as GL-CCCM). [00167] In an embodiment the CCCM model derivation stage can be placed before or after the SAO stages. [00168] In an embodiment the CCCM model derivation stage can be placed at any point before the luma ALF stage. [00169] In an embodiment the luma input to the CC-ALF stages can be the input to the ALF luma stage or the output of the ALF luma stage. [00170] In an embodiment, the filter may consist of auxiliary information in order to guide the filter in such a way that it improves areas or samples with certain characteristics better. Examples of such auxiliary information are as below: ^ Input to the filter may include residual information from the luma block; ^ Input to the filter may include prediction information of the luma block; ^ Input to the filter may include one or more of the transform coefficients of the luma block; ^ Input to the filter may include inputs and/or outputs of the earlier filtering operations. For example, input and/or output of deblocking filter, input and/or output of SAO filter, input and/or output of ALF filter; ^ Input to the filter may include the difference of the inputs and outputs of the earlier filtering operations. For example, difference of input and output of deblocking filter, difference of input and output of SAO filter, difference of input and output of ALF filter. A scaling operation may be also applied to the difference values before feeding them to the CCCM filter. [00171] In an embodiment the usage of the CCCM stage can be signalled from encoder to decoder. [00172] In an embodiment, the usage of the filter may be signalled in different granularities, for example the signalling may be done per each filtering unit such as 1x1, 2x2, etc. Alternatively, the signalling may be done in different granularity than filtering granularity. For example, the activation of the filter may be done in coarser granularity, for each per CTU, and when the filtering mode is enabled for that CTU, the filtering may be done in finer granularity (e.g., 1x1, 2x2, 4x4, …). In this case, in the encoder side, the decision whether to use the filter or not may be done for the whole CTU based on rate-distortion optimization. [00173] In an embodiment, different types and/or sizes of the filter such as conventional CCCM, GL-CCCM, CCLM or any other variant may be decided for each CTU, slice, sub-picture, picture, or sequence level and signalled accordingly. [00174] In an embodiment the CCCM stage can be applied to all chroma components or to a specified set of chroma components. [00175] In an embodiment the proposed ALF pipeline can be used in any color space. [00176] In an embodiment an additional chroma-to-chroma CCCM stage can be inserted between the ALF Cb and ALF Cr stages therefore making the chroma ALF also sequential. [00177] In an embodiment the CCCM model derivation and filtering stage can use down sampled and/or original luma samples. [00178] In an embodiment, filtering units may consist of non-overlapping blocks. In this case, the filter derivation process for all filtering units may be done in parallel. In addition, once parameters of certain unit are derived, then the CCCM filtering may begin without too much latency in the pipeline. [00179] FIG.13 illustrates operations which may be performed by a device such as, but not limited to, a device (e.g., the UE 10 as in FIG.12). As shown in step 1310 of FIG. 13 there is obtaining for a set of samples at least one convolutional cross- component model filter. As shown in step 1320 of FIG.13 wherein the set of samples are reconstructed samples of two channels for an image. As shown in step 1330 of FIG. 13 there is applying at least one filter to the set of reconstructed samples of the first channel. As shown in step 1340 of FIG.13 there is applying at least one filter to the set of reconstructed samples of the first channel. As shown in step 1350 of FIG.13 there is applying the at least one convolutional cross-component model filter using as input an output of the at least one filter to the set of reconstructed samples of the first channel; As shown in step 1360 of FIG.13 there is applying at least one filter to the output of the at least one convolutional cross-component model filter. As shown in step 1370 of FIG. 13 there is applying a cross component filter to the set of reconstructed samples of the first channel or to the output of the at least one filter to the set of reconstructed samples of the first channel, to obtain a correction for the output of the at least one filter using as input the output of the at least one convolutional cross-component model filter. [00180] In accordance with the example embodiments as described in the paragraphs above, there is adding an output of the the cross-component filter to the output of the at least one filter using as input the output of the at least one convolutional cross-component model filter. [00181] In accordance with the example embodiments as described in the paragraphs above, wherein the convolutional cross-component model filter is derived and applied for blocks that are not determined by any coding or prediction partitions. [00182] In accordance with the example embodiments as described in the paragraphs above, wherein said blocks have a square or rectangular shape, and wherein said blocks can be overlapped or distinct. wherein the usage of the convolutional cross-component model for each block is determined without additional signalling. [00183] A non-transitory computer-readable medium (MEM 12B as in FIG.12) storing program code (PROG 10C as in FIG.12), the program code executed by at least one processor (DP 10A and/or DP 10F as in FIG. 12) to perform the operations as at least described in the paragraphs above. [00184] In accordance with an example embodiment of the invention as described above there is an apparatus comprising: there is means for adding (one or more transceivers 10D and/or one or more transceivers 13D; MEM 10B and/or MEM 13B; PROG 10C and/or PROG 13C; and DP 10A and/or DP 13A as in FIG. 12) an output of the the cross-component filter to the output of the at least one filter using as input the output of the at least one convolutional cross-component model filter, wherein the convolutional cross-component model filter is derived and applied (one or more transceivers 10D and/or one or more transceivers 13D; MEM 10B and/or MEM 13B; PROG 10C and/or PROG 13C; and DP 10A and/or DP 13A as in FIG.12) for blocks that are not determined by any coding or prediction partitions, wherein said blocks have a square or rectangular shape, and wherein said blocks can be overlapped or distinct, wherein the usage of the convolutional cross-component model for each block is determined (one or more transceivers 10D and/or one or more transceivers 13D; MEM 10B and/or MEM 13B; PROG 10C and/or PROG 13C; and DP 10A and/or DP 13A as in FIG.12) without additional signalling. [00185] In the example aspect of the invention according to the paragraph above, wherein at least the means for adding, deriving, applying, and determining comprises a non-transitory computer readable medium [MEM 10B and/or MEM 13B as in FIG.12] encoded with a computer program [PROG 10C and/or PRPG 13C as in FIG. 12] executable by at least one processor [DP 10A and/or DP 13A as in FIG.12]. [00186] Further, in accordance with example embodiments of the invention there is circuitry for performing operations in accordance with example embodiments of the invention as disclosed herein. This circuitry can include any type of circuitry including content coding circuitry, content decoding circuitry, processing circuitry, image generation circuitry, data analysis circuitry, etc.). Further, this circuitry can include discrete circuitry, application-specific integrated circuitry (ASIC), and/or field- programmable gate array circuitry (FPGA), etc. as well as a processor specifically configured by software to perform the respective function, or dual-core processors with software and corresponding digital signal processors, etc.). Additionally, there are provided necessary inputs to and outputs from the circuitry, the function performed by the circuitry and the interconnection (perhaps via the inputs and outputs) of the circuitry with other components that may include other circuitry in order to perform example embodiments of the invention as described herein. [00187] In accordance with example embodiments of the invention as disclosed in this application this application, the “circuitry” provided can include at least one or more or all of the following: (a) hardware-only circuit implementations (such as implementations in only analog and/or digital circuitry); (b) combinations of hardware circuits and software, such as (as applicable): (i) a combination of analog and/or digital hardware circuit(s) with software/firmware; and (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions, such as functions or operations in accordance with example embodiments of the invention as disclosed herein); and (c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.” [00188] In accordance with example embodiments of the invention, there is adequate circuitry for performing at least novel operations in accordance with example embodiments of the invention as disclosed in this application, this `circuitry` as may be used herein refers to at least the following: (a) hardware-only circuit implementations (such as implementations in only analog and/or digital circuitry); and (b) to combinations of circuits and software (and/or firmware), such as (as applicable): (i) to a combination of processor(s) or (ii) to portions of processor(s)/software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions); and (c) to circuits, such as a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation, even if the software or firmware is not physically present. [00189] This definition of `circuitry` applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term "circuitry" would also cover an implementation of merely a processor (or multiple processors) or portion of a processor and its (or their) accompanying software and/or firmware. The term "circuitry" would also cover, for example and if applicable to the particular claim element, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or other network device. [00190] In general, the various embodiments may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof. [00191] Embodiments of the inventions may be practiced in various components such as integrated circuit modules. The design of integrated circuits is by and large a highly automated process. Complex and powerful software tools are available for converting a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate. [00192] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. All of the embodiments described in this Detailed Description are exemplary embodiments provided to enable persons skilled in the art to make or use the invention and not to limit the scope of the invention which is defined by the claims. [00193] The foregoing description has provided by way of exemplary and non- limiting examples a full and informative description of the best method and apparatus presently contemplated by the inventors for carrying out the invention. However, various modifications and adaptations may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of example embodiments of this invention will still fall within the scope of this invention. [00194] It should be noted that the terms "connected," "coupled," or any variant thereof, mean any connection or coupling, either direct or indirect, between two or more elements, and may encompass the presence of one or more intermediate elements between two elements that are "connected" or "coupled" together. The coupling or connection between the elements can be physical, logical, or a combination thereof. As employed herein two elements may be considered to be "connected" or "coupled" together by the use of one or more wires, cables and/or printed electrical connections, as well as by the use of electromagnetic energy, such as electromagnetic energy having wavelengths in the radio frequency region, the microwave region and the optical (both visible and invisible) region, as several non-limiting and non-exhaustive examples. [00195] Furthermore, some of the features of the preferred embodiments of this invention could be used to advantage without the corresponding use of other features. As such, the foregoing description should be considered as merely illustrative of the principles of the invention, and not in limitation thereof.

Claims

CLAIMS What is claimed is: 1. An apparatus comprising: at least one processor; and at least one non-transitory memory storing instructions, that when executed by the at least one processor, cause the apparatus at least to: obtain for a set of samples at least one convolutional cross-component model filter, wherein the set of samples are reconstructed samples of two channels for an image; apply at least one filter to the set of reconstructed samples of the first channel; apply the at least one convolutional cross-component model filter using as input an output of the at least one filter to the set of reconstructed samples of the first channel; apply at least one filter to the output of the at least one convolutional cross- component model filter; and apply a cross component filter to the set of reconstructed samples of the first channel or to the output of the at least one filter to the set of reconstructed samples of the first channel, to obtain a correction for the output of the at least one filter using as input the output of the at least one convolutional cross-component model filter.

2. The apparatus of claim 1, wherein the at least one non-transitory memory storing instructions is executed by the at least one processor, cause the apparatus to: add an output of the the cross-component filter to the output of the at least one filter using as input the output of the at least one convolutional cross-component model filter.

3. The apparatus of claim 1, wherein the convolutional cross-component model filter is derived and applied for blocks that are not determined by any coding or prediction partitions.

4. The apparatus of claim 3, wherein said blocks have a square or rectangular shape, and wherein said blocks can be overlapped or distinct.

5. The apparatus of claim 3, wherein the usage of the convolutional cross- component model for each block is determined without additional signalling.

6. A method, comprising: obtaining for a set of samples at least one convolutional cross-component model filter, wherein the set of samples are reconstructed samples of two channels for an image; applying at least one filter to the set of reconstructed samples of the first channel; applying the at least one convolutional cross-component model filter using as input an output of the at least one filter to the set of reconstructed samples of the first channel; applying at least one filter to an output of the at least one convolutional cross- component model filter; and applying a cross component filter to the set of reconstructed samples of the first channel or to an output of the at least one filter to the set of reconstructed samples of the first channel, for obtaining a correction for an output of the at least one filter using as input an output of the at least one convolutional cross-component model filter.

7. The apparatus of claim 6, wherein the at least one non-transitory memory storing instructions is executed by the at least one processor, cause the apparatus to: add an output of the the cross-component filter to an output of the at least one filter using as input an output of the at least one convolutional cross-component model filter.

8. The method of claim 6, wherein the convolutional cross-component model filter is derived and applied for blocks that are not determined by any coding or prediction partitions.

9. The method of claim 8, wherein said blocks have a square or rectangular shape, and wherein said blocks can be overlapped or distinct.

10. The method of claim 8, wherein the usage of the convolutional cross- component model for each block is determined without additional signalling.