RU2806282C2 - Interaction between in-loop transformation and tools for interframe coding - Google Patents

Abstract

FIELD: means for image encoding and decoding.

SUBSTANCE: first predictive block is determined in the first region from at least one reference image using unspecified motion information. The current video block is a block of the luminance component. A motion information refinement procedure is applied to obtain motion offset information. A second predictive block is generated from the samples on the specified at least one reference image using information about the motion offset. Based on the direct mapping procedure the second predictive block is transformed from the first region to the second region. Reconstructed samples of the current video block are generated based on the second predicted block in the second region. To display the readings of the second predictive block, a piecewise linear model is used, the scaling factors of which are determined based on the first variables, which are determined on the basis of the syntax elements included in the set of adaptation parameters, and the second variables, determined on the basis of bit depth.

EFFECT: increase in efficiency of image encoding.

14 cl, 74 dwg

Description

Cross references to related applications

Field of technology to which the invention relates

This patent document relates to video processing technologies, devices and systems.

State of the art

Despite all the advances in video compression, digital video still takes up the largest share of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and presenting video increases, bandwidth requirements for digital video use are expected to continue to increase.

Disclosure of the invention

Devices, systems and methods related to digital video processing and, in particular, to in-loop reshaping (ILR) for video processing are described. The described methods can be applied to both existing video processing standards (for example, High Efficiency Video Coding (HEVC)) and future video processing standards or video processors, including video codecs.

According to one exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing, for converting between a current video block of video and an encoded video representation, a procedure for refining motion information based on samples in a first region or a second region; and performing a transformation based on the result of the motion information refinement procedure, wherein during the transformation, samples for the current video block are obtained from the first predicted block in the first region using the unspecified motion information, and at least a second prediction block is generated in the second region using the refined motion information to determine a reconstructed block and generate reconstructed samples of the current video block based on at least the second predicted block.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video video block and an encoded video representation, wherein during the transformation the current video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component, wherein the transformation uses an encoding tool using parameters , obtained from at least the first set of samples in the video video area and the second set of samples in the reference image of the current video block, wherein the area for the first samples and the area for the second samples are combined.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining, for the current video block of the current video video region, a parameter for the encoding mode of the current video block based on one or more parameters for the encoding mode of the previous video region; and performing encoding on the current video block to generate an encoded video representation based on said definition, wherein said parameter for an encoding mode is included in a set of parameters in the encoded video representation, and performing encoding comprises transforming a representation of the current video block in the first region to a representation of the current video block in the second region , and when encoding is performed using the specified encoding mode, a current video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises receiving an encoded representation of a video containing a set of parameters including parameter information for an encoding mode; and performing decoding of the encoded representation using said parameter information to generate a current video block of the current video region of the video based on the encoded representation, wherein the parameter information for the encoding mode is based on one or more parameters for the encoding mode of the previous video region, wherein in said encoding mode constructing the current a video block based on the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video block of video and an encoded representation of the video, and the transformation comprises applying a filtering operation to a predicted block in a first region or in a second region different from the first region.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video video block and an encoded video representation, wherein the transformation determines a final reconstructed block for the current video block, generates a temporary reconstructed block using a prediction method, and presents the specified block in a second region.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video block of a video region of a video and an encoded representation of the video, wherein the transformation uses an encoding mode that constructs the current video block based on a first region and a second region and/or scales the remainder of the chroma component in a manner dependent on the luminance component, wherein the set parameters in encoded representation contains information about parameters for the specified encoding mode

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video block, which is a block of the color component of a video, and an encoded representation of the video, wherein during the transformation, the current video block is constructed based on a first region and a second region, wherein the transformation further comprises applying a forward reformation procedure and/or an inverse reformation procedure to one or more chrominance components of the current video block.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current chroma video block and an encoded video representation, wherein performing the transformation comprises: determining whether luma-dependent chroma residue scaling (LCRS) is enabled based on the rule, and reconstructing the current video block of the chrominance component based on the specified definition.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining, for conversion between a current video video block and an encoded video representation, whether to deactivate the use of an encoding mode based on one or more coefficient values for the current video block; and performing a conversion based on said definition, wherein when converting using said encoding mode, constructing a current video block based on the first block and the second block and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises partitioning, for converting between a current video block of video larger than a virtual pipeline data unit (VPDU) of video, the current video block into an area; and performing the conversion by applying the encoding mode separately to each area, wherein when converting using the corresponding encoding mode, constructing the current video block based on the first block and the second block and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining, for conversion between the current video block and an encoded video representation, whether to deactivate the use of an encoding mode based on the size or color format of the current video block; and performing a conversion based on said definition, wherein when converting using said encoding mode, constructing a current video block based on the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video block of video and an encoded representation of the video, wherein the transformation uses an encoding mode that constructs the current video block based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component, wherein at least one syntactic element in the encoded representation creates an indication of the use of the encoding mode and an indication of the model of the reshaping device.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining that an encoding mode is not activated for conversion between the current video block and the encoded video representation; and conditionally skipping direct reformation and/or reverse reformation based on the specified definition, wherein in the encoding mode the current video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video block of video and an encoded representation of the video, wherein the transformation uses an encoding mode that constructs the current video block based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component, using a plurality of operations forward reformatting and/or a plurality of reverse reformatting operations in a reformatting mode for the video domain.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining that an encoding mode is activated for conversion between a current video block of video and an encoded representation of the video; and performing a transformation using a palette mode, wherein at least a palette of representative sample values is used for the current video block, wherein in said encoding mode, constructing the current video block based on the samples in the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on brightness component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining, for conversion between a current video video block and an encoded video representation, that the current video block is encoded in a palette mode, wherein at least a palette of representative sample values is used to encode the current video block; and performing, based on the above definition, conversion by deactivating the encoding mode, wherein when the encoding mode is applied to the video block, constructing a video block based on the remainder of the chrominance component scaled in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a first video block of video and an encoded representation of the video, the transformation using a first encoding mode and a palette encoding mode in which at least a palette of representative pixel values is used to encode the current video block; and performing a transformation between a second video block of video that is encoded without using a palette encoding mode and the encoded video representation, wherein the transformation of the second video block uses the first coding mode, wherein if the first coding mode is applied to the video block, constructing a video block based on the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component, wherein said first encoding mode is applied in different ways to the first video block and the second video block.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining that an encoding mode is activated for conversion between the current video block and an encoded video representation, and performing the conversion using an intra-frame block copy mode in which a predicted block is generated using at least a block vector pointing to an image containing the current video block, at In this case, in the specified coding mode, the current video block is constructed based on the samples in the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining, for performing a conversion between the current video block and an encoded video representation, that the current video block is encoded in an intra block copy (IBC) mode generating a predicted block using at least a block vector pointing to a video frame containing specified current video block, for encoding the current video block; and, based on said determination, performing conversion by deactivating the encoding mode, wherein if said encoding mode is applied to the video block, constructing the video block based on the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a first video video block and an encoded representation of the video, the transformation using an intra-frame block copy mode generating a predicted block using at least a block vector pointing to a video frame containing the current video block, and a first encoding mode; and performing a transformation between a second video block of video encoded without using an intraframe block copy mode and an encoded representation of the video, wherein the transformation of the second video block uses the first coding mode, wherein if the first coding mode is applied to the video block, constructing a video block based on the first region and the second areas and/or scale the remainder of the chrominance component in a manner dependent on the luminance component, the first encoding mode being applied differently to the first video block and to the second video block.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining that an encoding mode is activated to convert between a current video video block and an encoded video representation; and performing the conversion using a block-based delta pulse code modulation (BDPCM) mode, wherein in said coding mode, constructing a current video block based on samples in the first region and the second region and/or scaling the remainder chrominance component in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining, for conversion between a current video video block and an encoded video representation, that the current video block is encoded using a block-based delta pulse code modulation (BDPCM) mode; and performing, based on said determination, a transformation by deactivating the encoding mode, wherein, if said encoding mode is applied to a video block, constructing a video block based on the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a first video block of video and an encoded representation of the video, wherein the transformation of the first video block uses a first coding mode and a block-based delta pulse code modulation (BDPCM) mode; and performing a transformation between the second video block of video and an encoded representation of the video, wherein the second video block is encoded without using a BDPCM modulation mode, and the transformation of the second video block uses the first encoding mode, wherein if the first encoding mode is applied to the video block, constructing the video block based on the first region and the second areas and/or scale the remainder of the chrominance component in a manner dependent on the luminance component, wherein the first encoding mode is applied differently to the first video block and to the second video block.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining that an encoding mode is activated to convert between a current video video block and an encoded video representation; and performing the transformation using the skip transformation mode, wherein the transformation of the prediction remainder is skipped when encoding the current video block, wherein in the specified encoding mode the current video block is constructed based on the samples in the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining, to perform a transformation between the current video block and an encoded video representation, that the current video block is encoded in a skip transformation mode, in which the transformation of the prediction residual is skipped when encoding the current video block; and performing, based on said determination, a conversion by canceling the encoding mode, wherein if said encoding mode is applied to the video block, constructing a video block based on the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a first video block of video and an encoded representation of the video, wherein the transformation of the first video block uses a first encoding mode and a skip transformation mode in which the transformation of the prediction residual is skipped when encoding the current video block; and performing a transformation between the second video block of video and the encoded representation of the video, wherein the second video block is encoded without using a skip transformation mode, and the transformation of the second video block uses the first encoding mode, wherein if the first encoding mode is applied to the video block, constructing a video block based on the first region and the second areas and/or scale the remainder of the chrominance component in a manner dependent on the luminance component, wherein the first encoding mode is applied differently to the first video block and to the second video block.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining that an encoding mode is activated to convert between a current video video block and an encoded video representation; and performing the transformation using the intra-frame prediction mode with pulse code modulation, wherein the current video block is encoded without applying transformation and quantization in the transformed area, and in the specified encoding mode, the current video block is constructed based on samples in the first area and the second area and/or the remainder is scaled chrominance component in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining, for the transformation between a current video video block and an encoded video representation, that the current block is encoded in an intra-frame prediction mode with pulse code modulation, in which the current video block is encoded without applying transformation and quantization in the transformed region; and performing, based on said determination, a transformation by deactivating the encoding mode, wherein, if said encoding mode is applied to a video block, constructing a video block based on the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a first video block of video and an encoded representation of the video, wherein the transformation of the first video block uses a first encoding mode and a PCM intra-frame prediction mode in which the current video block is encoded without applying transformation and quantization in the transformed region; and performing a transformation between the second video block of video and the encoded video representation, wherein the second video block is encoded without using an intra-frame prediction mode with pulse code modulation, wherein the transformation of the second video block uses a first encoding mode, wherein, if the first encoding mode is applied to the video block, constructing a video block on based on the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component, and the first encoding mode is applied differently to the first video block and to the second video block.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining that an encoding mode is activated to perform a conversion between a current video block and an encoded video representation; and performing the transformation using a modified transformation and quantization bypass mode, in which the current video block is encoded without loss and without applying transformation and quantization, and in the specified encoding mode, the current video block is constructed based on samples in the first region and the second region and/or the remainder of the chrominance component is scaled in a way that depends on the brightness component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises determining, for conversion between a current video video block and an encoded video representation, that the current video block is encoded in a transformation and quantization bypass mode, in which the current video block is encoded losslessly and without applying transformation and quantization; and performing, based on said determination, a transformation by deactivating the encoding mode, wherein, if the encoding mode is applied to the video block, constructing the video block based on the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a first video block of video and an encoded representation of the video, wherein to transform the first video block, a first encoding mode and a transformation and quantization bypass mode are used, in which the current video block is encoded losslessly and without applying transformation and quantization; and performing a transformation between the second video block of video and the encoded representation of the video, wherein the second video block is encoded without using a transform and quantization bypass mode and a first encoding mode is used to transform the second video block, wherein if the first encoding mode is applied to the video block, constructing a video block based on the first region and the second region and/or scale the remainder of the chrominance component in a manner dependent on the luminance component, wherein the first encoding mode is applied in different ways to the first video block and to the second video block.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video block of video and an encoded representation of the video, wherein the transformation uses an encoding mode that constructs the current video block based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component, the information used to encoding mode is signaled in a parameter set other than the sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS), or adaptation parameter set (APS) used to transmit adaptive loop filtering (ALF) parameters.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video block of video and an encoded representation of the video, wherein the transformation uses an encoding mode that constructs the current video block based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component, the information used to encoding mode is signaled in an adaptation parameter set (APS) together with adaptive loop filtering (ALF) information, wherein the information used for the encoding mode and the ALF filtering information are included in one network abstraction layer unit ( NAL)).

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video block of a video region of a video and an encoded representation of the video, wherein the transformation uses an encoding mode that constructs the current video block based on the first region and the second region and/or scales the remainder of the chroma component in a manner dependent on the luminance component, wherein the information used for the encoding mode is signaled in a first type adaptation parameter set (APS) different from a second type APS used for signaling with adaptive loop filtering (ALF) information.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video block of a video region of a video and an encoded representation of the video, wherein the transformation uses an encoding mode in which the current video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component, wherein for the specified The video domain is not allowed to access an adaptation parameter set or any parameter set signaled prior to a data structure of a specific type used for video processing, wherein the data structure of a specific type is signaled prior to the video domain.

According to another exemplary aspect, the proposed technology can be used to provide a video processing method. The method comprises performing a transformation between a current video block of video and an encoded representation of the video, wherein the transformation uses an encoding mode in which the current video block is constructed based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component, wherein the syntax element of the set parameters, containing parameters used for video processing, has specified values in the match data bit stream.

According to another exemplary aspect, the method described above is implemented as processor-executable code and stored on a computer-readable program medium.

According to another representative aspect, there is provided a device configured or operable to implement the method described above. This device may include a processor programmed to implement this method.

According to another exemplary aspect, the method described herein may be implemented in a video decoding apparatus.

The above and other aspects and features of the proposed technology are discussed in more detail in the drawings, in the description and in the claims.

Brief description of drawings

Fig. 1 shows an example of constructing a list of merged candidates.

Fig. 2 shows an example of spatial candidate positions.

Fig. 3 shows an example of redundancy-checked candidate pairs from a population of spatial merging candidates.

Fig. 4A and 4B show examples of the position of the second prediction unit (PU) based on the size and shape of the current block.

Fig. 5 shows an example of scaling motion vectors for temporal fusion candidates.

Fig. 6 shows an example of candidate positions for temporary merged candidates.

Fig. 7 shows an example of a combined bidirectionally interpolated (predicted) merge candidate.

Fig. 8 shows an example of candidate construction with motion vector prediction.

Fig. 9 shows an example of scaling motion vectors for spatial fusion candidates.

Fig. 10 shows an example of motion prediction using an alternative temporal motion vector prediction (ATMVP) algorithm for a coding unit (CU).

Fig. 11 shows an example of a coding unit (CU) with sub-blocks and neighboring blocks used by a spatial-temporal motion vector prediction (STMVP) algorithm.

Fig. 12 shows an example of adjacent samples for determining illuminance compensation (IC) parameters.

Fig. 13A and 13B show examples of a simplified 4-parameter affine model and a simplified 6-parameter affine model, respectively.

Fig. 14 shows an example of an affine motion vector field (MVF) per sub-block.

Fig. 15A and 15B show examples of 4-parameter and 6-parameter affine models, respectively.

Fig. 16 shows an example of motion vector prediction for AF_INTER mode for initially affine candidates.

Fig. 17 shows an example of motion vector prediction for AF_INTER mode for constructed affine candidates.

Fig. 18A and 18B show examples of candidate blocks and control-point motion vector (CPMV) predictor acquisition, respectively, for the AF_MERGE mode.

Fig. 19 shows an example of candidate positions for the affine object fusion mode.

Fig. 20 shows an example of the ultimate motion vector expression (UMVE) search procedure.

Fig. 21 shows an example of a UMVE expression search point.

Fig. 22 shows an example of decoder side motion vector refinement (DMVR) based on bilateral template matching.

Fig. 23 shows an example of a logic diagram of a decoding procedure using reshaping.

Fig. 24 shows an example of adjacent samples used in a two-way filter.

Fig. 25 shows an example of windows covering two samples used in weighting calculations.

Fig. 26 shows an example of a scanning circuit.

Fig. 27 shows an example of a decoding procedure using the inter-frame prediction mode.

Fig. 28 shows another example of a decoding procedure using the inter prediction mode.

Fig. 29 shows an example of a decoding procedure using the inter-prediction mode with filters applied after reconstruction.

Fig. 30 shows another example of a decoding procedure using the inter-prediction mode with filters applied after reconstruction.

Fig. 31A and 31B show logic diagrams of examples of video processing methods.

Fig. 32A - 32D show logic diagrams of examples of video processing methods.

Fig. 33 shows a logic diagram of an example of video processing methods.

Fig. 34A and 34B show logic diagrams of examples of video processing methods.

Fig. 35A - 35F show logic diagrams of examples of video processing methods.

Fig. 36A - 36C show logic diagrams of examples of video processing methods.

Fig. 37A - 37C show logic diagrams of examples of video processing methods.

Fig. 38A - 38L show logic diagrams of examples of video processing methods.

Fig. 39A - 39E show logic diagrams of examples of video processing methods.

Fig. 40A and 40B show examples of hardware platforms for implementing a visual information decoding method or a visual information encoding method described herein.

Carrying out the invention

Due to the growing need for video programs with ever higher resolution, video processing methods and technologies have become widespread in modern technology. Video codecs typically contain electronic circuitry or software that compresses or expands digital video, and are continually being improved to achieve ever-higher encoding efficiency. A video codec converts uncompressed video to a compressed format or vice versa. Video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access and end-to-end latency are all related in complex ways. The compressed format typically follows standard video compression specifications, such as High Efficiency Video Coding (HEVC) (also known as H.265 or MPEG-H Part 2), Versatile Video Coding (VVC) , which is still being finalized, or other current or future video coding standards.

Variations of the proposed technology can be applied to existing video coding standards (eg, HEVC, H.265) and to future standards to improve compression performance. The section headings in this document are used to enhance the readability of the description and in no way limit the discussion or options (and/or implementation options) to the relevant sections.

1. Examples of interframe prediction in the HEVC/H.265 coding standard

Video coding standards have been significantly improved in recent years and today they provide, in part, high coding efficiency and support higher resolutions. Recent standards such as HEVC and H.265 are based on a hybrid video coding structure that uses time prediction plus transformation coding.

1.1. Examples of forecast modes

Each PU prediction unit has motion parameters for one or two reference picture lists. In some embodiments, the plurality of motion parameters comprises a motion vector and a reference image index. In other embodiments, the use of one or two reference picture lists may also be signaled using the inter_pred_idc parameter. In yet another group of other options, the motion vectors may be explicitly encoded as increments relative to the predictors.

When a CU is encoded in the skip mode, there is one PU associated with the CU, and there are no significant residual coefficients or encoded motion vector increment or reference picture index. The fusion mode is specified such that the motion parameters for the current PU are obtained from neighboring PUs, including spatial and temporal candidates. The combining mode can be applied to any inter-prediction PU, not just the skip mode. An alternative to the fusion mode is to pass the motion parameters explicitly, where the motion vectors (more precisely, the motion vector differences (MVD) relative to the motion vector predictor), the corresponding reference picture index for each reference picture list, and the reference picture list utilization rate signaled explicitly for each PU. This type of mode is referred to in this document as advanced motion vector prediction (AMVP).

When signaling indicates that one of the two reference picture lists should be used, a PU is created from one block of samples. This is called 'uni-prediction'. Unidirectional prediction is available for both types of slices - P-slice or slice with reference to the previous slice (predicted slice) and B-slice or slice with reference to previous and subsequent slices (or bidirectionally interpolated slice ).

When signaling indicates that both reference picture lists should be used, a PU is created from two sample blocks. This is called 'bi-prediction'. Bidirectional interpolation is only available for B-slices.

List of reference images

In the HEVC coding standard, the term “inter-picture prediction” refers to prediction derived from data elements (eg, sample values or motion vectors) of reference pictures other than the currently decoded picture (current picture). Similar to the H.264/AVC coding standard, a picture can be predicted from multiple reference pictures. The reference pictures used for inter-picture prediction are organized in one or more reference picture lists. The support index identifies which of the reference pictures in the list should be used to create the predicted signal.

One reference picture list, List 0, is used for the P-slice, and two reference picture lists, List 0 and List 1, are used for the B-slice. It should be noted that the reference pictures from List 0/1 may be from past and future pictures in terms of capturing/displaying those pictures.

1.1.1. Options for constructing candidates for the merging mode

When a PU is predicted using the merging mode, an index indicating an input position in the merging candidate list is parsed from the data bit stream and used to extract motion information. The procedure for constructing the specified list can be summarized in accordance with the following sequence of steps:

Step 1: Obtaining an initial list of candidates

Step 1.1: Obtaining spatial candidates

Step 1.2: Redundancy control for spatial candidates

Stage 1.3: Obtaining temporary candidates

Step 2: Inserting Additional Candidates

Step 2.1: Generating Bidirectionally Interpolated Candidates

Step 2.2: Inserting Zero Motion Candidates

Fig. 1 shows an example of constructing a list of merging candidates based on the sequence of steps summarized above. To obtain spatial fusion candidates, a maximum of four fusion candidates are selected from a pool of candidates located at five different positions. To obtain temporary merge candidates, a maximum of one merge candidate is selected from two candidates. Since the decoder assumes a constant number of candidates for each PU, if the number of candidates does not reach the maximum number of candidates to be merged (MaxNumMergeCand) signaled in the slice (frame) header, additional candidates are generated. Since the number of candidates is constant, the index of the best merging candidate is encoded using truncated unary binarization (TU). If the CU size is 8, all PUs from the current CU share one merge candidate list, which is identical to the merge candidate list for a 2N×2N prediction unit.

1.1.2. Construction of spatial merging candidates

When obtaining spatial fusion candidates, a maximum of four fusion candidates are selected from the plurality of candidates located at the positions shown in FIG. 2. Candidates are selected in the following order: A _1, B _1, B _0, A ₀ and B ₂ . Position B ₂ is only taken into account when any of the PU units that should be in positions A ₁ , B ₁ , B ₀ , A ₀ is not available (for example, because that unit belongs to another slice or tile) or is coded with using intra-frame prediction. After adding a candidate at position A ₁ , the addition of the remaining candidates must be done with redundancy control, which ensures that candidates with the same motion information are excluded from the list, so that coding efficiency is improved.

To reduce computational complexity, not all possible candidate pairs are considered in the redundancy check process mentioned above. In contrast, only the pairs connected by the arrow in FIG. 3, and any candidate is added to the list only if the corresponding candidate used for redundancy control does not have the same traffic information. Another source of duplicate motion information is the “second PU” associated with partitions other than 2Nx2N. As an example, FIG. 4A and 4B show the second PU for the N×2N and 2N×N cases, respectively. When the current PU is partitioned as N×2N, the candidate at position A ₁ is not taken into account when constructing the list. In some embodiments, adding this candidate may result in two prediction units having the same motion information, which is redundant in terms of the requirement to have only one PU per coding unit. Likewise, position B ₁ is not taken into account when the current PU is partitioned as 2N×N.

1.1.3. Construction of temporary merging candidates

At this stage, only one candidate is added to the list. In particular, upon receiving this temporary fusion candidate, a scaled motion vector is generated based on the PU located at the same location, belonging to the image having the smallest difference in Picture Order Count (POC) relative to the current image in the reference picture list under consideration. The list of reference pictures that should be used to obtain a co-located PU is signaled explicitly in the slice header.

Fig. 5 shows an example of obtaining a scaled motion vector for a temporary fusion candidate (as shown by the dashed line), which is scaled from the motion vector for a co-located PU using sequence number distances (POC distances), tb and td, where distance tb is defined as the difference in POC numbers between the reference image for the current image and the current image itself; and the distance td is defined as the difference in POC numbers between the reference image for the colocated image and the colocated image itself. The reference image index for the temporary merge candidate is set to zero. For a B-slice, two motion vectors are obtained, one for reference picture list 0 and one for reference picture list 1, and these vectors are combined to obtain a bidirectionally interpolated fusion candidate.

In a co-located PU (Y) belonging to the reference frame, a position for a temporary candidate is selected between candidates C ₀ and C ₁ , as shown in FIG. 6. If the PU at position _C0 is not available, is intra-predictive encoded, or is outside the current CTU, position _C1 is used. Otherwise, position C ₀ is used to obtain a temporary merge candidate.

1.1.4. Constructing additional types of merging candidates

In addition to spatiotemporal merge candidates, there are two additional types of merge candidates: a combined bidirectionally interpolated merge candidate and a null merge candidate. Combined bidirectionally interpolated merge candidates are generated using spatiotemporal merge candidates. The combined bidirectionally interpolated merge candidate is used for B-slice only. Such combined bidirectionally interpolated candidates are generated by combining motion parameters from a first reference picture list for the original candidate with motion parameters from a second reference picture list for another candidate. If these two groups of parameters form different motion hypotheses, they will create a new bidirectionally interpolated candidate.

In fig. 7 shows an example of this procedure, where two candidates from the source list (710, left), having parameters mvL0 and refIdxL0 or mvL1 and refIdxL1, are used to create a combined bidirectionally interpolated merge candidate added to the final list (720, right). There are numerous rules regarding the construction of such combinations that are taken into account when generating such additional merging candidates.

Zero-motion candidates are inserted to fill the remaining input positions in the list of candidates to be merged, thereby achieving the maximum capacity of the MaxNumMergeCand list. These candidates have zero spatial displacement, and the reference image index starts at zero and is incremented each time a new candidate with zero motion is added to the list. The number of reference frames used by these candidates is one and two for unidirectional prediction and bidirectional prediction, respectively. In some embodiments, no redundancy control is performed for these candidates.

1.2. Advanced Motion Vector Prediction (AMVP) options

AMVP prediction uses the spatiotemporal correlation of the motion vector with neighboring PUs, which is used to convey motion parameters explicitly. When constructing a list of candidate motion vectors, they first check the availability of temporary neighboring PUs at top-left positions, eliminate redundant candidates, and add a null vector to make the candidate list of constant length. The encoder can then select the best predictor from the list of candidates and transmit a corresponding index indicating the selected candidate. Similar to the transmission of the pooling index through signaling, the best motion vector index is encoded using truncated unary binarization. The maximum value to be encoded in this case is 2 (see FIG. 8). The following sections provide details of the procedure for obtaining a candidate for motion vector prediction.

1.2.1. Examples of obtaining candidates when predicting AMVP

Fig. 8 summarizes the procedure for obtaining a motion vector prediction candidate, so that this procedure can be implemented for each reference image list using the refidx parameter as input.

When predicting motion vectors, two types of candidate motion vectors are considered: a spatial motion vector candidate and a temporal motion vector candidate. To generate a spatial motion candidate vector, two candidate motion vectors are ultimately obtained for each of the PUs located at five different positions, as previously shown in FIG. 2.

To generate a temporary motion vector candidate, one motion vector candidate is selected from two candidates obtained based on two different co-located positions. After generating the first list of spatiotemporal candidates, duplicate candidate motion vectors are eliminated from this list. If the number of potential candidates is greater than two, candidate motion vectors for which the reference picture index in the associated reference picture list is greater than 1 are excluded from the list. If this number of candidate spatiotemporal motion vectors is less than two, an additional zero candidate motion vector is added to the list.

1.2.2. Construction of spatial motion candidate vectors

To generate a spatial motion candidate vector, a maximum of two potential candidates are considered from a population of five candidate candidates obtained from the PUs located at the positions, as previously shown in FIG. 2, these positions are the same as when combining movement. The generation order for the left side of the current PU is specified as candidate A ₀ , candidate A ₁ , and scaled candidate A ₀ , scaled candidate A ₁ . The formation order for the upper side from the current PU is specified as candidate B ₀ , candidate B ₁ , candidate B ₂ , scaled candidate B ₀ , scaled candidate B ₁ , scaled candidate B ₂ . For each side, therefore, there are four cases that can be used as a candidate motion vector, where in two cases spatial scaling is not required to be used, and in two cases spatial scaling is applied. These four different cases are summarized as follows:

- No spatial scaling

(1) Same reference picture list and same reference picture index (same POC order)

(2) Different reference picture lists, but same reference picture (same POC order)

- Spatial scaling

(3) Same reference picture list, but different reference pictures (different POC order)

(4) Different reference picture lists and different reference pictures (different POC order)

Cases without spatial scaling are tested first, followed by cases that allow spatial scaling. Spatial scaling is considered when the POC order differs between the reference picture for a neighboring PU and the reference picture for the current PU regardless of the reference picture list. If all left candidate PUs are not available or are encoded using intra-frame prediction, motion vector scaling for the top unit is allowed to facilitate parallel determination of left and top candidate MV vectors. Otherwise, spatial scaling is not allowed for the unit motion vector from above.

As shown in the example shown in FIG. 9, for the case of spatial scaling, the motion vector for a neighboring PU is scaled in a manner similar to temporal scaling. One difference is that the reference picture list and the index of the current PU are used as input; the actual scaling procedure is the same as in the case of temporary scaling.

1.2.3. Constructing Temporal Motion Candidate Vectors

Apart from obtaining the reference image index, all procedures for generating temporal fusion candidates are the same as for generating spatial motion vector candidates (as shown in the example of FIG. 6). In some embodiments, the reference picture index is signaled to a decoding device.

2. An example of inter-frame forecasting methods within the framework of a joint research model (Joint Exploration Model (JEM))

In some embodiments, future video encoding technologies are explored using the Joint Exploration Model (JEM). In the JEM model, sub-block based prediction is adopted in a number of coding tools such as affine prediction, alternate temporal motion vector prediction (ATMVP), spatio-temporal motion vector prediction (STMVP), bi-directional optical flow ( BIO), Frame-Rate Up Conversion (FRUC), Locally Adaptive Motion Vector Resolution (LAMVR), Overlapped Block Motion Compensation (OBMC), Local illumination compensation (Local Illumination Compensation (LIC)) and refinement of motion vectors on the side of the decoding device (Decoder-side Motion vector Refinement (DMVR)).

2.1. Examples of motion vector prediction based on CU sub-units

In the JEM model using quadtrees plus binary trees (QTBT), each CU can have at most one set of motion parameters per prediction direction. In some embodiments, two methods are contemplated for predicting motion vectors at the sub-CU level in an encoder by dividing a large CU into sub-CUs and generating motion information for all sub-CUs from the large CU. The Alternate Temporal Motion Vector Prediction (ATMVP) method allows each CU to sample multiple sets of motion information from multiple blocks smaller than the current CU in a co-located reference image. When using the spatio-temporal motion vector prediction (STMVP) method, the motion vectors of CU sub-units are generated using a temporal motion vector predictor and a spatial neighbor motion vector. In some embodiments, and to maintain a more accurate motion field for predicting the motion of a CU sub-unit, motion compression for reference frames may not be activated.

2.1.1. Alternate Time Motion Vector Prediction (ATMVP) Examples

According to the ATMVP prediction method, the temporal motion vector prediction (TMVP) method is modified by sampling multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU.

Fig. 10 shows an example of a motion prediction procedure ATMVP for a CU 1000. The ATMVP prediction method predicts motion vectors of sub-units CU 1001 within a unit U 1000 in two steps. In the first step, a corresponding block 1051 in the time vector reference image 1050 is identified. The reference image 1050 is also called a motion source image. In the second step, the CU 1000 is divided into sub-CUs 1001 and the motion vectors as well as the reference indices of each CU sub-unit are obtained from the block corresponding to each CU sub-unit.

In the first step, the reference image 1050 and the corresponding block are determined by the motion information of spatially adjacent blocks for the current CU 1000. To avoid repeated scanning procedures for adjacent blocks, the first merge candidate from the list of merge candidates for the current CU 1000 is used. The first available motion vector , as well as the associated reference index is set as a temporary vector and index for the motion source image. In this way, the corresponding block can be more accurately identified, compared to the TMVP prediction method, where the corresponding block (sometimes called a co-located block) is always in the lower right or center position relative to the current CU.

In the second step, the corresponding block of the sub-CU 1051 is identified by a time vector in the motion source image 1050 by adding the time vector to the coordinate of the current CU. For each CU sub-unit, the motion information of its corresponding block (eg, the smallest motion grid covering the central reference) is used to obtain motion information for the CU sub-unit in question. After identifying the motion information of the corresponding N×N block, it is converted into motion vectors and reference indices of the current CU sub-unit, in the same way as the TMVP prediction method in the HEVC coding standard where motion scaling and other procedures are applied. For example, the decoder checks whether a low latency condition is satisfied (eg, the POC sequence numbers for all reference pictures for the current picture are less than the POC sequence number for the current picture) and optionally uses a motion vector MVx (eg, a motion vector corresponding to a list of reference pictures X ) to predict the motion vector MVy (for example, with X equal to 0 or 1 and Y equal to 1-X) for each CU sub-unit.

2.1.2. Examples of Spatio-Temporal Motion Vector Prediction (STMVP)

According to the STMVP prediction method, the motion vectors of CU sub-units are generated in a recursive manner following the raster scanning order. Fig. 11 shows an example of one CU with four sub-units and adjacent units. Consider an 8x8 CU 1100 containing four 4x4 sub-CUs, namely A(1101), B(1102), C(1103) and D(1104). Adjacent 4x4 blocks in the current frame are labeled a (1111), b (1112), c (1113), and d (1114).

Determining motion for a sub-unit CU A begins by identifying its two neighbors in space. The first neighbor is an N×N block located above the CU sub-unit A 1101 (block c 1113). If this block c (1113) is not available or is intra-predictive encoded, other N×N blocks above the CU sub-unit A (1101) are checked (from left to right, starting with block c 1113). The second neighbor is the block to the left of the CU sub-unit A 1101 (block b 1112). If block b (1112) is not available or is intra-prediction encoded, other blocks above the CU sub-unit A 1101 are checked (from top to bottom, starting from block b 1112). The motion information received from neighboring blocks for each list is scaled to the first reference frame for the particular list in question. Next, a temporal motion vector predictor (TMVP) is generated for sub-block A 1101 using the same TMVP predictor generation procedure as specified in the HEVC coding standard. Information about a block co-located with block D 1104 is selected and scaled accordingly. Finally, after recalling and scaling the motion information, all available motion vectors are averaged separately for each reference list. The average motion vector is assigned as the motion vector for the current CU sub-unit.

2.1.3. Examples of motion prediction mode signaling for a CU sub-unit

In some embodiments, the modes of the CU sub-unit are enabled as additional mergeable candidates such that no additional syntax element is required to signal these modes. Two additional merge candidates are added to the merging candidate list for each CU to represent the ATMVP prediction mode and the STMVP prediction mode. In other embodiments, up to seven merging candidates may be used if the sequence parameter set indicates that the ATMVP prediction mode and the STMVP prediction mode are activated. The encoding logic for additional merging candidates is the same as for merging candidates in the main HM model, which means, for each CU in a P-slice or in a B-slice, that two additional redundancy checks (RD) may be required for the two additional merging candidates . In some embodiments, such as the JEM model, all join index sections are encoded using Context-based Adaptive Binary Arithmetic Coding (CABAC). In other embodiments, such as HEVC encoding, only the first section is subjected to context-sensitive encoding and the remaining sections are context-independently encoded.

2.2. Example of Local Light Compensation (LIC) in JEM Model

Local Illumination Compensation (LIC) is based on a linear illumination variation model using a scale factor a and a shift b. Moreover, this compensation is activated and deactivated adaptively for a coding unit (CU) encoded using interframe prediction.

When LIC compensation is applied to a CU, a least squares method is used to obtain the parameters a and b using adjacent samples relative to the current CU and their corresponding reference samples. More specifically, as illustrated by FIG. 12, downsampled (2:1 downsampled) adjacent samples relative to the CU and corresponding samples (identified by motion information of the current CU or sub-CU) from the reference image are used.

2.2.1. Retrieving predicted blocks

IC compensation parameters are obtained and applied for each prediction direction separately. For each direction, a first prediction block is generated using the decoded motion information, and then a temporary prediction block is obtained by applying the LIC compensation model. Thereafter, these two temporary prediction blocks are used to obtain the final prediction block.

When a CU is encoded using the combining mode, the compensation flag LIC is copied from neighboring blocks in a manner similar to encoding motion information in the combining mode; otherwise, the LIC compensation flag is signaled to the CU to indicate whether LIC compensation is applied or not.

When LIC compensation is enabled for an image, an additional redundancy check (RD) must be performed at the CU level to determine whether LIC compensation is applied to any CU. If LIC compensation is activated for a CU, the mean-removed sum of absolute difference (MR-SAD) and the Hadamard-transformed sum of absolute differences (mean-removed sum of absolute) are used. Hadamard-transformed difference (MR-SATD) instead of sum of absolute differences (SAD) and Hadamard-transformed sum of absolute differences (SATD), to find integer motion of pixels and find fractional motion of pixels, respectively.

To reduce coding complexity, the JEM model uses the following coding scheme: LIC compensation is not activated for the entire image when there are no noticeable illumination changes between the current image and its corresponding reference images. To identify this situation, the encoder calculates histograms for the current image and for each reference image corresponding to that current image. If the histogram difference between the current image and each of the reference images for that current image is less than a specific threshold, LIC compensation for the current image is not activated; otherwise, this LIC compensation is activated for the current image.

2.3. Examples of inter-frame prediction methods in VVC coding

There are a number of new coding tools available to improve inter-frame prediction, such as Adaptive motion vector difference resolution (AMVR) for MVD difference signaling, affine prediction mode, Triangular prediction mode (TPM), ATMVP prediction mode, Generalized Bi-Prediction (GBI), Bi-directional Optical flow (BIO).

2.3.1. Examples of coding block structure in the VV coding standard

The VVC encoding standard adopts a QuadTree/BinaryTree/MulitpleTree (QT/BT/TT) structure for dividing an image into square or rectangular blocks. In addition to the QT/BT/TT structure, the VVC coding standard also adopts a split tree (known as dual tree coding) for I-frames (intra-prediction frames). In the case of a separate tree, the coding block structure is signaled separately for the luma and chrominance components.

2.3.2. Examples of adaptive resolution of motion vector differences

In some embodiments, motion vector differences (MVD) (between the motion vector and the predicted motion vector for the PU) are reported as signaling in units of quarter luminance samples when the use_integer_mv_flag is 0 in the slice header. Locally adaptive motion vector resolution (LAMVR) has been introduced into the JEM model. In the JEM model, the MVD difference can be encoded in units of quarter luminance samples, whole luminance samples, or quadruple luminance samples. MVD resolution is controlled at the coding unit (CU) level, and MVD resolution flags are conditionally signaled for each CU having at least one non-zero MVD component.

For a CU having at least one non-zero difference component MVD, a first flag is signaled to indicate whether one-quarter luminance unit precision is used for the MV vector in that CU. When the first flag (equal to 1) indicates that quarter luminance unit precision is not used for the MV vector, another flag is signaled to indicate that one whole luminance unit precision or four luminance unit precision is used for vector MV.

When the first MVD difference enable flag for a CU is zero, or is not encoded for a CU (meaning that all MVD differences for that CU are zero), a precision of one-quarter of the luminance sample unit for the MV vector is used for that CU. When a CU uses a precision of one luminance unit or four luminance units for a vector M, the MVP predictions in the AMVP prediction candidate list for that CU are rounded to the appropriate precision.

2.3.3. Examples of affine prediction with motion compensation

In the HEVC coding standard, motion compensation prediction (MCP) uses only the translational motion model. However, video cameras and objects can exhibit many types of motion, such as zooming in/out, rotation, perspective motion, and/or other irregular motion. The VVC coding standard uses simplified affine prediction with motion compensation with a 4-parameter affine model and a 6-parameter affine model. As shown in FIG. 13A and 13B, the block's affine motion field is described by two (in a 4-parameter affine model using the variables a, b, e and f) or three (in a 4-parameter affine model using the variables a, b, c, d, e and f) the motion vectors of the control point, respectively.

The motion vector field (MVF) for a block is described by the following equation according to the 4-parameter affine model and 6-parameter affine model, respectively:

$$\{\begin{array}{c}m{v}^h(x,y)=ax-by+e=\frac{(m{v}_1^h-m{v}_0^h)}{w}x-\frac{(m{v}_1^v-m{v}_0^v)}{w}y+m{v}_0^h\\ m{v}^v(x,y)=bx+ay+f=\frac{(m{v}_1^v-m{v}_0^v)}{w}x+\frac{(m{v}_1^h-m{v}_0^h)}{w}y+m{v}_0^v\end{array}$$ ...Lv. (1)

$$\{\begin{array}{c}m{v}^h(x,y)=ax+cy+e=\frac{(m{v}_1^h-m{v}_0^h)}{w}x+\frac{(m{v}_2^h-m{v}_0^h)}{h}y+m{v}_0^h\\ m{v}^v(x,y)=bx+dy+f=\frac{(m{v}_1^v-m{v}_0^v)}{w}x+\frac{(m{v}_2^v-m{v}_0^v)}{h}y+m{v}_0^v\end{array}$$ ...Lv. (2)

Here, (mv ^h ₀ , mv ^h ₀ ) denotes the motion vector of the motion control point (CP) in the upper left corner, and (mv ^h ₁ , mv ^h ₁ ) denotes the motion vector of the control point (CP) in the upper right corner, and (mv ^h ₂ , mv ^h ₂ ) denotes the motion vector of the reference point in the lower left corner, (x, y) represents the coordinates of the representative point relative to the upper left reference in the current block. The motion vectors of CPs can be signaled (as in AMVP affine prediction mode) or received in real time (as in affine fusion mode). The w and h parameters represent the width and height of the current block. In practice, partitioning is accomplished through the operation of right shift and rounding. In a VTM document, a representative point is defined as the center position of a sub-block, for example, when the coordinates of the upper left corner of a sub-block relative to the upper left reference in the current block are designated (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+ 2). For each sub-block (eg, 4x4 in a VTM document), a representative point is used to determine the motion vector for the sub-block as a whole.

Fig. 14 shows an example of an affine field MVF per sub-block as applied to block 1300, where affine transform prediction is applied to further simplify motion compensated prediction. To obtain the motion vector for each sub-block of size M×N, you can calculate the motion vector of the central sample of the corresponding sub-block according to Equations (1) and (2) and round it according to the fractional precision of the motion vector (for example, 1/16 in the model JEM). Motion compensation interpolation filters can then be applied to generate a prediction for each sub-block with the resulting motion vector. Affine mode introduces interpolation filters for the 1/16th image element. After MCP prediction, the high-precision motion vector for each sub-block is rounded and stored with the same accuracy as the regular motion vector.

2.3.3.1. Examples of affine prediction signaling

Similar to the translational model, there are also two modes of signaling additional information due to affine prediction. These are the AFFINE_INTER and AFFINE_Merge modes.

2.3.3.2. Examples of AF_INTER mode

For CUs that have both width and height greater than 8, the AF_INTER mode can be applied. CU-level affinity flag signaling is carried in the data bit stream to indicate whether AF_INTER mode is in use.

In this mode, for each reference image list (List 0 or List 1), a list of AMVP affine prediction candidates with three types of affine motion predictors is constructed in the following order, where each candidate contains an estimate of CPMV vectors for the current block. The differences between the best CPMV vectors found at the encoder side (such as in fig. 17) and estimates of these CPMV vectors. In addition, the AMVP affine prediction candidate index is obtained, from which the CPMV vector estimates are determined, and then transmitted as signaling.

Initial affine motion predictors

The verification procedure is similar to the case of spatial MVP prediction when constructing the AMVP prediction list in the HEVC encoding standard. First, the left initial affine motion predictor is obtained from the first block in the group {A1, A0} encoded in affine mode and having the same reference image as the current block. Secondly, the above initial affine motion predictor is obtained from the first block in the group {B1, B0, B2} encoded in affine mode and having the same reference image as the current block. In fig. Figure 16 shows five blocks A1, A0, B1, B0, B2.

When a neighboring block for affine mode encoding is found, the CPMV vectors for the coding unit covering that neighboring block are used to obtain predictors of the CPMV vectors for the current block. For example, if block A1 is encoded in non-affine mode and block A0 is encoded in 4-parameter affine mode, the left initial affine vector MV predictor will be obtained from block A0. In such a case, the CPMV vectors for the CU covering block A0, denoted as for the top left CPMV vector and for the top right CPMV vector in Fig. 18B are used to obtain estimates of the CPMV vectors for the current block, denoted by , for the top left (with coordinates (x0, y0)), top right (with coordinates (x1, y1)) and bottom right positions (with coordinates (x2, y2)) of the current block.

Construction of affine motion predictors

The constructed active motion predictor consists of control point motion vectors (CPMVs) obtained from neighboring blocks encoded using inter-frame prediction, as shown in FIG. 17, so that these blocks have the same reference image. If the current affine motion model is a 4-parameter affine model, the number of CPMV vectors is 2, otherwise, if the current affine motion model is a 6-parameter affine model, the number of CPMV vectors is 3. Top left CPMV vector is obtained by the MV vector in the first block of the group {A, B, C} encoded using inter-frame prediction and having the same reference picture as the current block. Top right CPMV vector is obtained by the MV vector in the first block of the group {D, E} encoded using inter-frame prediction and having the same reference picture as the current block. Lower left CPMV vector is obtained by the MV vector in the first block of the group {F, G} encoded using inter-frame prediction and having the same reference picture as the current block.

If the current affine motion model is a 4-parameter affine model, then the constructed affine motion predictor is inserted into the candidate list only if both vectors are found And , in other words, vectors And are used as estimates of the CPMV vectors for the top left (with coordinates (x0, y0)), and top right (with coordinates (x1, y1)) positions of the current block.

If the current affine motion model is a 4-parameter affine model, then the constructed affine motion predictor is inserted into the candidate list only if all three vectors are found , And , in other words, vectors , And are used as CPMV vector estimates for the top left (at coordinates (x0, y0)), top right (at coordinates (x1, y1)), and bottom right (at coordinates (x2, y2)) positions of the current block.

When inserting the constructed affine motion predictor into the candidate list, no truncation procedure is applied.

Conventional AMVP motion predictors

The following are applied until the number of affine motion predictors reaches a maximum.

Forming an affine motion predictor by setting all CPMV vectors equal to the vector , if available.

Forming an affine motion predictor by setting all CPMV vectors equal to the vector , if available.

Forming an affine motion predictor by setting all CPMV vectors equal to the vector , if available.

Generating an affine motion predictor by setting all CPMV vectors equal to the HEVC TMVP prediction, if any.

Forming an affine motion predictor by setting all CPMV vectors equal to the null MV vector.

Note that the vector already formed during the construction of the affine motion predictor.

In AF_INTER mode, if 4/6 parameter affine mode is used, 2/3 control points are required and therefore 2/3 of the MVD differences for these control points need to be encoded, as shown in FIG. 15A and 15B. In the existing embodiment, the MV vector can be formed as follows, for example, it predicts the differences mvd ₁ and mvd ₂ from the difference mvd ₀ .

Here are the parameters , mvd _i and mv ₁ represent the predicted motion vector, motion vector difference, and motion vector of the top left pixel (i = 0), top right pixel (i = 1), or bottom left pixel (i = 2), respectively, as shown in FIG. . 15B. In some embodiments, adding two motion vectors (eg, mvA(xA, yA) and mvB(xB, yB)) is equivalent to adding the two components separately. For example, the formula newMV = mvA + mvB implicitly assumes that the two components of the vector newMV are set equal to (xA + xB) and (yA + yB), respectively.

2.3.3.3. Examples of AF_Merge mode

When a CU is applied in AF_Merge mode, it receives the first affine-mode encoded block from the valid neighboring reconstructed blocks. In this case, the candidate block is selected in the following order - from left to top, top right, bottom left and top left, as shown in FIG. 18A (designated in order as A, B, C, D, E). For example, if the bottom left adjacent block is encoded in affine mode, as indicated by A0 in FIG. 18B, samples the motion vectors mv ₀ ^N , mv ₁ ^N , and mv ₂ ^N of control points (CPs) in the upper left corner, upper right corner, and lower left corner of the adjacent CU/PU that contains block A. And then the motion vector mv ₀ ^C , mv ₁ ^C and mv ₂ ^C (which is used only for the 6-parameter affine model) for the upper left corner/upper right, lower left corner for the current CU/PU unit are calculated based on the specified vectors mv ₀ ^N , mv ₁ ^N and mv ₂ ^N . It should be noted that in a VTM-2.0 document, a sub-block (for example, a 4x4 block in a VTM document) located in the upper left corner stores the vector mv0, a sub-block located in the upper right corner stores the vector mv1. if the current block is encoded in affine mode. If the current block is encoded according to the 6-parameter affine model, then the sub-block located in the lower left corner stores the vector mv2; otherwise (with 4-parameter affine moduli), the lower left block (LB) stores the vector mv2'. Other sub-blocks store MV vectors used for MC compensation.

After calculating the CPMV vectors for the current CU unit v0 and v1 according to the affine motion model according to Equations (1) and (2), the MVF field for the current CU unit can be generated. To identify whether the current CU is encoded in AF_Merge mode, an affinity flag may be signaled in the data bitstream if at least one adjacent block is encoded in affine mode.

In some embodiments (for example, JVET-L0142 and JVET-L0632), the list of affine mergeable candidates can be constructed according to the following steps:

1) Inserting initial affine candidates

The term "initial affine candidate" means that the candidate is derived from the affine motion model of its adjacent real block encoded in affine mode. From the affine motion model of neighboring blocks, a maximum of two initial affine candidates are obtained and inserted into the candidate list. For the left predictor, scanning is performed in the order {A0, A1}; for the top predictor, scanning is done in the order {B0, B1, B2}.

2) Insertion of constructed affine candidates

If the number of candidates in the affine merge candidate list is less than the MaxNumAffineCand parameter (set to 5 here), the constructed affine candidates are inserted into the candidate list. The term “constructed affine candidate” means that the candidate is constructed by combining the motion information of neighbors at each reference point.

a) Motion information for control points is derived primarily from the specified spatial neighbors and temporal neighbor shown in FIG. 19. The symbol CPk (k=1, 2, 3, 4) represents the k-th control point. The symbols A0, A1, A2, B0, B1, B2 and B3 show the spatial positions for predicting CPk (k=1, 2, 3); T shows the time position for CP4 prediction.

Points CP1, CP2, CP3 and CP4 have coordinates (0, 0), (W, 0), (H, 0) and (W, H), respectively, where W and H denote the width and height of the current block.

Traffic information for each control point is obtained according to the following order of priority:

For point CP1, the check priority is B2 → B3 → A2. Block B2 is used if available. Otherwise, if block B2 is not available, block B3 is used. If both blocks B2 and B3 are not available, block A2 is used. If all three candidates are unavailable, traffic information for CP1 cannot be obtained.

For point CP2, the check priority is B1 → B0.

For point CP3, the check priority is A1 → A0.

For point CP4, uses block T.

b) Secondly, a combination of control points is used to construct the affine merging candidate.

I. To construct a 6-parameter affine candidate, motion information for three control points is required. These three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). The combinations {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4} will be converted into a 6-parameter motion model represented by the upper left, upper right and lower left control points.

II. To construct a 4-parameter affine candidate, information about the movement of two control points is required. These two control points can be selected from one of the following six combinations ({CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}). The combinations {CP1, CP4}, {CP2, CP3}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4} will be converted into a 4-parameter motion model represented by the top left and top right control points.

III. Combinations of constructed affine candidates are inserted into the candidate list in the following order:

{CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}, {CP2, CP3}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}

i. For a reference list X (X is 0 or 1) combinations, the reference index with the highest utilization rate in reference points is selected as the reference index of this list X, and the motion vectors pointing to the difference reference image will be scaled.

Once a candidate is generated, a full truncation procedure is performed to check whether the same candidate has already been included in the list. If the same candidate exists, the newly formed candidate is discarded.

3) Filling with zero motion vectors

If the number of candidates in the affine merge candidate list is less than 5, zero motion vectors are inserted into the candidate list until the list is complete.

More specifically, for a list of sub-block merging candidates, the 4-parameter merging candidate has the MV vectors set to (0, 0) and the prediction direction is set to one side of list 0 (for P-slice) and to both sides (for B-cut).

2.3.4. Merge with Motion Vector Difference (MMVD) Examples

Document JVET-L0054 introduces the ultimate motion vector expression (UMVE, also known as MMVD). The UMVE parameter is used in conjunction with the skip mode or combine mode with the proposed motion vector expression method.

The UMVE parameter reuses a merge candidate that is the same as those included in the regular list of merge candidates in the VVC encoding standard. From the set of candidates to be combined, a base candidate can be selected and further expanded by the proposed motion vector expression method.

The UMVE parameter offers a new way of representing motion vector difference (MVD), whereby the starting point, the magnitude of the motion, and the direction of motion are used to represent the MVD.

This proposed method uses the list of candidates to be merged as is. But for the UMVE extension, candidates that have the default merge type (MRG_TYPE_DEFAULT_N) are considered.

The base candidate index determines the starting point. This base candidate index denotes the best candidate from the set of candidates included in the list as follows.

Table 1: Index (IDX) of the base candidate Index (IDX) of the base candidate 0 1 2 3 ^Nth MVP ^1st MVP ^2nd MVP ^3rd MVP ^4th MVP

If the base candidate number is 1, this IDX index is not signaled.

The distance index represents information about the magnitude of the movement. The distance index denotes a preset distance from the starting point information. This preset distance is defined as follows:

Table 2. Index (IDX) of distance Index (IDX) of distance 0 1 2 3 4 5 6 7 Distance in pixels (pel) 1/4-pel 1/2-pel 1-pel 2-pel 4-pel 8-pel 16-pel 32-pel

The direction index represents the direction of the MVD difference relative to the starting point. The direction index can represent four directions as shown below.

Table 3. Index (IDX) of direction Index (IDX) of direction 00 01 10 eleven x-axis + - Not applicable Not applicable y-axis Not applicable Not applicable + -

In some embodiments, the UMVE extension flag is sent as signaling immediately after the skip flag or merge flag is sent. If the skip or join flag is true, the UMVE flag is parsed. If the UMVE flag is 1, the UMVE extension syntax is parsed. But if this flag is not 1, the Affinity Flag is parsed. If the Affinity Flag is 1, it means AFFINE mode. But if this flag is not 1, the skip/merge index is parsed for the skip/merge mode in the VTM document.

No additional row buffer due to the emergence of UMVE candidates is required since the skip/merge candidate from the software is directly used as the base candidate. Using the input UMVE index, the complement of the MV vector just before motion compensation is determined. There is no need to keep a long line buffer for this.

In the current state of the shared text, either the first or second merge candidate from the list of merge candidates may be selected as the base candidate.

2.3.5. Examples of motion vector refinement on the decoder side (Decoder-side Motion vector Refinement (DMVR))

In bidirectional prediction, to predict an area of one block, two prediction blocks generated using a motion vector (MV) from list0 and an MV vector from list1 are combined, respectively, to form one prediction signal. According to the decoder-side motion vector refinement (DMVR) method, two motion vectors used in bidirectional prediction are further refined.

In the JEM model, motion vectors are refined through a two-way template matching procedure. This two-way template matching is used in the decoder to perform a distortion-based search between the two-way template and the reconstructed samples in the reference images to obtain a refined MV vector without transmitting additional motion information. An example is shown in Fig. 22. The two-way template is generated as a weighted (ie, averaged) combination of two predicted blocks from the original list0 vector MV0 and list1 vector MV1, respectively, as shown in FIG. 22. The pattern matching operation involves calculating costs between the generated pattern and the sample region (around the original predicted block) in the reference image. For each of the two reference images, the MV vector that produces the minimum template cost is considered as an updated MV vector from the corresponding list to replace the original vector. In the JEM model, nine candidate MV vectors are searched for each list. This set of nine candidate MV vectors contains the original MV vector and 8 surrounding MV vectors shifted by one luminance count from the original MV vector in either the horizontal direction or the vertical direction, or both. Finally, two new MV vectors, i.e. vectors MV0′ and MV1′, as shown in FIG. 22 are used to generate bidirectional prediction results. The sum of absolute differences (SAD) is used as a measure of cost. Please note that when calculating the cost of a prediction block generated by one of the surrounding MV vectors, the rounded (to whole pixels) MV vector is used instead of the actual MV vector to obtain the prediction block.

To further simplify the DMVR refinement procedure, JVET-M0147 proposes a number of changes to the structure of the JEM model. More specifically, the adopted DMVR refinement procedure in VTM-4.0 (due out soon) has the following main features:

ο Early termination from (0,0) position in the SAD sum between list0 and list1

ο Block size for the DMVR refinement procedure W*H>=64 && H>=8

ο Splitting the CU unit into several sub-blocks of size 16x16 for the DMVR refinement procedure when the CU unit size is > 16*16

ο Size of the reference block (W+7)*(H+7) (for the luminance component)

ο 25-point search based on sum of SAD over whole pixels (i.e. refinement search range (+-) 2, one stage)

ο DMVR refinement procedure based on bilinear interpolation

ο Mirror the MVD difference between list0 and list1 to allow two-way negotiation

ο “Surface parametric error equation” based on refinement at the level of fractions of a pixel

ο MC compensation of the brightness/chrominance component with filling of the reference block (if necessary)

ο Refined MV vectors are used only for MC compensation and TMVP prediction

2.3.6. Examples of combined intra and inter prediction (CIIR)

JVET-L0100 proposes multi-hypothesis prediction, where combining intra- and inter-frame prediction is one way to generate multiple hypotheses.

When multi-hypothesis prediction is used to improve the intra-prediction mode, multi-hypothesis prediction combines one intra-prediction and one indexed fusion prediction. At the merging CU, one flag is signaled for the merging mode to select an intra-prediction mode from the candidate list for intra-prediction when this flag is true. For the luminance component, the intra-prediction candidate list is formed from four intra-prediction modes, including DC mode, planar mode, horizontal mode and vertical mode, and the size of the intra-prediction candidate list can be 3 or 4 depending on the block shape. When the width of a CU is greater than twice the height of the CU, the horizontal mode is excluded from the intra prediction mode list, and when the height of the CU is greater than twice the width of the CU, the vertical mode is excluded from the intra prediction mode list. One intra-prediction mode selected by the intra-prediction mode index and one merging indexed prediction mode selected by the merging index are combined using weighted averaging. For the color component, the DM mode is always used without additional signaling. The weights for combining forecasts are described as follows. When DC mode or planar mode is selected, or the CB block width or height is less than 4, the same weighting factors are applied. For CB blocks with a width and height of at least 4, when the horizontal/vertical mode is selected, one CB block is first divided vertically/horizontally into four areas of equal area. Each set of weights, denoted by (w_intra _i , w_inter _i ), where i is from 1 to 4 and (w_intra ₁ , w_inter ₁ ) = (6, 2), (w_intra ₂ , w_inter ₂ ) = (5, 3) , (w_intra ₃ , w_inter ₃ ) = (3, 5) and (w_intra ₄ , w_inter ₄ ) = (2, 6), will be applied to the corresponding area. The coefficients (w_intra ₁ , w_inter ₁ ) are for the region closest to the reference samples, and the coefficients (w_intra ₄ , w_inter ₄ ) are for the region furthest from the reference samples. Then the combined prediction can be calculated by summing the two weighted prediction results and shifting 3 bits to the right. Moreover, the intra-prediction mode for the intra-prediction predictor hypotheses can be stored as a reference for the next neighboring CUs.

2.4. In-Loop Reshaping (ILR) in JVET-M0427

The basic idea of in-loop reshaping (ILR) is to transform the original (in the first region) signal (predicted/reconstructed signal) into a second region (reshaped region).

The in-loop luminance reshaping device is implemented as a pair of look-up tables (LUTs), but only one of the two LUTs needs to be signaled, while the other table can be calculated based on the reported LUT. Each LUT is a one-dimensional, 10-bit, mapping table with 1024 input positions (1D-LUT). One LUT is a forward LUT (direct LUT), FwdLUT, which converts quantities input color code into modified values : . The other LUT is the inverse LUT (inverse LUT), InvLUT, which converts the changed code values before : . ( represents the reconstructed quantities .).

2.4.1. Piece-wise linear (PWL) model

In some embodiments, the piecewise linear (PWL) model is implemented as follows:

Let x1, x2 be two input anchor points and y1, y2 be the corresponding output anchor points for one line segment. The output value y for any input value x between x1 and x2 can be interpolated using the following equation:

y = ((y2-y1)/(x2-x1)) * (x-x1) + y1

In a fixed-point implementation, this equation can be rewritten as:

y = ((m * x + 2FP_PREC-1) >> FP_PREC) + c

where m denotes a scalar, c denotes a shift, and FP_PREC is a constant for specifying precision.

Note that in the CE-12 software, the PWL model is used to pre-compute the forward FwdLUT and inverse InvLUT mapping tables, each with 1024 input positions; but the PWL model also allows implementations to calculate identical display values in real time without first computing LUTs.

2.4.2. Test CE12-2

2.4.2.1. Reshaping the brightness component

Benchmark 2 for in-loop luminance reshaping (i.e., CE12-2 in the proposal) creates a pipeline of lower complexity that also eliminates the decoding latency for block-by-block intra-frame prediction during slice reconstruction in inter-frame prediction. Intra-frame prediction is carried out in the reshaped area for both slices - with intra-frame and inter-frame prediction.

Intra-frame prediction is always performed in the reshaped region, regardless of the slice type. In such a configuration, the intra-prediction process can begin immediately after the reconstruction of the previous TU is completed. Such a configuration may also offer a unified procedure for intra-frame prediction instead of a slice-specific procedure. In fig. 23 is a flowchart of a mode-based decoding procedure CE12-2.

The CE12-2 system also tests 16-element piecewise linear (PWL) models for scaling luma and chrominance residuals instead of the 32-element PWL models in the CE12-1 system.

Reconstruction of a slice during intra-frame prediction using a device for intra-circuit reformation of the brightness component in the CE12-2 system (blocks shaded in light green indicate the signal in the reformed area: the remainder of the brightness component; the brightness component during intra-frame prediction; the reconstructed brightness component during intra-frame prediction).

2.4.2.2. Luma-dependent chroma residue scaling (LCRS)

The luminance component-dependent scaling of the chroma residual is a multiplicative process implemented by a fixed-point integer operation. Scaling the remaining chrominance component compensates for the interaction of the luminance signal with the chrominance signal. Chroma residual scaling is applied at the TU level. More specifically, the following applies:

ο For intra-frame prediction, the reconstructed luminance component is averaged.

ο For interframe prediction, the predicted luminance component is averaged.

Averaging is used to identify the index in the PWL model. This index identifies the scale factor cScaleInv. The remainder of the color component is multiplied by this coefficient.

Note that the scale factor for the chrominance component is calculated based on the forward-mapped luminance component values instead of the reconstructed luminance component values.

2.4.2.3. Signaling additional information about ILR reformation

These parameters are passed (currently) in the tile group header (similar to ALF filtering). These messages occupy 40 - 100 bits. A group of tiles could be another way to represent an image. The following table is based on version 9 of the JVET-L1001 document. Added syntax is in italics.

In 7.3.2.1. RBSP Sequence Parameter Set Syntax

In 7.3.3.1. General Tile Group Header Syntax

Adding a new model of the tile group reformation device in the syntax table:

Adding the following semantics to the general semantics of the RBSP sequence parameter set:

The sps_reshaper_enabled_flag equal to 1 specifies that the reshaper is used in a coded video sequence (CVS). The sps_reshaper_enabled_flag equal to 0 specifies that the reshaper is not used in the CVS sequence.

In the tile group header syntax, adding the following semantics

The tile_group_reshaper_model_present_flag equal to 1 specifies that the tile_group_reshaper_model() parameter is present in the tile group header. The tile_group_reshaper_model_present_flag equal to 0 specifies that the tile_group_reshaper_model() parameter is not present in the tile group header. When the tile_group_reshaper_model_present_flag is not present, it is considered to be 0.

The tile_group_reshaper_enabled_flag equal to 1 specifies that the reshaper is enabled for the current tile group. The tile_group_reshaper_enabled_flag equal to 0 specifies that the reshaper is not enabled for the current tile group. When the tile_group_reshaper_enable_flag is not present, it is considered to be 0.

The tile_group_reshaper_chroma_residual_scale_flag flag equal to 1 specifies that chroma residual scaling is enabled for the current tile group. The tile_group_reshaper_chroma_residual_scale_flag equal to 0 specifies that chroma residual scaling is not enabled for the current tile group. When the tile_group_reshaper_chroma_residual_scale_flag is not present, it is treated as 0.

Adding tile_group_reshaper_model() syntax

The reshape_model_min_bin_idx index specifies the index of the minimum fragment (or segment) to be used during the construction process of the reshaping device. The value of this index, reshape_model_min_bin_idx, must be in the range from 0 to the MaxBinIdx parameter, inclusive. The value of the MaxBinIdx parameter must be equal to 15.

The index reshape_model_delta_max_bin_idx specifies the index of the maximum valid fragment (or segment) MaxBinIdx minus the maximum fragment index to use in the reshaping device design process. The index value reshape_model_max_bin_idx is set equal to MaxBinIdx - reshape_model_delta_max_bin_idx.

The reshaper_model_bin_delta_abs_cw_prec_minus1 plus 1 parameter specifies the number of bits used to represent the reshape_model_bin_delta_abs_CW[ i ] syntax.

The reshape_model_bin_delta_abs_CW[ i ] parameter specifies the absolute value of the increment codeword (delta) for the i-th fragment.

The reshaper_model_bin_delta_sign_CW_flag[ i ] parameter specifies the sign of the reshape_model_bin_delta_abs_CW[ i ] parameter as follows:

- If the reshape_model_bin_delta_sign_CW_flag[ i ] flag is 0, the corresponding RspDeltaCW[ i ] variable has a positive value.

- Otherwise ( reshape_model_bin_delta_sign_CW_flag[ i ] is not equal to 0 ), the corresponding variable RspDeltaCW[ i ] has a negative value.

When the reshape_model_bin_delta_sign_CW_flag[ i ] flag is not present, it is considered equal to 0.

Variable RspDeltaCW[ i ] = (1 2*reshape_model_bin_delta_sign_CW [ i ]) * reshape_model_bin_delta_abs_CW [ i ];

The variable RspCW[ i ] is determined through the following steps:

The OrgCW variable is set to (1 << BitDepth _Y ) / ( MaxBinIdx + 1 ).

- If reshaper_model_min_bin_idx <= i <= reshaper_model_max_bin_idx

RspCW[ i ] = OrgCW + RspDeltaCW[ i ].

- Otherwise, RspCW[ i ] = 0.

The RspCW[i] value must be in the range from 32 to 2 * OrgCW - 1 if the BitDepth _Y value is 10.

InputPivot[ i ] variables with i in the range from 0 to MaxBinIdx + 1, inclusive, are formed as follows

InputPivot[ i ] = i * OrgCW

The ReshapePivot[ i ] variable for i in the range from 0 to MaxBinIdx + 1, inclusive, the ScaleCoef[ i ] variable and the InvScaleCoeff[ i ] variable for i in the range from 0 to MaxBinIdx, inclusive, are defined as follows:

shiftY = 14

ReshapePivot[ 0 ] = 0;

for( i = 0; i <= MaxBinIdx ; i++) {

ReshapePivot[ i + 1 ] = ReshapePivot[ i ] + RspCW[ i ]

ScaleCoef[ i ] = ( RspCW[ i ] * (1 << shiftY) + (1 << (Log2(OrgCW) - 1))) >> (Log2(OrgCW))
if ( RspCW[ i ] == 0 )

InvScaleCoeff[ i ] = 0

else
InvScaleCoeff[ i ] = OrgCW * (1 << shiftY) / RspCW[ i ]

}

The variable ChromaScaleCoef[i] with i in the range from 0 to MaxBinIdx, inclusive, is defined as follows:

ChromaResidualScaleLut[64] = {16384, 16384, 16384, 16384, 16384, 16384, 16384, 8192, 8192, 8192, 8192, 5461, 5461, 5461, 5461, 4096, 40 96, 4096, 4096, 3277, 3277, 3277, 3277, 2731, 2731, 2731, 2731, 2341, 2341, 2341, 2048, 2048, 2048, 1820, 1820, 1820, 1638, 1638, 1638, 1638, 1489, 1489, 1 489, 1489, 1365, 1365, 1365, 1365, 1260, 1260, 1260, 1260, 1170, 1170, 1170, 1170, 1092, 1092, 1092, 1092, 1024, 1024, 1024, 1024};

shiftC = 11

- if ( RspCW[ i ] == 0 )

ChromaScaleCoef [ i ] = (1 << shiftC)

- Otherwise (RspCW[ i ] != 0), ChromaScaleCoef[ i ] = ChromaResidualScaleLut[RspCW[ i ] >> 1]

2.4.2.4. Using ILR Reframing

On the encoder side, each image (or group of tiles) is first converted into a reshaped region. And the entire coding procedure is carried out in the reformed area. For intra-frame prediction, the adjacent block is in the reshaped region; For inter-frame prediction, reference blocks (generated from the original region from the decoded image buffer) are first converted into a reshaped region. The remainder is then generated and encoded in the data bit stream.

After encoding/decoding of the entire image (or group of tiles) is completed, the samples in the reshaped region are converted to the original region and then the deblocking filter and other filters are applied.

The procedure for direct conversion to a predicted signal is not activated in the following cases:

○ The current block is encoded using intra-frame prediction

○ The current block is encoded using the current picture referencing (CPR), also called intra-block copying (IBC)

○ The current block is encoded using combined inter-intra mode (CIIP) and direct reframing is not enabled for intra-prediction blocks

JVET-N0805

To avoid signaling additional ILR re-arrangement information in the tile group header, JVET-N0805 proposes to carry this signaling in the APS set. It contains the following main ideas:

- Optionally send the LMCS parameters in the SPS set. Here, LMCS stands for luma mapping with chroma scaling (LMCS) as defined in the relevant video coding standards.

- Define APS set types for ALF filtering parameters and LMCS parameters. Each APS set has only one type.

- Transfer of LMCS parameters in the APS set

- If the LMCS tool is activated, it has a flag in the TGH tile group header to indicate whether the LMCS aps_id is present or not. If there is no alarm about this, use a set of SPS parameters.

*Semantic constraints need to be added to ensure that there is always something valid that applies to the case when this tool is activated.

2.5.2.5.1. Implementation of the proposed framework on top of the JVET-M1001 document (VVC encoding, working draft 4)

The proposed changes are shown in italics below.

...

The sps_lmcs_enabled_flag flag equal to 1 specifies that the coded video sequence (CVS) uses luma mapping with chrominance scaling. The sps_lmcs_enabled_flag equal to 0 specifies that luma mapping with chrominance scaling is not used in the CVS sequence.

The sps_lmcs_default_model_present_flag equal to 1 specifies that lmcs data is present by default in this SPS set. The sps_lmcs_default_model_flag equal to 0 specifies that the default lmcs data is not present in this SPS set. When the sps_lmcs_default_model_present_flag is not present, it is considered to be 0.

...

The aps_params_type parameter specifies the type of APS set parameters passed in the APS set, as shown in the following table:

Table 7 - x -APS Parameter Set Type Codes and APS Parameter Types

Adding the following definitions to Art. 3:

ALF APS: A set of APS that has aps_params_type equal to ALF_APS.

LMCS APS: A set of APS that has aps_params_type equal to LMCS_APS.

The following semantic changes have been made:

...

The tile_group_alf_aps_id parameter specifies the adaptation_parameter_set_id parameter of the ALF APS set to which the tile group belongs. The TemporalId parameter of an ALF APS NAL unit that has an adaptation_parameter_set_id parameter equal to the tile_group_alf_aps_id parameter must be no greater than the TemporalId parameter of the encoded tile group NAL unit.

When multiple ALF APS sets with the same adaptation_parameter_set_id value refer to two or more tile groups in the same image, those multiple ALF APS sets with the same adaptation_parameter_set_id value shall have the same content.

...

The tile_group_lmcs_enabled_flag equal to 1 specifies that luma display with chrominance scaling is enabled for the current tile group. The tile_group_lmcs_enabled_flag equal to 0 specifies that luma display with chrominance scaling is not enabled for the current tile group. When the tile_group_lmcs_enable_flag is not present, it is considered to be 0.

The tile_group_lmcs_use_default_model_flag equal to 1 specifies that the chroma-scaled luma display operation for a group of tiles uses the default lmcs model. The tile_group_lmcs_use_default_model_flag equal to 0 specifies that the chrominance scaling luma mapping operation for a group of tiles uses the lmcs model in the LMCS APS set associated with the tile_group_lmcs_aps_id parameter. When the tile_group_reshaper_use_default_model_flag is not present, it is considered to be 0.

The tile_group_lmcs_aps_id parameter specifies the adaptation_parameter_set_id parameter of the LMCS APS set referenced by the tile group. The TemporalId parameter of an LMCS APS NAL unit that has an adaptation_parameter_set_id parameter equal to the tile_group_lmcs_aps_id parameter shall be no greater than the TemporalId parameter for the encoded tile group NAL unit.

When multiple LMCS APS sets with the same adaptation_parameter_set_id value refer to two or more tile groups in the same image, those multiple LMCS APS sets with the same adaptation_parameter_set_id value shall have the same content.

The tile_group_chroma_residual_scale_flag equal to 1 specifies that chroma residual scaling is enabled for the current tile group. The tile_group_chroma_residual_scale_flag equal to 0 specifies that chroma residual scaling is enabled for the current tile group. When the tile_group_chroma_residual_scale_flag is not present, it is treated as 0.

...

...

2.4.2.6. JVET-N0138

This section proposes an advanced use of the adaptation parameter set (APS) to pass device model parameters for reshaping, as well as ALF filtering parameters. At the last meeting, it was decided that the ALF filter parameters should be carried in the APS set instead of the tile group header in order to improve coding efficiency by avoiding unnecessary redundant signaling of parameters across multiple tile groups. For the same reason, it is proposed to pass the parameters of the reformer model to the APS model instead of the tile group header. To identify the type of parameters in an APS set (whether at least ALF filter parameters or reformer models), information about the APS set type is required in the APS set syntax as well as in the APS ID.

Syntax and semantics of the adaptation parameter set

The proposed changes are shown in italics below.

The adaptation_parameter_set_type parameter identifies the type of parameters in the APS set. The value of the adaptation_parameter_set_type parameter must be in the range from 0 to 1 inclusive. If adaptation_parameter_set_type is 0, the ALF filter parameters are signaled. Otherwise, the parameters of the device model for reformation are transmitted as a signal.

Generalized tile group header syntax and semantics

2.5. Virtual Pipelining Data Units (VPDUs)

Virtual Pipeline Data Units (VPDUs) are defined as non-overlapping MxM-luma(L)/NxN-chroma(C) data units in an image. In hardware decoders, successive VPDUs are processed in multiple pipeline stages simultaneously; different stages process different VPDUs simultaneously. The VPDU size is roughly proportional to the buffer size in most pipeline stages, so it can be said that it is very important to keep the VPDU size small. In hardware decoders for the HEVC encoding standard, the VPDU unit size is set equal to the maximum transform block (TB) size. Increasing the maximum TB block size from 32x32-L/16x16-C (as in the HEVC encoding standard) to 64x64-L/32x32-C (as in today's VVC encoding standard) can yield encoding gains, resulting in an expected 4x increase VPDU unit size (64x64-L/32x32-C) compared to the HEVC encoding standard. However, in addition to partitioning the coding unit (CU) according to a quadtree (QT), the VVC coding standard also adopts a ternary tree (TT) and a binary tree (BT) to achieve additional coding gain, and partitioning according to the TT and BT trees can be applied to the 128x128-L/64x64-C Coding Tree Units (CTUs) recursively, resulting in a 16x increase in VPDU unit size (128x128-L/64x64 -C) compared to the HEVC encoding standard.

In modern VVC coding configuration, the VPDU unit size is defined as 64x64-L/32x32-C.

2.6. Set of adaptation parameters

In the VVC coding standard, an adaptation parameter set (APS) is adopted to convey the ALF filtering parameters. The tile group header contains an aps_id parameter, which is conditionally present when ALF filtering is enabled. The APS set contains the aps_id parameter and ALF filtering parameters. The APS set has a new value of type NUT (NAL unit type, as in the AVC and HEVC encoding standards) (from JVET-M0132). For general test conditions, the VTM-4.0 document (to be released) suggests simply using aps_id = 0 and passing a set of APS with each image. As of today, an APS ID will have a range of 0..31 and APS sets can be shared between multiple images (and can vary across different groups within the same image). ID values must be, if present, encoded in a fixed-length code. ID values cannot be reused with other content in the same image.

2.7. Tools used

2.7.1. Diffusion filter (DF)

In JVET-L0157, a diffusion filter is proposed, where the intra/inter-prediction signal from a CU can be further modified with diffusion filters.

Homogeneous diffusion filter. A Uniform Diffusion Filter is implemented by convolving the predicted signal with a fixed mask specified as or how , which is defined below.

As an input to the filtered signal, in addition to the predicted signal itself, one line of reconstructed samples is used to the left and above of the block in question, where the use of these reconstructed samples can be avoided for blocks with inter-frame prediction.

Let denotes the predicted signal for a given block in question obtained by intra-frame prediction or motion-compensated prediction. To process cutoff points for filters, the predicted signal needs to be expanded to the predicted signal . This extended forecast signal can be generated in two ways:

Or, as an intermediate step, one line of reconstructed samples is added to the left and top of the predicted signal and then mirrored the resulting signal in all directions. Or only the predicted signal itself is mirrored in all directions. The last extension is used for inter-frame prediction blocks. In this case, only the forecast signal itself contains the input data for the extended forecast signal .

If a filter should be used , it is proposed to replace the predicted signal signal

using the boundary extension mentioned above. Here is the filter mask looks like

If you should use a filter , it is proposed to replace the predicted signal signal

Filter here given as

Directional scattering filter. Instead of using signal-adaptive scattering filters, directional filters are used, namely a horizontal filter and vertical filter , which still have a fixed mask. More precisely, uniform scattering filtering corresponding to the mask from the previous section are simply limited to application only along the vertical or along the horizontal direction. The vertical filter is implemented using a fixed filter mask

to the predicted signal and the horizontal filter is implemented using a transposed mask

2.7.2. Bilateral Filter (BF)

A two-way filter is proposed in JVET-L0406, and is always applied to luma blocks with non-zero transform coefficients and cutoff quantization parameters greater than 17. Therefore, there is no need to signal a two-way filter. The two-way filter, if applied, acts on the decoded samples immediately after the inversion. In addition to this, the filter parameters, i.e. weighting coefficients are explicitly derived from the encoded information.

The filtering procedure is defined as:

…(1)

Here, denotes the intensity of the current reading, and denotes the modified intensity of the current reading, And denote the intensity and weight parameter for the kth neighboring sample, respectively. An example of one current sample and its adjacent four samples (i.e., K=4) is shown in FIG. 24.

More specifically, the weighting factor , associated with the kth neighboring sample, is defined as follows:

. …(2)

Here,

And _.

Here, the parameter depends on the encoding mode and encoding block size. The described filtering procedure is applied to blocks encoded using intra-prediction and to blocks encoded using inter-prediction when the TU is further split to allow parallel processing.

To better capture the statistical properties of the video signal and improve filter performance, the weighting function obtained from Equation (2) is adjusted using the parameter given in Table 4 depending on the encoding mode and block partitioning parameters (minimum size).

Table 4. Parameter values for different block sizes and encoding mode Min (block width, block height) Intra-frame prediction mode Inter-frame prediction mode 4 82 62 8 72 52 Other 52 32

To further improve coding performance, for blocks encoded using inter-frame prediction when the TU is not divided, the intensity difference between the current sample and one of its adjacent samples is replaced by a representative intensity difference between two windows covering the current sample and said adjacent sample. Therefore, the equation for the filtration process is revised and reduced to the form:

…(4)

Here, And represent the magnitude of the mth sample in windows centered at points And , respectively. In this proposal, the window size is set to 3x3. Example of two windows covering samples And shown in Fig. 25.

2.7.3. Hadamard transform domain filter (HF)

JVET-K0068 proposes an in-loop filter in the domain of one-dimensional (1D) Hadamard transform applied at the CU level after reconstruction and implemented without the use of multiplication. The proposed filter is applied to all CUs satisfying a given condition, and the filter parameters are obtained from the encoded information.

The proposed filtering is always applied to the reconstructed luma blocks with non-zero transform coefficients, excluding 4x4 blocks and the case where the slice quantization parameter is greater than 17. The filter parameters are derived explicitly from the encoded information. The proposed filter, if applied, processes the decoded samples immediately after the inverse transformation.

For each pixel from the reconstructed block, processing contains the following steps:

○ Scanning 4 neighboring pixels around the processed pixel, including the current pixel, according to the scanning pattern

○ 4-point Hadamard transform of read pixels

○ Spectral filtering based on the following formula:

Here, (i) denotes the index of the spectral component in the Hadamard spectrum, R(i) denotes the spectral component of the reconstructed pixel corresponding to the index, σ denotes the filtering parameter obtained from the quantization parameter (QP) of the codec using the following equation:

.

An example of a scanning circuit is shown in FIG. 26, where A denotes the current pixel and {B,C,D} denotes the surrounding pixels.

For pixels lying on the boundary of a CU, the scanning pattern is adjusted to ensure that all required pixels are within the current CU.

3. Disadvantages of existing implementation options

In existing options for implementing ILR reformation, the following disadvantages may occur:

1) Signaling additional information to reshape the ILR in the tile group header is not suitable because it requires too many bits. Additionally, prediction between different images/tile groups is not possible. Therefore, additional ILR reshaping information needs to be transmitted for each group of tiles, which may cause encoding losses at low data bit rates, especially at low resolutions.

2) The interaction between ILR reformulation and DMVR refinement (or other newly introduced coding tools) remains unclear. For example, ILR reformation is applied to an inter-frame prediction signal to transform the original signal into a reformatted region, and the decoded residuals are in the reformatted region. Whereas DMVR refinement also uses the predicted signal to refine motion vectors for one block. Whether the DMVR refinement should be applied in the original domain or in the reshaped domain is unclear.

3) The interaction between ILR reshaping by screen content encoding tools such as palette, B-DPCM modulation, IBC copy, transform skip, transform and quantization bypass, I-PCM modulation modes is not clear.

4) When reformatting the ILR, scaling of the remainder of the chrominance component, depending on the luminance component, is used. Therefore, an additional delay is introduced here (due to the dependence between the luminance component and the chrominance component), which is not a favorable factor for hardware design.

5) The purpose of introducing VPDUs is to ensure that processing of one 64x64 square area is completed before processing of other 64x64 square areas begins. However, according to the ILR reshaping configuration, there is no restriction on the use of ILR reshaping, which may cause violation of VPDUs since the chrominance component relies on the predicted luminance component signal.

6) If all the coefficients for one CU are zero, the predicted block and the reconstructed block still go through the forward and backward reformation procedures, which leads to wasted computational complexity.

7) In document JVET-N0138, it is proposed to signal the ILR re-formation information in the APS set. This technical solution may create a number of new problems. For example, two types of APS kits have been developed. However, the adaptation_parameter_set_id signaled for ILR reshaping may refer to an APS set that does not contain ILR reshaping information. Likewise, the adaptation_parameter_set_id signaled for adaptive loop filtering (ALF) may refer to an APS set that does not contain ALF filtering information.

4. Examples of in-circuit reformation methods for video encoding

Variants of the technology proposed by the present invention overcome the shortcomings of existing implementations, thereby creating a video encoding system with higher encoding efficiency. In-loop reshaping methods based on the proposed technology that can enhance existing and future video coding standards are highlighted in the following examples described for possible implementations. The proposed technology examples below illustrate general concepts and should not be interpreted as limitations. In one example, unless explicitly stated to the contrary, a variety of features described in several examples may be combined. It should be noted that some of the proposed technologies can be applied to the existing candidate list building procedure.

Herein, the decoder side motion vector derivation (DMVD) method contains methods like DMVR refinement and FRUC transformation that perform motion estimation to obtain or refine block/sub-block motion information and flow BIO, which refines movement by samples. Below is a numbered list of various examples (Examples 1 - 42).

1. The procedure for refining traffic information in DMVD generation technologies, such as DMVR refinement, may depend on the information in the reformatted area.

a. In one example, predicted blocks generated from reference images in the original region may be first transformed into the reshaped region before they are used to refine motion information.

i. Alternatively, furthermore, cost calculations (eg, SAD differences, MR-SAD differences)/gradient calculations are performed on the reshaped region.

Alternatively, after the motion information has been refined, the reshaping procedure is not activated for prediction blocks generated using the motion information refined

b. Alternatively, the motion information refinement procedure in DMVD generation technologies such as DMVR refinement may depend on information in the source domain.

i. DMVD generation procedures can be invoked with predicted blocks in the source area.

ii. In one example, after the motion information has been refined, predicted blocks are obtained with the motion information refined, or the final predicted block (eg, a weighted average of two predicted blocks) can be further transformed into a reshaped region to generate the final reconstructed block.

iii. Alternatively, furthermore, after the motion information has been refined, the reformation procedure is not activated for prediction blocks generated with the motion information refined.

2. It is proposed to combine the region for the samples in the current tile/group of tiles/image and the samples obtained from the reference images (or both in the original region or in the reshaped region) used to determine the local light compensation (LIC) parameters.

a. In one example, the reshaped region is used to obtain LIC compensation parameters.

Alternatively, in addition, samples (e.g., reference samples of reference images (whether through interpolation or not), as well as adjacent/non-adjacent samples to reference samples (whether through interpolation or not)) may first be converted to the reshaped region before being used to derive parameters LIC compensation.

b. In one example, the source region is used to obtain LIC compensation parameters.

Alternatively, in addition, spatially adjacent/non-adjacent samples of the current block (eg, in the current tile group/image/tile) may be first converted to the original region before being used to obtain LIC compensation parameters.

c. It is assumed that when the LIC compensation parameters are obtained in one area, the same area of prediction blocks should be used when applying the LIC compensation parameters to the prediction blocks.

In one example, when point a. is invoked, the support blocks can be transformed into a reshaped region, and the LIC compensation model is applied to the reshaped support blocks.

In one example, when item b is invoked, reference blocks are stored in the source region, and the LIC compensation model is applied to the reference blocks in the source region.

d. In one example, the LIC compensation model is applied to the predicted blocks in the re-formed region (eg, the predicted block is first converted to the re-formed region through direct re-forming).

e. In one example, the LIC compensation model is first applied to the prediction blocks in the original region, after which the final prediction block, depending on the prediction blocks to which the LIC compensation is applied, can be converted into a reshaped region (eg, through direct reshaping) and used for obtaining the reconstructed block.

f. The above methods can be extended to other encoding tools that rely on both spatially adjacent/non-adjacent samples and reference samples in reference pictures.

3. For filters applied to the predicted signal (such as a diffuse filter (DF)), such a filter is applied to the predicted block in the source domain.

a. Alternatively, furthermore, reshaping is applied to the filtered predicted signal to generate a reconstructed block.

b. An example of a coding procedure using inter-frame prediction is shown in FIG. 27.

c. Alternatively, filters are applied to the predicted signal in the reshaped region.

Alternatively, in addition, the reformation is first applied to the predicted block; after which the filtering methods can be further applied to the re-formed predicted block to generate a reconstructed block.

ii. An example of a coding procedure using inter-frame prediction is shown in FIG. 28.

d. Filtering options may depend on whether ILR reshaping is enabled or not.

4. For filters applied to reconstructed blocks (eg, Bilateral Filter (BF), Hadamard Transform Domain (HF) filter), these filters are applied to the reconstructed blocks in the original domain instead of the reshaped domain.

a. Alternatively, in addition, the reconstructed block in the reshaped region is first converted to the original region, after which filters can be applied and used to generate the reconstructed block.

b. An example of a coding procedure using inter-frame prediction is shown in FIG. 29.

c. Alternatively, filters can be applied to the reconstructed block in the reshaped region.

Alternatively, filters may also be applied first before applying the inverse reformation. After this, the filtered reconstructed block can be converted to the original region.

ii. An example of a coding procedure using inter-frame prediction is shown in FIG. thirty.

d. Filtering options may depend on whether ILR reshaping is enabled or not.

5. It is proposed to apply a filtering procedure that can be applied to the reconstructed blocks (for example, after intra-frame/inter-frame or other types of prediction) in the reshaped area.

a. In one example, a deblocking filtering (DBF) procedure is performed on the reshaped region. In this case, the reverse reformation is not applied before DBF filtering.

In this case, the DBF filter parameters may be different depending on whether reshaping is applied or not.

ii. In one example, the DBF filtering procedure may depend on whether reshaping is applied or not.

1. In one example, this method is applied when DBF filtering is invoked in the source area.

2. Alternatively, this method is applied when the DBF filter is applied in the reshaped region.

b. In one example, a nonlinear sample adaptive offset (SAO) filtering procedure is performed on the reshaped region. In this case, the reverse reformation is not applied before SAO filtering.

c. In one example, an adaptive loop filtering (ALF) procedure is performed on the reshaped region. In this case, the inverse reformation is not applied before ALF filtering.

d. Alternatively, in addition, inverse reformation can be applied to blocks after DBF filtering.

e. Alternatively, in addition, inverse reformation can be applied to blocks after SAO filtering.

f. Alternatively, in addition, inverse reformation can be applied to the blocks after ALF filtering.

g. The above filtering method can be replaced by other filtering methods.

6. It is proposed to signal the ILR reshaping parameters in a new parameter set (such as the ILR APS set) instead of tile group headers.

a. In one example, the tile group title may contain an aps_id parameter. Alternatively, in addition, the aps_id parameter is conditionally present when ILR reshaping is enabled.

b. In one example, the ILR APS set contains the aps_id parameter and ILR reshaping parameters.

c. In one example, a new value of type NUT (NAL unit type, as in the AVC and HEVC encoding standards) is assigned to the APS ILR set.

d. In one example, the ILR APS ID values will be in the range 0...M (for example, M = 2K-1).

e. In one example, ILR APS sets may be shared between images (and may be different in different groups within the same image).

f. In one example, the ID value may, when present, be encoded in a fixed length code. Alternatively, it may be encoded using exponential-Golomb (EG), truncated unary, or other binarization.

g. In one example, ID values cannot be reused with other content within the same image.

h. In one example, the ILR APS set and the ALF filter parameter APS set may share the same NUT type.

i. Alternatively, the ILR reshaping parameters may be passed in the current APS set for the ALF filtering parameters. In this case, in the above methods mentioning the ILR APS set, it can be replaced by the current APS set.

j. Alternatively, the ILR reshaping parameters may be conveyed in the SPS set/VPS set/PPS set/sequence header/picture header.

k. In one example, the set of ILR reshaping parameters may contain information about the model of the reshaping device, the use of the ILR reshaping method, scaling factors, and chroma residuals.

l. Alternatively, in addition, ILR reshaping parameters may be signaled at one layer (such as in an APS set), and/or the use of ILR reframing may be further signaled at a second layer (such as in a group header tiles).

m. Alternatively, in addition, predictive coding can be applied to encoding ILR reshaping parameters with different APS set indices.

7. Instead of using luminance-dependent chroma residual scaling (LCRS) with conversion to chrominance blocks, it is proposed to use forward/reverse reformation procedures into chrominance blocks to eliminate the dependence between luminance and chrominance components.

a. In one example, one M-segment piecewise linear (PWL) model and/or forward/inverse lookup table may be used for one chrominance component. Alternatively, two PWL models and/or forward/inverse lookup tables can be used to encode the two chrominance components, respectively.

b. In one example, the chroma PWL model and/or forward/inverse lookup table may be derived from the PWL model or forward/inverse lookup tables for the luminance component.

In one example, there is no need for additional signaling of PWL models/lookup tables for the chrominance component.

c. In one example, an indication of the PWL model and/or forward/inverse lookup table may be signaled in the SPS set/VPS set/APS set/PPS set/sequence header/image header/tile group header/unit header/row CTUs/group of CTUs/areas.

8. In one example, the method of how to signal ILR reshaping parameters in one picture/tile group may depend on the ILR reframing parameters of previously encoded pictures/tile groups.

a. For example, the ILR reformation parameters of one image/tile group may be predicted by the ILR reformation parameters of one or more previously encoded images/tile groups.

9. It is proposed to disable luminance dependent chroma residual scaling (LCRS) for certain block sizes/time layers/tile group types/image types/encoding modes/certain motion type information.

a. In one example, even if the forward/backward resampling procedure is applied to the luma component blocks, the LCRS scaling procedure may not be applied to the corresponding chrominance component blocks.

b. Alternatively, even if the forward/backward resampling procedure is not applied to the luma blocks, the LCRS scaling procedure may still be applied to the corresponding chrominance blocks.

c. In one example, LCRS scaling is not used when cross-component linear model (CCLM) modes are used. The CCLM mode set contains LM, LM-A and LM-L.

d. In one example, LCRS scaling is not used when cross-component linear model (CCLM) modes are not applied. The CCLM mode set contains LM, LM-A and LM-L.

e. In one example, when the luminance component coded block exceeds one VPDU (eg, 64x64).

In one example, when the luminance component block size is less than M*H samples, such as 16 or 32 or 64 luminance samples, LCRS scaling is not allowed.

ii. Alternatively, when the minimum size is the width and/or height of a luminance component block less than or equal to X, LCRS scaling is not allowed. In one example, X is set to 8.

iii. Alternatively, when the minimum size is the width and/or height of a luminance component block not less than X, LCRS scaling is not allowed. In one example, X is set to 8.

iv. Alternatively, when block width > th1 or >=th1 and/or luminance block height > th2 or >=th2, LCRS scaling is not allowed. In one example, th1 and/or th2 are set to 8.

1. In one example, th1 and/or th2 are set to 128.

2. In one example, th1 and/or th2 are set to 64.

v. Alternatively, when luminance block width < th1 or <=th1 and/or luminance block height < th2 or <a=th2, LCRS scaling is not allowed. In one example, th1 and/or th2 are set to 8.

10. Whether the ILR reframing activation (forward reframing procedure and/or reverse reframing procedure) should be deactivated may depend on the coefficients.

a. In one example, when one block is encoded with all zero coefficients, the forward reformation procedure is skipped on the predicted block.

b. In one example, when one block is encoded with all zero coefficients, the inverse reshaping procedure is skipped on the reconstructed block.

c. In one example, when one block is encoded with only one non-zero coefficient located at certain positions (eg, a DC coefficient located in the top left position of one block, a coefficient located in the top left code group in one block), a direct reframing procedure in relation to the predicted block and/or the reverse reformation procedure in relation to the reconstructed block is skipped.

d. In one example, when one block is encoded such that it has only M (eg, M=1) non-zero coefficients, the forward reformation procedure on the predicted block and/or the backward reformation procedure on the reconstructed block is skipped.

11. It is proposed to split the scope of ILR reshaping into virtual pipeline data units (VPDUs) if the encoded block is larger than one VPDU. Each applied area (eg, with a maximum size of 64x64) is treated as an individual CU for the ILR reshaping operation.

a. In one example, when a block width > th1 or >=th1 and/or a block height > th2 or >=th2, that block may be divided into sub-blocks with a width < th1 or <= th1 and/or a height < th2 or <= th2, and ILR reshaping can be done for each sub-block.

In one example, the sub-blocks may have the same width and/or height.

ii. In one example, sub-blocks, except those located on the right border and/or the bottom border, may have the same width and/or height.

iii. In one example, sub-blocks, except those located on the left border and/or on the top border, may have the same width and/or height.

b. In one example, when the block size (i.e. width * height) is > th3 or >= th3, it can be split into sub-blocks of size < th3 or <= th3, and ILR reshaping can be performed for each sub-block.

In one example, the sub-blocks may be the same size.

ii. In one example, sub-blocks other than those located on the right border and/or the bottom border may be the same size.

iii. In one example, sub-blocks other than those located on the left border and/or on the top border may be the same size.

c. Alternatively, the use of ILR reshaping is limited only by the size of a particular block.

In one example, when the coded block is larger than one VPDU (eg, 64x64), ILR reshaping is not allowed.

ii. In one example, when a block contains fewer M*H samples, such as 16 or 32 or 64 luminance samples, ILR reshaping is not allowed.

iii. Alternatively, when the minimum block width and/or height dimension is less than or equal to X, ILR reshaping is not allowed. In one example, X is set to 8.

iv Alternatively, when the minimum block width and/or height dimension is not less than X, ILR reshaping is not allowed. In one example, X is set to 8.

v. Alternatively, when block width > th1 or >=th1 and/or block height > th2 or >=th2, ILR reshaping is not allowed. In one example, the value of th1 and/or th2 is set to 8.

l. In one example, the value of th1 and/or th2 is set to 128.

2. In one example, the value of th1 and/or th2 is set to 64.

vi. Alternatively, when block width < th1 or <=th1 and/or block height < th2 or <a=th2, ILR reshaping is not allowed. In one example, the value of th1 and/or th2 is set to 8.

12. The above methods (eg, whether to deactivate ILR reframing, and/or whether to deactivate LCRS scaling, and/or whether to signal the PWL model/lookup tables for chroma encoding) may depend on the color format, such as 4:4:4/4:2:0.

13. An ILR reshaper enable indication (eg, tile_group_reshaper_enable_flag) may be encoded subject to an indication of the present reshaper model (eg, tile_group_reshaper_model_present_flag).

a. Alternatively, the tile_group_reshaper_model_present_flag may be encoded subject to the tile_group_reshaper_enable_flag.

b. Alternatively, only one of two syntax elements, including the tile_group_reshaper_model_present_flag and the tile_group_reshaper_enable_flag, may be encoded. The value of the other element is set equal to the value of the element that can be signaled.

14. Various truncation techniques can be applied to the predicted signal and to the reconstruction procedure.

a. In one example, an adaptive truncation method may be applied, and the maximum and minimum values to be trimmed may be determined by the reshaped region.

b. In one example, adaptive truncation may be applied to the predicted signal in the reshaped region.

c. Alternatively, in addition, a fixed truncation (eg, according to the bit depth) may be applied to the reconstructed block.

15. Filter parameters (such as those used in DF, BF, HF filters) may depend on whether ILR reshaping is applied or not.

16. It is assumed that for blocks encoded in palette mode, ILR reshaping is not enabled at all or is applied differently.

a. In one example, when the block is encoded in palette mode, the reshaping and de-reshaping procedures are skipped.

b. Alternatively, when a block is encoded in palette mode, various reshaping and de-reshaping functions can be applied.

17. Alternatively, when ILR reframing is applied, the palette mode may be encoded differently.

a. In one example, when ILR reframing is applied, the palette mode may be encoded in the source region.

b. Alternatively, when ILR reshaping is applied, the palette mode may be encoded in the reshaped region.

c. In one example, when ILR reshaping is applied, the palette predictors may be signaled in the source region.

d. Alternatively, when ILR reshaping is applied, the palette predictors may be signaled in the reshaped region.

18. It is proposed that for blocks encoded in the IBC copy mode, the activation of ILR reframing is canceled or the reframing is applied differently.

a. In one example, when a block is encoded in IBC copy mode, the reshaping and de-reshaping operations are skipped.

b. Alternatively, when a block is encoded in the IBC copy mode, various reshaping and de-reshaping operations are applied.

19. Alternatively, when ILR reshaping is applied, IBC copy mode encoding can be performed in different ways.

a. In one example where ILR reshaping is applied, IBC copy mode encoding may be performed on the source region.

b. Alternatively, when ILR reshaping is applied, IBC copy mode encoding can be performed on the reshaped region.

20. It is proposed that for blocks encoded in the B-DPCM modulation mode, ILR reshaping should not be enabled or applied differently.

a. In one example, when a block is encoded in the B-DPCM modulation mode, the reshaping and de-reshaping operations are skipped.

b. Alternatively, when a block is encoded in the B-DPCM modulation mode, various reshaping and de-reshaping operations are applied.

21. Alternatively, when ILR reshaping is applied, B-DPCM mode coding can be performed in different ways.

a. In one example, when ILR reshaping is applied, B-DPCM encoding may be performed in the source domain.

b. Alternatively, when ILR reshaping is applied, B-DPCM encoding can be performed on the reshaped region.

22. It is proposed that for blocks encoded in skip transformation mode, ILR reshaping should not be enabled or applied differently.

a. In one example, when a block is encoded in skip-transformation mode, the reshaping and de-reshaping operations are skipped.

b. Alternatively, when a block is encoded in skip-transformation mode, various reshaping and de-reshaping operations may be applied.

23. Alternatively, when ILR reshaping is applied, the skip transformation mode can be encoded differently.

a. In one example, when ILR reshaping is applied, a skip transformation may be performed in the original region.

b. Alternatively, when ILR reshaping is applied, skipping the transformation can be performed on the reshaped region.

24. It is proposed that for blocks encoded in the I-PCM modulation mode, ILR reshaping should not be enabled or applied differently.

a. In one example, when the block is encoded in palette mode, the reshaping and de-reshaping operations are skipped.

b. Alternatively, when a block is encoded in palette mode, various reshaping and inverse reshaping functions can be applied.

25. Alternatively, when ILR reshaping is applied, I-PCM modulation mode coding can be performed in different ways.

a. In one example, when ILR reshaping is applied, I-PCM modulation mode coding can be performed in the source domain.

b. Alternatively, when ILR reshaping is applied, I-PCM modulation mode coding can be performed on the reshaped region.

26. It is proposed that for blocks encoded in transform and quantization bypass mode, ILR reshaping should not be enabled or applied differently.

a. In one example, when a block is encoded in the transform and quantization bypass mode, the inverse reshaping operations are skipped.

27. Alternatively, when a block is encoded in transform and quantization bypass mode, various reshaping and de-reshaping functions may be applied.

28. In the above positions, when ILR reframing is not activated, the forward reframing and/or reverse reframing procedures can be skipped.

a. Alternatively, the predicted signal and/or the reconstructed signal and/or the residual signal are in the original region.

b. Alternatively, the predicted signal and/or the reconstructed signal and/or the residual signal are in the reshaped region.

29. Multiple reframing/reframing functions (such as multiple PWL models) may be valid for encoding one picture/one group of tiles/one VPDU/one area/one row of CTUs/multiple CUs.

a. How exactly to choose from several functions may depend on the block size/encoding mode/picture type/low latency control flag/motion information/reference pictures/video content, etc.

b. In one example, multiple sets of additional ILR reframe information (eg, reframe/reverse reframe functions) may be signaled in the SPS set/VPS set/PPS set/sequence header/image header/tile group header/regions /VPDU units/etc.

Alternatively, predictive coding may also be used in relation to the additional ILR reshaping information.

c. In one example, more than one aps_idx may be signaled in a PPS set/picture header/tile group header/tile header/areas/VPDUs/etc.

30. In one example, the reframing information is signaled in a new syntax set other than the VPS set, SPS set, PPS set, or APS set. For example, reshaping information is signaled in a set designated inloop_reshaping_parameter_set() (IRPS, or by some other name).

a. An example syntax structure is shown below. Added syntax is in italics.

The inloop_reshaping_parameter_set_id parameter is an identifier for a set of IRPS parameters to be referenced by other syntax elements.

Note - IRPS sets can be used by multiple images and may vary between different groups of tiles within the same image.

An irps_extension_flag equal to 0 specifies that syntax elements with the irps_extension_data_flag flag are not present in the IRPS RBSP syntax structure. The irps_extension_flag flag equal to 1 specifies that syntax elements with the irps_extension_data_flag flag are present in the IRPS RBSP syntax structure.

The irps_extension_data_flag can have any value. Its presence and meaning do not affect the decoder's compliance with the profiles specified in this version of this Specification. Decoders conforming to this version of this Specification shall ignore all syntax elements with the irps_extension_data_flag.

b. An example syntax structure is shown below. Added syntax is in italics.

General syntax and semantics of a tile group header

The tile_group_irps_id parameter specifies the inloop_reshaping_parameter_set_id of the IRPS set to which the tile group in question belongs. The TemporalId of an IRPS NAL unit that has an inloop_reshaping_parameter_set_id equal to the tile_group_irps_id must be no greater than the TemporalId of the encoded tile group NAL unit.

31. In one example, IRL reshaping information is signaled along with ALF filtering information in the APS set.

a. An example syntax structure is shown below. Added syntax is in italics.

Syntax and semantics of the adaptation parameter set

In one example, a single tile_group_aps_id is signaled in the tile group header to specify the adaptation_parameter_set_id of the APS set to which the tile group in question belongs. In this specified set, APSs signal ALF filtering information and ILR reshaping information for the current tile group.

i. An example syntax structure is shown below. Added syntax is in italics.

32. In one example, ILR reshaping information and ALF filtering information are transmitted in different APS sets.

a. The first ID (may be called tile_group_aps_id_alf) is signaled in the tile group header to specify the first adaptation_parameter_set_id of the first APS set to which the tile group in question belongs. The ALF filtering information for the current tile group is signaled in the specified first set of APS.

b. A second ID (may be called tile_group_aps_id_irps) is signaled in the tile group header to specify the second adaptation_parameter_set_id of the second APS set to which the tile group in question belongs. The ILR reshaping information for the current tile group is signaled in the specified second set of APS.

c. In one example, the first APS set must contain ALF filtering information in the match data bitstream;

d. In one example, the second APS set must contain information about the ILR reshaping in the match data bit stream;

e. An example syntax structure is shown below. Added syntax is in italics.

33. In one example, some APS sets with the specified adaptation_parameter_set_id must have ALF filtering information. As another example, certain APS sets with the specified adaptation_parameter_set_id must have ILR reshaping information.

a. For example, APS sets with adaptation_parameter_set_id equal to 2N shall have ALF filtering information. Here N is an integer;

b. For example, APS sets with adaptation_parameter_set_id equal to 2N+1 shall have ILR reshaping information. Here N is an integer;

c. An example syntax structure is shown below. Added syntax is in italics.

For example, parameter 2* tile_group_aps_id_alf specifies the first adaptation_parameter_set_id of the first APS set to which the tile group in question belongs. The ALF filtering information for the current tile group is signaled in the specified first set of APS.

ii. For example, parameter 2* tile_group_aps_id_irps+1 specifies the second adaptation_parameter_set_id of the second APS set to which the tile group in question belongs. The ILR reshaping information for the current tile group is signaled in the specified second set of APS.

34. In one example, a tile group may not belong to an APS set (or IRPS set) signaled before a network abstraction layer (NAL) unit of the specified type that is signaled before the current tile group.

a. In one example, the tile group may not refer to an APS set (or IRPS set) signaled previously by a previously specified tile group type that is signaled before the current tile group.

b. For example, a tile group cannot belong to an APS set (or IRPS set) signaled before an SPS set that is signaled before the current tile group.

c. For example, a tile group cannot belong to an APS set (or IRPS set) signaled before a PPS set that is signaled before the current tile group.

d. For example, a tile group cannot belong to an APS set (or IRPS set) signaled before an Access unit delimiter NAL (AUD), which is signaled before the current tile group.

e. For example, a tile group cannot refer to an APS set (or IRPS set) signaled before the End of bitstream NAL(EoB) unit, which is signaled before the current tile group.

f. For example, a tile group cannot refer to an APS set (or IRPS set) signaled before the End of sequence NAL (EoS) unit, which is signaled before the current tile group.

g. For example, a tile group cannot belong to an APS set (or IRPS set) signaled before an instantaneous decoding refresh (IDR) NAL unit, which is signaled before the current tile group.

h. For example, a tile group cannot belong to an APS set (or IRPS set) signaled before a clean random access (CRA) NAL unit, which is signaled before the current tile group.

i. For example, a tile group cannot be an APS set (or IRPS set) signaled before the intra random access point (IRAP) one signaled before the current tile group.

j. For example, a tile group cannot be an APS set (or IRPS set) signaled before an I-type tile group (or image or slice) signaled before the current tile group.

k. The methods described in IDF-P1903237401H and IDF-P1903234501H can also be applied when ILR reshaping information is carried in an APS set or in an IRPS set.

35. The correspondence data bitstream must satisfy the condition that when the in-loop reshaping method is enabled for one unit of video data (such as a sequence), default ILR reshaping parameters such as a default model must be specified.

a. The sps_lmcs_default_model_present_flag must be set to 1 when the sps_lmcs_enabled_flag is set to 1.

b. Default parameters may be signaled provided that the ILR reframe enable flag is present instead of the default model present flag (such as the sps_lmcs_default_model_present_flag flag).

c. For each tile group, a default model use flag (such as tile_group_lmcs_use_default_model_flag) may be signaled without reference to the default model use flag from the SPS set.

d. The matching data bitstream must satisfy the condition that when there is no ILR reshaping information among the corresponding types in the APS set for ILR reshaping, and one video data unit (such as a group of tiles) is forced to use the ILR reshaping technology, the default model must be used.

e. Alternatively, the matching data bit stream must satisfy the condition that when there is no ILR reshaping information among the corresponding types in the APS set for ILR reshaping, and one unit of video data (such as a group of tiles) is forced to use the ILR reshaping technology (such as the equal 1 flag tile_group_lmcs_enable_flag), the default model use indication must be true, for example, the tile_group_lmcs_use_default_model_flag must be 1.

f. It is limited that default ILR reshaping parameters (such as a default model) must be transmitted in a video data unit (such as an SPS set).

i. Alternatively, in addition, ILR reframing parameters must be sent when the SPS set flag indicating the use of ILR reframing is true.

g. It is limited that at least one ILR APS set is transmitted in a video data unit (such as an SPS set).

i. In one example, the at least one ILR APS set contains default ILR reshaping parameters (such as a default model).

36. The default ILR reshaping parameters can be indicated by a single flag. When this flag indicates that the default ILR reshaping parameters are used, there is no need to additionally signal the ILR reshaping data.

37. Default ILR reshaping parameters may be preset when not signaled. For example, the default ILR reshaping parameters may correspond to the identity mapping.

38. Time layer information may be signaled along with ILR reshaping parameters such as the ILR APS set.

a. In one example, the index of the temporary layer may be signaled in the lmcs_data() parameter.

b. In one example, the index of the time layer minus 1 can be passed as a signal in the lmcs_data() parameter.

c. Alternatively, in addition, when encoding/decoding a single tile group/single tile, the situation is limited to references to those ILR APS sets associated with a lesser or equal time layer index.

d. Alternatively, when encoding/decoding a single tile group/single tile, the situation is limited to references to those ILR APS sets associated with a lower temporal layer index.

e. Alternatively, when encoding/decoding a single tile group/single tile, the situation is limited to references to those ILR APS sets associated with a large temporal layer index.

f. Alternatively, when encoding/decoding a single tile group/single tile, the situation is limited to references to those ILR APS sets associated with a greater or equal time layer index.

g. Alternatively, when encoding/decoding a single tile group/single tile, the situation is limited to references to those ILR APS sets associated with the same time layer index.

h. In one example, whether the above restrictions apply may depend on a segment of information that may be signaled to the decoding device or generated by the decoding device itself.

39. Time layer information can be signaled along with ALF filtering parameters, as in the ALF APS set.

a. In one example, the index of the temporary layer may be signaled in the alf_data() parameter.

b. In one example, the index of the time layer minus 1 can be passed as a signal in the alf_data() parameter.

c. Alternatively, in addition, when encoding/decoding one tile group/single tile or one CTU in one tile/tile group, the situation is limited to references to those ALF APS sets associated with a less than or equal time layer index.

d. Alternatively, in addition, when encoding/decoding a single tile group/single tile, the situation is limited to references to those ALF APS sets associated with a lower time layer index.

e. Alternatively, in addition, when encoding/decoding a single tile group/single tile, the situation is limited to references to those ALF APS sets associated with a large temporary layer index.

f. Alternatively, in addition, when encoding/decoding a single tile group/single tile, the situation is limited to references to those ALF APS sets associated with a greater or equal time layer index.

g. Alternatively, in addition, when encoding/decoding one tile group/one tile, the situation is limited to references to those ALF APS sets that are associated with the same time layer index.

h. In one example, whether the above restrictions apply may depend on a segment of information that may be signaled to the decoding device or generated by the decoding device itself.

40. In one example, the mapping of the reshaping between the original samples and the reshaping samples cannot be a positive relationship, in other words, the mapping of one larger quantity to a smaller quantity is not allowed.

a. For example, the reshaping mapping between the original samples and the reshaping samples may be a negative relationship, where, of two quantities, the larger value from the original region may be mapped to a smaller value in the reshaping region.

41. In the match data bitstream, the aps_params_type syntax element can only take a number of specified values, such as 0 and 1.

a. In another example, this element can only accept the values 0 and 7.

42. In one example, the default ILR reshaping information must be transmitted if the ILR reframing can be applied (eg, sps_lmcs_enabled_flag is true).

5. Examples of implementation of the proposed technology

In some embodiments, the tile_group_reshaper_enable_flag is conditionally present when the tile_group_reshaper_model_present_flag is enabled. Added syntax is in italics.

B 7.3.3.1 General Tile Group Header Syntax

Alternatively, the tile_group_reshaper_model_present_flag is conditionally present when the tile_group_reshaper_enable_flag is enabled.

Alternatively, only one of the two syntax elements may be signaled - tile_group_reshaper_model_present_flag or tile_group_reshaper_enable_flag. An element that is not passed can be inferred to be equal to the element that is passed. In this case, the specified one syntax element controls the use of ILR reframing.

Alternatively, the match data bitstream requires that the tile_group_reshaper_model_present_flag be equal to the tile_group_reshaper_enable_flag.

Alternatively, the tile_group_reshaper_model_present_flag and/or the tile_group_reshaper_enable_flag and/or the tile_group_reshaper_model() parameter and/or the tile_group_reshaper_chroma_residual_scale_flag may be signaled in the APS set instead of the tile group header.

Option #2 on top of document JVET-N0805. Added syntax is in italics.

...

The sps_lmcs_enabled_flag, equal to 1, specifies the chrominance scaling luma display used in the coded video sequence (CVS). The sps_lmcs_enabled_flag equal to 0 specifies that luma mapping with chrominance scaling is not used in the CVS sequence.

The sps_lmcs_default_model_present_flag equal to 1 specifies that lmcs mode data is present in the SPS set by default. The sps_lmcs_default_model_flag equal to 0 specifies that the default lmcs mode data is not present in the SPS set. When this sps_lmcs_default_model_present_flag is not present, it is considered to be 0.

...

The aps_params_type parameter specifies the type of APS set parameters carried in this APS set, as specified in the following table:

Table 7-x - APS Dial Parameter Type Codes and APS Dial Parameter Types aps_params_type Name aps_params_type APS Set Parameter Types 0 ALF_APS ALF filtering options 1 LMCS_APS LMCS mode parameters 2..7 Reserved Reserved

ALF APS: A set of APS having aps_params_type equal to ALF_APS.

LMCS APS: A set of APS having aps_params_type equal to LMCS_APS.

Making the following semantic changes:

...

The tile_group_alf_aps_id parameter specifies the adaptation_parameter_set_id of the ALF APS set to which the tile group belongs. The TemporalId of an ALF APS NAL unit that has an adaptation_parameter_set_id equal to the tile_group_alf_aps_id must be no greater than the TemporalId of the encoded tile group NAL unit.

When multiple ALF APS sets with the same adaptation_parameter_set_id value refer to two or more tile groups from the same image, those multiple ALF APS sets with the same adaptation_parameter_set_id value shall have the same content.

...

The tile_group_lmcs_enabled_flag flag, equal to 1, specifies that the luma display mode with chrominance scaling is enabled for the current tile group. The tile_group_lmcs_enabled_flag flag equal to 0 specifies that the luma component display mode with chrominance scaling is not enabled for the current tile group. When the tile_group_lmcs_enable_flag is not present, it is considered to be 0.

The tile_group_lmcs_use_default_model_flag equal to 1 specifies that the chroma-scaled luma display operation for a group of tiles uses the default lmcs model. The tile_group_lmcs_use_default_model_flag equal to 0 specifies that the chrominance scaling luma display operation for a group of tiles uses the lmcs model from the LMCS APS set associated with the tile_group_lmcs_aps_id. When the tile_group_reshaper_use_default_model_flag is not present, it is set to 0.

The tile_group_lmcs_aps_id parameter specifies the adaptation_parameter_set_id from the LMCS APS set to which the tile group belongs. The TemporalId parameter from an LMCS APS NAL unit that has an adaptation_parameter_set_id equal to tile_group_lmcs_aps_id must be at least equal to the TemporalId parameter of the NAL unit for the encoded tile group.

When multiple LMCS APS sets with the same adaptation_parameter_set_id value refer to two or more tile groups in the same image, those multiple LMCS APS sets with the same adaptation_parameter_set_id value shall have the same content.

The tile_group_chroma_residual_scale_flag equal to 1 specifies that chroma residual scaling is enabled for the current tile group. The tile_group_chroma_residual_scale_flag equal to 0 specifies that chroma residual scaling is not enabled for the current tile group. When the tile_group_chroma_residual_scale_flag is not present, it is treated as 0.

...

...

Luminance display mode data syntax with chrominance scaling

The examples described above may be included in the context of the method described below, for example, the methods as shown in FIG. 31A to 39E, which may be implemented in a de-video encoding apparatus or a video encoding apparatus.

Fig. 31A shows a logic diagram of an example of a video processing method. Method 3100 comprises, at step 3110, performing, to effect the conversion between the current video block and the encoded video representation, a procedure for refining motion information based on samples in the first region or in the second region. This method 3100 comprises, at step 3120, performing a conversion based on the result of the motion information refinement procedure. In some embodiments, the conversion process obtains samples for the current video block from a first prediction block in a first region using unrefined motion information, at least a second prediction block is generated in a second region using refined motion information used to determine the reconstructed block, and generating reconstructed samples of the current video block at least based on the second predicted block

Fig. 31B shows a logic diagram of an example of a video processing method. Method 3120 comprises, at step 3122, reconstructing, based on at least one predicted block in the second region, the current video block. In some implementations, during the conversion process, the current video block is constructed based on the first region and the second region and/or the remainder of the chroma component is scaled in a manner dependent on the luminance component. In some implementations, the conversion process employs an encoding tool derived from at least a plurality of first samples in the video video domain and a plurality of second samples in the reference picture for the current video block. In some implementations, the region for the first samples and the region for the second samples are combined.

Fig. 32A shows a logic diagram of an example of a video processing method. Method 3210 comprises, at step 3212, determining, for the current video block of the current video region, a parameter for the encoding mode of the current video block based on one or more encoding mode parameters of the previous video region. Method 3210 further comprises, at step 3214, performing encoding for the current video block to generate an encoded representation of the video based on this determination. In some implementations, the specified encoding mode parameter is included in a set of parameters in the encoded representation of the video. In some implementations, the encoding procedure comprises transforming a representation of a current video block in a first region to a representation of that current video block in a second region. In some embodiments, during the encoding process using the encoding mode, the current video block is constructed based on the first region and the second region and/or the remainder of the chroma component is scaled in a manner dependent on the luminance component.

Fig. 32B shows a logic diagram of an example of a video processing method. This method 3220 includes, at step 3222, receiving an encoded representation of the video containing a set of parameters, including parameter information for the encoding mode. Method 3220 further comprises, at step 3224, decoding the encoded representation using the parameter information to generate a current video block in the current video video region based on the encoded representation. In some implementations, the parameter information for the encoding mode is based on one or more encoding mode parameters of the previous video region. In some embodiments, in encoding mode, the current video block is constructed based on the first region and the second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component.

Fig. 32C shows a logic diagram of an example of a video processing method. Method 3230 includes, at step 3232, performing a conversion between the current video block of video and an encoded representation of the video. In some implementations, the transformation procedure comprises applying a filtering operation to a predicted block in a first region or in a second region different from the first region.

Fig. 32D shows a logic diagram of an example of a video processing method. Method 3240 comprises, at step 3242, performing a conversion between the current video block of video and an encoded representation of the video. In some implementations, the conversion process determines the final reconstructed block for the current video block. In some implementations, a temporary reconstructed block is generated using a prediction method and presented in a second region.

Fig. 33 shows a logic diagram of an example of a video processing method. Method 3300 comprises, at step 3302, performing a transformation between the current video block of the current video region of the video and an encoded representation of the video, where the transformation procedure uses an encoding mode in which the current video block is constructed based on the first region and the second region and/or scales the remainder of the chrominance component in a manner dependent from the luminance component, and where the parameter set in the encoded representation contains parameter information for the encoding mode.

Fig. 34A shows a logic diagram of an example of a video processing method. Method 3410 comprises, at step 3412, performing a transformation between a current video video block representing a chrominance component block and an encoded video representation, where, in the transformation process, the current video block is constructed based on the first region and the second region, and where the transformation procedure further comprises applying direct reformation procedures and/or reverse reformation procedures to one or more color components of the current video block.

Fig. 34B shows a logic diagram of an example of a video processing method. Method 3420 comprises, at step 3422, performing a conversion between a current chroma video block and an encoded video representation, where the conversion procedure comprises: determining whether luma-dependent chroma residual scaling (LCRS) is enabled or disabled, based on some rule, and reconstruction of the current video block of the chrominance component based on the determination result.

Fig. 35A shows a logic diagram of an example of a video processing method. Method 3510 comprises, at step 3512, determining, in order to perform a conversion between the current video block and an encoded video representation, whether a certain encoding mode should be deactivated based on the values of one or more coefficients for the current video block. This method 3510 further comprises, at step 3514, performing a conversion based on the specified determination. In some embodiments, during conversion using said encoding mode, the current video block is constructed based on the first region and the second region and/or the remainder of the chroma component is scaled in a manner dependent on the luminance component.

Fig. 35B shows a logic diagram of an example of a video processing method. Method 3520 comprises, at step 3522, partitioning, to convert between the current video block of video exceeding the virtual pipeline data unit (VPDU) for video, the current video block into areas. This method 3520 further comprises, at step 3524, performing the conversion by applying an encoding mode to each region separately. In some embodiments, during the conversion process using the encoding mode, the current video block is constructed based on the first region and the second region and/or the remainder of the chroma component is scaled in a manner dependent on the luminance component.

Fig. 35C shows a logic diagram of an example of a video processing method. Method 3530 comprises, at step 3532, determining, in order to perform a conversion between the current video block and an encoded video representation, whether to deactivate the use of an encoding mode based on the size or color format of the current video block. This method 3530 further comprises, at step 3534, performing a conversion based on the specified determination. In some embodiments, during conversion using said encoding mode, the current video block is constructed based on the first region and the second region and/or the remainder of the chroma component is scaled in a manner dependent on the luminance component.

Fig. 35D shows a logic diagram of an example of a video processing method. Method 3540 comprises, at step 3542, performing a transformation between the current video video block and an encoded video representation, where the transformation uses an encoding mode in which the current video block is constructed based on the first region and the second region and/or scales the remaining chrominance component in a luma dependent manner. component, and wherein the at least one syntactic element in said encoded representation creates an indication of the use of said encoding mode and an indication of a reshaping device model.

Fig. 35E shows a logic diagram of an example of a video processing method. Method 3550 comprises, at step 3552, determining that an encoding mode is not activated to perform the conversion between the current video block and the encoded video representation. This method 3550 further comprises, at step 3554, conditionally skipping forward reformation and/or reverse reformation based on the result of said determination. In some embodiments, in said encoding mode, the current video block is constructed based on the first region and the second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component.

Fig. 35F shows a logic diagram of an example of a video processing method. Method 3560 comprises, at step 3562, performing a transformation between the current video block of a video region and an encoded representation of the video, where the transformation procedure uses an encoding mode in which the current video block is constructed based on the first region and the second region and/or scales the remaining chrominance component in a manner dependent on luminance component, and wherein a plurality of forward reformatting operations and/or a plurality of reverse reformatting operations are applied in a reformatting mode for the video domain.

Fig. 36A shows a logic diagram of an example of a video processing method. Method 3610 comprises, at step 3612, determining that a certain encoding mode is activated to effect the conversion between the current video block and the encoded video representation. This method 3610 further comprises, at 3614, performing a transformation using a palette mode, where at least a palette of representative sample values is used for the current video block. In some embodiments, in encoding mode, the current video block is constructed based on the samples in the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

Fig. 36B shows a logic diagram of an example of a video processing method. Method 3620 comprises, at step 3622, determining, to effect a conversion between the current video block and an encoded representation of the video, that the current video block is encoded in a palette mode, in which at least a palette of representative sample values is used to encode the current video block. This method 3620 further comprises, at step 2624, carrying out, based on the results of the specified determination, a transformation by deactivating the encoding mode. In some implementations, when an encoding mode is applied to a video block, the video block is constructed based on the chrominance residual scaled in a manner dependent on the luminance component

Fig. 36C shows a logic diagram of an example of a video processing method. Method 3630 includes, at step 3632, performing a transformation between a first video block of video and an encoded representation of the video, where the transformation procedure uses a first encoding mode and a palette encoding mode in which at least a palette of representative pixel values is used to encode the current video block. This method 3630 further comprises, at step 3634, performing a conversion between a second video block of video encoded without using a palette encoding mode, and an encoded video representation, where the second video block conversion procedure uses the first encoding mode. When a first encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component. In some implementations, the first encoding mode is applied in different ways to the first video block and the second video block.

Fig. 37A shows a logic diagram of an example of a video processing method. Method 3710 comprises, at step 3712, determining that an encoding mode is activated to perform a conversion between the current video block and the encoded video representation. This method 3710 further comprises, at step 3714, performing a transform using an intra-frame block copy mode generating a predicted block using at least a block vector pointing to an image containing the current video block. In said encoding mode, the current video block is constructed based on the samples in the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

Fig. 37B shows a logic diagram of an example of a video processing method. Method 3720 comprises, at step 3722, determining, to effect a conversion between the current video block and an encoded video representation, that the current video block is encoded in an intra-block copy (IBC) mode, in which a predicted block is generated using at least a block vector pointing to a video frame containing the current video block to encode that current video block. This method 3720 further comprises, at step 3724, carrying out, based on the results of the specified determination, a transformation by deactivating the encoding mode. When this encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

Fig. 37C shows a logic diagram of an example of a video processing method. Method 3730 includes, at step 3732, performing a transformation between a first video block of video and an encoded representation of the video, where the transformation procedure uses an intra-frame block copy mode that generates a predicted block using at least a block vector pointing to the video frame containing the current video block and the first coding mode. This method 3730 further comprises, at step 3734, performing a conversion between a second video block of video encoded without using an intra-frame block copy mode, and an encoded video representation where the second video block conversion procedure uses the first encoding mode. When a first encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component, and the first encoding mode is applied in different ways to the first video block and the second video block.

Fig. 38A shows a logic diagram of an example of a video processing method. Method 3810 comprises, at step 3812, determining that an encoding mode is activated to perform a conversion between the current video block and the encoded video representation. This method 3810 further comprises, at 3814, performing conversion using a block-based delta pulse code modulation (BDPCM) mode. In this encoding mode, the current video block is constructed based on the samples in the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component

Fig. 38B shows a logic diagram of an example of a video processing method. Method 3820 comprises, at step 3822, determining, to perform a conversion between the current video block and an encoded video representation, that the current video block is encoded using a block-based delta pulse code modulation (BDPCM) mode. This method 3820 further comprises, at step 3824, carrying out, based on the results of the specified determination, a transformation by deactivation. When a coding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

Fig. 38C shows a logic diagram of an example of a video processing method. Method 3830 includes, at step 3832, performing a conversion between a first video block of video and an encoded representation of the video, where the first video block conversion procedure uses a first encoding mode and a block-based delta pulse code modulation (BDPCM) mode. This method 3830 further comprises, at step 3834, performing a conversion between the second video block of video and an encoded representation of the video, where the second video block is encoded without using a BDPCM modulation mode, and the second video block conversion procedure uses the first encoding mode. When a first encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component, and the first encoding mode is applied in different ways to the first video block and the second video block.

Fig. 38D shows a logic diagram of an example of a video processing method. Method 3840 comprises, at step 3842, determining that an encoding mode is activated to perform a conversion between the current video block and the encoded video representation. This method further comprises, at step 3844, performing the transformation using a skip transformation mode, in which the transformation of the prediction residual is skipped when encoding the current video block. In this encoding mode, the current video block is constructed based on the samples in the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

Fig. 38E shows a logic diagram of an example of a video processing method. Method 3850 comprises, at step 3852, determining, to perform a transformation between the current video block and an encoded video representation, that the current video block is encoded in a skip transformation mode, in which the transformation of the prediction residual is skipped when encoding the current video block. This method 3850 further comprises, at step 3854, carrying out, based on the results of this determination, a conversion by deactivating the encoding mode. When a coding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

Fig. 38F shows a logic diagram of an example of a video processing method. Method 3860 comprises, at step 3862, performing a transformation between a first video block of video and an encoded representation of the video, wherein the transformation procedure of the first video block uses a first encoding mode and a skip transformation mode in which the transformation of the prediction residual is skipped when encoding the current video block. This method 3860 further comprises, at step 3864, performing a transformation between the second video block of video and an encoded representation of the video, where the second video block is encoded without using a skip transformation mode, and the second video block transformation procedure uses the first encoding mode. When a first encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component, and the first encoding mode is applied in different ways to the first video block and the second video block

Fig. 38G shows a logic diagram of an example of a video processing method. Method 3870 includes, at step 3872, determining that an encoding mode is activated to perform a conversion between the current video block and the encoded video representation. This method 3870 further comprises, at step 3874, performing the transformation using an intra-frame prediction mode with pulse code modulation, in which the current video block without applying transformation and quantization in the transformed region. In this encoding mode, the current video block is constructed based on the samples in the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

Fig. 38H shows a logic diagram of an example of a video processing method. Method 3880 comprises, at step 3882, determining, to effect a transformation between the current video block and an encoded video representation, that the current video block is encoded in an intra-frame PCM prediction mode, in which the current video block is encoded without applying transformation and quantization to the transformed domain. . This method 3880 further comprises, at step 3884, carrying out, based on the results of the specified determination, a transformation by deactivating the encoding mode. When a coding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

Fig. 38I shows a logic diagram of an example of a video processing method. Method 3890 comprises, at step 3892, performing a transformation between a first video block of video and an encoded representation of the video, wherein the transformation procedure of the first video block uses a first encoding mode and a PCM intra-frame prediction mode in which the current video block is encoded without applying transformation and quantization to the transformed areas. This method 3890 further comprises, at step 3894, performing a transformation between the second video block of video and an encoded representation of the video, where the second video block is encoded without using an intra-frame PCM prediction mode, and the second video block transformation procedure uses the first encoding mode. When a first encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component, and the first encoding mode is applied in different ways to the first video block and the second video block.

Fig. 38J shows a logic diagram of an example of a video processing method. Method 3910 includes, at step 3912, determining that an encoding mode is activated to perform a conversion between the current video block and the encoded video representation. This method 3910 further comprises, at step 3914, performing the transformation using a modified transform and quantization bypass mode, in which the current video block is encoded without loss and without transformation and quantization. In this encoding mode, the current video block is constructed based on the samples in the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

Fig. 38K shows the logic diagram of an example video processing method. Method 3920 comprises, at step 3922, determining, to effect a transformation between the current video block and an encoded video representation, that the current video block is encoded in a bypass transformation and quantization mode, in which the current video block is encoded losslessly and without transformation and quantization. This method 3920 further comprises, at step 3924, carrying out, based on the determination, a conversion by deactivating the encoding mode. When a coding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

Fig. 38L shows a logic diagram of an example of a video processing method. Method 3930 comprises, at step 3932, performing a transformation between a first video block of video and an encoded representation of the video, wherein the transformation procedure of the first video block uses a first encoding mode and a transformation and quantization bypass mode in which the current video block is encoded losslessly and without transformation or quantization. This method 3930 further comprises, at step 3934, performing a transformation between the second video block of video and an encoded representation of the video, where the second video block is encoded without using a transform and quantization bypass mode, and the second video block's transformation procedure uses the first encoding mode. When a first encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component, and the first encoding mode is applied in different ways to the first video block and the second video block.

Fig. 39A shows a logic diagram of an example of a video processing method. Method 3940 comprises, at step 3942, performing a transformation between the current video video block and an encoded video representation, where the transformation procedure uses an encoding mode in which the current video block is constructed based on the first region and the second region and/or scales the remaining chroma component in a luminance dependent manner. component, and where the information used for the encoding mode is signaled in a parameter set that is different from the sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS) or adaptation parameter set (APS) used to transmit adaptive loop filtering (ALF) parameters.

Fig. 39B shows a logic diagram of an example of a video processing method. Method 3950 comprises, at step 3952, performing a transformation between the current video video block and an encoded video representation, where the transformation procedure uses an encoding mode in which the current video block is constructed based on the first region and the second region and/or scales the remainder of the chrominance component in a luminance dependent manner. component, and where the information used for the encoding mode is signaled as an adaptation parameter set (APS) together with adaptive loop filtering (ALF) information, where the information used for the encoding mode and the ALF filtering information are included in one NAL level unit.

Fig. 39C shows a logic diagram of an example of a video processing method. Method 3960 includes, at step 3962, performing a transformation between the current video block of a video region and an encoded representation of the video, where the transformation procedure uses an encoding mode in which the current video block is constructed based on the first region and the second region and/or scales the remainder of the chrominance component in a manner dependent on luminance component, and where the information used for the encoding mode is signaled in a first type of adaptation parameter set (APS) different from the second type of APS used for signaling the adaptive loop filtering (ALF) information in the signaling form.

Fig. 39D shows a logic diagram of an example of a video processing method. Method 3970 comprises, at step 3972, performing a transformation between the current video block of a video region and an encoded representation of the video, where the transformation procedure uses an encoding mode in which the current video block is constructed based on the first region and the second region and/or scales the remainder of the chrominance component in a manner dependent on luminance component, and where the video domain is not allowed to access an adaptation parameter set or a parameter set signaled prior to a data structure of a specified type used for video processing, and where the data structure of a specified type is signaled prior to the video domain.

Fig. 39E shows a logic diagram of an example of a video processing method. Method 3980 comprises, at step 3982, performing a transformation between the current video block and an encoded video representation, where the transformation procedure uses an encoding mode in which the current video block is constructed based on the first region and the second region and/or scales the remaining chroma component in a luminance dependent manner. component, and where a syntax element for a parameter set containing parameters used for video processing has a specified value in the matching data bit stream.

In fig. 40A is a block diagram of video processing hardware 4000. This apparatus 4000 may be used to implement one or more of the methods described herein. The hardware 4000 may be a smartphone, a tablet computer, a desktop computer, an Internet of Things (IoT) receiver, etc. The hardware 4000 may include one or more processors 4002, one or more storage devices 4004, and video processing hardware 4006. The processor(s) may be configured to implement one or more of the methods (including, but not limited to, the methods shown in FIGS. 31A - 39E) described herein. Storage device(s) 4004 may be used to store data and code used to implement the methods and technology described herein. Video processing hardware 4006 may be used to implement, in hardware, some of the methods and technologies described herein.

In fig. 40B shows another example block diagram of a video processing system in which the proposed technology may be implemented. In fig. 40B is a block diagram showing an example of a video processing system 4100 where various methods described herein may be implemented. Various implementations may include some or all of the components of the system 4100. The system 4100 may have an input 4102 for receiving video content. This video content may be received in a native or uncompressed format, such as 8 or 10-bit multi-pixel values, or may be in a compressed or encoded format. Input 4102 may represent a network interface, a peripheral bus interface, or a storage device interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

System 4100 may include an encoding component 4104 that can implement a variety of encoding techniques described herein. The encoding component 4104 may reduce the average bit rate of video data supplied from the input 4102 to the output of the encoding component 4104 to obtain an encoded representation of the video. Therefore, such encoding methods are sometimes called video compression methods or transvideo encoding methods. The output of encoding component 4104 may be either stored or transmitted over an attached communication link, as represented by component 4106. The representation of video received as input 4102, or stored in a storage device, in the form of a data bit stream (or encoded form) may be used by component 4106. This stored or bitstreamed (or encoded) representation of the video received at input 4102 may be used by component 4108 to generate pixel values or display video that is sent to display interface 4110. The process of generating user-viewable video from a data bitstream representation is sometimes called video expansion (or decompression). In addition, although certain video processing operations are referred to as "encoding" operations or tools, it should be understood that the encoding tools or operations are used in the encoding device, and the corresponding decoding tools or operations that reverse the encoding results will be performed in the decoding device.

Examples of peripheral bus interface or display interface include universal serial bus (USB) or high definition multimedia interface (HDMI) or Displayport, etc. Examples of storage interfaces include the serial advanced technology attachment (SATA) interface, the peripheral device interface (PCI), the IDE interface, and other similar interfaces. The methods described herein may be implemented in a variety of electronic devices, such as mobile phones, laptop computers, smartphones, or other devices capable of digital processing and/or displaying video.

In some embodiments, video encoding methods may be implemented using devices implemented on a hardware platform, as described with reference to FIGS. 40A or 40B.

A variety of methods and embodiments may be described using the following clause-by-clause format.

The first group of articles describes certain features and aspects of the methods discussed here, listed in the previous section, including, for example, Examples 1 and 2.

1. A video processing method, comprising: implementing, to perform a transformation between a current video block and an encoded video representation, a procedure for refining motion information based on samples in a first region or a second region; and performing a transformation based on the result of the motion information refinement procedure, where, in the transformation process, samples for the current video block are obtained from a first prediction block in a first region using the unspecified motion information, at least a second prediction block is generated in a second region using the refined motion information used to determine a reconstructed block, and generating reconstructed samples of the current video block based on at least the second predicted block.

2. The method according to article 1, characterized in that at least a second predicted block is generated from samples in reference images in the first region using updated motion information, and then a reshaping procedure is applied to transform the first region into a second region to at least the second predicted block.

3. The method according to article 2, characterized in that after the reformation procedure, the second predicted block is converted into a representation in the second region before being used to generate reconstructed samples of the current video block.

4. The method according to Article 1, characterized in that the implementation of the procedure for clarifying motion information is based on a method for obtaining the motion vector on the side of the decoding device (DMVD).

5. The method according to Article 4, characterized in that the method for obtaining DMVD comprises decoder-side motion vector refinement (DMVR) or frame rate upconversion (FRUC) or the use of bidirectional optical flow (BIO).

6. The method according to Article 4, characterized in that the cost calculation or gradient calculation during the DMVD procedure is carried out on the basis of samples in the first area.

7. The method according to Article 6, characterized in that the procedure for calculating the cost contains the determination of the sum of absolute differences (SAD) or the sum of absolute differences with the exception of the mean (MR-SAD).

8. The method according to Article 1, characterized in that the procedure for updating traffic information is carried out on the basis of samples converted from samples from at least the first predicted block in the first area to the second area, and characterized in that, after receiving updated information about traffic , deactivating the encoding mode in which the current video block is constructed based on the first region and the second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component for at least the second predicted block.

9. The method according to Article 4, characterized in that the procedure for clarifying traffic information is carried out on the basis of at least the first predicted block in the first area, and characterized in that the procedure for clarifying traffic information is involved with the first predicted block of the first area.

10. The method according to article 1, characterized in that the final prediction block is generated as a weighted average of the two second prediction blocks, and reconstructed samples of the current video block are generated based on this final prediction block.

11. The method according to Article 1, characterized in that the procedure for clarifying motion information is carried out on the basis of a predicted block in the first area, and characterized in that, after carrying out the procedure for clarifying motion information, activation of the encoding mode in which the current video block is constructed on based on the first region and the second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component for at least the second predicted blocks.

12. A video processing method, comprising: performing a transformation between a current video video block and an encoded video representation, where, in the transformation process, the current video block is constructed based on a first region and a second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component, where the encoding tool is applied to the conversion process using parameters derived from at least a plurality of first samples in a video video region and a plurality of second samples in a reference picture for the current video block, and wherein the region for the first samples and the region for the second samples are aligned with one another.

13. The method of Clause 12, wherein said encoding tool comprises a local illuminance compensation (LIC) model that uses a linear model of illuminance changes in the current video block during the conversion process, and applies a LIC compensation model based on these parameters.

14. The method according to article 12, characterized in that the video area contains the current tile, group of tiles or image.

15. The method of Clause 13, characterized in that the LIC compensation model is applied to the predicted block in the second region, and characterized in that the plurality of first samples and the plurality of second samples are in the second region.

16. The method of Clause 13, wherein the reference block is converted to a second region and the LIC compensation model is applied to the predicted block in the second region.

17. The method according to Article 15, characterized in that said plurality of first samples and plurality of second samples are converted into a second domain before being used to obtain parameters.

18. The method according to Article 17, characterized in that the specified set of second samples contains reference samples in the reference images and adjacent and/or non-adjacent samples with these reference samples.

19. The method of Clause 13, characterized in that the LIC compensation model is applied to the predicted block in the first region, and characterized in that the plurality of first samples and the plurality of second samples are in the first region.

20. The method of Clause 13, characterized in that the reference block is supported in the first region and the LIC compensation model is applied to the predicted block in the first region.

21. The method according to Article 19, characterized in that the plurality of first samples are converted into the first region before they are used to obtain parameters.

22. The method according to Article 21, characterized in that the set of first samples contains spatially adjacent and/or non-adjacent samples relative to the current video block.

23. The method according to Article 12, characterized in that the area used to obtain the parameters is also used to apply the parameters to the predicted block.

24. The method according to Article 13, characterized in that the LIC compensation model is applied to the forecast block in the second region.

25. The method of Clause 20 or Clause 21, wherein after applying the LIC compensation model to a prediction block in the first region, the final prediction block dependent on said prediction block is converted to the second region.

26. The method as set forth in any one of Clauses 1 to 25, wherein the first region is the original region and the second region is a reshaped region using a luminance component scaling (LMCS) display method that displays luminance samples into specific values.

27. The method according to article 26, characterized in that the LMCS method uses a piecewise linear model to map the luminance component samples into specific values.

28. The method according to any of articles 1 - 27, characterized in that the implementation of the transformation includes generating an encoded representation from the current block.

29. The method according to any of articles 1 - 27, characterized in that the conversion procedure includes generating the current block from the encoded representation.

30. A device in a video system containing a processor and a non-volatile storage device with commands written therein, characterized in that when executing these commands the processor implements a method in accordance with any one of Articles 1 to 29.

31. A computer program product stored on a non-volatile computer-readable storage medium, this computer program product contains program code for implementing a method according to any one of Articles 1 to 29.

The second group of articles describes certain features and aspects of the proposed methods listed in the previous section, including, for example, Examples 3 - 5, 8 and 15.

1. A method for processing video, comprising: determining, for the current video block of the current video video region, a parameter for the encoding mode of this current video block based on one or more parameters for the encoding mode of the previous video region; and performing encoding on the current video block to generate an encoded video representation based on the determination result, wherein said parameter for an encoding mode is included in a set of encoded video representation parameters, and wherein performing the encoding procedure comprises transforming a representation of the current video block in the first region into a representation of the current video block in a second region, and where, in the process of performing encoding using said encoding mode, the current video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

2. A method for processing video, comprising: receiving an encoded representation of the video containing a set of parameters, which includes information about the parameters for the encoding mode; and performing decoding of the received encoded representation using said parameter information to generate a current video block of the current video region of the video from the encoded representation, and where parameter information for an encoding mode based on one or more parameters for an encoding mode of the previous video region, where in said encoding mode, the current the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

3. The method according to article 1 or 2, characterized in that the specified set of parameters differs from the header of the tile group.

4. The method according to article 1 or 2, characterized in that the specified set of parameters is an adaptation parameter set (APS).

5. The method according to article 1 or 2, characterized in that the current video area contains one tile of a video image of a video or a video of a video image;

6. The method according to article 1 or 2, characterized in that the previous video area contains one or more image tiles.

7. The method according to article 1 or 2, characterized in that the previous video area contains one or more video images of the video.

8. A video processing method, comprising: performing a transformation between a current video block of video and an encoded representation of the video, and characterized in that the transformation procedure comprises applying a filtering operation to a predicted block in a first region or in a second region different from the first region.

9. The method of Clause 8, characterized in that a filtering operation is performed on a predicted block in the first region to generate a filtered predicted signal, an encoding mode is applied to this filtered predicted signal to generate a reformatted predicted signal located in the second region, and a current video block using a reformatted predicted signal.

10. The method of Clause 8, wherein the encoding mode is applied to the predicted block before applying a filtering operation to generate a reshaped predicted signal located in the second region, and said filtering operation is performed using the reshaped predicted signal to generate a filtered predicted signal, and the current video block is constructed using the reformatted predicted signal.

11. The method according to Article 9 or 10, characterized in that, in encoding mode, the current video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

12. The method according to any of articles 8 - 11, characterized in that the filtering operation includes the use of a scattering filter.

13. The method according to any one of clauses 8 to 11, characterized in that the parameters associated with the filtering operation depend on whether the filtering operation is applied to a block in the first region or in the second region.

14. The method according to Article 8, characterized in that the conversion procedure further comprises: applying, before performing the filtering operation, motion-compensated prediction to the current video block to obtain a predicted signal; applying, after performing the filtering operation, an encoding mode to the filtered prediction signal to generate a reformatted prediction signal, the filtered prediction signal being generated by applying the filtering operation to the prediction signal; and constructing a current video block using the reshaped predicted signal.

15. The method according to Article 8, characterized in that the conversion procedure further comprises: applying, before performing the filtering operation, motion-compensated prediction for the current video block in order to obtain a predicted signal; applying an encoding mode to the predicted signal to generate a reshaped predicted signal; and constructing, after applying the filtering operation, a current video block using the filtered deformed predicted signal, and characterized in that the filtered deformed predicted signal is generated by applying a filtering operation to the deformed predicted signal.

16. A video processing method, comprising: performing a transformation between a current video video block and an encoded video representation, where, in the transformation process, a final reconstructed block for the current video block is determined, and where a temporary reconstructed block is generated using a prediction method and presented in a second region.

17. The method according to Article 16, characterized in that the transformation procedure further comprises: applying motion-compensated prediction to the current video block in order to obtain a predicted signal; applying direct reshaping to the predicted signal to generate a reshaped predicted signal used to generate a temporary reconstructed block; and applying an inverse reformation to the temporary reconstructed block to obtain a reverse reconstructed block, and characterized in that filtering is applied to the reverse reconstructed block to generate a final reconstructed block.

18. The method according to Article 16, characterized in that the transformation procedure further comprises: applying prediction to motion compensation to the current video block to obtain a predicted signal; applying direct reshaping to the predicted signal to generate a reshaped predicted signal used to generate a temporary reconstructed block; applying an inverse reformation to the filtered reconstructed block to obtain a final reconstructed block, and characterized in that the filtered reconstructed block is generated by applying filtering to the temporary reconstructed block.

19. The method according to any of articles 16 - 18, characterized in that the conversion procedure further comprises the use of a luminance component mapping procedure with residual chrominance component scaling (LMCS), which maps the luminance component samples to specific values.

20. The method of Clause 16, characterized in that a filter is applied to the temporary reconstructed block in the first region, first the temporary reconstructed block in the second region is converted to the first region using an inverse reformation procedure before applying the filter, and the final reconstructed block depends on the filtered temporary reconstructed block.

21. The method of Clause 16, wherein the filter is applied directly to the temporary reconstructed block in the second region, and thereafter, a reverse reformation operation is applied to generate the final reconstructed block.

22. The method according to article 16, characterized in that the filter contains a bidirectional filter (BF) or a Hadamard transform filter (HF).

23. The method of Clause 16, wherein the filter uses a deblocking filtering (DBF) operation, a nonlinear adaptive offset filtering (SAO) operation, or an adaptive loop filtering (ALF) operation.

24. The method according to any one of articles 1 to 23, characterized in that the parameters of the filter used for the filtering operation depend on whether or not the encoding mode is activated for the current video block, where, in the encoding mode, the current video block is designed to based on the first region and the second region and/or scale the remainder of the chrominance component in a manner dependent on the luminance component.

25. The method as set forth in any one of Clauses 1 to 25, wherein the first region is the original region and the second region is the reshaped region using a luminance component scaling (LMCS) mapping method that maps luminance component samples to specific values .

26. The method according to article 25, characterized in that the LMCS method uses a piecewise linear model to map the luminance component samples to specific values.

27. The method according to any of articles 8 - 26, characterized in that the implementation of the transformation includes generating an encoded representation from the current block.

28. The method according to any of articles 8 - 26, characterized in that the implementation of the transformation includes generating the current block from the encoded representation.

29. A device in a video system containing a processor and a non-volatile storage device with commands written therein, where, when executing these commands, the processor implements a method in accordance with any one of Articles 1 to 28.

30. A computer program product stored on a non-volatile computer-readable storage medium, this computer program product contains program code for implementing a method according to any one of Articles 1 to 28.

The third group of articles describes certain features and aspects of the proposed methods listed in the previous section, including, for example, Example 6.

1. A video processing method, comprising: performing a transformation between a current video block of a video region and an encoded representation of the video, where the transformation procedure uses an encoding mode in which the current video block is constructed based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on luminance component, and where the parameter set in the encoded representation contains parameter information for the encoding mode.

2. The method according to article 1, characterized in that the specified set of parameters differs from the header of the tile group.

3. The method according to Article 2, characterized in that the specified set of parameters is an adaptation parameter set (APS).

4. The method according to Clause 3, characterized in that the APS set for encoding mode information is called a luminance component scaling (LMCS) APS parameter set.

5. The method of Clause 3, wherein the identifier for the APS set to be used for the current video block is included in the encoded video representation.

6. The method of Article 5, characterized in that whether the specified identifier is present in the encoded video representation depends on whether the encoding mode for the video domain is activated.

7. The method according to Article 3, characterized in that the specified set of parameters contains the APS set identifier.

8. The method according to article 1, characterized in that a NAL unit type value is assigned to the set of parameters.

9. The method according to Article 1, characterized in that the value of the parameter set identifier is in the range between 0 and M, where M is equal to 2 ^K -1.

10. The method according to article 1, characterized in that the specified set of parameters is shared by images from the video.

11. The method according to Article 1, characterized in that the parameter set identifier has a value encoded in a fixed-length code.

12. The method of Clause 1, wherein the parameter set identifier is encoded using exponential-Golomb (EG), truncated unary, or binarization code.

13. The method according to Article 1, characterized in that, for two sub-regions in the same image, the specified set of parameters contains identifiers having two different values.

14. The method of Clause 3, wherein said parameter set and the APS set for adaptive loop filtering (ALF) information share the same network abstraction layer (NAL) unit type (NUT).

15. The method according to clause 1, characterized in that the parameter information is transmitted with the current set of APS for adaptive loop filter (ALF) information.

16. The method according to article 1, characterized in that the parameter information is transmitted in a sequence parameter set (SPS), in a video parameter set (VPS), in a picture parameter set (PPS), in a sequence, in a header or in a header picture.

17. The method according to Article 1, characterized in that the information about the parameters contains at least one of information about the model of the reshaping device, information about the use of the encoding mode, or scale factors for the remainder of the color component.

18. The method according to Article 1, characterized in that information about the parameters is transmitted in the form of signaling at one level.

19. The method according to article 1, characterized in that the parameter information contains information about the use of the encoding mode, which is reported in the form of signaling at the second level.

20. The method according to Article 18 and 19, characterized in that the parameter information is transmitted as signaling in the APS set and information about the use of the encoding mode is transmitted as signaling at the video domain level.

21. The method according to article 1, characterized in that they perform parsing of information about parameters at one level.

22. The method according to Article 1, characterized in that the information about the parameters contains information about the use of the encoding mode, subjected to parsing at the second level.

23. The method according to Article 21 or 22, characterized in that the parameter information is parsed in the APS set and the encoding mode usage information is parsed at the video domain level.

24. The method according to Article 1, characterized in that predictive coding is used to encode information about parameters with different APS set indices.

25. The method as set forth in any one of Clauses 1 to 24, wherein the first region is the original region and the second region is the reshaped region using a luminance component scaling (LMCS) display method that displays luminance component samples on specific meanings.

26. The method according to Article 25, characterized in that the LMCS method uses a piecewise linear model to display the samples.

27. The method according to any of articles 1 to 26, characterized in that the video area is an image or a group of tiles.

28. The method according to any of articles 1 to 26, characterized in that the video area level is an image header or a tile group header.

29. The method as set forth in any one of Clauses 1 to 28, wherein the first region is the original region and the second region is the reshaped region using a luminance component scaling (LMCS) mapping method that maps luminance component samples to specific values in accordance with reformation models.

30. The method according to article 29, characterized in that the LMCS method uses a piecewise linear model to map the luminance component samples to specific values.

31. The method according to any of articles 1 - 30, characterized in that the conversion procedure contains the generation of an encoded representation from the current block.

32. The method according to any of articles 1 to 30, characterized in that the conversion procedure includes generating the current block from the encoded representation.

33. A device in a video system containing a processor and a non-volatile storage device with commands written therein, where, when executing these commands, the processor implements a method in accordance with any one of Articles 1 to 32.

34. A computer program product stored on a non-volatile computer-readable storage medium, this computer program product contains program code for implementing a method according to any one of Articles 1 to 32.

The fourth group of articles describes certain features and aspects of the proposed methods listed in the previous section, including, for example, Examples 7 and 19.

1. A video processing method, comprising: converting between a current video video block, which is a chrominance component block, and an encoded video representation, wherein, in the conversion process, the current video block is constructed based on a first region and a second region, and wherein the transformation procedure further comprises applying the procedure direct reformation and/or reverse reformation procedure to one or more color components of the current video block.

2. The method of Clause 1, wherein the method further proposes to refrain from applying luminance-dependent chroma residual scaling (LCRS) to one or more chroma components of the current video block.

3. The method according to article 1, characterized in that for the chrominance component at least one of the following is used - a piecewise linear model (PWL), a lookup table for a forward transformation, or a lookup table for an inverse transformation.

4. The method according to article 3, characterized in that the specified PWL model, conversion table for direct conversion and conversion table for inverse conversion for the chrominance component are derived, respectively, from the PWL model, conversion table for direct conversion and conversion table for inverse conversion for the corresponding brightness component.

5. The method of Clause 3, wherein the PWL model is signaled in a Sequence Parameter Set (SPS), Video Parameter Set (VPS), Adaptation Parameter Set (APS), Picture Parameter Set (PPS), Sequence Header, Header image, tile group header, tile header, in a row of coding tree units (CTUs), a group of CTUs, or in areas.

6. The method of Clause 3, wherein the forward transform lookup table and the reverse lookup table are signaled in a Sequence Parameter Set (SPS), Video Parameter Set (VPS), Adaptation Parameter Set (APS), Parameter Set picture (PPS), sequence header, picture header, tile group header, tile header, row of coding tree units (CTUs), group of CTUs or in regions.

7. A video processing method, comprising: performing a transformation between a current chroma video block and an encoded video representation, wherein performing the transformation comprises: determining whether luminance-dependent chroma residual scaling (LCRS) is enabled or not, based on a rule, and reconstructing the current chrominance video block based on the result of this determination.

8. The method of Clause 7, wherein said rule specifies that LCRS mode is not enabled for certain block sizes, time levels, tile group types, image types, encoding modes, certain types of motion information.

9. The method according to Article 7, characterized in that the said rule specifies that the LCRS mode is not activated for blocks of the chroma component and the forward and/or reverse reformation procedure is applied to the corresponding blocks of the luminance component.

10. The method according to Article 7, characterized in that the said rule specifies that the LCRS mode is applied to blocks of the chrominance component and the forward and/or reverse reformation procedure is not applied to the corresponding blocks of the luminance component.

11. The method of Clause 7, wherein said rule specifies that LCRS mode is not enabled for the current chrominance video block encoded using a cross component linear model (CCLM).

12. The method of Clause 7, wherein said rule specifies that the LCRS mode is not enabled for the current chrominance video block not encoded using a cross-component linear prediction model (CCLM).

13. The method of Clause 7, wherein said rule specifies that deactivation of the LCRS mode is based on the video block size being larger than a virtual pipeline data unit (VPDU).

14. The method according to Article 13, characterized in that the LCRS mode is not allowed when the number of video samples in a video block is less than M*H.

15. The method according to Article 13, characterized in that the LCRS mode is not allowed in the case where the minimum size - width and/or height - of the video block is less than or equal to a certain value.

16. The method according to Article 13, characterized in that the LCRS mode is not allowed in the case where the minimum size - width and/or height - of the video block is not less than a certain size.

17. The method according to Article 15 or 16, characterized in that the specified specific value is equal to 8.

18. The method according to Article 13, characterized in that the LCRS mode is not allowed in the case when the width of the video block is equal to or greater than the first value, and/or when the height of the video block is equal to or greater than the second value.

19. The method according to Article 13, characterized in that the LCRS mode is not allowed in the case when the width of the video block is equal to or less than the first value, and/or when the height of the video block is equal to or less than the second value.

20. The method according to Article 18 or 19, characterized in that at least one of the values - the first value and/or the second value - is equal to 8, 64 or 128.

21. The method according to any of articles 13 - 20, characterized in that the specified video block is a block of a luminance component or a block of a chrominance component.

22. The method as set forth in any one of Clauses 1 to 21, wherein the first region is the original region and the second region is the reshaped region using a luminance component scaling (LMCS) mapping method that maps luminance samples to specific values in accordance with reformation models.

23. The method according to Article 22, characterized in that the LMCS method uses a piecewise linear model to map the luminance component samples into specific values.

24. The method according to any of articles 1 to 23, characterized in that the remainder of the chrominance component is scaled in a luminance-dependent manner by performing a luminance-dependent scaling operation of the remainder of the chrominance component, which involves scaling the remainder of the chrominance component before they are used to form the reconstruction video block of the chrominance component, and the scaling parameters are obtained from the luminance component samples.

25. The method according to any of articles 1 - 24, characterized in that the implementation of the transformation involves generating an encoded representation from the current block.

26. The method according to any of articles 1 - 24, characterized in that the implementation of the transformation includes generating the current block from the encoded representation e.

27. A device in a video system containing a processor and a non-volatile storage device with commands written therein, where, when executing these commands, the processor implements a method in accordance with any one of Articles 1 to 26.

28. A computer program product stored on a non-volatile computer-readable storage medium, this computer program product contains program code for implementing a method according to any one of Articles 1 to 26.

The fifth group of articles describes certain features and aspects of the proposed methods listed in the previous section, including, for example, Examples 10 - 14, 28, 29 and 40.

1. A video processing method, comprising: determining, in order to perform a conversion between a current video block and an encoded video representation, whether to deactivate the use of an encoding mode based on one or more coefficient values of the current video block; and performing conversion based on the result of this determination, wherein, in the conversion process using the encoding mode, the current video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

2. The method according to Article 1, characterized in that the reformation procedure contains: selective application of the direct reformation procedure to readings in the first area, which are then converted into readings in the second area; and selectively applying the inverse reshaping procedure to the samples in the second region, which are then converted to a representation in the first region.

3. The method according to article 1 or 2, characterized in that the reformation procedure further contains: selective application of the procedure for scaling the remainder of the chrominance component depending on the brightness component.

4. The method according to any of articles 1 - 3, characterized in that the specified determination is made based on whether the current video block is encoded with all zero coefficients.

5. The method according to Article 2, characterized in that the direct reformation procedure is skipped based on whether the current video block is encoded with all zero coefficients.

6. The method according to Article 2, characterized in that the current video block is encoded with all zero coefficients, and characterized in that the reverse reformation procedure is skipped.

7. The method according to Article 2, characterized in that the current video block is encoded with all zero coefficients, and characterized in that the procedure for scaling the remainder of the chrominance component depending on the luminance component is skipped.

8. The method according to Article 2, characterized in that said determination is based on whether the current video block is encoded with only one non-zero coefficient located at a certain position.

9. The method according to Article 2, characterized in that the current video block is encoded with only one non-zero coefficient located in a certain position, and at least one of the following procedures is skipped - a direct reformation procedure, an inverse reformation procedure and/or a luminance component-dependent procedure scaling the remainder of the chrominance component.

10. The method according to Article 2, characterized in that said determination is based on whether the current video block is encoded with M non-zero coefficients.

11. The method according to Article 2, characterized in that the current video block is encoded with M non-zero coefficients, and at least one of the following procedures is skipped - a direct reformation procedure, a reverse reformation procedure and/or a procedure for scaling the remainder of the chrominance component depending on the luminance component.

12. Method according to Article 11, characterized in that M is equal to 1.

13. A video processing method, comprising: partitioning, for converting between a current video block of a video larger than a video virtual pipeline data unit (VPDU), that current video block into areas; and performing conversion by applying the encoding mode separately to each area, characterized in that, in the conversion process by applying the encoding mode, the current video block is constructed based on the first area and the second area and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

14. The method according to Article 13, characterized in that each area corresponds to an individual coding unit (CU) for the coding mode.

15. The method of Article 13, characterized in that the width of the current video block is equal to or greater than the first value, the current video block is divided into sub-blocks such that the width of one or more of these sub-blocks is equal to or less than the first value, and the encoding mode is activated for each sub-block.

16. The method according to Article 13, characterized in that the height of the current video block is equal to or greater than the second value, the current video block is divided into sub-blocks having one or more heights that are equal to or less than the specified second value, and the encoding mode is activated for each sub-block. block.

17. The method according to Article 13, characterized in that the size of the current video block is equal to or greater than the third value, the current video block is divided into sub-blocks having one or more sizes equal to or less than the specified third value, and the encoding mode is activated for each sub-block.

18. The method according to any one of Articles 15 - 17, characterized in that the specified sub-blocks have the same width or the same height.

19. A video processing method, comprising: determining, to perform a conversion between a current video block and an encoded video representation, whether to deactivate the use of an encoding mode based on the size or color format of the current video block; and performing conversion based on the result of said determination, wherein, in the conversion process using the encoding mode, the current video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

20. The method according to article 19, characterized in that in the said determination step it is decided that the encoding mode for the current video block exceeding the virtual pipeline data unit (VPDU) should be deactivated.

21. The method according to Article 19, characterized in that at the said determination stage it is decided that the encoding mode should be deactivated for the current video block having a size containing samples the number of which is less than M*H.

22. The method according to Article 19, characterized in that at the stage of the specified determination it is decided that the activation of the encoding mode for the current video block should be canceled in the case where the minimum size of the width and/or height of the current video block is equal to or less than the value X, which is an integer.

23. The method according to Article 19, characterized in that at the stage of the specified determination it is decided that the activation of the encoding mode for the current video block should be canceled in the case where the minimum size of the width and/or height of the current video block is not less than the value X, which is an integer.

24. Method according to Article 22 or 23, characterized in that X is equal to 8.

25. The method according to Article 19, characterized in that at the stage of the specified determination it is decided that the activation of the encoding mode for the current video block should be canceled in the case where the current video block has a height and/or width, where this width is equal to or greater than the first value and this height equal to or greater than the second value.

26. The method according to Article 19, characterized in that at the stage of the specified determination it is decided that the activation of the encoding mode for the current video block should be canceled in the case where the current video block has a height and/or width, where this width is equal to or less than the first value and this height equal to or less than the second value.

27. The method according to Article 25 or 26, characterized in that at least one of the values - the first value and/or the second value is equal to 8

28. The method according to any of articles 19 - 27, characterized in that the procedure for canceling the activation of the encoding mode includes canceling the activation of at least one of: 1) direct reformation to convert samples in the first area to the second area; 2) reverse reformation to transform samples in the second region into the first region; 3) scaling of the chrominance component depending on the brightness component.

29. A video processing method, comprising: performing a transformation between a current video video block and an encoded video representation, where the transformation procedure uses an encoding mode in which the current video block is constructed based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component, and wherein the at least one syntactic element in the encoded representation creates an indication of the use of an encoding mode and an indication of the model of the reshaping device.

30. The method according to Article 29, characterized in that the indication of the use of the encoding mode is encoded based on the indication of the model of the reshaping device.

31. The method according to Article 29, characterized in that the indication of the model of the reshaping device is encoded based on the indication of the encoding mode.

32. The method according to Article 29, characterized in that only one of the syntactic elements is encoded.

33. The method according to any of articles 1 to 32, characterized in that different truncation methods are applied to the predicted signal and to the reconstructed signal.

34. The method according to Article 33, characterized in that an adaptive truncation procedure is applied to the predicted signal, allowing the use of different truncation parameters within the video.

35. The method according to Article 34, characterized in that in the second area the maximum value and minimum value for adaptive truncation are determined.

36. The method according to Article 33, characterized in that a fixed truncation is applied to the reconstructed signal.

37. A method for processing video, comprising: determining that an encoding mode is not activated to perform a conversion between a current video block and an encoded representation of the video; and conditional skipping of direct reformation and/or reverse reformation based on the result of the specified determination, characterized in that, in the encoding mode, the current video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

38. The method according to article 37, characterized in that at least one of the signals - a predicted signal, a reconstructed signal and/or a residual signal - is located in the first region.

39. The method according to article 37, characterized in that at least one of the signals - a predicted signal, a reconstructed signal and/or a residual signal - is located in the second region.

40. A video processing method, comprising: performing a transformation between a current video block of a video region and an encoded representation of the video, where the transformation procedure uses an encoding mode in which the current video block is constructed based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on luminance component, and wherein a plurality of forward reformatting operations and/or a plurality of reverse reformatting operations are applied in a reformatting mode for the video domain.

41. The method of Clause 40, wherein said video area comprises an image, a group of tiles, a virtual pipeline data unit (VPDU), a series of coding tree units (CTU), or multiple coding units.

42. The method according to Article 40 or 41, characterized in that how to select multiple forward reframing operations and/or multiple reverse reframing operations depends on at least one of the following factors i) block size or video block size, ii) encoding mode of the current video block or video area, iii) the image type of the current video block or video area, iv) the low latency check flag of the current video block or video area, v) the motion information of the current video block or video area, vi) reference images for the current video block or video area, or vii) the video content of the current video block or video area.

43. The method according to any of articles 1 - 42, characterized in that, during the conversion process, the reading in the first area is mapped to a reading in the second area, which has a value less than the reading in the first area.

44. The method as set forth in any of Clauses 1 to 43, wherein the first region is the original region and the second region is a reshaped region using a luma mapping method with chrominance scaling (LMCS) to map the chrominance samples to specific values .

45. The method according to Article 44, characterized in that the LMCS method uses a piecewise linear model to map the chrominance component samples to specific values.

46. The method according to any of articles 1 to 45, characterized in that the implementation of the transformation involves generating an encoded representation from the current block.

47. The method according to any of articles 1 to 45, characterized in that the implementation of the transformation includes generating the current block from the encoded representation.

48. A device in a video system containing a processor and a non-volatile storage device with commands written therein, where, when executing these commands, the processor implements a method in accordance with any one of Articles 1 to 47.

49. A computer program product stored on a non-volatile computer-readable storage medium, this computer program product contains program code for implementing a method according to any one of Articles 1 to 47.

The sixth group of articles describes certain features and aspects of the proposed methods listed in the previous section, including, for example, Examples 16 and 17.

1. A video processing method, comprising: determining that an encoding mode is activated to perform a conversion between a current video block and an encoded video representation; and performing the conversion using a palette mode, wherein at least a palette of representative sample values is used for the current video block, and where, in the encoding mode, the current video block is constructed based on the samples in the first region and the second region and/or scales the remainder of the chrominance component in a manner dependent on the brightness component.

2. The method according to article 1, characterized in that the palette of representative sample values contains at least one of 1) palette predictors or 2) outlier samples.

3. The method according to article 1, characterized in that the representative sample values represent values in the first region.

4. The method according to article 1, characterized in that the representative sample values represent values in the second region.

5. The method of claim 1 or 2, wherein the palette predictors used in the palette mode and included in the encoded representation are in the first region or in the second region.

6. The method according to clause 1 or 2, characterized in that the outlier samples used in the palette mode and included in the encoded representation are in the first region or in the second region.

7. The method of claim 1 or 2, characterized in that when the palette predictors and/or outlier samples used in the palette mode and included in the encoded representation are in the second region, first generating a first reconstructed block in the second region and then using it to encoding subsequent blocks.

8. The method of Clause 7, characterized in that when the palette predictors and/or outlier samples used in the modified palette mode and included in the encoded representation are in the second region, generating a final reconstructed block in the first region using the first reconstructed block and procedure reverse reformation.

9. The method according to Article 8, characterized in that the reverse reformation procedure is used immediately before deblocking filtration.

10. The method according to any of articles 1 - 9, characterized in that the conversion is carried out based on the color component of the current video block.

11. The method according to Article 10, characterized in that the color component is the brightness component.

12. A video processing method, comprising: determining, to perform a conversion between a current video block and an encoded video representation, that the current video block is encoded in a palette mode, in which at least a palette of representative sample values is used to encode the current video block; and performing, according to the results of said determination, the conversion by deactivating the encoding mode, where when the encoding mode is applied to a video block, the video block is constructed based on the remainder of the chrominance component scaled in a manner dependent on the luminance component.

13. The method according to Article 12, characterized in that when the current video block is encoded in palette mode, the encoding mode is not activated.

14. A video processing method, comprising: performing a transformation between a first video block of video and an encoded representation of the video, wherein the transformation procedure uses a first encoding mode and a palette encoding mode, in which at least a palette of representative pixel values is used to encode the current video block; and performing a conversion between a second video block of said video encoded without using a palette encoding mode and an encoded video representation, and where the second video block conversion procedure uses a first encoding mode, where the first encoding mode is applied to the video block, the video block is constructed based on the first region and the second region and /or scaling the remainder of the chrominance component in a manner dependent on the luminance component, and wherein the first encoding mode is applied in different ways for the first video block and the second video block.

15. The method according to Article 14, characterized in that the first encoding mode applied to the first video block is different from the first encoding mode applied to the second video block due to the cancellation of the activation of the use of forward reframing and reverse reframing, which is used to convert samples between the first area and the second area.

16. The method of Article 14, characterized in that the first encoding mode applied to the first video block is different from the first encoding mode applied to the second video block due to the use of a different reframing function and/or a different inverse reframing function used to transform the samples between the first region and the second region.

17. The method according to any of articles 1 - 11 and 14 - 16, characterized in that the first area is the original area, and the second area is the reshaped area using a luminance component display method with chroma component scaling (LMCS), which displays samples brightness component to specific values.

18. The method according to Article 17, characterized in that the LMCS mode uses a piecewise linear model to map the luminance component samples to specific values.

19. The method according to any of articles 1 - 18, characterized in that the conversion procedure contains the generation of an encoded representation from the current block.

20. The method according to any of articles 1 - 18, characterized in that the conversion procedure includes generating the current block from the encoded representation.

21. A device in a video system containing a processor and a non-volatile storage device with commands written therein, where, when executing these commands, the processor implements a method in accordance with any one of Articles 1 to 21.

22. A computer program product stored on a non-volatile computer-readable storage medium, this computer program product contains program code for implementing a method according to any one of Articles 1 to 21.

The seventh group of articles describes certain features and aspects of the proposed methods listed in the previous section, including, for example, Examples 18 and 19.

1. A video processing method, comprising: determining that an encoding mode is activated to perform a conversion between a current video block and an encoded video representation; and performing the conversion using an intra-frame block copy mode that generates a predicted block using at least a block vector pointing to an image containing the encoded video block, and wherein, in the encoding mode, the current video block is constructed based on the samples in the first region and the second region and /or scale the remainder of the chrominance component in a manner dependent on the luminance component.

2. The method according to article 1, characterized in that the predicted block is generated in the first area.

3. The method according to article 1, characterized in that the remainder block is presented as an encoded representation in the first area.

4. The method according to article 1, characterized in that the predicted block is generated in the second area.

5. The method according to article 1, characterized in that the remainder block is presented as an encoded representation in the second area.

6. The method of claim 4 or 5, wherein the first building block for the current video block is obtained based on the sum of the remainder block and the predicted block in the second region, and this first building block is used for conversions between the subsequent video block and the encoded video representation.

7. The method of claim 4 or 5, wherein the final building block for the current video block is obtained based on an inverse reformation applied to the first building block to transform the first building block from the second region to the first region.

8. The method according to any one of articles 1 - 7, characterized in that the conversion is carried out based on the color component of the current video block.

9. The method according to Article 8, characterized in that the specified color component is the brightness component.

10. A video processing method, comprising: determining, to perform a transformation between a current video block and an encoded video representation, that the current video block is encoded in an intra-block copy (IBC) mode that generates a predicted block using at least a block vector indicating a video frame, containing the current video block, for encoding the current video block; and, based on the results of said determination, performing a transformation by deactivating the encoding mode, where when the encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

11. The method according to Article 10, characterized in that when the current video block is encoded in the IBC copy mode, activation of the encoding mode is canceled.

12. A video processing method, comprising: performing a transformation between a first video block of video and an encoded representation of the video, where the transformation procedure uses an intra-frame block copy mode that generates a predicted block using at least a block vector pointing to a video frame containing the current video block and the first coding mode; and performing a transformation between a second video block of the same video encoded without using an intra-frame block copy mode and an encoded representation of the video, wherein the second video block's transformation procedure uses a first encoding mode, where when the first encoding mode is applied to a video block, the video block is constructed based on the first region and a second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component, and wherein the first encoding mode is applied in different ways to the first video block and the second video block.

13. The method according to Article 12, characterized in that the first encoding mode applied to the first video block is different from the first encoding mode applied to the second video block, since the activation of the use of forward reframing and inverse reframing used to convert samples between the first region and second area.

14. The method of Article 12, characterized in that the first encoding mode applied to the first video block is different from the first encoding mode applied to the second video block by using different forward reframing functions and/or different inverse reframing functions used to convert samples between the first area and the second area.

15. The method according to any of Articles 1 - 14, characterized in that the first area is the original area, and the second area is a reshaped area using a luminance component mapping method with chrominance component scaling (LMCS), which maps luminance component samples to specific quantities.

16. The method according to Article 15, characterized in that the LMCS mode uses a piecewise linear model to display samples.

17. The method according to any of articles 1 - 16, characterized in that the conversion procedure contains the generation of an encoded representation from the current block.

18. The method according to any of articles 1 - 16, characterized in that the conversion procedure includes generating the current block from the encoded representation.

19. A device in a video system containing a processor and a non-volatile storage device with commands written therein, where, when executing these commands, the processor implements a method in accordance with any one of Articles 1 to 18.

20. A computer program product stored on a non-volatile computer-readable storage medium, this computer program product contains program code for implementing a method according to any one of Articles 1 to 18.

The eighth group of articles describes certain features and aspects of the proposed methods listed in the previous section, including, for example, Examples 20 - 27.

1. A video processing method, comprising: determining that an encoding mode is activated to perform a conversion between a current video block and an encoded video representation; and performing the conversion using a block-based delta pulse code modulation (BDPCM) mode, wherein, in said encoding mode, the current video block is constructed based on samples in the first region and the second region and/or scales the remaining chrominance component in a luminance-dependent manner component.

2. The method according to article 1, characterized in that the predicted block for the current video block is generated in the first area.

3. The method according to article 1, characterized in that the remainder block for the current video block is presented in an encoded representation in the first area.

4. The method according to article 1, characterized in that the predicted block for the current video block is generated in the second area.

5. The method according to article 1, characterized in that the remainder block for the current video block is presented in an encoded representation in the second area.

6. The method of claim 4 or 5, characterized in that the first building block for the current video block is obtained based on the sum of the remainder block and the predicted block in the second region, and uses this first building block for conversions between the subsequent video block and the encoded video representation.

7. The method of Clause 4 or 5, wherein the final building block for the current video block is obtained based on an inverse reformation applied to the first building block to transform the first building block from the second region to the first region.

8. A video processing method, comprising: determining, to perform a conversion between a current video block and an encoded video representation, that the current video block is encoded using a block-based delta pulse code modulation (BDPCM) mode; and, based on the results of said determination, performing a transformation by deactivating the encoding mode, where when said encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

9. The method according to Article 8, characterized in that when the current video block is encoded in the BDPCM modulation mode, the encoding mode is not activated.

10. A video processing method, comprising: performing a conversion between a first video block of video and an encoded representation of the video, wherein the conversion procedure of the first video block uses a first coding mode and a block-based delta pulse code modulation (BDPCM) mode; and performing a conversion between a second video block of said video and an encoded representation of the video, where the second video block is encoded without using a BDPCM modulation mode, and the second video block conversion procedure uses a first encoding mode, where when the first encoding mode is applied to the video block, the video block is constructed based on the first region and a second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component, and wherein the first encoding mode is applied in different ways to the first video block and the second video block.

11. The method according to Article 10, characterized in that the first encoding mode applied to the first video block differs from the first encoding mode applied to the second video block due to the cancellation of the activation of the use of forward reframing and reverse reframing used to convert samples between the first region and the second region.

12. The method according to Article 10, characterized in that the first encoding mode applied to the first video block differs from the first encoding mode applied to the second video block due to the use of a different forward reformation and/or a different inverse reformation relative to those used for the sample conversion between the first region and the second region.

13. A method for processing video, comprising: determining that an encoding mode is activated to perform a conversion between a current video block and an encoded representation of the video; and performing the transformation using a skip transformation mode, in which the transformation of the prediction residual is skipped when encoding the current video block, where, in the encoding mode, the current video block is constructed based on the samples in the first region and the second region and/or the chroma residual is scaled in a manner dependent on the luminance component.

14. The method according to article 13, characterized in that the predicted block for the current video block is generated in the first region.

15. The method according to Article 13, characterized in that the remainder block relative to the current video block is presented in an encoded representation in the first area.

16. The method according to article 13, characterized in that the predicted block for the current video block is generated in the second region.

17. The method according to Article 13, characterized in that the remainder block is presented in an encoded representation in a second area.

18. The method of Clause 16 or 17, wherein the first building block for the current video block is derived from the sum of the remainder block and the predicted block in the second region, and this first building block is used for conversions between the subsequent video block and the encoded video representation.

19. The method of Clause 16 or 17, wherein the final building block for the current video block is obtained based on an inverse reformation applied to the first building block to transform the first building block from the second region to the first region.

20. A video processing method, comprising: determining, to perform a transformation between a current video block and an encoded video representation, that the current video block is encoded in a skip transformation mode, in which the transformation of the prediction residual is skipped when encoding the current video block; and, based on the results of said determination, performing a transformation by deactivating the encoding mode, where when the encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

21. The method according to Article 20, characterized in that when the current video block is encoded in the skip transformation mode, the encoding mode is not activated.

22. A video processing method, comprising: performing a transformation between a first video block of video and an encoded representation of the video, wherein the transformation procedure of the first video block uses a first encoding mode and a skip transformation mode, in which the transformation of the prediction residual is skipped when encoding the current video block; and performing a transformation between the second video block and the encoded video representation, where the second video block is encoded without using a skip transformation mode, and the transformation procedure of the second video block uses the first encoding mode, where when the first encoding mode is applied to the video block, the video block is constructed based on the first region and the second areas and/or scale the remainder of the chrominance component in a manner dependent on the luminance component, and wherein the first encoding mode is applied in different ways to the first video block and the second video block.

23. The method according to Article 22, characterized in that the first encoding mode applied to the first video block is different from the first encoding mode applied to the second video block due to the cancellation of the activation of the use of forward reframing and reverse reframing used to convert samples between the first region and the second area.

24. The method according to Article 22, characterized in that the first encoding mode applied to the first video block differs from the first encoding mode applied to the second video block due to the use of a different forward reformation method and/or a different reverse reformulation method relative to those used for converting samples between the first region and the second region.

25. A method for processing video, comprising: determining that an encoding mode is activated to perform a conversion between a current video block and an encoded representation of the video; and performing the transformation using an intra-frame prediction mode with pulse code modulation, in which the current video block is encoded without applying transformation and quantization in the transformed region, where, in the encoding mode, the current video block is constructed based on samples in the first region and the second region and/or scaled the remainder of the chrominance component in a manner dependent on the luminance component.

26. The method according to article 25, characterized in that the predicted block for the current video block is generated in the first region.

27. The method of Article 25, characterized in that the remainder block for the current video block is represented in an encoded representation in the first region.

28. The method according to article 25, characterized in that the predicted block for the current video block is generated in the second region.

29. The method of Article 25, characterized in that the remainder block for the current video block is presented in an encoded representation in the second region.

30. The method of Clause 28 or 29, wherein the first building block for the current video block is obtained based on the sum of the remainder block and the predicted block in the second region, and this first building block is used for conversion between the subsequent video block and the encoded video representation.

31. The method of Clause 28 or Clause 29, wherein the final building block for the current video block is obtained based on an inverse reformation applied to the first building block to transform the first building block from the second region to the first region.

32. A method of video processing, comprising: making a determination to perform a transformation between a current video block and an encoded representation of video that the current video block is encoded in an intra-frame pulse code modulation prediction mode, in which the current video block is encoded without applying transformation and quantization in the transformed region ; and, based on the results of said determination, performing a transformation by deactivating the encoding mode, where when the encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chroma component is scaled in a manner dependent on the luminance component.

33. The method according to Article 32, characterized in that when the current video block is encoded in the PCM intra-prediction mode, the encoding mode is not activated.

34. A method of video processing, comprising: performing a transformation between a first video block of video and an encoded representation of the video, where the transformation procedure of the first video block uses a first encoding mode and an intra-frame prediction mode with pulse code modulation, in which the current video block is encoded without applying transformation and quantization in the transformed region ; and performing a transformation between the second video block of video and an encoded representation of the video, where the second video block is encoded without using an intra-frame prediction mode with pulse code modulation, and the transformation procedure of the second video block uses a first encoding mode, where, when the first encoding mode is applied to the video block, the video block constructed based on the first region and the second region and/or scale the remainder of the chrominance component in a manner dependent on the luminance component, and wherein the first encoding mode is applied in different ways to the first video block and to the second video block.

35. The method of Article 34, characterized in that the first encoding mode applied to the first video block is different from the first encoding mode applied to the second video block due to the cancellation of the activation of the use of forward reframing and reverse reframing used to convert samples between the first region and the second region.

36. The method of Article 34, characterized in that the first encoding mode applied to the first video block is different from the first encoding mode applied to the second video block due to the use of a different forward reframing mode and/or a different inverse reframing mode applied to the transformation samples between the first region and the second region.

37. A method for processing video, comprising: determining that an encoding mode is activated to perform a conversion between a current video block and an encoded representation of the video; and performing the conversion using a modified transformation and quantization bypass mode, in which the current video block is encoded losslessly and without transformation and quantization, where, in the encoding mode, the current video block is constructed based on samples in the first region and the second region and/or the remainder of the chrominance component is scaled in a way that depends on the brightness component.

38. The method according to article 37, characterized in that the predicted block for the current video block is generated in the first region.

39. The method of Article 37, characterized in that the remainder block for the current video block is represented in an encoded representation in the first region.

40. The method according to article 37, characterized in that the predicted block for the current video block is generated in the second region.

41. The method of Article 37, characterized in that the remainder block for the current video block is represented in an encoded representation in a second region.

42. The method of Clause 40 or 41, wherein the first building block for the current video block is obtained based on the sum of the remainder block and the predicted block in the second region, and using this first building block to convert between the subsequent video block and the encoded video representation.

43. The method of Clause 40 or Clause 41, wherein the final building block for the current video block is obtained based on an inverse reformation applied to the first building block to transform the first building block from the second region to the first region.

44. A video processing method, comprising: determining, to perform a transformation between a current video block and an encoded video representation, that the current video block is encoded in a transformation and quantization bypass mode, in which the current video block is encoded losslessly and without applying transformation and quantization; and, based on the results of said determination, performing a transformation by deactivating the encoding mode, where when the encoding mode is applied to a video block, the video block is constructed based on the first region and the second region and/or the remainder of the chrominance component is scaled in a manner dependent on the luminance component.

45. The method according to Article 44, characterized in that when the current video block is encoded in the intra-frame prediction mode with pulse code modulation, this encoding mode is not activated.

46. A video processing method, comprising: performing a transformation between a first video block of video and an encoded representation of the video, wherein the transformation procedure of the first video block uses a first encoding mode and a transformation and quantization bypass mode, in which the current video block is encoded losslessly and without transformation and quantization; and performing a transformation between a second video block of said video and an encoded representation of the video, wherein the second video block is encoded without using a transform and quantization bypass mode, and the second video block transformation procedure uses a first encoding mode, where when the first encoding mode is applied to the video block, the video block is constructed based on a first region and a second region and/or scaling the remainder of the chrominance component in a manner dependent on the luminance component, and wherein the first encoding mode is applied in different ways to the first video block and the second video block.

47. The method according to Article 46, characterized in that the first encoding mode applied to the first video block is different from the first encoding mode applied to the second video block due to the cancellation of the activation of the use of forward reframing and reverse reframing used to convert samples between the first region and the second region.

48. The method of Article 46, characterized in that the first encoding mode applied to the first video block is different from the first encoding mode applied to the second video block due to the use of a different forward reshaping and/or a different inverse reframing used to convert the samples between the first area and the second area.

49. The method according to any one of articles 1 - 48, characterized in that the conversion is carried out based on the color component of the current video block.

50. The method according to Article 49, characterized in that the specified color component is a chromatic component.

51. The method according to any of clauses 1 to 50, characterized in that the first region is the original region and the second region is a reshaped region using a luminance component scaling (LMCS) display method that maps luminance component samples to specific values .

52. The method according to Article 51, characterized in that the LMCS mode uses a piecewise linear model to map the luminance component samples to specific values.

53. The method according to any of articles 1 - 52, characterized in that the conversion procedure contains the generation of an encoded representation from the current block.

54. The method according to any of articles 1 - 52, characterized in that the conversion procedure includes generating the current block from the encoded representation.

55. A device in a video system containing a processor and a non-volatile storage device with commands written therein, where, when executing these commands, the processor implements a method in accordance with any one of Articles 1 to 54.

56. A computer program product stored on a non-volatile computer-readable storage medium, this computer program product contains program code for implementing a method according to any one of Articles 1 to 54.

The ninth group of articles describes certain features and aspects of the proposed methods listed in the previous section, including, for example, Examples 30 - 34 and 41.

1. A video processing method, comprising: performing a transformation between a current video video block and an encoded video representation, where the transformation procedure uses an encoding mode in which the current video block is constructed based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component, and where the information used for the encoding mode is signaled in a parameter set other than the sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS) or adaptation parameter set (APS) used to transmit adaptive loop filtering (ALF) parameters.

2. The method according to article 1, characterized in that the specified set of parameters is shared by several images.

3. The method according to article 1, characterized in that the specified set of parameters contains one or more syntactic elements, including at least one of the identifiers of this parameter set or a flag indicating the presence of extension data of this parameter set.

4. The method according to article 1, characterized in that the specified set of parameters is specific to a group of tiles in the image.

5. A video processing method, comprising: performing a transformation between a current video video block and an encoded video representation, where the transformation operation uses an encoding mode in which the current video block is constructed based on a first region and a second region and/or scales the remainder of the chrominance component in a luminance-dependent manner component, and where the information used for the encoding mode is signaled in an adaptation parameter set (APS) together with the adaptive loop filtering (ALF) information, where the information used for the encoding mode and the ALF filtering information are included in one unit NAL level.

6. The method of Clause 5, wherein the APS set identifier is signaled in the tile group header.

7. A video processing method, comprising: performing a transformation between a current video block of a video region and an encoded representation of the video, where the transformation procedure uses an encoding mode in which the current video block is constructed based on the first region and the second region and/or scales the remainder of the chrominance component in a manner dependent on luminance component, and where the information used for the encoding mode is transmitted as signaling in a first type of adaptation parameter set (APS) different from the second type APS used for transmitting the adaptive loop filtering (ALF) information in the form of signaling.

8. The method of Clause 7, wherein the second type APS set identifier is communicated as signaling at the video domain level.

9. The method according to Clause 7, characterized in that the APS set identifier of the first type is communicated in the form of signaling at the video domain level.

10. The method of Clause 7, wherein the first type APS set included in the encoded representation contains a second type APS set that includes ALF filtering information in the match data bitstream.

11. The method of Clause 7, wherein the second type APS set included in the encoded representation contains the first type APS set, which includes information used for the encoding mode in the correspondence data bit stream.

12. The method according to Article 7, characterized in that the set of APS of the first type and the set of APS of the second type are associated with different identifiers.

13. The method according to Article 12, characterized in that the APS set of the second type has an identifier equal to 2N, where N is an integer.

14. The method according to Article 13, characterized in that the APS set of the first type has an identifier equal to 2N+1, where N is an integer.

15. A video processing method, comprising: performing a transformation between a current video block of a video region and an encoded representation of the video, where the transformation procedure uses an encoding mode in which the current video block is constructed based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on luminance component, and where it is not permissible for said video region to access an adaptation parameter set or a set of parameters signaled before a data structure of a specified type used for video processing, and where this data structure of a specified type is signaled before the video region.

16. The method of Clause 15, wherein said data structure comprises at least one of a network abstraction layer (NAL) unit, a tile group, a sequence parameter set (SPS), a picture parameter set (PPS), an access unit delimiter NAL Level (AUD), NAL End of Data Bitstream (EoB) Unit, NAL End of Sequence Unit (NAL), Instant Decoding Update (IDR) NAL Unit, Pure Random Access (CRA) NAL Unit, Intra Prediction Random Access Point Unit (IRAP), I-type tile group, image or slice.

17. A video processing method, comprising: performing a transformation between a current video block and an encoded video representation, where the transformation procedure uses an encoding mode in which the current video block is constructed based on a first region and a second region and/or scales the remainder of the chrominance component in a manner dependent on the luminance component , and where a syntax element of a parameter set containing parameters used for video processing has specified values in the matching data bit stream.

18. The method according to Article 17, characterized in that the specified specified values are equal to 0 and 1.

19. The method according to Article 17, characterized in that the specified specified values are 0 and 7.

20. The method according to any of articles 1 - 19, characterized in that the specified video area contains at least one - a group of tiles, an image, a slice and/or a tile.

21. The method according to any of articles 1 to 20, characterized in that the first region is the original region and the second region is a reshaped region using a luminance component scaling (LMCS) display method that maps luminance component samples to specific values .

22. The method according to Article 21, characterized in that the LMCS method uses a piecewise linear model to map the luminance component samples to specific values.

23. The method according to any of articles 1 - 22, characterized in that the conversion procedure contains the generation of an encoded representation from the current block.

24. The method according to any of articles 1 - 22, characterized in that the conversion procedure includes generating the current block from the encoded representation.

25. A device in a video system containing a processor and a non-volatile storage device with commands written therein, where, when executing these commands, the processor implements a method in accordance with any one of Articles 1 to 24.

26. A computer program product stored on a non-volatile computer-readable storage medium, this computer program product contains program code for implementing a method according to any one of Articles 1 to 24.

It will be understood from the foregoing that specific embodiments of the technology provided herein have been described herein for purposes of illustration, but various modifications may be made without departing from the scope of the invention. Accordingly, the technology presented herein is not limited in any way except by the attached Claims.

Embodiments of the subject matter of the present invention and the functional operations described herein may be implemented in a variety of systems, digital electronic circuits or computer downloadable software, firmware or hardware, including the structures discussed herein and their structural equivalents, or in a combination of one or more of the listed components. The embodiments of the subject matter of the present invention discussed in this description may be implemented in the form of one or more computer program products, i.e. one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable storage medium for execution by a data processing device or for controlling the operation of that device. Such a computer-readable storage medium may be a computer-readable storage device, a computer-readable storage board, a storage device, a composition of objects implementing a computer-readable propagated signal, or a combination of one or more of these components. The term "data processing module" or "data processing device" covers all data processing equipment, devices and machines, including, by way of example, a programmable processor, a computer, or multiple processors or computers. The hardware may contain, in addition to hardware, code constituting the execution environment for the subject computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these components.

A computer program (also known as a program, software, application, macro, or code) may be written in any programming language, including compiled or interpreted languages, and may be deployed in in any form, including a stand-alone program, program module, component, subroutine, or other similar object suitable for use in a computer environment. A computer program does not necessarily correspond to a single file on a computer system. A program may be stored in a portion of a file that stores other programs or data, (for example, one or more macros stored in a markup language document) in a single file dedicated to the program in question, or in multiple coordinated files (for example, files storing one or more modules, subroutines or pieces of code). A computer program may be deployed to run on a single computer or on multiple computers located in one location or distributed over multiple locations and connected by a communications network.

The procedures and logic discussed herein may be implemented by one or more programmable processors executing one or more programs to implement a function by operating on input data and generating output data. These procedures and logic circuits may also be implemented by and devices may be implemented as special-purpose logic circuits, such as field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs). .

Processors suitable for executing a computer program may include, for example, microprocessors, both general purpose and special purpose, and one or more digital computer processors of some type. In general, the processor will accept instructions and data from read-only memory or random access memory, or both types of storage devices. The essential elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. In general, the computer will also comprise or be operatively coupled to receive and/or transmit data with mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks. However, the computer does not need to have such devices. Computer-readable storage media for storing instructions and data includes all forms of non-volatile memory, storage media, and storage devices, including, by way of example, semiconductor storage devices such as EPROM, electrically erasable read-only memory, and programmable read-only memory. storage device (EEPROM) and flash memory devices. The processor and storage device can be complemented or included in special-purpose logic circuitry.

This description, together with the accompanying drawings, should be considered as examples only. As used here, the application of the conjunction "or" must also include "and/or" unless the context clearly indicates otherwise.

Although this patent document contains many specific details, they should not be construed as any limitation on the scope of any innovation or what may be claimed as an invention, but merely as descriptions of features that may be specific to particular embodiments of particular innovations . Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of one embodiment may also be implemented in multiple embodiments individually or in some suitable subcombination. More specifically, although features may be described above as operating in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be excluded from that claimed combination such that the claimed combination may be converted into a subcombination or variations of such a subcombination.

Likewise, while operations are depicted in the drawings in a particular order, this should not be construed as a requirement that such operations must be performed in the specific order shown or in a sequential order or that all illustrations shown must be performed to achieve the desired results. Moreover, the separation of the various system components described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few embodiments and examples are described herein, however, other embodiments, improvements and variations may be created based on what is described and illustrated herein.

Claims (45)

1. A method for processing video data, comprising the steps of: determining, for conversion between the current video video block and the video data bit stream, a first predicted block in a first region of at least one reference image using the unspecified motion information, wherein the current video block is a luminance component block; applying, at least based on the first predicted block, a motion information refinement procedure to the unspecified motion information to obtain motion offset information; generating a second predicted block from the samples on the at least one reference image using the motion offset information; converting, based on a direct mapping procedure, a second prediction block from the first region to a second region, wherein samples of the second prediction block are mapped to specific values; And generating reconstructed samples of the current video block based on the second predicted block in the second region, in this case, to map the samples of the second predicted block to specific values during the direct mapping procedure, a piecewise linear model is used, wherein the scaling factors of the piecewise linear model are determined based on first variables that are determined based on syntactic elements included in the set of adaptation parameters and second variables that are determined based on bit depth. 2. The method of claim 1, wherein during the motion information refinement procedure, motion offset information is generated based on the bilinear interpolation and cost calculation procedure, wherein the motion offset information is further used to predict a temporal motion vector for subsequent blocks. 3. The method of claim 2, wherein the cost calculation comprises sum of absolute differences (SAD) or sum of absolute differences plus mean (MR-SAD). 4. The method of claim 1, wherein during the motion information refinement procedure, motion displacement information is generated based on gradient calculations in different directions based on samples in the first region. 5. The method of claim 1, wherein the reconstructed samples of the current video block are converted into the first region based on a demapping procedure. 6. The method according to claim 5, in which a filtering procedure is applied to the transformed reconstructed samples in the first region. 7. The method according to claim 1, in which the samples of the remainder of the chroma component block corresponding to the current video block are determined by applying a scaling procedure based on the reconstructed samples in the second region related to the luminance component of the image to which the current video block belongs. 8. The method of claim 1, wherein the motion information refinement procedure comprises at least one of a decoder-side motion vector refinement procedure, a frame rate upconversion procedure, or a bidirectional optical flow procedure. 9. The method according to claim 1, in which, at the stage of applying the procedure for clarifying motion information to unspecified motion information to obtain information about the displacement of motion: checking a plurality of motion vectors determined based on the unspecified motion information, and selecting one vector with the lowest cost to determine the motion offset information. 10. The method according to claim 1, in which, at the conversion stage, the current video block is encoded into a stream of data bits. 11. The method according to claim 1, in which, at the conversion stage, the current video block is decoded from the data bit stream. 12. A video data processing device comprising a processor and a non-volatile storage device with commands written therein, wherein the commands, when executed by the processor, cause the processor to execute: determining, for conversion between the current video video block and the video data bit stream, a first predicted block in a first region from at least one reference image using the unspecified motion information, wherein the current video block is a luminance component block; applying, at least based on the first predicted block, a motion information refinement procedure to the unspecified motion information to obtain motion offset information; generating a second predicted block from the samples on the at least one reference image using the motion offset information; transforming, based on a direct mapping procedure, a second prediction block from a first region to a second region, the samples of the second prediction block being mapped to specific values; And generating reconstructed samples of the current video block based on the second predicted block in the second region, in this case, to map the samples of the second predicted block to specific values during the direct mapping procedure, a piecewise linear model is used, wherein the scaling factors of the piecewise linear model are determined based on first variables that are determined based on syntactic elements included in the set of adaptation parameters and second variables that are determined based on bit depth. 13. A non-volatile computer-readable storage medium that stores instructions that cause the processor to execute: determining, for conversion between the current video video block and the video data bit stream, a first predicted block in a first region from at least one reference image using the unspecified motion information, wherein the current video block is a luminance component block; applying, at least based on the first predicted block, a motion information refinement procedure to the unspecified motion information to obtain motion offset information; generating a second predicted block from the samples on the at least one reference image using the motion offset information; transforming, based on a direct mapping procedure, a second prediction block from a first region to a second region, the samples of the second prediction block being mapped to specific values; And generating reconstructed samples of the current video block based on the second predicted block in the second region, in this case, to map the samples of the second predicted block to specific values during the direct mapping procedure, a piecewise linear model is used, wherein the scaling factors of the piecewise linear model are determined based on first variables that are determined based on syntactic elements included in the set of adaptation parameters and second variables that are determined based on bit depth. 14. A method for storing a video data bit stream, comprising the steps of: determining, for the current video video block, a first predicted block in a first region of at least one reference image using the unspecified motion information, wherein the current video block is a luminance component block; applying, at least based on the first predicted block, a motion information refinement procedure to the unspecified motion information to obtain motion offset information; generating a second predicted block from the samples on the at least one reference image using the motion offset information; converting, based on a direct mapping procedure, a second prediction block from the first region to a second region, wherein samples of the second prediction block are mapped to specific values; And generating a stream of data bits based on the second predicted block in the second region; And storing the stream of data bits in a non-volatile computer-readable storage medium; wherein reconstructed samples of the current video block are generated based on the second predicted block in the second region, in this case, to map the samples of the second predicted block to specific values during the direct mapping procedure, a piecewise linear model is used, wherein the scaling factors of the piecewise linear model are determined based on first variables that are determined based on syntactic elements included in the set of adaptation parameters and second variables that are determined based on bit depth.