RU2787812C2 - Method and equipment for video image prediction - Google Patents

Abstract

FIELD: image encoding.

SUBSTANCE: invention relates to the field of image encoding technologies, in particular to devices for video image prediction.Video images prediction equipment contains: a syntactic analysis unit configured for syntactic analysis, from a sequence parameter set (hereinafter – SPS) in a bit stream, of a value of the first sps_affine_enabled_flag indicator and a value of the third sps_sbtmvp_enabled_flag indicator; wherein the syntactic analysis unit is configured for syntactic analysis of a value of the first indicator from the mentioned SPS, when the value of the first indicator is equal to the first value; a determination unit configured for determination of the maximum length of the first list of candidate motion vectors based on the second indicator, when the value of the first indicator is equal to the first value, wherein the first list of candidate motion vectors is a list of candidate motion vectors, created for a block to be processed, a mode of prediction of subblock merge is used for the block to be processed. The determination unit is configured for determination of the maximum length of the first list of candidate motion vectors based on the value of the third indicator, when the value of the first indicator is equal to the second value, wherein the second value differs from the first value.

EFFECT: invention provides determination of the maximum length of a list of candidate motion vectors, corresponding to a subblock merge mode.

18 cl, 5 tbl, 23 dwg

Description

[0001] The present application claims priority based on Chinese Patent Application No. 201811268188.2 filed with the National Intellectual Property Administration of China on October 29, 2018 and entitled "INTER PREDICTION METHOD AND APPARATUS", which is hereby incorporated by reference in its entirety. The present application claims priority based on Chinese Patent Application No. 201811642717.0, filed with the China Patent Office on December 29, 2018, entitled "VIDEO PICTURE PREDICTION METHOD AND APPARATUS", which is hereby incorporated by reference in its entirety.

Technical field

[0002] This application relates to the field of image coding technologies and, in particular, to a method and apparatus for predicting video images.

State of the art

[0003] With the development of information technology, video services such as high-definition television, web conferencing, IPTV, and 3D television are developing rapidly. Due to the advantages of intuitiveness and high efficiency, video signals are becoming the main method of obtaining information in people's daily lives. Video signals contain a large amount of data and therefore take up most of the transmission bandwidth and storage space. For efficient transmission and storage of video signals, it is necessary to perform compression encoding on the video signals. Video compression technology has gradually become an indispensable key technology in the video application field.

[0004] The main principle of video coding compression is to reduce redundancy as much as possible by using correlations between the spatial domain, the time domain, and the codeword. At present, a common technique is to implement video coding compression using a picture block-based hybrid video coding structure and by performing steps such as prediction (including intra prediction and inter prediction), transformation, quantization, and entropy coding.

[0005] In various video encoding/decoding solutions, inter-prediction motion estimation/motion compensation is a key technology that affects encoding/decoding performance. In the existing inter-prediction, sub-block-based merging motion vector prediction is added based on a block-based motion compensation (MC) prediction in which a rectilinear motion model is used. In the existing technology, there is no acceptable method for determining the maximum length of the list of candidate motion vectors corresponding to the subblock merge mode.

The essence of the invention

[0006] This application provides a video prediction method and apparatus to provide a method for determining the maximum length of a list of candidate motion vectors in a subblock merge mode.

[0007] According to a first aspect, an embodiment of this application provides a method for predicting video images, including:

parsing the first pointer (eg, sps_affine_enable_flag) from the bitstream; when the first pointer indicates that the candidate mode used for inter-prediction of the block to be processed contains affine mode, parsing the second pointer (e.g., five_minus_max_num_subblock_merge_cand or six_minus_max_num_subblock_merge_cand) from the bitstream, with the second pointer used to indicate the maximum length of the first candidate list motion vectors, and the first motion vector candidate list is a motion vector candidate list created for a block to be processed, the sub-block merge prediction mode is used for the block to be processed; and determining a maximum length of the first list of candidate motion vectors based on the second pointer.

[0008] The above method provides a method for determining the maximum length of a list of candidate motion vectors in a subblock merge mode. It's simple and easy to implement.

[0009] In a possible implementation, before determining the maximum length of the first list of candidate motion vectors based on the second pointer, the method further comprises: parsing a third pointer (eg, sps_sbtmvp_enabled_flag) from the bitstream, where the third pointer is used to indicate the presence state of the extended temporal motion vector prediction mode in subblock merge prediction mode.

[0010] In an exemplary embodiment, the subblock merge prediction mode comprises at least one of planar motion vector prediction mode, extended temporal motion vector prediction mode, or affine mode; and when the third pointer indicates that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode, determining the maximum length of the first candidate motion vector list based on the second pointer comprises: determining the first number based on the third pointer and determining the maximum length of the first candidate vector list movements based on the second pointer and the first number.

[0011] For example, when sps_sbtmvp_enabled_flag=0, this indicates that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode. For example, the first number is equal to the number of motion vectors that are supported in prediction performed using the extended temporal motion vector prediction mode. When sps_sbtmvp_enabled_flag=0, the first number is equal to the number of motion vectors that are supported in prediction performed using the extended temporal motion vector prediction mode.

[0012] In an exemplary embodiment, before determining the maximum length of the first list of candidate motion vectors based on the second pointer, the method further comprises: parsing a fourth pointer (e.g., sps_planar_enabled_flag) from the bitstream, where the fourth pointer is used to indicate the presence state of the planar motion vector prediction mode in subblock merge prediction mode.

[0013] In an exemplary embodiment, when the third pointer indicates that the extended temporal motion vector prediction mode is present in the subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is not present in the subblock merge prediction mode, determining the maximum length of the first list of candidate motion vectors based on the second pointer comprises: determining the second number based on the fourth pointer and determining the maximum length of the first list of candidate motion vectors based on the second pointer and the second number.

[0014] For example, when sps_planar_enabled_flag=0, this indicates that the planar motion vector prediction mode is not present in the subblock merge prediction mode. For example, the second number is equal to the number of motion vectors that are supported in prediction performed using the planar motion vector prediction mode.

[0015] In an exemplary embodiment, when the third pointer indicates that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is present in the subblock merge prediction mode, determining the maximum length of the first list of candidate motion vectors based on the second pointer comprises: determining the maximum length of the first list of candidate motion vectors based on the second pointer and the first number.

[0016] In an exemplary embodiment, when the third pointer indicates that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is not present in the subblock merge prediction mode, determining the maximum length of the first the list of candidate motion vectors based on the second pointer comprises: determining the maximum length of the first list of candidate motion vectors based on the second pointer, the first number, and the second number.

[0017] In an exemplary embodiment, the maximum length of the first list of candidate motion vectors is obtained from the following formula:

MaxNumSubblockMergeCand=K − K_minus_max_num_subblock_merge_cand,

where MaxNumSubblockMergeCand represents the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand represents the second pointer, and K is a preset non-negative integer.

[0018] In an exemplary embodiment, when the maximum length of the first candidate motion vector list is determined based on the second pointer and the first number, the maximum length of the first candidate motion vector list is obtained according to the following formula:

MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L1,

where MaxNumSubblockMergeCand represents the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand represents the second pointer, L1 represents the first number, and K is a preset non-negative integer.

[0019] In an exemplary embodiment, when the maximum length of the first candidate motion vector list is determined based on the second pointer and the second number, the maximum length of the first candidate motion vector list is obtained according to the following formula:

MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L2,

where MaxNumSubblockMergeCand is the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand is the second pointer, L2 is the second number, and K is a preset non-negative integer.

[0020] In an exemplary embodiment, when the maximum length of the first candidate motion vector list is determined based on the second pointer, the first number, and the second number, the maximum length of the first candidate motion vector list is obtained according to the following formula: MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L1-L2, where MaxNumSubblockMergeCand is the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand is the second pointer, L1 is the first number, and L2 is the second number, and K is a preset non-negative integer.

[0021] In a possible implementation, parsing the second pointer from the bitstream comprises:

parsing a second pointer from a bitstream sequence parameter set; or parsing a second pointer from a slice header for a slice in the bitstream, the block to be processed is contained in that slice.

[0022] In a possible implementation, the method further comprises: when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, and the third pointer (for example, sps_sbtmvp_enabled_flag) indicates that the extended temporal prediction mode a motion vector is present in the subblock merge prediction mode, determining a third number based on the third pointer, and determining the maximum length of the first list of candidate motion vectors based on the third number. For example, when sps_sbtmvp_enabled_flag=1, it indicates that the extended motion vector prediction mode is present in the subblock merge prediction mode. The third number is equal to the number of motion vectors that are supported in prediction performed using the advanced temporal motion vector prediction mode. For example, the maximum length of the first list of candidate motion vectors is the third number.

[0023] In an exemplary embodiment, when the fourth indicator (eg, sps_planar_enabled_flag) indicates that the planar motion vector prediction mode is present in the subblock merge prediction mode, determining the maximum length of the first list of candidate motion vectors based on the first number comprises: determining the fourth number based on the fourth a pointer and determining a maximum length of the first list of candidate motion vectors based on the third number and the fourth number. For example, the maximum length of the first list of candidate motion vectors is the sum of the third number and the fourth number.

[0024] For example, when sps_planar_enabled_flag=1, this indicates that the planar motion vector prediction mode is present in the subblock merge prediction mode. The fourth number is equal to the number of motion vectors that are supported in the prediction performed using the list of planar motion vectors.

[0025] In an exemplary embodiment, the method further comprises: when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, the third pointer indicates that the extended temporal motion vector prediction mode does not is present in the subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is present in the subblock merge prediction mode, determining the fourth number based on the fourth pointer, and determining the maximum length of the first motion vector candidate list based on the fourth number. For example, the maximum length of the first list of candidate motion vectors is the fourth number.

[0026] In a possible implementation, when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, and the third pointer indicates that the extended temporal motion vector prediction mode is not present in sub-block merge prediction mode, the maximum length of the first list of candidate motion vectors is zero.

[0027] In an exemplary embodiment, when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, the third pointer indicates that the extended temporal motion vector prediction mode is not present in the mode. subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is not present in the subblock merge prediction mode, the maximum length of the first motion vector candidate list is zero.

[0028] In a possible implementation, the third pointer is equal to the first value, and the first number is equal to 1.

[0029] In a possible implementation, the fourth pointer is equal to the third value, and the second number is equal to 1.

[0030] In a possible implementation, the third pointer is equal to the second value, and the third number is equal to 1.

[0031] In an exemplary embodiment, the fourth pointer is equal to the fourth value, and the fourth number is equal to 1.

[0032] According to a second aspect, an embodiment of this application provides a video prediction apparatus, including:

a parser configured to: parse the first pointer from the bitstream; and when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains the affine mode, parsing the second pointer from the bitstream, where the second pointer is used to indicate the maximum length of the first list of candidate motion vectors, and the first list candidate motion vectors is a list of candidate motion vectors created for the block to be processed, the sub-block merge prediction mode is used for the block to be processed; and

a determiner configured to determine a maximum length of the first list of candidate motion vectors based on the second pointer.

[0033] In an exemplary embodiment, the parser is further configured to parse a third pointer from the bitstream before the maximum length of the first list of candidate motion vectors is determined based on the second pointer, where the third pointer is used to indicate the presence state of the extended temporal motion vector prediction mode in subblock merge prediction mode.

[0034] In an exemplary embodiment, the subblock merge prediction mode comprises at least one of planar motion vector prediction mode, extended temporal motion vector prediction mode, or affine mode; and when the third pointer indicates that the extended temporal motion vector prediction mode is not present in the sub-block merge prediction mode, the determining unit is specifically configured to:

determine the first number based on the third pointer, and

determine the maximum length of the first list of candidate motion vectors based on the second pointer and the first number.

[0035] In an exemplary embodiment, before the maximum length of the first list of candidate motion vectors is determined based on the second pointer, the parser is further configured to:

parsing a fourth pointer from the bitstream, where the fourth pointer is used to indicate the presence state of the planar motion vector prediction mode in the subblock merge prediction mode.

[0036] In an exemplary embodiment, when the third pointer indicates that the extended temporal motion vector prediction mode is present in the subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is not present in the subblock merge prediction mode, the determining unit is specifically configured, to:

determine the second number based on the fourth pointer, and

determine the maximum length of the first list of candidate motion vectors based on the second pointer and the second number.

[0037] In an exemplary embodiment, when the third pointer indicates that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is present in the subblock merge prediction mode, the determining unit is specifically configured, to:

determine the maximum length of the first list of candidate motion vectors based on the second pointer and the first number.

[0038] In an exemplary embodiment, when the third pointer indicates that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is not present in the subblock merge prediction mode, the determining unit is specifically configured , to:

determine the maximum length of the first list of candidate motion vectors based on the second pointer, the first number, and the second number.

[0039] In an exemplary embodiment, the maximum length of the first list of candidate motion vectors is obtained from the following formula:

MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand,

where MaxNumSubblockMergeCand represents the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand represents the second pointer, and K is a preset non-negative integer.

[0040] In an exemplary embodiment, the maximum length of the first list of candidate motion vectors is obtained from the following formula:

MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L1,

where MaxNumSubblockMergeCand represents the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand represents the second pointer, L1 represents the first number, and K is a preset non-negative integer.

[0041] In an exemplary embodiment, the maximum length of the first list of candidate motion vectors is obtained from the following formula:

MaxNumSubblockMergeCand=K - K_minus_max_num_subblock_merge_cand - L2,

where MaxNumSubblockMergeCand is the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand is the second pointer, L2 is the second number, and K is a preset non-negative integer.

[0042] In an exemplary embodiment, the maximum length of the first list of candidate motion vectors is obtained by the following formula:

MaxNumSubblockMergeCand=K - K_minus_max_num_subblock_merge_cand - L1 - L2,

where MaxNumSubblockMergeCand represents the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand represents the second pointer, L1 represents the first number, L2 represents the second number, and K is a preset non-negative integer.

[0043] In an exemplary embodiment, when parsing the second pointer from the bitstream, the parser is specifically configured to:

parse a second pointer from a set of sequence parameters in the bitstream; or parse a second pointer from a slice header for a slice in the bitstream, wherein the block to be processed is contained in said slice.

[0044] In a possible implementation, when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, and the third pointer indicates that the extended temporal motion vector prediction mode is present in the mode sub-block merge prediction, the determining unit is further configured to: determine the third number based on the third pointer, and determine the maximum length of the first list of candidate motion vectors based on the third number.

[0045] In an exemplary embodiment, when the fourth pointer indicates that the planar motion vector prediction mode is present in the subblock merge prediction mode, when determining the maximum length of the first list of candidate motion vectors based on the first number, the determining unit is specifically configured to:

determine the fourth number based on the fourth pointer, and

determine the maximum length of the first list of candidate motion vectors based on the first number and the fourth number.

[0046] In an exemplary embodiment, when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, the third pointer indicates that the extended temporal motion vector prediction mode is not present in the mode. subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is present in the subblock merge prediction mode, the determining unit is further configured to: determine the fourth number based on the fourth pointer, and determine the maximum length of the first list of candidate motion vectors based on the fourth number.

[0047] In a possible implementation, when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, and the third pointer indicates that the extended temporal motion vector prediction mode is not present in sub-block merge prediction mode, the maximum length of the first list of candidate motion vectors is zero.

[0048] In an exemplary embodiment, when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, the third pointer indicates that the extended temporal motion vector prediction mode is not present in the mode. subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is not present in the subblock merge prediction mode, the maximum length of the first motion vector candidate list is zero.

[0049] In an exemplary embodiment, the maximum length of the first list of candidate motion vectors is a third number.

[0050] In an exemplary embodiment, the maximum length of the first list of candidate motion vectors is equal to the sum of the third number and the fourth number.

[0051] In an exemplary embodiment, the maximum length of the first list of candidate vectors is a fourth number.

[0052] Optionally, the third pointer is equal to the first value, and the first number is 1.

[0053] In a possible implementation, the fourth pointer is equal to the third value, and the second number is equal to 1.

[0054] In a possible implementation, the third pointer is equal to the second value, and the third number is equal to 1.

[0055] In an exemplary embodiment, the fourth pointer is equal to the fourth value, and the fourth number is equal to 1.

[0056] According to a third aspect, an embodiment of this application provides hardware. The hardware may be a decoder and include a processor and memory. The memory is configured to store the instruction. When the hardware is running, the processor executes an instruction stored in memory to enable the hardware to perform the method according to any of the first aspect or implementations of the first aspect. It should be noted that the memory may be integral to the processor or may be independent of the processor.

[0057] According to a fourth aspect, an embodiment of this application provides a video prediction method used at the encoder side, including:

encoding the first pointer in the bitstream; and

when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains affine mode, encoding the second pointer in the bitstream, the second pointer being used to indicate the maximum length of the first list of candidate motion vectors, and the first list of candidate vectors motion vector is a list of candidate motion vectors created for the block to be processed, the sub-block merge prediction mode is used for the block to be processed.

[0058] According to a fifth aspect, an embodiment of this application provides a video prediction engine. The hardware may be an encoder and include a processor and memory. The memory is configured to store the instruction. When the hardware is running, the processor executes an instruction stored in the memory to enable the hardware to execute the method according to the fourth aspect. It should be noted that the memory may be integral to the processor or may be independent of the processor.

[0059] According to a sixth aspect of this application, a computer-readable storage medium is provided. The computer-readable storage medium stores the instruction. When an instruction is executed on a computer, the computer is able to execute the method according to each of the above aspects.

[0060] According to a seventh aspect of this application, a computer program product including an instruction is provided. When the computer program product is executed on the computer, the computer is able to perform the method according to each of the above aspects.

[0061] It should be understood that the technical solutions described in aspects from the second aspect to the seventh aspect of this application are consistent with the technical solutions described in the first aspect of this application, and the benefits achieved in all aspects and corresponding implementations are similar. Therefore, details are not described again.

BRIEF DESCRIPTION OF THE DRAWINGS

[0062] FIG. 1A is a block diagram of an example of a video encoding and decoding system 10 for implementing embodiments of this application;

[0063] FIG. 1B is a block diagram of an example of a video coding system 40 for implementing embodiments of this application;

[0064] FIG. 2 is a block diagram of an exemplary structure of an encoder 20 for implementing embodiments of this application;

[0065] FIG. 3 is a block diagram of an exemplary structure of a decoder 30 for implementing embodiments of this application;

[0066] FIG. 4 is a block diagram of an example of a video encoding apparatus 400 for implementing embodiments of this application;

[0067] FIG. 5 is a block diagram of an example of other encoding equipment or decoding equipment for implementing embodiments of this application;

[0068] FIG. 6A is a schematic diagram of a candidate location for traffic information for implementing embodiments of this application;

[0069] FIG. 6B is a schematic diagram of a legacy reference point motion vector prediction for implementing embodiments of this application;

[0070] FIG. 6C is a schematic diagram of a built-in reference point motion vector prediction for implementing embodiments of this application;

[0071] FIG. 6D is a schematic diagram of a procedure for combining the movement information of a checkpoint to obtain information about the movement of a constructed checkpoint for implementing embodiments of this application;

[0072] FIG. 6E is a schematic diagram of an ATMVP prediction method for implementing embodiments of this application;

[0073] FIG. 7 is a schematic diagram of a planar motion vector prediction method for implementing embodiments of this application;

[0074] FIG. 8A is a flowchart of an inter prediction method for implementing embodiments of this application;

[0075] FIG. 8B is a schematic diagram of building a list of candidate motion vectors for implementing embodiments of this application;

[0076] FIG. 8C is a schematic diagram of a motion compensation unit for implementing embodiments of this application;

[0077] FIG. 9 is a schematic block diagram of a video prediction method for implementing embodiments of this application;

[0078] FIG. 10 is a schematic block diagram of another video prediction method for implementing embodiments of this application;

[0079] FIG. 11 is a schematic block diagram of yet another video prediction method for implementing embodiments of this application;

[0080] FIG. 12A and FIG. 12B is a schematic block diagram of yet another video prediction method for implementing embodiments of this application;

[0081] FIG. 13 is a schematic diagram of apparatus 1300 for implementing embodiments of this application;

[0082] FIG. 14 is a schematic diagram of apparatus 1400 for implementing embodiments of this application; and

[0083] FIG. 15 is a schematic diagram of apparatus 1500 for implementing embodiments of this application.

Description of Embodiments

[0084] Embodiments of this application will now be described with reference to the accompanying drawings in embodiments of this application. In the following description, reference is made to the accompanying drawings, which form part of this disclosure and show, by way of illustration, specific aspects of embodiments of this application or specific aspects in which embodiments of this application may be used. It should be understood that embodiments of this application may be used in other aspects and may contain structural or logical changes not shown in the accompanying drawings. Therefore, the following detailed descriptions are not to be taken in a limiting sense, and the scope of this application is defined by the appended claims. For example, it should be understood that the disclosure in connection with the described method may also be true for the corresponding device or system configured to perform the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may comprise one or more blocks, such as functional blocks, for performing the one or more method steps described (for example, one block performing one or more steps; or a plurality of blocks, each of which performs one or more of the plurality of steps), even if such one or more units are not explicitly described or illustrated in the accompanying drawings. On the other hand, for example, if a particular device is described in terms of one or more blocks, such as functional blocks, the corresponding method may include one step used to perform the functionality of one or more blocks (for example, one step used to perform the functionality of one or multiple blocks or multiple steps, each of which is used to perform the functionality of one or more of the multiple blocks), even if such one or more steps are not explicitly described or illustrated in the accompanying drawings. In addition, it should be understood that features of the various exemplary embodiments and/or aspects described in this specification may be combined with each other, unless specifically indicated otherwise.

[0085] The technical solutions in the embodiments of this application are not only applicable to an existing video coding standard (eg, H.264 or HEVC), but also applicable to a future video coding standard (eg, H.266). The terms used in the DESCRIPTION OF EMBODIMENTS of this application are used only to explain specific embodiments of this application, but are not intended to limit this application. The following first briefly describes related concepts in embodiments of this application.

[0086] Video coding generally refers to the processing of a sequence of images that form a video or video sequence. The term "image (picture)", "frame (frame)" or "image (image)" can be used interchangeably in the field of video coding. Video encoding in this specification is video encoding or video decoding. Video encoding is performed at the source side and typically involves processing (eg, by compressing) the source video to reduce the amount of data required to represent the video for more efficient storage and/or transmission. Video decoding is performed on the destination side and usually contains inverse processing compared to the encoder for video reconstruction. "Encoding" a video image in the embodiments should be understood as "encoding" or "decoding" a video sequence. The combination of an encoding part and a decoding part is also called a codec (encoding and decoding).

[0087] The video sequence contains a series of images (picture), the image is further divided into slices (slice), and the slice is further divided into blocks (block). Video encoding processing is performed in blocks. In some new video coding standards, the concept of a block is further expanded. For example, a macroblock (MB) is introduced into the H.264 standard. A macroblock may be further divided into a plurality of prediction blocks (segments) that may be used for predictive coding. The high efficiency video coding (HEVC) standard uses basic concepts such as a coding unit (CU), a prediction unit (PU), and a transform unit (TU). The set of block units is obtained through functional partitioning and is described using a new tree structure. For example, a CU may be split into smaller CUs based on a quadtree, and the smaller CU may be further split to generate a quadtree structure. CU is the basic unit used to partition and encode an encoded picture. PU and TU also have similar tree structures. The PU may correspond to a prediction block and is the base unit used for predictive coding. The CU is further partitioned into a plurality of PUs based on the partition pattern. A TU may correspond to a transform unit and is the base unit used to transform the prediction residual. However, essentially all CUs, PUs, and TUs are conceptually blocks (or image blocks).

[0088] For example, in HEVC, the CTU is partitioned into multiple CUs using a quadtree structure represented as a coding tree. The decision on whether to encode an image area using inter (temporal) or intra frame (spatial) prediction is made at the CU level. Each CU may be further partitioned into one, two, or four PUs based on a PU partitioning pattern. Within one PU, the same prediction process is applied and the corresponding information is transmitted to the decoder based on the PU. After obtaining the residual block by applying the prediction process based on the PU splitting pattern, the CU can be divided into transform units (TUs) based on a different quadtree structure similar to the coding tree used for the CU. Recent developments in video compression technologies use a Quad-tree and binary tree (QTBT) split frame to split a coding block. In the block structure of QTBT, the CU can be square or rectangular.

[0089] In this specification, for ease of description and understanding, an image block to be encoded in the current encoded image may be referred to as a current block. For example, when encoding, the current block is the block currently being encoded; and when decoding, the current block is the block currently being decoded. A decoded image block, as a reference image used to predict the current block, is called a reference block. In other words, a reference block is a block that provides a reference signal for the current block. The reference signal is a pixel value in the image block. A block that provides a prediction signal for the current block in a reference picture may be referred to as a prediction block. The prediction signal represents a pixel value, a sample value, or a sample signal in a prediction block. For example, after passing through a plurality of reference blocks, an optimal reference block is found, and the optimal reference block provides prediction for the current block, and this block is referred to as a prediction block.

[0090] In the case of lossless video coding, the original video images can be recovered. That is, the reconstructed video images have the same quality as the original video images (assuming no transmission loss or other data loss occurs during storage or transmission). In the case of lossy video coding, additional compression is performed, for example by quantization, to reduce the amount of data required to represent video images, and the video images cannot be fully reconstructed at the decoder side. That is, the quality of the reconstructed video images is lower or worse than the quality of the original video images.

[0091] Some H.261 video coding standards are designed for "hybrid lossy video codecs" (ie, spatial and temporal sample-domain prediction is combined with 2D transform coding to apply transform-domain quantization). Each picture of a video sequence is usually divided into a set of non-overlapping blocks, and encoding is usually done at the block level. In other words, on the encoder side, the video is usually processed, that is, encoded at the block (video block) level. For example, a prediction block is generated by spatial (intra-frame) prediction and temporal (inter-) prediction, the prediction block is subtracted from the current block (the block currently being processed or to be processed) to obtain a residual block, and the residual block is transformed and quantized into conversion area to reduce the amount of data that must be transferred (compressed). On the decoder side, the reverse processing of the encoder is performed on the encoded or compressed block in order to reconstruct the current block for presentation. In addition, the encoder duplicates the processing cycle of the decoder, so that the encoder and decoder generate the same predictions (eg, intra prediction and inter prediction) and/or reconstruction for processing, i.e., for encoding the next block.

[0092] The system architecture used in the embodiments of this application is described below. Fig. 1A is a schematic block diagram of an example of a video encoding and decoding system 10 used in embodiments of this application. As shown in FIG. 1A, video encoding and decoding system 10 may include a source device 12 and a destination device 14. Source device 12 generates encoded video data, and therefore source device 12 may be referred to as video encoding equipment. Destination device 14 may decode encoded video data generated by source device 12, and therefore, destination device 14 may be referred to as video decoding equipment. In various implementations, source device 12, destination device 14, or both source device 12 and destination device 14 may comprise one or more processors and memory associated with one or more processors. Memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store the desired program code in the form of an instruction or data structure accessible to a computer as described in this specification. Source device 12 and destination device 14 may comprise a variety of devices, including a desktop computer, a mobile computing device, a portable computer (for example, a laptop computer), a tablet computer, a set-top box, a handset such as a so-called "smart" phone, television, camera, display device, digital media player, game console, vehicle-mounted computer, wireless communication device, and the like.

[0093] Although FIG. 1A depicts source device 12 and destination device 14 as separate devices, an embodiment of the device may alternatively comprise both source device 12 and destination device 14, or the functionality of both source device 12 and destination device 14, then there is a source device 12 or corresponding functionality and a destination device 14 or corresponding functionality. In such an embodiment, source device 12 or related functionality and destination device 14 or related functionality may be implemented using the same hardware and/or software, separate hardware and/or software, or any combination thereof.

[0094] The communication connection between the source device 12 and the destination device 14 may be implemented via the communication line 13, and the destination device 14 may receive encoded video data from the source device 12 via the communication line 13. Link 13 may include one or more media or devices that can transmit encoded video data from source device 12 to destination device 14. In one example, link 13 may include one or more media that allows source device 12 to directly transmit the encoded video data to the destination device 14 in real time. In this example, source device 12 may modulate encoded video data in accordance with a communication standard (eg, a wireless communication protocol) and may transmit the modulated video data to destination device 14. One or more communication media may comprise a wireless communication medium and/or a wired communication medium. , for example, the radio frequency (RF) spectrum or one or more physical transmission cables. One or more communication media may be part of a packet-based network, and the packet-based network is, for example, a local area network, a wide area network, or a wide area network (eg, the Internet). One or more communication media may include a router, switch, base station, or other device that provides communication from source device 12 to destination device 14.

[0095] Source device 12 includes an encoder 20. Optionally, source device 12 may further comprise an image source 16, an image preprocessor 18, and a communications interface 22. In a specific implementation, encoder 20, image source 16, image preprocessor 18, and communication interface 22 may be hardware components in source device 12, or may be software programs in source device 12. Descriptions are provided separately as follows:

[0096] The image source 16 may comprise or may be any type of image capturing device configured to, for example, capture an image of the real world; and/or any type of device for generating an image or comment (to encode screen content, some text on the screen is also considered part of the picture or image to be encoded), such as a computer graphics processor configured to generate a computer animation image; or any type of device configured to receive and/or provide a real-world image or a computer-animated image (eg, screen content or virtual reality (VR) image); and / or any combination of them (for example, an image of augmented reality (augmented reality, AR)). Image source 16 may be a camera configured to capture an image or a memory configured to store an image. Image source 16 may further comprise any type of interface (internal or external) through which a previously captured or generated image is stored and/or an image is acquired or received. When the image source 16 is a camera, the image source 16 may be, for example, a local camera or an integrated camera built into the source device. When the image source 16 is a memory, the image source 16 may be a local memory or, for example, a built-in memory built into the source device. When the image source 16 contains an interface, this interface may be, for example, an external interface for receiving an image from an external video source. The external video source is, for example, an external image capture device such as a camera, external memory, or an external image generating device. The external image generating device is, for example, an external computer graphics processor, a computer or a server. The interface may be any type of interface, such as a wired or wireless interface, or an optical interface, in accordance with any proprietary or standardized interface protocol.

[0097] An image can be viewed as a two-dimensional array or matrix of pixel elements (image element). A pixel element in an array can also be called a selection. The number of samples in the horizontal and vertical directions (or axes) of an array or image determines the size and/or resolution of the image. Three color components are usually used to represent a color, in particular, an image can be represented as three arrays of samples or contain them. For example, in RBG format or color space, an image contains corresponding arrays of red, green, and blue color samples. However, in video encoding, each pixel is typically represented in a luma/chrominance format or color space. For example, a YUV image contains a luma component labeled Y (sometimes labeled L instead) and two chrominance components labeled U and V. The luminance (luma) component Y represents the brightness or intensity of the gray level (for example, both are the same in a grayscale), and the two chroma components U and V are chrominance or color information components. Accordingly, a YUV image contains an array of luma samples of luminance sample values (Y) and two chrominance sample arrays of chrominance values (U and V). Images in RGB format can be converted or converted to YUV format and vice versa. This process is also called color conversion or conversion. If the image is monochrome, it can only contain an array of brightness samples. In this embodiment of this application, the image transmitted by the image source 16 to the image processor may also be referred to as raw image data 17 .

[0098] The image preprocessor 18 is configured to receive the raw image data 17 and perform pre-processing of the raw image data 17 to obtain a pre-processed image 19 or pre-processed image data 19. For example, the pre-processing performed by the image preprocessor 18 may include cropping, color format conversion (eg, RGB to YUV format), color correction, or noise reduction.

[0099] An encoder 20 (or referred to as a video encoder 20) is configured to receive preprocessed image data 19 and process the preprocessed image data 19 using an appropriate prediction mode (eg, prediction mode in each embodiment of this specification) to provide encoded image data 21 (structural details of encoder 20 are further described below based on FIG. 2, FIG. 4, or FIG. 5). In some embodiments, encoder 20 may be configured to perform each embodiment described below to implement encoder-side application of the chrominance block prediction method described in this application.

[00100] Communication interface 22 may be configured to receive encoded image data 21 and transmit encoded image data 21 to destination device 14 or any other device (eg, memory) over communication line 13 for storage or direct restoration (reconstruction). The other device may be any device used for decoding or storage. The communication interface 22 may, for example, be configured to package the encoded image data 21 into an appropriate format, such as a data packet, for transmission over the communication line 13.

[00101] Destination device 14 includes a decoder 30. Optionally, destination device 14 may further comprise a communication interface 28, an image post-processor 32, and a display device 34. Descriptions are provided separately as follows:

[00102] Communication interface 28 may be configured to receive encoded image data 21 from source device 12 or any other source. Any other source is, for example, a storage device. The storage device is, for example, an encoded image data storage device. Communication interface 28 may be configured to transmit or receive encoded image data 21 over communication line 13 between source device 12 and destination device 14, or over any type of network. Link 13 is, for example, a direct wired or wireless connection. Any type of network is, for example, a wired or wireless network, or any combination thereof, or any type of private or public network, or any combination thereof. The communication interface 28 may, for example, be configured to decompress the data packet transmitted via the communication interface 22 to obtain encoded image data 21 .

[00103] Both communication interface 28 and communication interface 22 may be configured as unidirectional communication interfaces or bidirectional communication interfaces, and may be configured, for example, to send and receive messages to establish a connection, and to acknowledge and exchange any other information related to to a communication link and/or data transmission, such as the transmission of encoded image data.

[00104] Decoder 30 (or referred to as decoder 30) is configured to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (structural details of decoder 30 are further described below based on Fig. 3, Fig. 4 or 5). In some embodiments, decoder 30 may be configured to perform each embodiment described below to implement on the decoder side the chrominance block prediction method described in this application on the decoder side.

[00105] The image post-processor 32 is configured to post-process decoded image data 31 (also referred to as reconstructed (reconstructed) image data) to obtain post-processed image data 33. The post-processing performed by the image post-processor 32 may include color format conversion (eg, from YUV form to RGB format), color correction, cropping, resampling, or any other processing. The image post-processor 32 may be further configured to transfer the post-processed image data 33 to the display device 34 .

[00106] The display device 34 is configured to receive post-processed image data 33 to display the image, for example, to a user or viewer. The display device 34 may be or may include any type of display for presenting the reconstructed image, such as a built-in or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), digital light processor (DLP) or any other display.

[00107] Although FIG. 1A depicts source device 12 and destination device 14 as separate devices, an embodiment of the device may alternatively comprise both source device 12 and destination device 14 or the functionality of both source device 12 and destination device 14, then there is a source device 12 or corresponding functionality and a destination device 14 or corresponding functionality. In such an embodiment, source device 12 or related functionality and destination device 14 or related functionality may be implemented using the same hardware and/or software, separate hardware and/or software, or any combination thereof.

[00108] As will be apparent to one of skill in the art based on the descriptions, existence, and (exact) separation of the functionality of the various blocks or functionality of source device 12 and/or destination device 14 shown in FIG. 1A may differ depending on the actual device and application. Source device 12 and destination device 14 may comprise any of a wide range of devices, including any type of portable or stationary device such as a laptop or laptop computer, mobile phone, smartphone, tablet or tablet computer, camcorder, desktop computer, set-top box, television , a camera, a vehicle-mounted device, a display device, a digital media player, a game console, a video streaming device (such as a content service server or a content delivery server), a broadcast receiver device, or a broadcast transmitter device, and may or may not use any type of operating system.

[00109] Encoder 20 and decoder 30 may each be implemented as one of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs). ), field-programmable gate array (FPGA), discrete logic, hardware, or any combination of these. If the techniques are implemented in part using software, the device may store the program instruction on a suitable non-volatile computer-readable storage medium and may execute the instruction using hardware, such as one or more processors, to implement the techniques of this disclosure. Any of the above content (including hardware, software, combination of hardware and software, etc.) may be considered one or more processors.

[00110] In some cases, the video encoding and decoding system 10 shown in FIG. 1A is merely an example, and the techniques of this application are applicable to video encoding settings (eg, video encoding or video decoding) that do not necessarily involve any communication between an encoding device and a decoding device. In another example, data may be retrieved from local memory, streamed over a network, or the like. The video encoding device may encode the data and store the data in memory, and/or the video decoding device may retrieve the data from the memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data into memory and/or retrieve data from memory and decode the data.

[00111] FIG. 1B is a diagram illustrating an example of a video encoding system 40 including encoder 20 in FIG. 2 and/or decoder 30 in FIG. 3 according to an exemplary embodiment. Video coding system 40 may implement a combination of different technologies in embodiments of this application. In the illustrated implementation, the video coding system 40 may include an imager 41, an encoder 20, a decoder 30 (and/or a video encoder/decoder implemented by the logic circuit 47 of the processing unit 46), an antenna 42, one or more processors 43, one or more memory devices 44 and/or display device 45.

[00112] As shown in FIG. 1B, imager 41, antenna 42, processing unit 46, logic circuit 47, encoder 20, decoder 30, processor 43, memory 44, and/or display device 45 may communicate with each other. As described, although video coding system 40 is illustrated with both encoder 20 and decoder 30, in various examples, video coding system 40 may include only encoder 20 or only decoder 30.

[00113] In some examples, antenna 42 may be configured to transmit or receive an encoded video bitstream. In addition, in some examples, display device 45 may be configured to present video data. In some examples, logic circuit 47 may be implemented by processing unit 46. The processing unit 46 may comprise application-specific integrated circuit (ASIC) logic, a graphics processor, a general purpose processor, and the like. The video encoding system 40 may also include an additional processor 43. Similarly, the additional processor 43 may contain application-specific integrated circuit (ASIC) logic, a graphics processor, a general purpose processor, and the like. In some examples, logic circuitry 47 may be implemented in hardware, such as dedicated video coding hardware, and processor 43 may be implemented using general purpose software, an operating system, or the like. In addition, the memory 44 may be any type of memory, such as volatile memory (for example, static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM)), or non-volatile memory (for example, flash memory). In a non-limiting example, memory 44 may be implemented with a cache. In some examples, logic circuitry 47 may access memory 44 (eg, to implement an image buffer). In other examples, logic circuitry 47 and/or processing unit 46 may contain a memory (eg, cache) for implementing an image buffer or the like.

[00114] In some examples, encoder 20 implemented by logic may include an image buffer (eg, implemented by processing unit 46 or memory 44) and a graphics processing unit (eg, implemented by processing unit 46). The graphics processor may be communicatively coupled to the image buffer. The graphics processing unit may include an encoder 20 implemented by logic 47 to implement various modules which are described with reference to FIG. 2 and/or any other encoder system or subsystem described in this specification. The logic circuit can be configured to perform the various operations described in this specification.

[00115] In some examples, decoder 30 may be implemented by logic 47 in a similar manner to implement various modules that are described with reference to decoder 30 in FIG. 3 and/or any other decoder system or subsystem described in this specification. In some examples, the decoder 30 implemented by the logic circuit may include an image buffer (eg, implemented by processing unit 2820 or memory 44) and a graphics processing unit (eg, implemented by processing unit 46). The graphics processor may be communicatively coupled to the image buffer. The graphics processing unit may include a decoder 30 implemented by logic 47 to implement various modules which are described with reference to FIG. 3 and/or any other decoder system or subsystem described in this specification.

[00116] In some examples, antenna 42 may be configured to receive an encoded video bitstream. As described, the encoded bitstream may contain data, a pointer, an index value, mode selection data, and the like, related to the encoding of video frames described in this specification, such as data related to coding separation (for example, a transform coefficient or a quantized coefficient conversions, an optional pointer (as described), and/or data that defines an encoding split). Video coding system 40 may further comprise a decoder 30 coupled to antenna 42 and configured to decode the encoded bitstream. The display device 45 is configured to display a video frame.

[00117] It should be understood that in this embodiment of this application for the example described with reference to encoder 20, decoder 30 may be configured to perform the reverse process. With regard to syntax element signaling, decoder 30 may be configured to receive and parse such syntax element and decode associated video data accordingly. In some examples, encoder 20 may entropy encode the syntax element in the encoded video bitstream. In such examples, decoder 30 may parse the syntax element and decode the associated video data accordingly.

[00118] It should be noted that the method described in the embodiments of this application is mainly used in the inter prediction process. This process is performed by both encoder 20 and decoder 30. Encoder 20 and decoder 30 in the embodiments of this application may be, for example, an encoder and decoder conforming to a video standard protocol, such as H.263, H.264, HEVC, MPEG-2 , MPEG-4, VP8, or VP9, or a next generation video standard protocol (such as H.266).

[00119] FIG. 2 is a schematic/conceptual block diagram of an example encoder 20 for implementing embodiments of this application. In the example in FIG. 2, the encoder 20 includes a residual calculation block 204, a transform processing block 206, a quantization block 208, an inverse quantization block 210, an inverse transform processing block 212, a reconstruction block 214, a buffer 216, a loop filter block 220, a decoded picture buffer, DPB) 230, a prediction processing unit 260, and an entropy encoding unit 270. The prediction processing unit 260 may comprise an inter prediction unit 244, an intra prediction unit 254, and a mode selection unit 262. Inter prediction block 244 may include a motion estimator and a motion compensation block (not shown). Encoder 20 shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

[00120] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy encoding unit 270 form a direct signal path of the encoder 20, while, for example, the inverse quantization unit 210, the processing unit 212 demapper, decoder 214, buffer 216, loop filter 220, decoded picture buffer (DPB) 230, and prediction processor 260 form the return signal path of the encoder. The reverse encoder signal path corresponds to the decoder signal path (referring to decoder 30 in FIG. 3).

[00121] Encoder 20 receives, for example, via input 202, image 201 or image block 203 for image 201, such as an image in a sequence of images constituting a video or video sequence. The picture block 203 may also be referred to as the current picture block or the picture block to be encoded. Picture 201 may be referred to as the current picture or the picture to be encoded (particularly when encoding a video to distinguish the current picture from other pictures, e.g. previously encoded and/or decoded pictures of the same video sequence, i.e. a video sequence that also includes current image).

[00122] An embodiment of the encoder 20 may include a splitter (which is not shown in FIG. 2) configured to split the image 201 into a plurality of blocks, such as image blocks 203. Image 201 is typically divided into a plurality of non-overlapping blocks. The partition block may be configured to use the same block size for all images in a video sequence and a corresponding block size grid, or to change the block size between images or subsets or groups of images and divide each image into appropriate blocks.

[00123] In one example, prediction processor 260 in encoder 20 may be configured to perform any combination of the separation techniques described above.

[00124] Like image 201, image block 203 is also or may be considered as a two-dimensional array or matrix of samples with sample values, although the size of image block 203 is smaller than the size of image 201. In other words, image block 203 may contain, for example, one array samples (eg, a luminance array in the case of a monochrome image 201), three arrays of samples (eg, one luminance array and two chrominance arrays in the case of a color image), or any other number and/or type of arrays depending on the color format used. The number of samples in the horizontal and vertical directions (or axes) of the image block 203 determines the size of the image block 203 .

[00125] The encoder 20 shown in FIG. 2 is configured for block-by-block encoding of the image 201. For example, encoding and prediction are performed for each block 203 of the image.

[00126] Residual block 204 is configured to calculate residual block 205 based on image block 203 and predictor 265 (more details about predictor 265 are provided below), for example, by subtracting sample values of predictor 265 from sample values of image block 203 per sample. sampling (pixel by pixel) to obtain a residual block 205 in the sampling region.

[00127] The transform processing unit 206 is configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values of the residual block 205 to obtain transform coefficients 207 in the transform domain . The transform coefficient 207 may also be referred to as the residual transform coefficient and represents the residual block 205 in the transform region.

[00128] Transform processing unit 206 may be configured to apply DCT/DST integer approximations, such as the transforms specified in HEVC/H.265. Compared to the orthogonal DCT, such integer approximations are typically scaled based on a factor. To preserve the norm of the residual block that is processed using the forward and inverse transforms, an additional scale factor is applied as part of the transform process. The scaling factor is usually chosen based on some constraints, for example, the scaling factor is a power of two for the shift operation, the bit depth of the transform factor, and a trade-off between accuracy and implementation cost. For example, a particular scaling factor is indicated for inverse transform, for example, by the inverse transform processing unit 212 on the decoder 30 side (and the corresponding inverse transform, for example, by the inverse transform processing unit 212 on the encoder 20 side), and accordingly, the corresponding scale factor can be indicated for direct conversion by the conversion processing unit 206 on the side of the encoder 20.

[00129] The quantizer 208 is configured to quantize the transform coefficients 207 to obtain quantized transform coefficients 209, for example, by applying scalar quantization or vector quantization. The quantized transform coefficient 209 may also be referred to as the quantized residual coefficient 209. The quantization process may reduce the bit depth related to some or all of the transform coefficients 207. For example, an n-bit transform factor may be rounded to an m-bit transform factor during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, scalar quantization may use different scales to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, and a larger quantization step size corresponds to coarser quantization. The appropriate quantization step size may be indicated by a quantization parameter (QP). For example, the quantization parameter may be an index to a predetermined set of suitable quantization step sizes. For example, a smaller quantization parameter may correspond to finer quantization (smaller quantization step size), and a larger quantization parameter may correspond to coarser quantization (larger quantization step size), or vice versa. The quantization may comprise dividing by the quantization step size and the corresponding quantization or inverse quantization, such as performed by inverse quantizer 210, or may comprise multiplying by the quantization step size. In embodiments according to some standards, such as HEVC, the quantization parameter may be used to determine the quantization step size. Typically, the quantization step size can be calculated based on the quantization parameter using a fixed-point approximation of an equation involving division. Additional scaling factors may be introduced for quantization and dequantization to recover the residual block norm, where the residual block norm may be modified due to the scale used in the fixed-point approximation of the equation for the quantization step size and the quantization parameter. In an exemplary implementation, the inverse transform scale may be combined with the dequantization scale. Alternatively, a tuned quantization table may be used and signaled from an encoder to a decoder, eg in a bitstream. Quantization is a lossy operation in which the loss increases as the quantization step size increases.

[00130] The inverse quantizer 210 is configured to apply the inverse quantization of the quantizer 208 to the quantized coefficient to obtain the dequantized coefficient 211, for example, applying based on or using the same quantization step size as the quantizer 208, an inversion of the quantization scheme applied by 208 quantization. The dequantized coefficient 211 may also be referred to as the dequantized residual coefficient 211 and correspond to the transform coefficient 207, although the dequantized coefficient 211 usually differs from the transform coefficient due to losses caused by quantization.

[00131] The inverse transform processing unit 212 is configured to apply an inverse transform to the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST) to obtain block 213 inverse transform in the area of the sample. The inverse transform block 213 may also be referred to as an inverse transform dequantization block 213 or a residual inverse transform block 213 .

[00132] Restoration block 214 (e.g., adder 214) is configured to add inverse transform block 213 (namely, restored residual block 213) to prediction block 265, for example, by adding sample values of recovered residual block 213 and sample values of prediction block 265, to get a reconstructed block 215 in the sample area.

[00133] Optionally, buffer block 216 ("buffer" 216 for short), eg, linear buffer 216, is configured to buffer or store the reconstructed block 215 and the corresponding sample value, eg, for intra prediction. In other embodiments, the encoder may be configured to use the unfiltered reconstructed block and/or the corresponding sample value stored in buffer block 216 to perform any type of estimation and/or prediction, such as intra prediction.

[00134] For example, in an embodiment, encoder 20 may be configured such that buffer block 216 is configured to store a reconstructed block 215 not only used for intra prediction block 254, but also used for loop filter block 220 (which is not shown in FIG. 2), and/or such that, for example, buffer block 216 and decoded picture buffer block 230 form one buffer. In other embodiments, the filtered block 221 and/or a block or sample (not shown in FIG. 2) from the decoded picture buffer 230 is used as a basis or basis for the intra prediction block 254.

[00135] A loop filter block 220 ("loop filter" 220 for short) is configured to filter the reconstructed block 215 to obtain a filtered block 221, smooth pixel transitions, or improve video quality. The loop filter block 220 is designed to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, such as a two-sided filter, an adaptive loop filter (ALF). ), a sharpening or smoothing filter, or a joint filter. Although the loop filter unit 220 is shown in FIG. 2, in another implementation, the loop filter block 220 may be implemented as a post-loop filter. The filtered block 221 may also be referred to as the filtered reconstructed block 221. The decoded picture buffer 230 may store the reconstructed coded block after the loop filter block 220 performs a filtering operation on the reconstructed coded block.

[00136] In an embodiment, encoder 20 (respectively, loop filter block 220) may be configured to output a loop filter parameter (e.g., sample adaptive bias information), for example, directly or after entropy encoding performed by entropy encoding block 270 or any other block. entropy encoding so that decoder 30 can receive and apply the same loop filter parameter for decoding.

[00137] Decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use in encoding video data by encoder 20. DPB 230 may be formed by any of a variety of storage devices, such as dynamic memory with random access (dynamic random access memory, DRAM) (including synchronous DRAM (synchronous DRAM, SDRAM), magnetoresistive RAM (magnetoresistive RAM, MRAM), resistive RAM (resistive RAM, RRAM)) or other types of storage devices. DPB 230 and buffer 216 may be provided by the same storage device or separate storage devices. In the example, the decoded picture buffer (DPB) 230 is configured to store the filtered block 221. The decoded picture buffer 230 can be further configured to store other previously filtered blocks, such as previously restored and filtered blocks 221, of the same current picture. or different pictures, e.g., previously reconstructed pictures, and may provide fully previously reconstructed, i.e., decoded pictures (and corresponding reference blocks and samples) and/or partially reconstructed current picture (and corresponding reference blocks and samples), e.g. for inter prediction . In one example, if the reconstructed block 215 is reconstructed without in-loop filtering, the decoded picture buffer (DPB) 230 is configured to store the reconstructed block 215.

[00138] The prediction processing unit 260, also referred to as the block prediction processing unit 260, is configured to receive or obtain an image block 203 (the current image block 203 of the current image 201) and reconstructed image data, such as reference samples of the same (current) image from the buffer 216 and/or reference picture data 231 of one or more previously decoded pictures from decoded picture buffer 230, and processing such data for prediction, namely to provide prediction block 265, which may be inter prediction block 245 or intra prediction block 255.

[00139] The mode selector 262 may be configured to select a prediction mode (e.g., intra or inter prediction mode) and/or an appropriate prediction block 245 or 255 to be used as prediction block 265, to compute the residual block 205, and to recover refurbished unit 215.

[00140] In an embodiment, mode selector 262 may be configured to select a prediction mode (e.g., from among the prediction modes supported by prediction processor 260) where the prediction mode provides the best match, or in other words, minimum residual (minimum residual means better compression). for transmission or storage), or provides minimal signaling overhead (lower signaling overhead means better compression for transmission or storage), or considers or balances both. The mode selector 262 may be configured to determine a prediction mode based on rate distortion optimization (RDO), i.e., selecting a prediction mode that provides minimal rate distortion, or selecting a prediction mode for which the associated rate distortion is at least , satisfies the prediction mode selection criterion.

[00141] The following describes in detail the prediction processing (eg, performed by prediction processing block 260) and mode selection (eg, performed by mode selection block 262) that are performed by the example for encoder 20.

[00142] As described above, encoder 20 is configured to determine or select the best or optimal prediction mode from a set of (predetermined) prediction modes. The set of prediction modes may include, for example, an intra prediction mode and/or an inter prediction mode.

[00143] The intra prediction mode set may contain 35 different intra prediction modes, such as non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in H.265, or may contain 67 different modes intra prediction, such as non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in H.266 under development.

[00144] In an exemplary implementation, the set of inter prediction modes depends on the available reference pictures (namely, for example, at least partially decoded pictures stored in DBP 230 as described above), and other inter prediction parameters, for example, depends on , the entire reference image or just a part of the reference image, e.g., the area of the search box around the area of the current block, is used to find the most suitable reference block and/or, for example, depends on whether pixel interpolation is applied, such as half-pixel and/or quarter-pixel interpolation . The set of inter prediction modes may include, for example, an Advanced Motion Vector Prediction (AMVP) mode and a merge mode. In a particular implementation, the set of inter prediction modes may comprise an AMVP mode based on a refined checkpoint and a merge mode based on a refined checkpoint in embodiments of this application. In an example, intra prediction block 254 may be configured to perform any combination of inter prediction techniques described below.

[00145] In addition to the aforementioned prediction modes, the skip mode and/or direct mode may also be applied in the embodiments of this application.

[00146] Prediction processing unit 260 may be further configured to divide image block 203 into smaller block parts or sub-blocks, for example, by iteratively using quad-tree (QT), binary tree (binary-tree, BT), splitting a ternary tree (triple-tree, TT) or any combination thereof, and performing, for example, a prediction for each of the sections of blocks or subblocks. The mode selection comprises selecting the tree structure of the divided image block 203 and selecting the prediction mode used for each of the block partitions or subblocks.

[00147] Inter prediction block 244 may comprise a motion estimation (ME) block (which is not shown in FIG. 2) and a motion compensation (MC) block (which is not shown in FIG. 2). The motion estimator is configured to receive or obtain an image block 203 (the current image block 203 of the current image 201) and a decoded image 231 or at least one or more previously reconstructed blocks, e.g., one or more reconstructed blocks of other/different previously decoded images 231 for movement estimates. For example, the video sequence may comprise the current picture and the previously decoded picture 31. In other words, the current picture and the previously decoded picture 31 may be part of or form a sequence of pictures that make up the video sequence.

[00148] For example, encoder 20 may be configured to select a reference block from a plurality of reference blocks of the same image or different images from a plurality of other images, and provide a reference image and/or offset (spatial offset) between location (X, Y coordinate) of the reference block and the location of the current block as the inter prediction parameter for the motion estimation block (which is not shown in FIG. 2). This offset is also called the motion vector (MV).

[00149] The motion compensation block is configured to obtain an inter prediction parameter and perform inter prediction based on or by using the inter prediction parameter to obtain the inter prediction block 245 . The motion compensation performed by the motion compensator (which is not shown in FIG. 2) may comprise sampling or generating a prediction block based on the motion vector/block determined by the motion estimation (perhaps performing subpixel-accurate interpolation). Interpolation filtering may generate additional pixel samples from known pixel samples. This potentially increases the number of candidate prediction blocks that can be used to encode an image block. After receiving the motion vector for the PU of the current picture block, the motion compensator 246 may find the prediction block pointed to by the motion vector in one of the reference picture lists. The motion compensation block 246 may also generate a syntax element associated with the video block and slice, such that decoder 30 uses the syntax element to decode the image block in the video slice.

[00150] In particular, inter prediction block 244 may pass the syntax element to entropy encoding block 270 . The syntax element contains an inter prediction parameter (eg, inter prediction mode selection indication information used to predict the current block after passing a plurality of inter prediction modes). In an exemplary application scenario, if there is only one inter-prediction mode, the inter-prediction parameter may not be carried in the syntax element. In this case, the decoder side 30 can directly perform decoding using the default prediction mode. It can be appreciated that inter predictor 244 may be configured to perform any combination of inter prediction technologies.

[00151] The intra prediction unit 254 is configured to obtain, for example, receiving an image block 203 (the current image block) and one or more previously reconstructed blocks, such as reconstructed adjacent blocks, of the same image for intra estimation. For example, encoder 20 may be configured to select an intra prediction mode from a plurality of (predetermined) intra prediction modes.

[00152] In an embodiment, encoder 20 may be configured to select an intra prediction mode according to an optimization criterion, such as based on minimum residual (eg, intra prediction mode providing prediction block 255 that is most similar to current picture block 203) or minimum distortion speed.

[00153] The intra prediction block 254 is further configured to determine the intra prediction block 255 based on, for example, an intra prediction parameter in the selected intra prediction mode. In any case, after selecting the intra prediction mode for the block, the intra prediction block 254 is further configured to provide an intra prediction parameter, that is, information indicating the selected intra prediction mode for the block, to the entropy encoding block 270 . In an example, intra predictor 254 may be configured to perform any combination of intra prediction technologies.

[00154] In particular, the intra prediction block 254 may pass the syntax element to the entropy encoding block 270 . The syntax element contains an intra prediction parameter (eg, information indicating selection of an intra prediction mode used to predict the current block after passing through a plurality of intra prediction modes). In an exemplary application scenario, if there is only one intra prediction mode, the intra prediction parameter may not be carried in the syntax element. In this case, the decoder side 30 can directly perform decoding using the default prediction mode.

[00155] Entropy encoding unit 270 is configured to apply (or not apply) an entropy encoding algorithm or scheme (e.g., variable length coding (VLC) scheme, context adaptive VLC (CAVLC) scheme), arithmetic coding schemes, context adaptive binary arithmetic coding (CABAC) schemes, syntax-based context-adaptive binary arithmetic coding (SBAC) schemes, entropy coding schemes with probabilistic interval division (probability interval partitioning entropy, PIPE) or other entropy coding methodology or technology) to one or all of the quantized residual coefficient 209, an inter prediction parameter, an intra prediction parameter, and/or a loop filter parameter to obtain encoded image data 21 that may not be output via output 272, for example, in the form of an encoded bitstream 21. The encoded bitstream may be transmitted to video decoder 30 or stored for later transmission or retrieval by video decoder 30. The entropy encoding unit 270 may be further configured to entropy encode another syntax element for the current video fragment being encoded.

[00156] Other structural variations of the video encoder 20 may be used to encode the video stream. For example, non-transform-based encoder 20 may quantize the residual signal directly without transform processing unit 206 for some blocks or frames. In another implementation, the encoder 20 may have a quantizer 208 and an inverse quantizer 210 combined into a single block.

[00157] In particular, in this embodiment of this application, the encoder 20 may be configured to implement the inter prediction method described in the following embodiments.

[00158] It should be understood that other structural variations of the video encoder 20 may be used to encode the video stream. For example, for some image blocks or image frames, video encoder 20 may directly quantize the residual signal. In this case, processing by the conversion processing unit 206 is not required, and accordingly, processing by the inverse conversion processing unit 212 is also not required. Alternatively, for some image blocks or image frames, video encoder 20 does not generate residual data. Accordingly, in this case, processing by the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 is not required. Alternatively, video encoder 20 may directly store the reconstructed image block as a reference block. In this case, processing by the filter 220 is not required. Alternatively, quantizer 208 and inverse quantizer 210 in video encoder 20 may be combined. Loop filter 220 is optional. In addition, in the case of lossless compression coding, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 are also optional. It should be understood that inter predictor 244 and intra predictor 254 may be selectively used in different application scenarios.

[00159] FIG. 3 is a schematic/conceptual block diagram of an example decoder 30 for implementing embodiments of this application. Video decoder 30 is configured to receive, for example, encoded image data (eg, encoded bitstream) 21 obtained by encoding by encoder 20 to obtain decoded image 231. In the decoding process, video decoder 30 receives video data from video encoder 20, a video stream that represents an image block in an encoded video slice and its associated syntax element.

[00160] In the example of FIG. 3, the decoder 30 includes an entropy decoding block 304, an inverse quantization block 310, an inverse transform processing block 312, a reconstruction block 314 (e.g., adder 314), a buffer 316, a loop filter 320, a decoded image buffer 330, and a prediction processing block 360. The prediction processing unit 360 may comprise an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. In some examples, video decoder 30 may perform a decoding step that is typically the reverse of the encoding step described with reference to video encoder 20 in FIG. 2.

[00161] The entropy decoding unit 304 is configured to entropy decode the encoded image data 21 to obtain, for example, a quantized coefficient 309 and/or a decoded coding parameter (which is not shown in FIG. 3), for example, any one or all of the inter prediction parameter, an intra prediction parameter, a loop filter parameter, and/or another syntax element (which are decoded). The entropy decoding block 304 is further configured to forward an inter prediction parameter, an intra prediction parameter, and/or another syntax element to the prediction processing block 360 . Video decoder 30 may receive a syntax element at a video slice level and/or a picture block level.

[00162] Inverse quantizer 310 may be identical in function to inverse quantizer 110, inverse transform processor 312 may be identical in function to inverse transform processor 212, restorer 314 may be identical in function to restorer 214, buffer 316 may be identical in function of buffer 216, loop filter 320 may be identical in function to loop filter 220, and decoded image buffer 330 may be identical in function to decoded image buffer 230.

[00163] The prediction processing block 360 may comprise an inter prediction block 344 and an intra prediction block 354 . Inter prediction block 344 may be similar in function to inter prediction block 244, and intra prediction block 354 may be similar in function to intra prediction block 254. Prediction processing unit 360 is typically configured to perform block prediction and/or obtain prediction block 365 from encoded data 21, and to receive or obtain (explicitly or implicitly) a prediction-related parameter and/or information about the selected prediction mode, for example, from block 304 entropy decoding.

[00164] When a video slice is encoded into an intra (I) slice, intra prediction block 354 in prediction processing block 360 is configured to generate prediction block 365 for a picture block in the current video slice based on the signaled intra prediction mode and data of the previously decoded block of the current frame or image. When a video frame is encoded into an inter-coded (namely, B or P) slice, an inter prediction block 344 (e.g., a motion compensation block) in the prediction processing block 360 is configured to generate a video block predictor 365 in the current video slice based on the motion vector and another a syntax element that is received from the entropy decoder 304 . In inter prediction, a prediction block may be generated from a reference picture in a reference picture list. The video decoder 30 may create the reference frame lists, list 0 and list 1, using the default generation technique and based on the reference pictures stored in the DPB 330.

[00165] The prediction processing unit 360 is configured to determine prediction information of a video block in the current video slice by parsing the motion vector and another syntax element, and generating, using the prediction information, the prediction block of the current video block that is being decoded. In an example of this application, prediction processing unit 360 determines, using some received syntax elements, a prediction mode (eg, intra prediction or inter prediction) for encoding a video block in a video slice, an inter prediction slice type (eg, B-slice, P-slice, or GPB slice), information about the construction of one or more reference picture lists for the slice, the motion vector of each externally coded video block in the slice, the inter prediction status of each externally coded video block in the slice, and other information for decoding the video block in the current video slice. In another example of this disclosure, the syntax element received by the video decoder 30 from the bitstream contains the syntax element in one or more adaptive parameter sets (APS), a sequence parameter set (SPS), a picture parameter set (picture parameter set, PPS) or fragment header.

[00166] The inverse quantizer 310 may be configured to perform inverse quantization (namely, dequantization) of the quantized transform coefficient provided in the bitstream and decoded by the entropy decoder 304 . The inverse quantization process may comprise using a quantization parameter calculated by video encoder 20 for each video block in the video slice to determine the amount of quantization to be applied as well as the amount of inverse quantization to be applied.

[00167] The inverse transform processing unit 312 is configured to apply an inverse transform (eg, inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficient to generate a residual block in the pixel region.

[00168] A reconstruction (reconstruction) block 314 (e.g., an adder 314) is configured to add an inverse transform block 313 (namely, a reconstructed residual block 313) to a prediction block 365, for example, by adding sample values of the reconstructed residual block 313 and block sample values 365 prediction to obtain a reconstructed block 315 in the sample area.

[00169] The loop filter block 320 (either in the encoding loop or after the encoding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321, smooth pixel transitions, or improve video quality. In one example, loop filter unit 320 may be configured to perform any combination of the filter technologies described below. The loop filter block 320 is designed to represent one or more loop filters, such as a deblocking filter, a sample-adaptive bias (SAO) bias filter, or other filters, such as a two-sided filter, an adaptive loop filter. , ALF), sharpen or smooth filter, or collaborative filter. Although the loop filter unit 320 is shown in FIG. 3, in another implementation, the loop filter block 320 may be implemented as a post-loop filter.

[00170] Then, the decoded video block 321, in a given frame or picture, is stored in the decoded picture buffer 330, which stores a reference picture used for subsequent motion compensation.

[00171] Decoder 30 is configured to output, for example, decoded image 31 via output 332 for presentation or viewing to a user.

[00172] Other variants of video decoder 30 may be used to decode the compressed bitstream. For example, decoder 30 may generate an output video stream without loop filter block 320 . For example, a non-transform-based decoder 30 may directly inverse-quantize the residual signal without the inverse-transform processing unit 312 for some blocks or frames. In another implementation, the video decoder 30 may have an inverse quantizer 310 and an inverse transform processor 312 combined into one block.

[00173] In particular, in this embodiment of this application, the decoder 30 may be configured to implement the inter prediction method described in the following embodiments.

[00174] It should be understood that other structural variations of the video decoder 30 may be used to decode the encoded video bitstream. For example, video decoder 30 may generate an output video stream without processing by filter 320. Alternatively, for some image blocks or image frames, entropy decoding module 304 in video decoder 30 does not obtain a quantized coefficient through decoding. Accordingly, in this case, the processing by the inverse quantizer 310 and the inverse transform processor 312 is not required. Loop filter 320 is optional. In addition, in the case of lossless compression, the inverse quantization unit 310 and the inverse transform processing unit 312 are also optional. It should be understood that in different application scenarios, the inter prediction block and the intra prediction block may be selectively used.

[00175] It should be understood that in the encoder 20 and decoder 30 in this application, the processing result of a step may be further processed and then output to the next step. For example, after a step such as interpolation filtering, motion vector derivation, or loop filtering, an additional operation such as clipping or shifting is performed on the processing result of the corresponding step.

[00176] For example, a motion vector that is a reference point of the current image block and that is derived based on the motion vector of an adjacent affine-coded block may be further processed. In this application, it is expressly not limited. For example, the value of the motion vector is limited to a specific bit range. Assuming the allowed bit depth of the motion vector is bitDepth, the value of the motion vector is in the range -2^(bitDepth-1) to 2^(bitDepth-1)-1, where the symbol "^" represents exponentiation. If bitDepth is 16, the value is in the range -32768 to 32767. If bitDepth is 18, the value is in the range -131072 to 131071. The motion vector value can be limited by one of the following two methods:

[00177] Method 1: The most significant bit of the motion vector overflow is removed:

[00178] For example, the value of vx is -32769, and 32767 is output in accordance with the formulas above. The value is stored on the computer in two's complement representation, the two's complement representation of -32769 is 1.0111,1111,1111,1111 (17 bits), and the computer's overflow processing discards the most significant bit. Therefore, the value of vx is 0111,1111,1111,1111, which is 32767. This value is consistent with the result obtained from formula processing.

[00179] Method 2: Clipping is performed on the motion vector, and the following formulas are used:

vx = Clip3(–2 ^{bitDepth – 1} , 2 ^{bitDepth – 1} – 1, vx)

vy = Clip3(–2 ^{bitDepth – 1} , 2 ^{bitDepth – 1} – 1, vy)

[00180] In the formulas above, Clip3 is defined as the truncation of the z value to the range [x, y].

Clip3(x, y, z) =

[00181] FIG. 4 is a schematic block diagram of a video encoding apparatus 400 (eg, a video encoding apparatus 400 or a video decoding apparatus 400) according to an embodiment of this application. The video encoding device 400 is suitable for implementing the embodiments described in this specification. In an embodiment, video encoding device 400 may be a video decoder (eg, decoder 30 in FIG. 1A) or a video encoder (eg, encoder 20 in FIG. 1A). In another embodiment, video encoding device 400 may be one or more components of decoder 30 in FIG. 1A or encoder 20 in FIG. 1A.

[00182] The video encoding device 400 includes: input ports 410 and a receiver (Rx) 420 that are configured to receive data; a processor, logical unit, or central processing unit (CPU) 430 that is configured to process data; transmitter (Tx) 440 and output ports 450 that are configured to transmit data; and a memory 460 configured to store data. The video encoding device 400 may further comprise opto-electrical components and electrical-optical (EO) components that are connected to input ports 410, receiver 420, transmitter 440, and output ports 450 to output or input an optical or electrical signal.

[00183] The processor 430 is implemented in hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (eg, a multi-core processor), FPGA, ASIC, and DSP. Processor 430 communicates with input ports 410, receiver 420, transmitter 440, output ports 450, and memory 460. Processor 430 includes an encoding module 470 (eg, encoding module 470 or decoding module 470). The encoding/decoding module 470 implements the embodiments disclosed in this specification to implement the chrominance block prediction method provided in the embodiments of this application. For example, the encoding/decoding module 470 implements, processes, or provides various encoding operations. Therefore, the encoding/decoding unit 470 provides a significant improvement in the function of the video encoding device 400 and affects the transition (conversion) of the video encoding device 400 to another state. Alternatively, encoder/decoder module 470 is implemented as an instruction that is stored in memory 460 and executed by processor 430.

[00184] Memory 460 includes one or more disks, tape drives, and solid state drives and can be used as an overflow storage device, for storing programs when such programs are selected for execution, and for storing instructions and data read during program execution. The memory 460 may be volatile and/or nonvolatile and may be read only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and/or static random access memory (SRAM). ).

[00185] FIG. 5 is a simplified block diagram of a device 500 that can be used as one or both of source device 12 and destination device 14 in FIG. 1A according to an exemplary embodiment. Device 500 may implement the technologies of this application. In other words, FIG. 5 is a schematic block diagram of an implementation of an encoder or decoder (called encoder 500 for short) according to an embodiment of this application. Encoder 500 may include a processor 510, a memory 530, and a bus system 550. The processor and memory are connected via a bus system. The memory is configured to store the instruction. The processor is configured to execute an instruction stored in memory. The program code is stored in the memory of the encoder. The processor may call the program code stored in memory to perform various video encoding or decoding methods described in this application. To avoid repetition, the details are not described here.

[00186] In this embodiment of this application, processor 510 may be a central processing unit (Central Processing Unit, "CPU" for short). Alternatively, processor 510 may be another general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistorized logic device, discrete hardware component, and etc. A general purpose processor may be a microprocessor, any conventional processor, or the like.

[00187] The memory 530 may comprise a read only memory (ROM) device or a random access memory (RAM) device. Alternatively 530, any other suitable type of storage device may be used. Memory 530 may contain code and data 531 that is accessed by processor 510 via bus 550. Memory 530 may further comprise an operating system 533 and application program 535. Application program 535 contains at least one program that allows processor 510 to perform an encoding method or video decoding described in this application. For example, application programs 535 may comprise applications 1 through N. The applications further comprise a video encoding or decoding application (referred to as a video encoding application for short) that performs the video encoding or decoding method described in this application.

[00188] The bus system 550 may not only include a data bus, but also include a power bus, a control bus, a status signal bus, and the like. However, for a clear description, the various types of tires are referred to in the drawing as busbar system 550.

[00189] Optionally, encoder 500 may further comprise one or more output devices, such as a display 570. In one example, display 570 may be a touch display that combines a display and a touch sensor unit that can accept touch input. Display 570 may be connected to processor 510 via bus 550.

[00190] The following first describes the concepts in this application.

[00191] 1. Inter prediction mode:

[00192] HEVC uses two inter-prediction modes: advanced motion vector prediction (AMVP) mode and merge mode.

[00193] In the AMVP mode, the spatial or temporal adjacent coded blocks (denoted as contiguous blocks) of the current block are traversed first; a list of candidate motion vectors (which may also be called a list of candidate motion information) is created based on the motion information of neighboring blocks; and then the optimal motion vector is determined from the list of candidate motion vectors based on the velocity distortion cost. The candidate motion information corresponding to the minimum distortion cost is used as the motion vector predictor (MVP) of the current block. Both locations of adjacent blocks and the order in which they pass are predetermined. The rate distortion cost is calculated according to formula (1), where J is the rate distortion cost (RD cost), SAD is the sum of absolute differences (SAD) between the original sample values and the prediction sample values obtained by motion estimation using a candidate motion vector predictor, R represents the bit rate and λ represents the Lagrange multiplier. The encoder side transmits the index value of the selected motion vector predictor in the list of candidate motion vectors and the index value of the reference frame to the decoder side. Additionally, a motion search is performed in the neighborhood centered on the MVP to obtain the actual motion vector of the current block. The encoder side transmits the difference (motion vector difference) between the MVP and the actual motion vector to the decoder side.

(1)

(1)

[00194] In the merge mode, a list of candidate motion vectors is first created based on the motion information of the spatial or temporal neighboring coded blocks of the current block. Then, the optimal motion information is determined from the list of candidate motion vectors as the motion information of the current block based on the rate distortion cost. An index value (hereinafter referred to as the merge index) of the location of the optimal motion information in the list of candidate motion vectors is transmitted to the decoder side. Information about the spatial and temporal candidate motion of the current block is shown in FIG. 6A. Information about the candidate motion in space comes from five spatial neighboring blocks (A0, A1, B0, B1 and B2). If a neighbor block is not available (the neighbor block does not exist, or the neighbor block is not encoded, or the prediction mode used for the neighbor block is not an inter prediction mode), motion information of the neighbor block is not added to the list of candidate motion vectors. The temporal information about the candidate motion of the current block is obtained by scaling the MV block at the corresponding location in the reference frame based on the picture order count (POC) of the reference frame and the current frame. First, it is determined whether the block at location T in the reference frame is available. If the block is not available, the block at location C is selected.

[00195] Similar to the AMVP mode, in the merge mode, both locations of adjacent blocks and their traversal order are also predetermined. In addition, the locations of neighboring blocks and the order in which they are passed may differ in different modes.

[00196] It can be recognized that one list of candidate motion vectors needs to be maintained in both AMVP mode and merge mode. Before adding new motion information to the list of candidates each time, it is first checked whether the same motion information exists in the list. If the same traffic information exists in the list, the traffic information is not added to the list. This validation process is referred to as shortening the list of candidate motion vectors. The reduction of the list is intended to avoid having the same motion information in the list to avoid overcomputing the speed distortion overhead.

[00197] In HEVC inter-prediction, the same motion information is used for all coding block samples, and then motion compensation is performed based on the motion information to obtain coding block sample predictors. However, in a coding block, not all samples have the same motion characteristics. Using the same motion information can result in inaccurate motion compensation prediction and more residual information.

[00198] Existing video coding standards use block matching motion estimation based on a straight-line motion model, and it is assumed that the motion of all samples in a block is consistent. However, in the real world, there are many movements. Many objects are in non-rectilinear motion, such as a spinning object, multi-directional roller coasters, fireworks, and some movie stunts, especially a moving object in a UGC scenario. For these moving objects, if the block motion compensation technology based on the rectilinear motion model in existing coding standards is used for encoding, the encoding efficiency may suffer greatly. In this regard, to improve the coding efficiency, a model of non-rectilinear motion is introduced, for example, an affine motion model.

[00199] Based on this, in terms of different traffic patterns, the AMVP mode can be classified into a straight-line model based AMVP mode and a non-rectiline model based AMVP mode, and the merge mode can be classified into a straight-line model based merge mode and a merge mode based on a non-rectilinear model.

[00200] 2. Non-rectilinear motion model:

[00201] Non-rectilinear motion model prediction means that the same motion model is used on both the encoder side and the decoder side to obtain the motion information of each sub-block of the current block, and motion compensation is performed based on the motion information of the sub-block for get a prediction block. Thus, the prediction efficiency is improved. Conventional curved motion models include the 4-parameter affine motion model and the 6-parameter affine motion model.

[00202] A sub-block in embodiments of this application may be a sample or block of an N1×N2 sample obtained using a particular partitioning method, where both N1 and N2 are positive integers, and N1 may or may not be equal to N2.

[00203] An affine motion model with 4 parameters is expressed by formula (2):

(2)

(2)

[00204] A 4-parameter affine motion model can be represented by motion vectors of two samples and their coordinates relative to the top left sample of the current block. The sample used to represent a motion model parameter is called a control point. If the top left sample (0, 0) and the top right sample (W, 0) are used as control points, first determine the corresponding motion vectors (vx0, vy0) and (vx1, vy1) top left vertex and top right vertex of the current block. Then, the motion information of each sub-block of the current block is obtained according to Formula (3), where (x, y) represents the coordinates of the sub-block relative to the top left sample of the current block, and W represents the width of the current block.

(3)

(3)

[00205] An affine motion model with 6 parameters is expressed by formula (4):

(4)

(four)

[00206] A 6-parameter affine motion model can be represented by motion vectors of three samples and their coordinates relative to the top left sample of the current block. If the top left sample (0, 0), top right sample (W, 0) and bottom left sample (0, H) of the current block are used as control points, the corresponding vectors are first determined movements (vx0, vy0), (vx1, vy1) and (vx2, vy2) upper left breakpoint, top right breakpoint, and bottom left breakpoint of the current block. Then, the motion information of each sub-block of the current block is obtained according to Formula (5), where (x, y) represents the coordinates of the sub-block relative to the top left sample of the current block, and W and H represent the width and height of the current block, respectively.

(5)

(five)

[00207] A coding block that is predicted using an affine motion model is called an affine-coded block.

[00208] Typically, the motion information of an affine-coded block checkpoint can be obtained using an affine motion model based on an Advanced Motion Vector Prediction (AMVP) mode or a merge mode based on an affine model. movement.

[00209] The checkpoint motion information of the current coding block can be obtained using a checkpoint inherited motion vector prediction method or a checkpoint constructed motion vector prediction method.

[00210] 3. Method for predicting the legacy motion vector of the checkpoint:

[00211] A method for predicting a legacy checkpoint motion vector is to use a motion model of an adjacent encoded affine-coded block to determine candidate checkpoint motion vectors of the current block.

[00212] The current block shown in FIG. 6B is used as an example. Blocks at neighboring locations around the current block are traversed in a specific order, for example, A1->B1->B0->A0->B2, to find an affine-coded block that includes the block at the current block's neighboring location and to obtain information about movement of the control point of the affine-coded block. In addition, a checkpoint motion vector (used in the merge mode) or checkpoint motion vector predictor (used in the AMVP mode) of the current block is derived using a motion model built based on the checkpoint motion information of the affine-coded block. The order A1->B1->B0->A0->B2 is just used as an example. This application may also use a different combination order. In addition, blocks at adjacent locations are not limited to A1, B1, B0, A0, and B2.

[00213] The block at an adjacent location may be a sample or a sample block of a predetermined size obtained using a particular partitioning method. For example, the sample block may be a 4×4 sample block, a 4×2 sample block, or other size sample block. It's not limited here.

[00214] The following describes the determination process using A1, and similar processes can be used for other cases.

[00215] As shown in FIG. 4, if the coding block in which A1 is located is an affine-coded block with 4 parameters, the motion vector (vx4, vy4) of the upper left vertex (x4, y4) and the motion vector (vx5, vy5) of the upper right vertex of the affine-encoded block. The motion vector (vx0, vy0) of the upper left vertex (x0, y0) of the current affine-coded block is calculated in accordance with formula (6), and the motion vector (vx1, vy1) of the upper right vertex (x1, y1) of the current affine-coded block calculated according to Formula (7).

(6)

(6)

(7)

(7)

[00216] The combination of the motion vector (vx0, vy0) of the upper left vertex (x0, y0) and the motion vector (vx1, vy1) of the upper right vertex (x1, y1) of the current block, which are obtained based on the affine-coded block in which the A1, these are the candidate motion vectors of the control point of the current block.

[00217] If the coding block in which A1 is located is an affine-coded block with 6 parameters, the motion vector (vx4, vy4) of the upper left vertex (x4, y4), the motion vector (vx5, vy5) of the upper right vertex (x5 , y5) and the motion vector (vx6, vy6) of the lower left vertex (x6, y6) of the affine-coded block. The motion vector (vx0, vy0) of the upper left vertex (x0, y0) of the current block is calculated according to Formula (8), the motion vector (vx1, vy1) of the upper right vertex (x1, y1) of the current block is calculated according to Formula (9), and the motion vector (vx2, vy2) of the lower left vertex (x2, y2) of the current block is calculated according to Formula (10).

(8)

(8)

(9)

(nine)

(10)

(10)

[00218] Combination of the motion vector (vx0, vy0) of the upper left vertex (x0, y0), the motion vector (vx1, vy1) of the upper right vertex (x1, y1), and the motion vector (vx2, vy2) of the lower left vertex (x2, y2 ) of the current block, which are derived from the affine-coded block in which A1 resides, is a candidate motion vector of the current block checkpoint.

[00219] It should be noted that other traffic patterns, candidate locations, and search and traverse orders may also be used in this application. Details are not described in the embodiments of this application.

[00220] It should be noted that this application can also use methods to represent motion models of neighboring and current coding blocks based on other breakpoints. Details are not described here.

[00221] 4. Method 1 for predicting the built checkpoint motion vector(s) built checkpoint motion vectors:

[00222] A method for predicting the built checkpoint motion vector is to combine the motion vectors of neighboring coded blocks around the current block's checkpoint as the checkpoint motion vector of the current affine-coded block, regardless of whether the neighboring coded blocks are affine-coded blocks.

[00223] The motion vectors of the top left vertex and top right vertex of the current block are determined based on information about the movement of adjacent coded blocks around the current coding block. Fig. 6C is used as an example to describe a method for predicting a built reference point motion vector. It should be noted that FIG. 6C is just being used as an example.

[00224] As shown in FIG. 6C, the motion vectors of neighboring coded blocks A2, B2 and B3 in the upper left corner are used as candidate motion vectors for the motion vector of the upper left vertex of the current block; and the motion vectors of neighboring encoded blocks B1 and B0 in the upper right corner are used as candidate motion vectors for the motion vector of the upper right vertex of the current block. The candidate top left and top right motion vectors are combined to form a 2-tuple set of spaces. The motion vectors of the two encoded blocks contained in the 2-tuple space can be used as candidate motion vectors of the current block reference point, as shown in the following Formula (11A):

(11A)

(11A)

представляет вектор движения A2,

представляет вектор движения B1,

представляет вектор движения B0,

представляет вектор движения B2, а

представляет вектор движения B3.where

represents the motion vector A2,

represents the motion vector B1,

represents the motion vector B0,

represents the motion vector B2, and

represents the motion vector B3.

[00225] As shown in FIG. 6C, the motion vectors of neighboring coded blocks A2, B2 and B3 in the upper left corner are used as candidate motion vectors for the motion vector of the upper left vertex of the current block; the motion vectors of neighboring encoded blocks B1 and B0 in the upper right corner are used as candidate motion vectors for the motion vector of the upper right vertex of the current block; and the motion vectors of neighboring encoded blocks A0 and A1 in the lower left corner are used as candidate motion vectors for the motion vector of the lower left vertex of the current block. The candidate motion vectors of the top left vertex, top right vertex, and bottom left vertex are combined into a 3-tuple space. The motion vectors of the three encoded blocks contained in the 3-tuple space can be used as candidate motion vectors of the current block reference point, as shown in the following formulas (11B) and (11C):

(11B)

(11B)

(11C)

(11C)

представляет вектор движения A2,

представляет вектор движения B1,

представляет вектор движения B0,

представляет вектор движения B2,

представляет вектор движения B3,

представляет вектор движения A0, и

представляет вектор движения A1.where

represents the motion vector A2,

represents the motion vector B1,

represents the motion vector B0,

represents the motion vector B2,

represents the motion vector B3,

represents the motion vector A0, and

represents the motion vector A1.

[00226] It should be noted that other methods of combining motion vectors of control points can also be used in this application. Details are not described here.

[00227] It should be noted that this application can also use methods to represent motion models of neighboring and current coding blocks based on other breakpoints. Details are not described here.

[00228] 5. Method 2 for predicting the built reference point motion vector(s) as shown in FIG. 6D:

[00229] Step 601: Get information about the movement of control points of the current block.

[00230] For example, in FIG. 6C CPk (k=1, 2, 3, 4) represents the k-th control point. A0, A1, A2, B0, B1, B2, and B3 are spatial neighbor locations of the current block and are used to predict CP1, CP2, or CP3. T is the temporal neighbor location of the current block and is used for CP4 prediction.

[00231] The coordinates CP1, CP2, CP3, and CP4 are assumed to be (0, 0), (W, 0), (H, 0), and (W, H), respectively, where W and H represent the width and height of the current block .

[00232] For each checkpoint, checkpoint movement information is obtained in the following order:

[00233] (1) For CP1, the check order is: B2->A2->B3. If B2 is available, B2 traffic information is used for CP1. Otherwise, A2 and B3 are checked. If traffic information of all three locations is not available, traffic information of CP1 cannot be obtained.

[00234] (2) For CP2, the check order is B0->B1. If B0 is available, B0 motion information is used for CP2. Otherwise, B1 is checked. If traffic information of both locations is not available, CP2 traffic information cannot be obtained.

[00235] (3) For CP3, check order: A0->A1.

[00236] (4) For CP4, motion information T is used.

[00237] Here, that X is available means that a block in X (X is A0, A1, A2, B0, B1, B2, B3, or T) has already been encoded, and inter prediction mode is used for the block. Otherwise, X is not available.

[00238] It should be noted that this application can also use other methods to obtain information about the movement of the control point. Details are not described here.

[00239] Step 602: Combine checkpoint motion information to obtain motion information of the constructed checkpoints.

[00240] The motion information of the two control points is combined into a 2-tuple space to build a 4-parameter affine motion model. Combinations of two breakpoints can be {CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3} and {CP3, CP4}. For example, a 4-parameter affine motion model built using a 2-tuple space including control points CP1 and CP2 can be referred to as affine (CP1, CP2).

[00241] The motion information of the three control points is combined into a 3-tuple space to construct a 6-parameter affine motion model. The combinations of the three breakpoints can be {CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4} and {CP1, CP3, CP4}. For example, a 6-parameter affine motion model built using a 3-tuple space including control points CP1, CP2, and CP3 can be referred to as affine (CP1, CP2, CP3).

[00242] The motion information of the four control points is combined into a 4-tuple space to construct an 8-parameter bilinear motion model. An 8-parameter bilinear model built using a 4-tuple space including control points CP1, CP2, CP3, and CP4 can be referred to as bilinear (CP1, CP2, CP3, CP4).

[00243] In embodiments of this application, for ease of description, the combination of motion information of two checkpoints (or two coded blocks) is simply referred to as a 2-tuple space, the combination of motion information of three checkpoints (or three coded blocks) is simply referred to as a 3-space. tuples, and the combination of the motion information of four control points (or four coded blocks) is simply called a 4-tuple space.

[00244] These models are run in a given order. If information about the movement of the control point corresponding to the combined model is not available, the model is considered to be unavailable. Otherwise, the index of the reference frame of the model is determined and the motion vector of the control point is scaled. If the motion information of all control points after scaling is consistent, the model is invalid. If all the motion information of the control points driving the model is available and the model is valid, the motion information of the control points used to build the model is added to the motion information candidate list.

[00245] The method of scaling the motion vector of the reference point is shown in Formula (12):

(12)

(12)

представляет собой POC номер текущего кадра,

представляет собой POC номер опорного кадра текущего блока,

представляет собой POC номер опорного кадра контрольной точки,

представляет собой вектор движения, полученный после масштабирования, MV представляет вектор движения контрольной точки.where

is the POC number of the current frame,

represents the POC number of the key frame of the current block,

is the POC number of the reference frame of the reference point,

represents the motion vector obtained after scaling, MV represents the motion vector of the control point.

[00246] It should be noted that different combinations of checkpoints can be converted to checkpoints at the same location.

[00247] For example, a 4-parameter affine motion model derived from a combination of {CP1, CP4}, {CP2, CP3}, {CP2, CP4}, {CP1, CP3}, or {CP3, CP4} is represented as {CP1 , CP2} or {CP1, CP2, CP3} after conversion. The conversion method includes: substitution of motion vectors and information about the coordinates of control points in Formula (2) to obtain model parameters; and then substituting the coordinate information {CP1, CP2} into Formula (3) to obtain motion vectors {CP1, CP2}.

) представляет вектор движения CP1, (

) представляет вектор движения CP2, (

) представляет вектор движения CP3, а (

) представляет собой вектор движения CP4.[00248] More specifically, the conversion may be performed in accordance with formulas (13) - (21), where W represents the width of the current block, and H represents the height of the current block. In formulas (13)-(21) (

) represents the motion vector CP1, (

) represents the motion vector CP2, (

) represents the motion vector of CP3, and (

) is the motion vector of CP4.

[00249] {CP1, CP2} can be converted to {CP1, CP2, CP3} according to Formula (13). In other words, the motion vector CP3 in {CP1, CP2, CP3} can be determined according to Formula (13):

(13)

(13)

[00250] {CP1, CP3} can be converted to {CP1, CP2} or {CP1, CP2, CP3} according to formula (14):

(14)

(fourteen)

[00251] {CP2, CP3} can be converted to {CP1, CP2} or {CP1, CP2, CP3} according to formula (15):

(15)

(15)

[00252] {CP1, CP4} can be converted to {CP1, CP2} or {CP1, CP2, CP3} according to formula (16) or (17):

(16)

(sixteen)

(17)

(17)

[00253] {CP2, CP4} can be converted to {CP1, CP2} according to formula (18), and {CP2, CP4} can be converted to {CP1, CP2, CP3} according to formulas (18) and (19 ):

(18)

(eighteen)

(19)

(nineteen)

[00254] {CP3, CP4} can be converted to {CP1, CP2} according to formula (20), and {CP3, CP4} can be converted to {CP1, CP2, CP3} according to formulas (20) and (21 ):

(20)

(20)

(21)

(21)

[00255] For example, a 6-parameter affine motion model derived from a combination of {CP1, CP2, CP4}, {CP2, CP3, CP4}, or {CP1, CP3, CP4} is converted and represented using {CP1, CP2, CP3}. The conversion method includes: substitution of motion vectors and information about the coordinates of control points in Formula (4) to obtain model parameters; and then substituting the coordinate information {CP1, CP2, CP3} into formula (5) to obtain motion vectors {CP1, CP2, CP3}.

) представляет вектор движения CP1, (

) представляет вектор движения CP2, (

) представляет вектор движения CP3, а (

) представляет собой вектор движения CP4.[00256] More specifically, the conversion may be performed in accordance with formulas (22) - (24), where W represents the width of the current block, and H represents the height of the current block. In formulas (13) - (21) (

) represents the motion vector CP1, (

) represents the motion vector CP2, (

) represents the motion vector of CP3, and (

) is the motion vector of CP4.

[00257] {CP1, CP2, CP4} can be converted to {CP1, CP2, CP3} according to formula (22):

(22)

(22)

[00258] {CP2, CP3, CP4} can be converted to {CP1, CP2, CP3} according to formula (23):

(23)

(23)

[00259] {CP1, CP3, CP4} can be converted to {CP1, CP2, CP3} according to formula (24):

(24)

(24)

[00260] 6. Advanced Temporal Motion Vector Prediction (Advanced Temporal Motion Vector Prediction, ATMVP):

[00261] In inter-prediction in HEVC, motion compensation is performed on all samples of the current block based on the same motion information to obtain predictors of the samples of the block to be processed. However, not all samples of the block to be processed have the same motion characteristics. Using the same motion information to predict all samples of a block to be processed can reduce motion compensation accuracy and increase residual information.

[00262] To solve the above problems, the current solution provides advanced temporal motion vector prediction (ATMVP) technology.

[00263] The process of performing prediction using ATMVP technology mainly comprises the following steps, as shown in FIG. 6E:

[00264] (1) Determine the offset vector of the block to be processed.

[00265] (2) Determine in the corresponding target image, based on the location of the sub-block to be processed, the block to be processed, and the offset vector, the sub-block corresponding to the sub-block to be processed, where the target image is one of the encoded images.

[00266] (3) Determine the motion vector of the sub-block to be processed based on the motion vector of the corresponding sub-block.

[00267] For example, the motion vector of the current sub-block to be processed may be determined using a scaling method. For example, the scaling method is implemented by Formula (25):

(25)

(25)

представляет собой POC номер кадра, в котором, блок, подлежащий обработке, расположен,

представляет собой POC номер кадра, в котором соответствующий субблоку находится,

представляет собой POC номер опорного кадра соответствующего субблока,

представляет вектор движения, полученный посредством масштабирования, а

представляет вектор движения соответствующего субблока.where

represents the POC number of the frame in which the block to be processed is located,

represents the POC number of the frame in which the corresponding subblock resides,

is the POC number of the reference frame of the corresponding subblock,

represents the motion vector obtained by scaling, and

represents the motion vector of the corresponding sub-block.

[00268] (4) Perform motion compensation prediction for the sub-block to be processed based on the motion vector of the sub-block to be processed to obtain a prediction sample value for the sub-block to be processed.

[00269] 7. Planar Motion Vector Prediction (PLANAR):

[00270] In planar motion vector prediction, motion information is obtained at the top neighbor spatial location, left neighbor spatial location, right location, and bottom location of each sub-block to be processed for the block to be processed, and the average value of the motion information is calculated and converted to information about the movement of the current sub-block to be processed.

согласно Формуле (26):[00271] For a sub-block to be processed whose coordinates are (x, y), the motion vector P(x, y) of the sub-block to be processed is calculated based on the horizontal motion vector Ph(x, y) and the horizontal motion vector interpolation Pv(x, y)

according to Formula (26):

(26)

(26)

where H represents the height of the block to be processed and W represents the width of the block to be processed.

(27) и (28):[00272] The horizontal interpolation motion vector Ph (x, y) and the horizontal interpolation motion vector Pv (x, y) can be calculated based on the motion vectors of the sub-blocks to the left, right, top and bottom of the current sub-block to be processed, according to the formulas

(27) and (28):

(27)

(27)

(28)

(28)

представляет вектор движения субблока слева от субблока, который должен быть обработан, R(w, y) представляет вектор движения субблока справа от субблока, который должен быть обработан,

представляет вектор движения субблока сверху от субблока, подлежащего обработке, а

представляет вектор движения субблока снизу от субблока, подлежащего обработке. where

represents the motion vector of the sub-block to the left of the sub-block to be processed, R(w, y) represents the motion vector of the sub-block to the right of the sub-block to be processed,

represents the motion vector of the sub-block above the sub-block to be processed, and

represents the motion vector of the sub-block below the sub-block to be processed.

и

блоков кодирования в предварительно заданных местоположениях (-1, y) и (x, -1) получаются на основе координат (x, y) субблока, подлежащего обработке.[00273] The left block motion vector L and the top block motion vector A are obtained based on the spatial neighboring block of the current coding block. Motion vectors

and

coding blocks at predetermined locations (-1, y) and (x, -1) are obtained based on the coordinates (x, y) of the sub-block to be processed.

[00274] As shown in FIG. 7, the motion vector R(w, y) of the right block and the motion vector B(x, H) of the bottom block can be extracted using the following method:

[00275] (1) Retrieve temporal motion information BR at the lower right spatial neighbor location of the block to be processed.

[00276] (2) Obtain the motion vector R(w, y) of the right block by performing a weighting calculation based on the extracted motion vector AR at the upper right spatial neighbor location and the extracted temporal motion information BR at the lower right spatial neighbor location, as shown in Formula (29):

(29)

(29)

[00277] (3) Obtain the lower block motion vector B(x, H) by performing a weighting calculation based on the extracted motion vector BL at the lower left spatial neighbor location and the extracted temporal motion information BR at the lower right spatial neighbor location, as shown in Formula (30):

(30)

(thirty)

[00278] It should be noted that the motion vector used in the calculation is a motion vector obtained by scaling after the motion vector is scaled to indicate the first reference frame in a particular reference frame queue.

[00279] 8. Affine motion model-based advanced motion vector prediction mode (Affine AMVP mode):

[00280] (1) Create a list of candidate motion vectors.

[00281] A list of candidate motion vectors corresponding to the affine motion model based AMVP mode is generated using a checkpoint legacy motion vector prediction method and/or a checkpoint built motion vector prediction method. In embodiments of this application, a list of candidate motion vectors corresponding to an affine motion model based AMVP mode may be referred to as a control point motion vectors predictor candidate list. The motion vector predictor of each control point contains the motion vectors of two control points (for a 4-parameter affine motion model) or the motion vectors of three control points (for a 6-parameter affine motion model).

[00282] Optionally, the list of candidate checkpoint motion vector predictors is reduced and sorted according to a specific rule, and may be truncated or padded to a specific number.

[00283] (2) Determine the optimal motion vector predictor of the reference point.

[00284] At the encoder side, the motion vector of each subblock of the current coding block is obtained based on each motion vector predictor of the checkpoint in the checkpoint motion vector predictor candidate list according to formula (3) or (5). The resulting motion vector is used to obtain a sample value at the corresponding location in the reference frame pointed to by the subblock's motion vector. The resulting sample value is used as a predictor to perform motion compensation using an affine motion model. The average difference between the original value and the prediction value of each sample of the current coding block is calculated. The motion vector predictor of the checkpoint corresponding to the minimum average difference is selected as the optimal motion vector predictor of the checkpoint and used as the motion vector predictors of two or three breakpoints of the current coding block. An index number representing the location of the cue motion vector predictor in the cue motion vector predictor candidate list is encoded in the bitstream and sent to the decoder.

[00285] On the decoder side, the index number is analyzed and a control point motion vector predictor (CPMVP) is determined from a list of control point motion vector predictor candidates based on the index number.

[00286] (3) Determine the motion vector of the reference point.

[00287] At the encoder side, the control point motion vector predictor is used as a search starting point to search for motion within a specific search range to obtain control point motion vectors (CPMV). The difference between the control point motion vector and the control point motion vectors differences (CPMVD) predictor is transmitted to the decoder side.

[00288] On the decoder side, the checkpoint motion vector difference is analyzed and added to the checkpoint motion vector predictor to obtain the checkpoint motion vector.

[00289] 9. Sub-block based merging mode:

[00290] A sub-block based merging candidate list is created using at least one of: extended temporal motion vector prediction, a checkpoint legacy motion vector prediction method, a checkpoint constructed motion vector prediction method, and a planar prediction.

[00291] Optionally, the subblock-based merge candidate list is reduced and sorted according to a specific rule, and may be truncated or padded to a specific number.

[00292] On the encoder side, if extended temporal motion vector prediction is used, the motion vector of each sub-block (sample or N1×N2 sample block obtained using a specific partitioning method) is obtained using the method described in the previous descriptions in “7. Planar motion vector prediction”. If the planar prediction method is used, the motion vector of each sub-block is obtained using the method described in the foregoing descriptions in “8. Advanced Motion Vector Prediction Mode Based on Affine Motion Model.

[00293] If a checkpoint legacy motion vector prediction method or a checkpoint constructed motion vector prediction method is used, the motion vector of each sub-block (sample or N1×N2 sample block obtained using a specific partitioning method) of the current block is obtained according to Formula (3), or (five). After the motion vector of each sub-block is obtained, a sample value at a location in the reference frame pointed to by the motion vector of the sub-block points is further obtained, and the sample value is used as a sub-block predictor for affine motion compensation. The average difference between the original value and the predictor of each sample of the current coding block is calculated. The breakpoint motion vector corresponding to the minimum average difference is selected as the motion vectors of two or three breakpoints of the current coding block. An index number representing the location of the motion vector of the checkpoint in the candidate list is encoded in the bitstream and sent to the decoder.

[00294] On the decoder side, the index number is parsed and a control point motion vector (control point motion vectors, CPMV) is determined from a list of control point motion vector fusion candidates based on the index number.

[00295] In addition, it should be noted that in this application, "at least one" means one or more, and "many" means two or more than two. The term "and/or" describes an association relationship for describing associated objects and represents that there may be three relationships. For example, A and/or B may represent the following cases: only A exists, both A and B exist, and only B exists, where A and B can be singular or plural. The "/" character usually represents an "or" relationship between related entities. "At least one element (part) of the following" or a similar expression means any combination of these elements, including a single element (part) or any combination of multiple elements (parts). For example, at least one element (part) of a, b, or c may represent: a, b, c, ab, ac, bc, or abc, where a, b, and c may be singular or plural.

[00296] In this application, when the inter prediction mode is used to decode the current block, the syntax element can be used to signal the inter prediction mode.

[00297] See Table 1 for some currently used inter-prediction mode syntax structures used to parse the current block. It should be noted that a syntax element in a syntax structure may alternatively be represented by other pointers. In this application, this is not specifically limited.

Table 1.

coding_unit(x0, y0, cbWidth, cbHeight, treeType) { Descriptor if(slice_type != I) { cu_skip_flag [x0][y0] ae(v) if(cu_skip_flag[x0][y0] == 0) pred_mode_flag [x0][y0] ae(v) } if(CuPredMode[x0][y0] == MODE_INTRA) { if(treeType == SINGLE_TREE | | treeType == DUAL_TREE_LUMA) { intra_luma_mpm_flag [x0][y0] ae(v) if(intra_luma_mpm_flag[x0][y0]) intra_luma_mpm_idx [x0][y0] ae(v) else intra_luma_mpm_remainder [x0][y0] ae(v) } if(treeType == SINGLE_TREE | | treeType == DUAL_TREE_CHROMA) intra_chroma_pred_mode [x0][y0] ae(v) } else { /* MODE_INTER (inter mode) */ if(cu_skip_flag[x0][y0]) { if(MaxNumSubblockMergeCand > 0 && cbWidth >= 8 && cbHeight >= 8) merge_subblock_flag[x0][y0] ae(v) if(merge_subblock_flag[x0][y0] == 0 && MaxNumMergeCand > 1) merge_idx[x0][y0] ae(v) if(merge_subblock_flag[x0][y0] == 1 && MaxNumSubblockMergeCand > 1) merge_subblock_idx[x0][y0] ae(v) } else { merge_flag [x0][y0] ae(v) if(merge_flag[x0][y0]) { if((sps_affine_enabled_flag | | sps_sbtmvp_enabled_flag) && cbWidth >= 8 && cbHeight >= 8) merge_subblock_flag [x0][y0] ae(v) if(merge_subblock_flag[x0][y0] == 0 && MaxNumMergeCand > 1) merge_idx [x0][y0] ae(v) if(merge_subblock_flag[x0][y0] == 1) merge_subblock_idx [x0][y0] ae(v) } else { if(slice_type == B) inter_pred_idc [x0][y0] ae(v) if(sps_affine_enabled_flag && cbWidth >= 16 && cbHeight >= 16) { inter_affine_flag [x0][y0] ae(v) if(sps_affine_type_flag && inter_affine_flag[x0][y0]) cu_affine_type_flag [x0][y0] ae(v) } if(inter_pred_idc[x0][y0] != PRED_L1) { if(num_ref_idx_l0_active_minus1 > 0) ref_idx_l0[x0][y0] ae(v) mvd_coding(x0, y0, 0, 0) if(MotionModelIdc[x0][y0] > 0) mvd_coding(x0, y0, 0, 1) if(MotionModelIdc[x0][y0] > 1) mvd_coding(x0, y0, 0, 2) mvp_l0_flag[x0][y0] ae(v) } else { MvdL0[x0][y0][0] = 0 MvdL0[x0][y0][1] = 0 } if(inter_pred_idc[x0][y0] != PRED_L0) { if(num_ref_idx_l1_active_minus1 > 0) ref_idx_l1 [x0][y0] ae(v) if(mvd_l1_zero_flag && inter_pred_idc[x0][y0] == PRED_BI) { MvdL1[x0][y0][0] = 0 MvdL1[x0][y0][1] = 0 MvdCpL1[x0][y0][0][0] = 0 MvdCpL1[x0][y0][0][1] = 0 MvdCpL1[x0][y0][1][0] = 0 MvdCpL1[x0][y0][1][1] = 0 MvdCpL1[x0][y0][2][0] = 0 MvdCpL1[x0][y0][2][1] = 0 } else { mvd_coding(x0, y0, 1, 0) if(MotionModelIdc[x0][y0] > 0) mvd_coding(x0, y0, 1, 1) if(MotionModelIdc[x0][y0] > 1) mvd_coding(x0, y0, 1, 2) mvp_l1_flag [x0][y0] ae(v) } else { MvdL1[x0][y0][0] = 0 MvdL1[x0][y0][1] = 0 } if(sps_amvr_enabled_flag && inter_affine_flag == 0 &&
(MvdL0[x0][y0][0] != 0 | | MvdL0[x0][y0][1] != 0 | |
MvdL1[x0][y0][0] != 0 | | MvdL1[x0][y0][1] != 0)) amvr_mode [x0][y0] ae(v) } } } if(CuPredMode[x0][y0] != MODE_INTRA && cu_skip_flag[x0][y0] == 0) cu_cbf ae(v) if(cu_cbf) { transform_tree(x0, y0, cbWidth, cbHeight, treeType) }

[00298] The variable treeType indicates the type of coding tree used to encode the current block.

[00299] The variable slice_type is used to indicate the type of slice in which the current block is located, for example, type P, type B, or type I.

[00300] The cu_skip_flag[x0][y0] syntax element may be used to indicate whether the current block has a remainder. For example, when cu_skip_flag[x0][y0] = 1, this indicates that the current block has a remainder; or when cu_skip_flag[x0][y0] = 0, it indicates that the current block has no remainder.

[00301] It should be noted that the skip mode is a special merge mode. After using the merge mode for motion vector search, if the encoder determines by some method that the current block is basically the same as the reference block, the residual data need not be transmitted, and only the motion vector index and cu_skip_flag need to be transmitted. Therefore, if cu_skip_flag[x0][y0] = 0, this indicates that the current block has no remainder and the residual data does not need to be parsed.

[00302] The pred_mode_flag[x0][y0] syntax element is used to indicate whether the prediction mode for the current block is inter prediction or intra prediction mode.

[00303] The variable CuPredMode[x0][y0] is determined based on pre_mode_flag[x0][y0]. MODE_INTRA indicates the intra prediction mode.

[00304] The syntax element merge_flag[x0][y0] can be used to indicate whether the merge mode is used for the current block. For example, when merge_flag[x0][y0] = 1, this indicates that the merge mode is being used for the current block; or when merge_flag[x0][y0] = 0, this indicates that the merge mode is not used for the current block, where x0 and y0 represent the coordinates of the current block in the video image.

[00305] cbWidth specifies the width of the current box, and cbHeight specifies the height of the current box.

[00306] The sps_affine_enabled_flag syntax element can be used to indicate whether an affine motion model can be used for inter prediction of an image block contained in a video image. For example, when sps_affine_enabled_flag=0, this indicates that the affine motion model cannot be used for inter prediction of an image block contained in a video image; or when sps_affine_enabled_flag=1, this indicates that the affine motion model can be used for inter prediction of an image block contained in a video image.

[00307] The merge_subblock_flag[x0][y0] syntax element can be used to indicate whether a subblock-based merge mode is used for the current block. The type (slice_type) of the slice in which the current block is located is type P or type B. For example, when merge_subblock_flag[x0][y0] = 1, this indicates that the current block uses subblock-based merge mode; or when merge_subblock_flag[x0][y0] = 0, it indicates that the subblock-based merge mode is not used for the current block, but the rectilinear motion model-based merge mode may be used.

[00308] The merge_idx[x0][y0] syntax element may be used to specify the index of the merge candidate list.

[00309] The merge_subblock_idx[x0][y0] syntax element may be used to specify the index of a subblock-based merge candidate list.

[00310] The sps_sbtmvp_enabled_flag may be used to indicate whether the ATMVP mode can be used for intra prediction of an image block contained in a video image. For example, when sps_sbtmvp_enabled_flag=1, this indicates that the ATMVP mode can be used for intra prediction of a picture block contained in a video picture; or when sps_sbtmvp_enabled_flag=0, this indicates that the ATMVP mode cannot be used for intra prediction of a picture block contained in a video picture.

[00311] The inter_affine_flag[x0][y0] syntax element may be used to indicate whether the affine motion model based AMVP mode is used for the current block when the slice in which the current block is located is a P-type slice or a B-type slice . For example, when inter_affine_flag[x0][y0] = 0, this indicates that the affine motion model based AMVP mode is used for the current block; or when inter_affine_flag[x0][y0] = 1, it indicates that the affine motion model based AMVP mode is not used for the current block, but the rectilinear motion model based AMVP mode can be used.

[00312] The cu_affine_type_flag[x0][y0] syntax element can be used to indicate whether a 6-parameter affine motion model is used to perform motion compensation for the current block when the slice in which the current block is located is a P-type slice or a B-type. When cu_affine_type_flag[x0][y0] = 0, it indicates that a 6-parameter affine motion model is not used to perform motion compensation for the current block, but only a 4-parameter affine motion model can be used to perform motion compensation; or when cu_affine_type_flag[x0][y0] = 1, this indicates that a 6-parameter affine motion model is used to perform motion compensation for the current block.

[00313] As shown in Table 2, when MotionModelIdc[x0][y0] = 1, this indicates that a 4-parameter affine motion model is being used; when MotionModelIdc[x0][y0] = 2, this indicates that an affine motion model with 6 parameters is used; or when MotionModelIdc[x0][y0] = 0, this indicates that a rectilinear motion model is being used.

Table 2.

MotionModelIdc [x0] [y0] Motion model for motion compensation 0 Rectilinear motion 1 motion with 4 affine parameters (4-parameter affine motion) 2 motion with 6 affine parameters (6-parameter affine motion)

[00314] The variable MaxNumMergeCand is used to indicate the maximum length of the list of candidate motion vectors to merge, the variable MaxNumSubblockMergeCand is used to indicate the maximum length of the list of candidate motion vectors to merge on a sub-block basis, inter_pred_idc[x0][y0] is used to indicate the prediction direction, PRED_L0 indicates forward prediction, num_ref_idx_l0_active_minus1 specifies the number of reference frames in the front reference frame list, ref_idx_l0[x0][y0] indicates the index value for the current block's front reference frame, mvd_coding(x0, y0, 0, 0) indicates the first motion vector difference, mvp_l0_flag[x0] [y0] indicates the index value for the forward MVP candidate list, PRED_L1 is used to indicate backward prediction, num_ref_idx_l1_active_minus1 indicates the number of reference frames in the reverse reference frame list, ref_idx_l1[x0][y0] indicates the index value for the reverse reference frame of the current block, and mvp_l1_flag[ x0 ][y0] specifies the index value for the reverse MVP candidate list.

[00315] In Table 1, ae(v) represents a syntax element encoded using context-based adaptive binary arithmetic coding (CABAC).

[00316] Next, the inter prediction process will be described in detail as shown in FIG. 8A.

[00317] Step 801: Parse the bitstream based on the syntax structure shown in Table 1 and determine the inter prediction mode for the current block.

[00318] If it is determined that the inter prediction mode for the current block is the affine motion model based AMVP mode, step 802a is performed.

[00319] The sps_affine_enabled_flag=1, merge_flag=0, and inter_affine_flag=1 syntax elements indicate that the inter prediction mode for the current block is an affine motion model based AMVP mode.

[00320] If it is determined that the inter-prediction mode for the current block is the sub-block merge mode, step 802b is executed.

[00321] The sps_affine_enabled_flag=1 syntax element or the sps_sbtmvp_enabled_flag=1 syntax element, and the merge_flag=1 and merge_subblock_flag=1 syntax elements indicate that the inter prediction mode for the current block is the subblock merge mode.

[00322] Step 802a: create a list of candidate motion vectors corresponding to the AMVP mode based on the affine motion model. Execute step 803a.

[00323] A checkpoint candidate motion vector of the current block is output using the checkpoint inherited motion vector prediction method and/or checkpoint built motion vector prediction method, and added to the list of candidate motion vectors.

[00324] The list of candidate motion vectors may be a 2-tuple list (a 4-parameter affine motion model is used for the current coding block) or a 3-tuple list. The 2-tuple list contains one or more 2-tuples used to build an affine motion model with 4 parameters. The 3-tuple list contains one or more 3-tuples used to build an affine motion model with 6 parameters.

[00325] Optionally, the 2/3-tuple list of candidate motion vectors is reduced and sorted according to a specific rule, and may be truncated or padded to a specific number.

[00326] A1: A process for constructing a list of candidate motion vectors using a checkpoint legacy motion vector prediction method is described.

[00327] FIG. 4 is used as an example. In the example in FIG. 4, blocks at neighboring locations around the current block are searched in the order A1->B1->B0->A0->B2 to find an affine-coded block including the block at the neighboring location of the current block, and to obtain motion information of the reference points of the affine-coded block. Additionally, information about the movement of the affine-coded block checkpoint is used to build a movement model to obtain information about the candidate movement to the current block's checkpoint. For details, see the previous description in "3. A way to predict the legacy motion vector of a checkpoint. Details are not described here again.

[00328] For example, the affine motion model used for the current block is a 4-parameter affine motion model (ie MotionModelIdc=1). If a 4-parameter affine motion model is used for an adjacent affine decoder block, the motion vectors of the two control points of the affine decoder block are obtained: the motion vector (vx4, vy4) of the upper left control point (x4, y4) and the motion vector (vx5, vy5) of the upper right control point (x5, y5). An affine decoding block is an affine encoding block for which prediction is performed using an affine motion model during encoding.

[00329] The motion vectors of the two control points, namely the upper left and upper right control points of the current block, are derived respectively according to formulas (6) and (7) corresponding to the 4-parameter affine motion model using the 4-parameter affine motion model, which includes two checkpoints of a neighboring affine decoding block.

[00330] If a 6-parameter affine motion model is used for a neighbor affine decoder, the motion vectors of the three control points of the neighbor affine decoder are obtained, e.g. , vy5) of the upper right control point (x5, y5) and the motion vector (vx6, vy6) of the lower left vertex (x6, y6) in FIG. four.

[00331] The motion vectors of the two control points, namely the upper left and upper right control points of the current block, are derived respectively according to formulas (8) and (9) corresponding to the 6-parameter affine motion model using the 6-parameter affine motion model, which includes three control points of the adjacent affine decoding block.

[00332] For example, the affine motion model used for the current decoding block is a 6-parameter affine motion model (ie, MotionModelIdc=2). If the affine motion model used for the neighboring affine decoding block is a 6-parameter affine motion model, the motion vectors of the three control points of the neighboring affine decoding block are obtained, for example, the motion vector (vx4, vy4) of the upper left control point (x4, y4), vector motion (vx5, vy5) of the upper right control point (x5, y5) and motion vector (vx6, vy6) of the lower left vertex (x6, y6) in FIG. four.

[00333] The motion vectors of the three control points, namely, the upper left, upper right and lower left control points of the current block, are output respectively in accordance with formulas (8), (9) and (10) corresponding to the 6-parameter affine motion model using a 6-parameter affine motion model including three control points of a neighboring affine decoding block.

[00334] If the affine motion model used for the neighboring affine decoding block is a 4-parameter affine motion model, the motion vectors of the two control points of the adjacent affine decoding block are obtained: motion vector (vx4, vy4) of the upper left control point (x4, y4) and the motion vector (vx5, vy5) of the upper right control point (x5, y5).

[00335] The motion vectors of the three control points, namely the upper left, upper right, and lower left control points of the current block, are derived respectively according to formulas (6) and (7) corresponding to the 4-parameter affine motion model using the 4-parameter affine motion model. parameters, which includes two checkpoints of the neighboring block of affine decoding.

[00336] It should be noted that other traffic patterns, possible locations, and search orders may also be used in this application. Details are not described here. It should be noted that this application can also use methods to represent motion models of adjacent and current coding blocks based on other breakpoints. Details are not described here.

[00337] A2: A process for constructing a list of candidate motion vectors using a checkpoint constructed motion vector prediction method is described.

[00338] For example, the affine motion model used for the current decoding block is a 4-parameter affine motion model (i.e., MotionModelIdc=1) and the motion vectors of the top left vertex and top right vertex of the current coding block are determined based on the motion information of the adjacent encoded block of the current encoding block. In particular, the list of candidate motion vectors may be constructed using the reference point constructed motion vector prediction method 1 or the reference point constructed motion vector prediction method 2. For information on a specific method, see the previous description in section “4. Method 1 for predicting the constructed motion vector of the control point”; 5. Method 2 for predicting the constructed motion vector of the control point. Details are not described here again.

[00339] For example, the affine motion model used for the current decoding block is a 6-parameter affine motion model (i.e., MotionModelIdc=2) and the motion vectors of the top left vertex, top right vertex, and bottom left vertex of the current coding block are determined based on motion information of a neighboring encoded block of the current encoding block. In particular, the list of candidate motion vectors may be constructed using the reference point constructed motion vector prediction method 1 or the reference point constructed motion vector prediction method 2. For information on a specific method, see the previous description in section “4. Method 1 for predicting the constructed motion vector of the control point”; 5. Method 2 for predicting the constructed motion vector of the control point. Details are not described here again.

[00340] It should be noted that other combinations of checkpoint movement information may also be used in this application. Details are not described here.

[00341] Step 803a: Parse the bitstream and determine an optimal checkpoint motion vector predictor. Execute step 804a.

[00342] B1: If the affine motion model used for the current decoding block is a 4-parameter affine motion model (MotionModelIdc=1), the index number is parsed and the optimal motion vector predictor for the two control points is determined from a list of candidate motion vectors based on index number.

[00343] For example, the index number is mvp_l0_flag or mvp_l1_flag.

[00344] B2: If the affine motion model used for the current decoding block is a 6-parameter affine motion model (MotionModelIdc=2), the index number is parsed and the optimal motion vector predictor for the three control points is determined from the list of candidate motion vectors at serial number basis.

[00345] Step 804a: parse the bitstream and determine the motion vector of the checkpoint.

[00346] C1: If the affine motion model used for the current decoding block is a 4-parameter affine motion model (MotionModelIdc=1), the motion vector differences of the two control points of the current block are obtained by decoding the bitstream, and the motion vectors of the two control points are then are obtained based on the differences of the motion vectors and the motion vector predictors of the control points. Using forward prediction as an example, the motion vector differences of two control points are mvd_coding(x0, y0, 0, 0) and mvd_coding(x0, y0, 0, 1), respectively.

[00347] For example, the motion vector differences of the top left breakpoint and top right breakpoint are obtained by bitstream decoding and added to the respective motion vector predictors to obtain the top left breakpoint and top right breakpoint motion vectors of the current block.

[00348] C2: If the affine motion model used for the current decoding block is a 6-parameter affine motion model (MotionModelIdc=2), the motion vector differences of the three control points of the current block are obtained by decoding the bitstream, and the motion vectors of the three control points are then are obtained based on the differences of the motion vectors and the motion vector predictors of the control points. Using direct prediction as an example, the motion vector differences of the three control points are mvd_coding (x0, y0, 0, 0), mvd_coding (x0, y0, 0, 1), and mvd_coding (x0, y0, 0, 2), respectively.

[00349] For example, the motion vector differences of the top left breakpoint, top right breakpoint, and bottom left breakpoint are obtained by bitstream decoding and added to the respective motion vector predictors to obtain the top left breakpoint, top right checkpoint, and bottom motion vectors. the left control point of the current block.

[00350] Step 805a: Obtain the motion vector of each sub-block of the current block based on the checkpoint motion information and the affine motion model used for the current decoding block.

[00351] One sub-block in the current affine decoding block may be equivalent to one motion compensation block, and the width and height of the sub-block is less than the width and height of the current block. The motion information of a sample at a predetermined location in the motion compensation block may be used to represent the motion information of all samples of the motion compensation block. Assuming the size of the motion compensation block is M×N, the sample at the preset location can be center sample (M/2, N/2), top left sample (0, 0), top right sample (M - 1, 0 ) or by sampling at another location in the motion compensation block. The center sample of the motion compensation block is used as an example for the description below, as shown in FIG. 8C. In FIG. 8C, V0 represents the motion vector of the upper left control point, and V1 represents the motion vector of the top right control point. Each small rectangle represents one block of motion compensation.

[00352] The coordinates of the center sample of the motion compensation block relative to the upper left sample of the current affine decoding block are calculated according to formula (31). In formula (31), i represents the i-th movement compensation unit in the horizontal direction (left to right), j represents the j-th movement compensation unit in the vertical direction (top to bottom), and (x(i, j), y(i, j )) represents the coordinates of the center sample of the (i, j)th motion compensation block relative to the upper left sample of the current affine decoding block.

[00353] If the affine motion model used for the current affine decoding block is a 6-parameter affine motion model, (x(i, j), y(i, j)) is substituted into formula (32) corresponding to the 6-parameter affine motion model parameters to obtain a motion vector (vx (i, j), vy (i, j)) of the center sample of each motion compensation block, and the motion vector is used as the motion vectors of all samples of the motion compensation block.

[00354] If the affine motion model used for the current affine decoding block is a 4-parameter affine motion model, (x(i, j), y(i, j)) is substituted into formula (33) corresponding to the affine motion model with 4 parameters to obtain the motion vector (vx(i, j), vy(i, j)) of the center sample of each motion compensation block, and the motion vector is used as the motion vectors of all samples of the motion compensation block.

(31)

(31)

(32)

(32)

(33)

(33)

[00355] Step 806a: Perform motion compensation for each subblock based on the determined motion vector of the subblock to obtain a subblock prediction sample value.

[00356] Step 802b: Create a motion information candidate list corresponding to the subblock merge mode.

[00357] Specifically, a motion information candidate list corresponding to a sub-block based merging candidate list may be constructed using one or more of extended temporal motion vector prediction, a checkpoint legacy motion vector prediction method, a predicting the constructed motion vector of the control point or the planar method. In this embodiment of this application, the list of motion information candidates corresponding to the subblock merge mode may be referred to as a subblock merge candidate list for short.

[00358] Optionally, the list of motion information candidates is reduced and sorted according to a specific rule, and may be truncated or padded to a specific number.

[00359] D0: If sps_sbtmvp_enabled_flag=1, a motion vector candidate is obtained using ATMVP and added to the subblock-based merge candidate list. For details, see the previous description in “6. Advanced temporal motion vector prediction”.

[00360] D1: If sps_affine_enabled_flag=1, rfylblfnyfz the checkpoint motion information of the current block is output using the checkpoint legacy motion vector prediction method and added to the subblock-based merge candidate list. For details, see the previous description in “3. A way to predict the legacy motion vector of a checkpoint.

[00361] For example, in FIG. 6C, blocks at neighboring locations around the current block are traversed in the order A1->B1->B0->A0->B2 to find an affine-coded block including the block at the neighboring location of the current block and obtain checkpoint movement information. affine-coded block. Additionally, a motion model corresponding to the affine-coded block is used to derive motion information about the candidate checkpoint of the current block.

[00362] If the subblock-based merge candidate list is empty, candidate checkpoint movement information is added to the subblock-based merge candidate list. Otherwise, the motion information in the sub-unit merge candidate list is sequentially passed to check whether there is motion information that is the same as the candidate checkpoint motion information in the sub-unit merge candidate list. If no motion information is present in the sub-block-based merge candidate list that is the same as the checkpoint motion candidate information, the checkpoint motion candidate information is added to the sub-block-based merge candidate list.

[00363] Determining whether two pieces of candidate motion information are the same should be implemented by determining whether the forward and reverse reference frames of the two pieces of candidate motion information and the horizontal and vertical components of each forward motion vector and reverse motion vector are the same. Two pieces of candidate motion information are considered different only when all of these elements are different.

[00364] If the number of motion information pieces in the subblock-based merge candidate list reaches the maximum list length MaxNumSubblockMergeCand (MaxNumSubblockMergeCand is a positive integer such as 1, 2, 3, 4, or 5, where 5 is used as an example in the following description, and the details are not described here), the construction of the list of candidates is completed. Otherwise, the block is traversed at the next adjacent location.

[00365] D2: If sps_affine_enabled_flag=1, checkpoint motion candidate information for the current block is output using the checkpoint motion vector built prediction method and added to the subblock-based merge candidate list as shown in FIG. 8B.

[00366] Step 801c: Get information about the movement of checkpoints of the current block. For example, for details, refer to step 601 in the preceding description in "5. A method for predicting the constructed motion vector of a control point. The details are again not described here.

[00367] Step 802c: Combine checkpoint motion information to obtain motion information of the constructed checkpoints. See step 602 in FIG. 6d. The details are again not described here.

[00368] Step 803c: Add the built checkpoint motion information to the subblock-based merge candidate list.

[00369] If the length of the subblock-based merge candidate list is less than the maximum length of the MaxNumSubblockMergeCand list, the combinations of checkpoint motion information are passed through in a predetermined order, and the resulting valid combination is used as candidate checkpoint motion information. In this case, if the sub-block-based merge candidate list is empty, checkpoint movement candidate information is added to the sub-block-based merge candidate list. Otherwise, the motion information in the list of candidate motion vectors is searched sequentially to check if there is motion information that is the same as the motion information of the candidate point in the candidate list of sub-block merge candidates. If no motion information is present in the sub-block-based merge candidate list that is the same as the candidate checkpoint motion information, the candidate checkpoint motion information is added to the sub-block-based merge candidate list.

[00370] For example, the preset order is: Affine (CP1, CP2, CP3) -> Affine (CP1, CP2, CP4) -> Affine (CP1, CP3, CP4) -> Affine (CP2, CP3, CP4) - > Affine (CP1, CP2) -> Affine (CP1, CP3). There are six combinations in total.

[00371] For example, if sps_affine_type_flag=1, the preset order would be: Affine (CP1, CP2, CP3) -> Affine (CP1, CP2, CP4) -> Affine (CP1, CP3, CP4) -> Affine (CP2, CP3 , CP4) -> Affine (CP1, CP2) -> Affine (CP1, CP3). There are six combinations in total. The order of adding six combinations to the list of candidate motion vectors is not specifically limited in this embodiment of this application.

[00372] If sps_affine_type_flag=0, the preset order is: Affine (CP1, CP2) -> Affine (CP1, CP3). There are two combinations in total. The order of adding two combinations to the list of candidate motion vectors is not specifically limited in this embodiment of this application.

[00373] If the checkpoint motion information corresponding to a combination is not available, the combination is considered unavailable. If a combination is available, the combination key frame index is determined (if there are two breakpoints, the minimum key frame index is chosen as the combination key frame index; if there are more than two breakpoints, the index of the key frame with the maximum frequency of presence is chosen as the combination key frame index , and if the plurality of reference frame indexes have the same occurrence frequency, the minimum reference frame index is selected as the combination reference frame index). The control point motion vectors are further scaled. If the motion information of all control points after scaling is the same, the combination is invalid.

[00374] D3: Optionally, if sps_planar_enabled_flag=1, traffic information generated using ATMVP is added to the subblock-based merge candidate list. For details, see the previous description in “7. Planar motion vector prediction”.

[00375] Optionally, in this embodiment of this application, the list of candidate motion vectors may be extended. For example, after the above traversal process, if the length of the list of candidate motion vectors is less than the maximum length of the list MaxNumSubblockMergeCand, the list of candidate motion vectors may be padded until the length of the list becomes equal to MaxNumSubblockMergeCand.

[00376] The padding may be performed using a zero motion vector padding method, or using a pooling or weighted average method of existing candidate motion information in an existing list. It should be noted that this application may also use other methods to populate the list of candidate motion vectors. Details are not described here.

[00377] Step 803b: Parse the bitstream and determine optimal checkpoint movement information.

[00378] The merge_subblock_idx index number is parsed and the optimal motion information is determined from the subblock-based merge candidate list based on the index number.

[00379] merge_subblock_idx is typically binarized using a TR (truncated unary code) code. In other words, merge_subblock_idx is mapped to different bin rows based on the maximum index value. The maximum index value is preconfigured or passed. For example, if the maximum index value is 4, merge_subblock_idx is converted to binary form according to table 3.

Table 3

index Bean string 0 0 1 1 0 2 1 1 0 3 1 1 1 0 four 1 1 1 1

[00380] When the maximum index value is 2, merge_subblock_idx is converted to binary form according to Table 4.

Table 4

index Bean string 0 0 1 1 0 2 1 1

[00381] If merge_subblock_idx is transmitted using a bin string, the decoder side may determine the index number based on the maximum index value and table 2 or table 3. For example, the maximum index value is 4. When decoding is performed to obtain an index number, if 0 or a number is received index obtained by decoding is equal to the maximum value of the index, the index number is determined. For example, when decoding is performed to obtain an index number, if the first bit is 0, the decoding that is performed to obtain an index number ends and the index number is determined to be 0. In another example, if the first bit is 1 and the second bit is 0, the decoding that is performed to obtain the index number ends and the index number is determined to be 1.

[00382] Setting the maximum value of the index and binarization table is not specifically limited in this embodiment of this application.

[00383] Step 804b: If the optimal motion information is ATMVP or planar motion information, obtain a motion vector for each subblock using ATMVP or planar method directly.

[00384] If the motion mode indicated by the optimal motion information is the affine mode, the motion vector of each subblock of the current block is obtained based on the optimal checkpoint motion information and the affine motion model used for the current decoding block. This process is similar to step 805a.

[00385] Step 805b: Perform motion compensation for each subblock based on the determined motion vector of the subblock to obtain a subblock prediction sample value.

[00386] In the existing technology, there is no acceptable way to determine the maximum length (MaxNumSubblockMergeCand) of the list of candidate motion vectors corresponding to the subblock merge mode.

[00387] In view of this, embodiments of this application provide a video prediction method and apparatus to provide a method for determining a maximum length (MaxNumSubblockMergeCand) of a list of candidate motion vectors corresponding to a subblock merge mode. The method and apparatus are based on the same inventive concept. Since the problem solving principle of the method is similar to that of the hardware, a mutual reference can be made between hardware and method implementations. There are no repeated descriptions.

[00388] The method for determining the maximum length of the list of candidate motion vectors corresponding to the subblock merge mode is described in detail below in two cases.

[00389] In the first case, the sub-block merging mode may comprise at least one of an affine mode, an extended temporal motion vector prediction mode, or a planar motion vector prediction mode.

[00390] In the second case, it is considered that the planar motion vector prediction mode is not present in the subblock merge mode. In other words, the sub-block merging mode may comprise at least one of an affine mode or an extended motion vector temporal prediction mode.

[00391] The following is a detailed description of the implementation of this application from the point of view of the decoder with reference to the accompanying drawings. In particular, the method may be performed by the video decoder 30, or may be implemented by a motion compensation module in the video decoder, or may be performed by a processor.

[00392] For the first case, several possible implementations are presented as examples, from the first implementation to the fifth.

[00393] FIG. 9 shows a description of the first implementation.

[00394] Step 901: parse the first pointer from the bitstream. Execute step S902 or step S904.

[00395] The first pointer is used to indicate whether the candidate mode used for inter-prediction of the block to be processed contains the affine mode. In other words, pointer 1 is used to indicate whether affine mode can (or is allowed) be used to perform motion compensation on the block to be processed.

[00396] For example, the first pointer may be configured in the SPS in the bitstream. Based on this, parsing the first pointer from the bitstream can be implemented as follows: parsing pointer 1 from the SPS in the bitstream. Alternatively, the first pointer may be configured in a slice header for a slice in the bitstream, with the block to be processed contained in the slice. Based on this, parsing the first pointer from the bitstream can be implemented as follows: parsing the first pointer from the slice header for a slice in the bitstream, with the block to be processed contained in the slice.

[00397] For example, the first pointer may be represented by the sps_affine_enabled_flag syntax element, and the sps_affine_enabled_flag is used to indicate whether the affine mode can be used for inter prediction of an image block contained in a video image. For example, when sps_affine_enabled_flag=0, this indicates that the affine mode cannot be used for intra-prediction of an image block contained in a video image. When sps_affine_enabled_flag=1, this indicates that the affine motion model can be used for inter prediction of an image block contained in a video image.

[00398] Step S902: When the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains affine mode, parse the second pointer from the bitstream, where the second pointer is used to indicate (or determine) the maximum length the first candidate motion vector list, and the first candidate motion vector list is the candidate vector list created for the block to be processed using the sub-block merge prediction mode. The first list of candidate motion vectors may be called MaxNumSubblockMergeCand.

[00399] For example, the second pointer may be configured in the SPS, PPS, or slice header. Based on this, parsing the second pointer from the bitstream can be implemented as follows: parsing the second pointer from the set of sequence parameters in the bitstream; or parsing the second pointer from the slice header to the slice, including the block to be processed, in the bitstream.

[00400] For example, the second pointer may be represented by K_minus_max_num_subblock_merge_cand.

[00401] For example, the value of K_minus_max_num_subblock_merge_cand can range from 0 to 4.

[00402] For example, the maximum value of MaxNumSubblockMergeCand may be 5.

[00403] When the maximum value of MaxNumSubblockMergeCand may be 5, the second pointer may be represented by five_minus_max_num_subblock_merge_cand.

[00404] Table 5 shows an example of a syntax structure for parsing a second pointer.

Table 5

slice_header( ) { descriptor ... if(sps_affine_enable_flag) five_minus_max_num_subblock_merge_cand ue(v) ... }

[00405] Step S903: Determine the maximum length of the first list of candidate motion vectors based on the second pointer.

[00406] In an example where the first pointer indicates that the candidate mode used for inter-prediction of a block to be processed contains affine mode, the maximum length (MaxNumSubblockMergeCand) of the first list of candidate vectors can be obtained according to the following formula: MaxNumSubblockMergeCand= K K_minus_max_num_subblock_merge_cand,

where MaxNumSubblockMergeCand represents the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand represents the second pointer, and K is a preset non-negative integer.

[00407] Step S904: When the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, and the third pointer is used to indicate that the ATMVP mode is present in the merge prediction mode subblocks, determine the third number based on the third pointer, and determine the maximum length of the first list of candidate motion vectors based on the third number.

[00408] That the candidate mode used for inter-prediction of the block to be processed contains only the rectilinear motion vector prediction mode means that the candidate mode used for inter-prediction of the block to be processed cannot (not allowed ) contain an affine mode. The third pointer is used to indicate whether the ATMVP mode is present in the subblock merge prediction mode. In other words, the third pointer is used to indicate whether the use of the ATMVP mode for inter-prediction of a block to be processed is allowed. The third pointer may be configured in the SPS, PPS, or slice header.

[00409] In an example, when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, and the third pointer is used to indicate that the ATMVP mode is present in the merge prediction mode subblocks, the maximum length (MaxNumSubblockMergeCand) of the first list of candidate motion vectors is equal to the third number.

[00410] For example, the third pointer may be represented by sps_sbtmvp_enabled_flag. For example, when sps_sbtmvp_enabled_flag is equal to the first value, it indicates that the ATMVP mode is not present in the subblock merge prediction mode; or when sps_sbtmvp_enabled_flag is equal to the second value, it indicates that the ATMVP mode is present in the subblock merge prediction mode. For example, the first value is 0 and the second value is 1.

[00411] For example, the third number may be used to indicate the maximum number of motion vectors that are supported in prediction performed using the ATMVP mode. For example, when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, if sps_sbtmvp_enabled_flag=0, the third number is 0, or if sps_sbtmvp_enabled_flag=1 , the third number is equal to the maximum the number of motion vectors that are supported in the prediction performed using the ATMVP mode. Additionally, when the maximum number of motion vectors that are supported in prediction performed using the ATMVP mode is 1, the third number may be equal to the value of sps_sbtmvp_enabled_flag. For example, if sps_sbtmvp_enabled_flag=0, the third number is 0; or if sps_sbtmvp_enabled_flag=1, the third number is 1.

[00412] That the maximum value of MaxNumSubblockMergeCand can be 5 is used as an example. If sps_affine_enable_flag=0, MaxNumSubblockMergeCand is obtained from the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag.

[00413] If sps_affine_enable_flag=1, MaxNumSubblockMergeCand is obtained from the following formula:

MaxNumSubblockMergeCand=5 - five_minus_max_num_subblock_merge_cand.

[00414] For example, five_minus_max_num_subblock_merge_cand can be defined as subtracting the maximum length of the subblock-based merge motion vector prediction list in the slice from 5 (five_minus_max_num_subblock_merge_cand indicates the maximum number of subblock-based merge motion vector prediction (MVP) candidates supported in the slice that is subtracted from 5 ).

[00415] The maximum number of subblock-based merge MVP candidates, MaxNumSubblockMergeCand, is defined as follows:

-- If sps_affine_enabled_flag is 0:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag;

-- Otherwise (sps_affine_enabled_flag is 1):

MaxNumSubblockMergeCand=5 - five_minus_max_num_subblock_merge_cand.

[00416] The MaxNumSubblockMergeCand value must be between 0 and 5 inclusive.

[00417] FIG. 10 shows a description of the second implementation.

[00418] Step S1001: For details of step S1001, see step S901. The details are again not described here. Execute step S1002 or step S1004.

[00419] Step S1002: When the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains affine mode, parse the second pointer and the third pointer from the bitstream, where the second pointer is used to indicate (or determine ) the maximum length of the first candidate motion vector list, and the first candidate motion vector list is a candidate motion vector list created for a block to be processed using the sub-block merge prediction mode.

[00420] Step S1003: When the third pointer indicates that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode, determine the first number based on the third pointer, and determine the maximum length of the first motion vector candidate list based on the second pointer and the first number.

[00421] For an explanation of the first pointer, the second pointer, and the third pointer, refer to the embodiment corresponding to FIG. 9. Details are not described here again.

[00422] When the third pointer indicates that the ATMVP mode is not present in the subblock merge prediction mode, the first number may be equal to the maximum number of motion vectors that are supported in the prediction performed using the ATMVP mode. For example, the third pointer could be sps_sbtmvp_enabled_flag. When sps_sbtmvp_enabled_flag=0, it indicates that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode. In this case, the first number is equal to the maximum number of motion vectors that are supported in the prediction performed using the ATMVP mode. On the contrary, when sps_sbtmvp_enabled_flag=1, it indicates that the extended temporal motion vector prediction mode is present in the subblock merge prediction mode. In this case, the first number is 0. For example, the maximum number of motion vectors that are supported in prediction performed using the ATMVP mode may be 1. In this case, the first number may be equal to the value of the third pointer. For details, refer to the description of the embodiment corresponding to FIG. 9. Details are not described here again.

[00423] In an example, when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains the affine mode, the maximum length of the first list of candidate motion vectors can be obtained according to the following formula:

MaxNumSubblockMergeCand=K - K_minus_max_num_subblock_merge_cand - L1,

where MaxNumSubblockMergeCand represents the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand represents the second pointer, L1 represents the first number, and K is a preset non-negative integer.

[00424] For example, the value of K_minus_max_num_subblock_merge_cand can range from 0 to 3.

[00425] For example, the maximum value of MaxNumSubblockMergeCand can be 5.

[00426] When the maximum value of MaxNumSubblockMergeCand may be 5, the second pointer may be represented by five_minus_max_num_subblock_merge_cand.

[00427] For example, L1 can be obtained by the following formula:

L1=sps_sbtmvp_enabled_flag == 1? 0 : 1. If sps_sbtmvp_enabled_flag=1, L1=0. If sps_sbtmvp_enabled_flag=0, L1=1.

[00428] Step S1004: For details of step S1004, see step S904. The details are again not described here.

[00429] That the maximum value of MaxNumSubblockMergeCand can be 5 is used as an example. If sps_affine_enable_flag=0, MaxNumSubblockMergeCand is obtained from the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag.

[00430] If sps_affine_enable_flag=1, MaxNumSubblockMergeCand is obtained from the following formula:

MaxNumSubblockMergeCand=5 - five_minus_max_num_subblock_merge_cand - (sps_sbtmvp_enabled_flag == 1? 0 : 1).

[00431] For example, five_minus_max_num_subblock_merge_cand can be defined as subtracting the maximum length of the merge motion vector prediction list based on a subblock in a slice from 5 (five_minus_max_num_subblock_merge_cand indicates the maximum number of merge motion vector prediction (MVP) candidates that are subtracted from 5). The maximum number of subblock-based merge MVP candidates, MaxNumSubblockMergeCand, is defined as follows:

-- If sps_affine_enabled_flag is 0:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag;

-- Otherwise (sps_affine_enabled_flag is 1):

MaxNumSubblockMergeCand=5 - five_minus_max_num_subblock_merge_cand - (sps_sbtmvp_enabled_flag == 1? 0 : 1).

[00432] The MaxNumSubblockMergeCand value must be between 0 and 5 inclusive.

[00433] FIG. 11 shows a description of the third implementation.

[00434] Step S1101 - Step S1103: For details of Step S1101 - Step S1103, refer to Step S901 - Step S903. The details are again not described here.

[00435] It should be noted that, in a third implementation, the maximum value of MaxNumSubblockMergeCand may be allowed to be 5. For example, the second pointer may be represented by K_minus_max_num_subblock_merge_cand, where the value of K_minus_max_num_subblock_merge_cand may range from 0 to 4. When the maximum value of MaxNumSubblockMergeCand may be 5, the second a pointer can be represented with five_minus_max_num_subblock_merge_cand.

[00436] Step S1104: When the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, parse the third pointer and the fourth pointer from the bitstream. Execute step S1105, step S1106 or step S1107.

[00437] The third pointer is used to indicate the presence state of the ATMVP mode in the subblock merge prediction mode. For related descriptions of the third pointer, see the descriptions in the embodiment corresponding to FIG. 9. Details are not described here again.

[00438] The fourth pointer is used to indicate the presence state of the PLANAR mode of the motion vector in the subblock merge prediction mode. In other words, the fourth pointer is used to indicate whether the planar mode is allowed to inter-predict the block to be processed.

[00439] Step S1105: When the third pointer indicates that the ATMVP mode is present in the subblock merge prediction mode, and the fourth pointer indicates that the planar mode is not present in the subblock merge prediction mode, determine the third number based on the third pointer, and determine the maximum length of the first list candidate motion vectors based on the third number only.

[00440] For example, when the fourth pointer is the third value, it indicates that the planar mode is not present in the subblock merge prediction mode; or when the fourth pointer is the fourth value, it indicates that the planar mode is present in the subblock merge prediction mode. For example, the third value is 0 and the fourth value is 1. For example, the fourth pointer may be configured in the SPS, PPS, or slice header. The fourth pointer may be represented by sps_planar_enabled_flag.

[00441] Step S1106: When the third pointer indicates that the ATMVP mode is not present in the subblock merge prediction mode, and the fourth pointer indicates that the planar mode is present in the subblock merge prediction mode, determine the fourth number based on the fourth pointer, and determine the maximum length of the first list candidate motion vectors based on the fourth number.

[00442] For example, when the fourth pointer indicates that the planar mode is present in the subblock merge prediction mode, the fourth number is equal to the maximum number of motion vectors that are supported in the prediction performed using the planar mode.

[00443] For example, in step S1106, the maximum length of the first motion vector candidate list is the fourth number. For example, if the maximum number of motion vectors that are supported in the prediction performed using the planar mode is 1, the maximum length of the first motion vector candidate list is 1. In another example, when the fourth pointer is 1, this indicates that the planar mode is present in subblock merge prediction mode. In this case, the maximum length of the first list of candidate motion vectors is equal to the fourth pointer.

[00444] Step S1107: When the third pointer indicates that the ATMVP mode is present in the subblock merge prediction mode, and the fourth pointer indicates that the planar mode is present in the subblock merge prediction mode, determine the third number based on the third pointer, determine the fourth number based on the fourth pointer and determine the maximum length of the first list of candidate motion vectors based on the third number and the fourth number.

[00445] In an example where the first pointer indicates that the candidate mode used for inter-prediction of a block to be processed contains only the rectilinear motion vector prediction mode, the maximum number of motion vectors that are supported in the prediction performed using the planar mode is 1, and the maximum number of motion vectors that are supported in the prediction performed using the ATMVP mode is 1, if the third pointer is 1, this indicates that the ATMVP mode is present in the subblock merge prediction mode; or if the third pointer is 0, this indicates that the ATMVP mode is not present in the subblock merge prediction mode; and if the fourth pointer is 1, it indicates that the PLANAR mode is present in the subblock merge prediction mode; or if the fourth pointer is 0, it indicates that the planar mode is present in the subblock merge prediction mode. In this case, the maximum length of the first list of candidate motion vectors may be equal to the sum of the third pointer and the fourth pointer.

[00446] The third pointer is represented by sps_sbtmvp_enabled_flag and the fourth pointer is represented by sps_planar_enabled_flag. The maximum length of the first list of candidate motion vectors can be obtained from the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag.

[00447] When sps_sbtmvp_enabled_flag=1 (indicates that the ATMVP mode is present in the subblock merge prediction mode), and sps_planar_enabled_flag=0 (indicates that the planar mode is not present in the subblock merge prediction mode), MaxNumSubblockMergeCand = sps_sbtmvp_enabled_flag = 1, which corresponds to step S1105 . When sps_sbtmvp_enabled_flag=0 (indicating that the ATMVP mode is not present in the subblock merge prediction mode) and sps_planar_enabled_flag=1 (indicating that the planar mode is present in the subblock merge prediction mode), MaxNumSubblockMergeCand = sps_planar_enabled_flag = 1, which corresponds to step S1106. When sps_sbtmvp_enabled_flag=1 and sps_planar_enabled_flag=1, MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag=2, which corresponds to step S1107.

[00448] Of course, when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains only the rectilinear motion vector prediction mode, the third pointer indicates that the ATMVP mode is not present in the subblock merge prediction mode, and the fourth the pointer indicates that the planar mode is not present in the subblock merge prediction mode, MaxNumSubblockMergeCand=0.

[00449] That the maximum value of MaxNumSubblockMergeCand can be 5 is used as an example.

[00450] If sps_affine_enable_flag=0, MaxNumSubblockMergeCand is obtained from the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag.

[00451] If sps_affine_enable_flag=1, MaxNumSubblockMergeCand is obtained from the following formula:

MaxNumSubblockMergeCand=5 - five_minus_max_num_subblock_merge_cand.

[00452] For example, five_minus_max_num_subblock_merge_cand can be defined as subtracting the maximum length of the subblock-based merge motion vector prediction list in the slice from 5 (five_minus_max_num_subblock_merge_cand indicates the maximum number of subblock-based merge motion vector prediction (MVP) candidates supported in the slice that is subtracted from 5 ).

[00453] The maximum number of subblock-based merge MVP candidates, MaxNumSubblockMergeCand, is defined as follows:

-- If sps_affine_enabled_flag is 0:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag;

-- Otherwise (sps_affine_enabled_flag is 1):

MaxNumSubblockMergeCand=5 - five_minus_max_num_subblock_merge_cand.

[00454] The MaxNumSubblockMergeCand value must be between 0 and 5 inclusive.

[00455] FIG. 12A and FIG. 12B shows descriptions of the fourth implementation.

[00456] Step S1201: For details of step S1201, see step S901. The details are again not described here. Execute step S1202 or step S1206.

[00457] Step S1202: When the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains the affine mode, parse the second pointer, the third pointer, and the fourth pointer from the bitstream, where the second pointer is used to indicate (or determining) the maximum length of the first candidate motion vector list, and the first candidate motion vector list is a candidate motion vector list created for a block to be processed using the sub-block merge prediction mode. Execute steps S1203, S1204 or step S1205.

[00458] Step S1203: When the third pointer indicates that the extended temporal motion vector prediction mode is present in the subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is not present in the subblock merge prediction mode, determine the second number based on the fourth pointer and determine the maximum length of the first list of candidate motion vectors based on the second pointer and the second number.

[00459] Step S1204: When the third pointer indicates that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is present in the subblock merge prediction mode, determine the first number based on the third pointer and determine the maximum length of the first list of candidate motion vectors based on the second pointer and the first number.

[00460] Step S1205: When the third pointer indicates that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode, and the fourth pointer indicates that the planar motion vector prediction mode is not present in the subblock merge prediction mode, determine the first number based on the third pointer, determine the second number based on the fourth pointer, and determine the maximum length of the first list of candidate motion vectors based on the second pointer, the first number, and the second number.

[00461] When the fourth pointer indicates that the planar mode is not present in the subblock merge prediction mode, the second number may be equal to the maximum number of motion vectors that are supported in the prediction performed using the planar mode. For example, the fourth pointer could be represented by sps_planar_enabled_flag. When sps_planar_enabled_flag=0, it indicates that the planar motion vector prediction mode is not present in the subblock merge prediction mode. In this case, the second number is equal to the maximum number of motion vectors that are supported in the prediction performed using the ATMVP mode. On the contrary, when sps_planar_enabled_flag=1, it indicates that the planar motion vector prediction mode is present in the subblock merge prediction mode. In this case, the second number is 0. For example, the maximum number of motion vectors that are supported in prediction performed using planar mode may be 1. In this case, the second number may be equal to the value of sps_planar_enabled_flag.

[00462] In an example, when the first pointer indicates that the candidate mode used for inter-prediction of the block to be processed contains the affine mode, the maximum length of the first list of candidate motion vectors can be obtained according to the following formula:

MaxNumSubblockMergeCand=K - K_minus_max_num_subblock_merge_cand - L1-L2,

[00463] For example, in the fourth implementation, the value of K_minus_max_num_subblock_merge_cand may range from 0 to 2, or from 0 to 3.

[00464] For example, the maximum value of MaxNumSubblockMergeCand can be 5 or 6.

[00465] When the maximum value of MaxNumSubblockMergeCand may be 5, the second pointer may be represented by five_minus_max_num_subblock_merge_cand. When the maximum value of MaxNumSubblockMergeCand can be 6, the second pointer can be represented with six_minus_max_num_subblock_merge_cand.

[00466] For example, L1 can be obtained by the following formula:

L1=sps_sbtmvp_enabled_flag == 1? 0 : 1. If sps_sbtmvp_enabled_flag=1, L1=0. If sps_sbtmvp_enabled_flag=0, L1=1.

[00467] In the example, L2 can be obtained by the following formula:

L2=sps_planar_enabled_flag == 1? 0 : 1. If sps_planar_enabled_flag=1, L2=0. If sps_sbtmvp_enabled_flag=0, L2=1.

[00468] Steps S1206 - S1209: For details on steps S1206 - S1209, refer to steps S1104 - S1107. The details are again not described here.

[00469] For example, the maximum value of MaxNumSubblockMergeCand may be 5. The value of five_minus_max_num_subblock_merge_cand ranges from 0 to 2.

[00470] If sps_affine_enable_flag=0, MaxNumSubblockMergeCand is obtained from the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag.

[00471] Otherwise (if sps_affine_enable_flag=1) MaxNumSubblockMergeCand is obtained from the following formula:

MaxNumSubblockMergeCand=5 - five_minus_max_num_subblock_merge_cand - (sps_sbtmvp_enabled_flag == 1? 0 : 1) - (sps_planar_enabled_flag == 1? 0 : 1),

where (five_minus_max_num_subblock_merge_cand indicates the maximum number of subblock-based merge motion vector prediction (MVP) candidates supported in a slice subtracted from 5), the maximum number of subblock-based merge MVP candidates, MaxNumSubblockMergeCand, is defined as follows:

-- If sps_affine_enabled_flag is 0:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag;

-- Otherwise (sps_affine_enabled_flag is 1):

MaxNumSubblockMergeCand=5 - five_minus_max_num_subblock_merge_cand - (sps_sbtmvp_enabled_flag == 1? 0 : 1) - (sps_planar_enabled_flag == 1? 0 : 1).

[00472] The MaxNumSubblockMergeCand value must be between 0 and 5 inclusive.

[00473] For example, the maximum value of MaxNumSubblockMergeCand may be 6. The value of five_minus_max_num_subblock_merge_cand ranges from 0 to 3.

[00474] If sps_affine_enable_flag=0, MaxNumSubblockMergeCand is obtained from the following formula:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag;

[00475] Otherwise (if sps_affine_enable_flag=1) MaxNumSubblockMergeCand is obtained from the following formula:

MaxNumSubblockMergeCand=6 - six_minus_max_num_subblock_merge_cand - (sps_sbtmvp_enabled_flag == 1? 0 : 1) - (sps_planar_enabled_flag == 1? 0 : 1).

where (six_minus_max_num_subblock_merge_cand indicates the maximum number of subblock-based merge motion vector prediction (MVP) candidates supported in a slice subtracted from 6), the maximum number of subblock-based merge MVP candidates, MaxNumSubblockMergeCand, is defined as follows:

-- If sps_affine_enabled_flag is 0:

MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag;

-- Otherwise (sps_affine_enabled_flag is 1):

MaxNumSubblockMergeCand=6 - six_minus_max_num_subblock_merge_cand - (sps_sbtmvp_enabled_flag == 1? 0 : 1) - (sps_planar_enabled_flag == 1? 0 : 1).

[00476] The MaxNumSubblockMergeCand value must be between 0 and 6 inclusive.

[00477] For the second case, the planar motion vector prediction mode is not considered to be present in the subblock merge mode. In other words, when the sub-block merging mode may comprise at least one of the affine mode and the extended motion vector temporal prediction mode, the first implementation or the second implementation may be used. Details are not described here again.

[00478] Based on the same inventive concept as in the embodiments of said method, an embodiment of this application further provides hardware. As shown in FIG. 13, hardware 1300 may in particular be a processor in a video decoder, a chip, chip system, or module in a video decoder, such as an entropy decoder 304 and/or an inter prediction 344.

[00479] For example, the hardware may include a parser 1301 and a determiner 1302. The parser 1301 and the determiner 1302 perform the method steps described in the embodiments corresponding to FIG. 9 - fig. 12A and FIG. 12b. For example, parser 1301 may be configured to parse the pointers (e.g., first pointer, second pointer, third pointer, or fourth pointer) contained in the bitstream, and determiner 1302 is configured to determine the maximum length of the first list of candidate motion vectors.

[00480] An embodiment of this application further provides a different hardware structure used in the decoder. As shown in FIG. 14, hardware 1400 may include a communications interface 1410 and a processor 1420. Optionally, hardware 1400 may further include a memory 1430. Memory 1430 may be located inside or outside the hardware. Both the parser 1301 and the determiner 1302 shown in FIG. 13 may be implemented by processor 1420. Processor 1420 sends or receives a video stream or bit stream via communication interface 1410 and is configured to implement the methods shown in FIG. 9 - fig. 12A and FIG. 12b. During implementation, the steps in the processing procedure may be performed using a hardware integrated logic circuit in processor 1420 or an instruction in the form of software to complete the methods shown in FIG. 9 - fig. 12A and FIG. 12b.

[00481] The communication interface 1410 in this embodiment of this application may be a circuit, a bus, a transceiver, or any other hardware that can be configured to exchange information. For example, the other equipment may be a device connected to the equipment 1400. For example, if the equipment is a video encoder, the other equipment may be a video decoder.

[00482] In this embodiment of this application, processor 1420 may be a general purpose processor, a digital signal processor, an application specific integrated circuit, a field-programmable gate array or other programmable logic device, a discrete gate or transistorized logic device, or a discrete hardware component and may implement or perform the methods, steps, and flowcharts disclosed in the embodiments of this application. A general purpose processor may be a microprocessor, any conventional processor, or the like. The steps of the methods disclosed with reference to embodiments of this application may be performed directly by a hardware processor or may be performed using a combination of hardware in the processor and a software module. Program code executed by processor 1420 to implement the above methods may be stored in memory 1430. Memory 1430 is associated with processor 1420.

[00483] The connection in this embodiment of this application may be an indirect connection or communication connection between devices, blocks or modules in electrical, mechanical or other form and is used to exchange information between devices, blocks, or modules.

[00484] Processor 1420 may work in conjunction with memory 1430. Memory 1430 may be non-volatile memory, such as a hard disk drive (HDD) or solid state drive (solid-state drive, SSD), or may be volatile memory (volatile memory ), for example, RAM (random-access memory, RAM). Memory 1430 is any other medium that can be configured to carry or store expected program code in the form of an instruction or data structure and that can be accessed by, but is not limited to, a computer.

[00485] In this embodiment of this application, the specific communication medium between the communication interface 1410, the processor 1420, and the memory 1430 is not limited. In this embodiment of this application, memory 1430, processor 1420, and communications interface 1410 are connected via the bus in FIG. 14. The tire is represented by the bold line in FIG. 14. The mode of connection between other components is simply described schematically and is not limited thereto. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used to represent the tire in Fig. 14, but this does not mean that there is only one tire or only one type of tire.

[00486] The above possible implementations and specific embodiments related to FIG. 9 - fig. 12A and FIG. 12B provide descriptions of one or more video decoding apparatuses in this application. It should be understood that, according to the foregoing description, the encoder side typically determines the inter prediction mode and encodes the inter prediction mode in the bitstream. After the inter prediction mode is finally selected, the pointer (for example, the first pointer, the second pointer, the third pointer, or the fourth pointer described above) for the inter prediction mode is encoded in the bitstream using an encoding process that is the complete inverse of the above decoding method ( the encoding process corresponds to the decoding process of parsing the first pointer, second pointer, third pointer, or fourth pointer). It should be understood that the definition of the maximum length of the first list of candidate motion vectors by the encoder side is fully consistent with the definition by the decoder side. The specific embodiment of the encoder side is not described. However, it should be understood that the video prediction method described in this application is also used in the encoding device.

[00487] An embodiment of this application further provides a device used in an encoder. As shown in FIG. 15, hardware 1500 may include a communication interface 1510 and a processor 1520. Optionally, hardware 1500 may further include a memory 1530. Memory 1530 may be located inside or outside the hardware. The processor 1520 sends or receives a video stream or bitstream through the communication interface 1510.

[00488] In one aspect, processor 1520 is configured to: encode a first pointer in a bitstream; and when the first pointer indicates that the candidate mode used for inter prediction of the block to be processed contains an affine mode, encode the second pointer in the bitstream, the second pointer being used to indicate the maximum length of the first list of candidate motion vectors, and the first list of candidate motion vectors of motion vectors is a list of candidate motion vectors created for a block to be processed using the sub-block merge prediction mode.

[00489] The communication interface 1510 in this embodiment of this application may be a circuit, a bus, a transceiver, or any other hardware that can be configured to exchange information. For example, the other equipment may be a device connected to the equipment 1500. For example, if the equipment is a video encoder, the other equipment may be a video decoder.

[00490] In this embodiment of this application, the processor 1520 may be a general purpose processor, a digital signal processor, an application specific integrated circuit, a field-programmable gate array or other programmable logic device, a discrete gate or transistorized logic device, or a discrete hardware component and may implement or perform the methods, steps, and flowcharts disclosed in the embodiments of this application. A general purpose processor may be a microprocessor, any conventional processor, or the like. The steps of the methods disclosed with reference to embodiments of this application may be performed directly by a hardware processor or may be performed using a combination of hardware in the processor and a software module. Program code executed by processor 1520 to implement the above methods may be stored in memory 1530. Memory 1530 is associated with processor 1520.

[00491] The connection in this embodiment of this application may be an indirect connection or communication connection between apparatuses, blocks, or modules in electrical, mechanical, or other form, and is used to exchange information between apparatuses, blocks, or modules.

[00492] The processor 1520 may operate in conjunction with the memory 1530. The memory 1530 may be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or may be a volatile memory (volatile memory), for example, RAM (random-access memory, RAM). Memory 1530 is any other medium that can be configured to carry or store expected program code in the form of an instruction or data structure and that can be accessed by, but is not limited to, a computer.

[00493] In this embodiment of this application, the specific communication medium between the communication interface 1510, the processor 1520, and the memory 1530 is not limited. In this embodiment of this application, memory 1530, processor 1520, and communication interface 1510 are connected via the bus in FIG. 15. The tire is represented by the bold line in FIG. 15. The mode of connection between other components is simply described schematically and is not limited thereto. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used to represent the tire in Fig. 15, but this does not mean that there is only one tire or only one type of tire.

[00494] Based on the embodiments described above, an embodiment of this application further provides a computer storage medium. The media stores the software. When the software is read and executed by one or more processors, the method provided in any one or more of the above embodiments may be implemented. A computer storage medium may include any medium that can store program code, such as a USB flash drive, removable hard disk, read only memory, random access memory, magnetic disk, or optical disk.

[00495] Based on the embodiments described above, an embodiment of this application further provides a chip. The chip contains a processor configured to implement the functions in any one or more of the above embodiments, such as receiving or processing information or messages used in the above methods. Optionally, the chip further comprises memory. The memory is configured to store program instructions and data that are needed and executed by the processor. A microcircuit may contain a microcircuit or may contain a microcircuit and other discrete device.

[00496] While specific aspects of this application have been described with reference to video encoder 20 and video decoder 30, it should be understood that the technologies of this application may be used by many other video encoding and/or decoding modules, processors, processing modules, and hardware decoding units, etc. .p., for example encoders/decoders (CODEC). In addition, it should be understood that the steps described and shown in FIG. 8A in FIG. 12A and FIG. 12B are presented merely as possible implementations. In other words, the steps described in the possible implementations in FIG. 8A in FIG. 12A and FIG. 12B are not necessarily executed in the order shown in FIG. 8A in FIG. 12A and FIG. 12B, and fewer steps, additional steps, or alternative steps may be performed.

[00497] In addition, it should be understood that, depending on possible implementations, specific actions or events in any of the ways described in this specification may be performed in different orders, an action or event may be added, or actions or events may be combined, or omitted (eg, not all actions or events described are necessary to implement the methods). In addition, in a particular possible implementation, actions or events may (for example) be threaded or interrupted, or may be processed by multiple processors simultaneously rather than sequentially. Furthermore, while specific aspects of this application are described as being performed by a single unit or device for clarity, it should be understood that the technologies of this application may be performed by a combination of units or modules associated with a video decoder.

[00498] In one or more possible implementations, the described functions may be implemented using hardware, software, firmware, or any combination thereof. If the functions are implemented using software, the functions may be stored on a computer-readable medium as one or more instructions or code, or transmitted over a computer-readable medium and executed by a hardware processing unit. The computer-readable medium may comprise a computer-readable storage medium or communication medium. A computer-readable storage medium corresponds to a tangible medium, such as a storage medium. A communication medium includes any medium that facilitates the transfer of a computer program (for example) from one place to another in accordance with a communication protocol.

[00499] Thus, a computer-readable medium may correspond to, for example, (1) a non-volatile tangible computer-readable storage medium, or (2) a communication medium such as a signal or a medium. A storage medium can be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the technologies described in this application. The computer program product may include a computer-readable medium.

[00500] As a possible implementation, and not limitation, a computer-readable storage medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage equipment, magnetic disk storage equipment or other magnetic storage equipment, flash memory, or any other media. A that can be used to store the required code in the form of an instruction or data structure and that can be accessed by a computer. Similarly, any connection can be appropriately referred to as a computer-readable medium. For example, if an instruction is transmitted from a website, server, or other remote source via coaxial cable, optical fiber, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio and microwave, coaxial cable, optical fiber, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are contained in the media definition.

[00501] However, it should be understood that a computer-readable storage medium and a storage medium may not contain a connection, medium, signal, or other temporary medium, but, alternatively, mean non-volatile tangible storage media. The magnetic disc and optical disc described in this specification include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc. A magnetic disc usually reproduces data magnetically, while an optical disc reproduces data optically using a laser. The combination of the aforementioned magnetic disk and optical disk should also be included in the scope of the computer-readable medium.

[00502] An instruction may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used in this specification may be any of the above structures, or another structure that is used to implement the technologies described in this specification. In addition, in some aspects, the functions described in this specification may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or may be included in a combined codec. Likewise, all technologies may be implemented in one or more circuits or logic elements.

[00503] The technologies in this application may be implemented in various apparatuses or devices, including a wireless mobile phone, an integrated circuit (IC), or an IC stack (eg, a chipset). Various components, modules, or blocks are described in this application to emphasize the functional aspects of a device configured to perform the disclosed technologies, but are not necessarily implemented in different hardware blocks. More specifically, as described above, the various blocks may be combined into a codec hardware block or provided with cooperating hardware blocks (including one or more of the processors described above) in combination with an appropriate set of software and/or firmware.

[00504] The above descriptions are merely examples of specific implementations of this application, but are not intended to limit the scope of protection of this application. Any change or replacement readily detectable by a person skilled in the art within the technical scope disclosed in this application should/should fall within the scope of this application. Therefore, the scope of this application is protected by the claims.

Claims (37)

1. Video prediction equipment, comprising: a parser configured to parse from the sequence parameter set, SPS, in the bitstream, the value of the first pointer sps_affine_enabled_flag and the value of the third pointer sps_sbtmvp_enabled_flag; wherein the parser is configured to parse the second pointer value of said SPS when the first pointer value is equal to the first value; a determining unit configured to determine the maximum length of the first motion vector candidate list based on the second pointer when the value of the first pointer is equal to the first value, the first motion vector candidate list being the motion vector candidate list created for the block to be processed, merge prediction mode subblocks is used for the block to be processed; wherein the determiner is configured to determine the maximum length of the first list of candidate motion vectors based on the value of the third pointer when the value of the first pointer is equal to the second value, the second value being different from the first value. 2. Apparatus according to claim 1, wherein the maximum length of the first list of candidate motion vectors is obtained according to the following formula: MaxNumSubblockMergeCand=K−K_minus_max_num_subblock_merge_cand, wherein MaxNumSubblockMergeCand represents the maximum dyne of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand represents the second pointer, and K is a predetermined non-negative integer. 3. The apparatus according to claim 1 or 2, wherein determining the maximum length of the first list of candidate motion vectors based on the value of the third pointer comprises: determining that the maximum length of the first list of candidate motion vectors is 0 when the value of the third pointer is 0. 4. Apparatus according to claim 1 or 3, wherein determining the maximum length of the first list of candidate motion vectors based on the value of the third pointer comprises: determining that the maximum length of the first list of candidate motion vectors is 1 when the value of the third pointer is 1. 5. The apparatus of claim 4, wherein the value of the third indicator is 1, indicating that the extended temporal motion vector prediction mode is present in the sub-block merge prediction mode. 6. The apparatus of claim 3, wherein the value of the third pointer is 0 indicating that the extended temporal motion vector prediction mode is not present in the subblock merge prediction mode. 7. The apparatus of claim 5 or 6, wherein the extended temporal motion vector prediction mode is a prediction mode based on the temporal motion vector of a sub-unit. 8. Equipment according to any one of paragraphs. 1-7, wherein the value of the first pointer indicates whether the candidate mode used for inter-prediction of the block to be processed contains the affine mode. 9. Equipment according to any one of paragraphs. 1-8, where the first value is 1, the second value is 0. 10. Decoder, containing: CPU; and memory storing instructions and connected to the processor; moreover, instructions, when executed by the processor, force the processor: parse, from the sequence parameter set, SPS, in the bitstream, the value of the first pointer sps_affine_enabled_flag and the value of the third pointer sps_sbtmvp_enabled_flag; parse the value of the second pointer from the SPS when the value of the first pointer is equal to the first value; determine the maximum length of the first motion vector candidate list based on the second pointer, when the value of the first pointer is equal to the first value, the first motion vector candidate list being the motion vector candidate list created for the block to be processed, the sub-block merge prediction mode is used for the block, which is to be processed; and determine the maximum length of the first list of candidate motion vectors based on the value of the third pointer when the value of the first pointer is equal to the second value, and the second value is different from the first value. 11. The decoder of claim 10, wherein the maximum length of the first list of candidate motion vectors is obtained according to the following formula: MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand, where MaxNumSubblockMergeCand represents the maximum length of the first list of candidate motion vectors, K_minus_max_num_subblock_merge_cand represents the second pointer, and K is a preset non-negative integer. 12. The decoder according to claim 10 or 11, and the processor is configured to: determining that the maximum length of the first list of candidate motion vectors is 0 when the value of the third pointer is 0. 13. Decoder according to any one of paragraphs. 10-12, wherein the processor is configured to: determining that the maximum length of the first list of candidate motion vectors is 1 when the value of the third pointer is 1. 14. The decoder of claim 13, wherein the value of the third indicator is 1, indicating that the extended temporal motion vector prediction mode is present in the sub-block merge prediction mode. 15. The decoder of claim 12, wherein the value of the third pointer is 0 indicates that the extended temporal motion vector prediction mode is not present in the sub-block merge prediction mode. 16. The decoder of claim 14 or 15, wherein the extended temporal motion vector prediction mode is a prediction mode based on the sub-block temporal motion vector. 17. Decoder according to any one of paragraphs. 10-16, wherein the value of the first pointer indicates whether the candidate mode used for inter-prediction of a block to be processed contains an affine mode. 18. Decoder according to any one of paragraphs. 10-17, where the first value is 1, the second value is 0.

Images