RU2810126C2 - Method and device for image prediction and computer readable data medium - Google Patents

Abstract

FIELD: video encoding technologies.

SUBSTANCE: invention relates to a method and device for image prediction. The technical result is to increase the efficiency of video coding by reducing redundancy in the image prediction process. The result is achieved in that the image prediction method includes: determining whether a merge mode is used for the current image block; when the merge mode is used for the current image block, determining whether the level-1 merge mode is available for the current image block; if the first merge mode is not available, and the high-level syntax element corresponding to the first merge mode indicates that the first merge mode is prohibited from being used, directly defining the second merge mode as the target merge mode applicable to the current image block, wherein both the first merge mode and the second merge mode belong to the level-2 merge mode; and predicting the current image block based on the target fusion mode.

EFFECT: increased efficiency of video coding by reducing redundancy in the image prediction process.

24 cl, 18 dwg, 7 tbl

Description

[0001] This application claims priority to Chinese Patent Application No. 201910341218.6, filed with the National Intellectual Property Office of China on April 25, 2019, entitled “METHOD AND DEVICE FOR ENCODING/DECODING VIDEO IMAGES,” which is incorporated herein by reference in its entirety.

[0002] This application claims priority to Chinese Patent Application No. 201910474007.X, filed with the National Intellectual Property Office of China on June 2, 2019, entitled "METHOD AND APPARATUS FOR IMAGE PREDICTION, COMPUTER READABLE DATA MEDIUM", which is incorporated herein in its entirety via link.

FIELD OF TECHNOLOGY TO WHICH THE INVENTION RELATES

[0003] The present application relates to the field of video encoding technology, and more particularly to an image prediction method and apparatus, as well as to a computer readable storage medium.

BACKGROUND OF THE ART

[0004] Digital video capabilities can be included in a variety of devices, including digital television, digital direct broadcast system, wireless broadcast system, personal digital assistant (PDA), laptop or desktop computer, tablet, reader e-readers, digital camera, digital recorder, digital media player, video game device, game console, cellular or satellite radio telephone (namely, “smartphone”), video conferencing device, video streaming device, etc. The digital video device implements video compression technologies, such as the video compression technologies described in the standards defined in MPEG-2, MPEG-4, ITU-T H.263, and ITU-T H.264/MPEG-4 Part 10 Advanced Coding video (AVC, advanced video coding), H.265 video coding standard/high efficiency video coding (HEVC) standard and extensions of such standards. A video device can more efficiently transmit, receive, encode, decode and/or store digital video information by implementing such video compression technologies.

[0005] Video compression technologies are used to perform spatial (within-image) prediction and/or temporal (between-image) prediction to reduce or remove inherent redundancy in video sequences. In block video encoding, a video fragment (namely, a video frame or part of a video frame) can be divided into image blocks, and the image block can also be called a tree block, a coding unit (CU, coding unit) and/or a coding unit. A block of an image in a sub-image to be intra-coded (I) is encoded by spatial prediction based on a reference sample in an adjacent block in the same image. For a picture block in a sub-picture to be intra-frame encoded (P or B), spatial prediction based on a reference sample in an adjacent block in the same picture or temporal prediction based on a reference sample in another reference picture may be used. The image may be called a frame, and the reference image may be called a reference frame.

[0006] When a fusion mode is used to predict an image block, there are generally many optional fusion modes. In the traditional solution, the merge mode applicable to the current image block is usually determined from a plurality of possible merge modes one after another. When a merge mode is not available, it continues to determine whether the next merge mode is available. In the traditional solution, there is redundancy where the merge mode applicable to the current block is determined from the last two remaining merge modes.

SUMMARY OF THE INVENTION

[0007] The present application provides an image prediction method and apparatus, as well as a computer readable storage medium, to reduce redundancy in the image prediction process as much as possible.

[0008] According to a first aspect, an image prediction method is provided. The method includes: determining whether a merge mode is used for the current image block; when the merge mode is used for the current image block, continuing to determine whether the layer-1 merge mode is available; when the level-1 merge mode is not available and the high-level syntax element corresponding to the first merge mode indicates that the first merge mode is prohibited from being used, defining the second merge mode as the target merge mode applicable to the current image block; and predicting the current image block based on the target fusion mode.

[0009] Both the first merging mode and the second merging mode belong to the layer-2 merging mode, and the layer-2 merging mode includes the first merging mode and the second merging mode. In addition, for the current image block, the level-1 fusion mode and level-2 fusion mode already include all the optional fusion modes of the current image block, and for the current image block, it is necessary to determine the final target fusion mode from the level-1 fusion mode and the fusion mode level-2.

[0010] Optionally, the priority of the layer-1 merge mode is higher than the priority of the layer-2 merge mode.

[0011] That the priority of the layer-1 merge mode is higher than the priority of the layer-2 merge mode means that in the process of determining the target merge mode of the current image block, the target merge mode is preferably determined from the layer-1 merge mode. If there is no merge mode available in the level-1 merge mode, the target merge mode is then determined from the level-2 merge mode.

[0012] Optionally, determining whether a merge mode is used for the current image block includes: when a merge_flag corresponding to the current image block is 1, determining that the merge mode is used for the current image block; and when the merge_flag corresponding to the current image block is 0, determining that the merge mode is not used for the current image block.

[0013] It should be understood that when it is determined that the merge mode is not used for the current image block, a mode other than the merge mode may be used to predict the current image block. For example, when it is determined that the fusion mode is not used for the current image block, the extended motion vector AMVP mode may be used to predict the current image block.

[0014] In the present application, when the high-level syntax element of the first merge mode indicates that the use of the first merge mode is prohibited, there is no need to analyze the available status information of the remaining second merge mode, and the second merge mode can be directly determined as the final target merge mode. This can reduce, as much as possible, the redundancy generated due to determining the target fusion mode in the image prediction process.

[0015] Optionally, the method further includes: determining whether the layer-1 merge mode is available.

[0016] Specifically, whether the layer-1 merge mode is available is determined based on a high-level syntax element corresponding to the layer-1 merge mode and/or available status information corresponding to the layer-1 merge mode.

[0017] With respect to the first aspect, in some implementations of the first aspect, when the layer-1 merge mode is not available and the high-level syntax element corresponding to the first merge mode indicates that the first merge mode is enabled, the target merge mode is determined based on the syntax element a high level corresponding to the second merge mode, and/or information about the available status of the second merge mode.

[0018] Information about the available status of the second merge mode is used to indicate whether the second merge mode is used when the current image block is predicted.

[0019] For example, the second merge mode is the CIIP mode, and the available status information of the second merge mode is the ciip_flag value. When ciip_flag is 0, CIIP mode is not available for the current image block. When ciip_flag is 1, CIIP mode is enabled for the current image block.

[0020] It should be understood that for the CIIP mode, if the CIIP mode is to be selected as the target merge mode, the high-level syntax element corresponding to CIIP must indicate that the CIIP mode is enabled for use and available status information indicating the available status CIIP mode must indicate that CIIP is available.

[0021] For example, when sps_ciip_enabled_flag=1 and ciip_flag=1, the CIIP mode may be determined as the merge target mode of the current image block.

[0022] With respect to the first aspect, in some implementations of the first aspect, that the target merge mode is determined based on a high-level syntax element corresponding to the second merge mode and/or information about the available status of the second merge mode includes: when the syntax element high level corresponding to the second merge mode, and/or information about the available status of the second merge mode indicates that the second merge mode is prohibited from being used, the first merge mode is determined to be the target merge mode.

[0023] That the high-level syntax element corresponding to the second merge mode and/or the second merge mode available status information indicates that use of the second merge mode is prohibited includes:

[0024] The high-level syntax element corresponding to the second merge mode indicates that the second merge mode is prohibited from being used, and information about the available status of the second merge mode indicates that the second merge mode cannot be used; and a high-level syntax element corresponding to the second merge mode indicates that the second merge mode is allowed to be used, and information about the available status of the second merge mode indicates that the second merge mode cannot be used.

[0025] Optionally, the target merge mode is determined based on the high-level syntax element corresponding to the second merge mode and/or information about the available status of the second merge mode, further including: when the high-level syntax element corresponding to the second merge mode indicates that the second merge mode is enabled for use, and the available status information of the second merge mode indicates that the second merge mode is available, the second merge mode is determined to be the target merge mode.

[0026] With respect to the first aspect, in some implementations of the first aspect, before the target merge mode is determined based on a high-level syntax element corresponding to the second merge mode and/or information about the available status of the second merge mode, the method further includes : determining that at least one of the following conditions is true: the size of the current image block meets a specified condition; and skip mode is not used to predict the current image block.

[0027] In other words, before determining the target merge mode, it is necessary to further ensure that the size of the current image block meets the condition, and the skip mode is not used for the current image block. Otherwise, a mode other than the merge mode may be used to predict the current image block.

[0028] With respect to the first aspect, in some implementations of the first aspect, the size of the current image block meets a given condition includes: the current image block satisfies the following three conditions:

(cdWidth * cbHeight) ≥ 64;

cbWidth < 128; And

cbHeight < 128.

[0029] cdWidth is the width of the current image block, and cbHeight is the height of the current image block.

[0030] With respect to the first aspect, in some implementations of the first aspect, the first fusion mode includes a triangular division TPM mode, and the second fusion mode includes a combined intra- and inter-frame prediction CIIP mode.

[0031] Optionally, when a high-level syntax element corresponding to a TPM mode indicates that the TPM mode is prohibited from being used, the CIIP mode is defined as the target merge mode.

[0032] In the present application, when a high-level syntax element corresponding to a TPM mode indicates that the TPM mode is prohibited from being used, it is not necessary to determine by analyzing the high-level syntax corresponding to the CIIP mode and/or the available status information that indicates the available status CIIP mode, whether CIIP mode is available. Instead, the CIIP mode can be directly defined as the target merge mode. This can reduce redundancy in the process of determining the target merge mode.

[0033] Optionally, when the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is enabled for use, the target merge mode is determined based on the high-level syntax element corresponding to the CIIP mode and/or available status information indicating available CIIP mode status.

[0034] Optionally, when the high-level syntax element corresponding to the CIIP mode and/or the available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, the TPM mode is determined to be the target merge mode.

[0035] Optionally, when a high-level syntax element corresponding to a CIIP mode indicates that the CIIP mode is enabled for use, and the available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is available, the CIIP mode is determined to be the target mode mergers.

[0036] With respect to the first aspect, in some implementations of the first aspect, before the target merge mode is determined based on a high-level syntax element corresponding to the second merge mode and/or information about the available status of the second merge mode, the method further includes: determining that the type of the tile or group of tiles in which the current image block is located is B; and determining that the maximum number of possible TPM modes supported by the tile or group of tiles in which the current image block is located is greater than or equal to 2.

[0037] Optionally, before the target merge mode is determined based on a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode, the method further includes: determining that the type is fragment or the group of fragments in which the current image block is located is B; and determining that the maximum number of possible TPM modes supported by the tile or group of tiles in which the current image block is located is greater than or equal to 2.

[0038] With respect to the first aspect, in some implementations of the first aspect, the first fusion mode is a triangular division TPM mode, and the second fusion mode is a combined intra- and inter-frame prediction CIIP mode. The method further includes: when the layer-1 merge mode is not available, and the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is enabled for use, but the current image block does not meet at least one of condition A and condition B, defining CIIP mode as the target merge mode.

[0039] Condition A and condition B are as follows:

[0040] Condition A: The fragment type in which the current image block is located is B.

[0041] Condition B: The maximum number of possible TPM modes supported by the tile or group of tiles in which the current image block is located is greater than or equal to 2.

[0042] The TPM mode may be selected as the target fusion mode finally used to predict the current image block only when both condition A and condition B are satisfied.

[0043] On the one hand, if condition A or condition B is not satisfied, the CIIP mode is determined as the target merge mode.

[0044] On the other hand, when a high-level syntax element corresponding to a TPM mode specifies that the TPM mode is prohibited from being used if condition A or condition B is not satisfied, the CIIP mode is determined to be the target merge mode.

[0045] On the other hand, when a high-level syntax element corresponding to a TPM mode indicates that the TPM mode is enabled for use if condition A or condition B is not satisfied, the CIIP mode is determined to be the target merge mode.

[0046] In other words, the CIIP mode can be defined as the target merge mode under the condition that one of sps_trangle_enabled_flag=1, condition A and condition B is not satisfied.

[0047] On the other hand, if sps_trangle_enabled_flag=1, condition A and condition B are all satisfied, the target merge mode must be determined based on the ciip_flag according to several conditions in the prior art.

[0048] With respect to the first aspect, in some implementations of the first aspect, the high-level syntax element is a syntax element at at least one of the sequence level, the image level, the fragment level, and the fragment group level.

[0049] With respect to the first aspect, in some implementations of the first aspect, the layer-1 merging mode includes a conventional merging mode, a motion vector difference MMVD merging mode, and a sub-block merging mode.

[0050] When it is determined whether the layer-1 merging mode is available, whether these modes are available can be sequentially determined in a sequence of the normal merging mode, the MMVD mode, and the sub-block merging mode.

[0051] For example, it may first be determined whether the normal merge mode is available. When normal merge mode is not available (if normal merge mode is available, normal merge mode can directly be used as the final target merge mode), it continues to determine whether MMVD mode is available. When MMVD mode is not available, it continues to determine whether subblock merging mode is available.

[0052] With respect to the first aspect, in some implementations of the first aspect, the method further includes: when a layer-1 merge mode is not available, determining a target merge mode from a layer-2 merge mode, wherein the layer-2 merge mode includes a TPM mode and a CIIP; and when CIIP mode is enabled and one of the following conditions is not met, specifying CIIP mode as the target merge mode:

[0053] Condition D: TPM mode is enabled.

[0054] Condition E: Skip mode is not used to predict the current image block.

[0055] Condition F: (cbWidth * cbHeight) ≥ 64.

[0056] Condition G: cbWidth < 128.

[0057] Condition H: cbHeight < 128.

[0058] cbWidth is the width of the current image block, and cbHeight is the height of the current image block.

[0059] With respect to the first aspect, in some implementations of the first aspect, a prediction method is applied to the encoder side to encode the current image block.

[0060] Regarding the first aspect, in some implementations of the first aspect, a prediction method is applied to the decoder side to decode the current image block.

[0061] According to a second aspect, an image prediction method is provided. The method includes: determining whether a merge mode is used for the current image block; when the merge mode is used for the current image block, determining whether the level-1 merge mode is available; when the layer-1 merge mode is not available, determining the target merge mode from the layer-2 merge mode, where the layer-2 merge mode includes a TPM mode and a CIIP mode; When CIIP mode is enabled and one of the following conditions is not met (condition 1 - condition 5), specify CIIP mode as the target merge mode:

[0062] Condition 1: TPM mode is allowed.

[0063] Condition 2: The type of fragment or group of fragments in which the current image block is located is B.

[0064] Condition 3: The maximum number of possible TPM modes supported by the tile or group of tiles in which the current image block is located is determined to be greater than or equal to 2.

[0065] Condition 4: The size of the current image block meets the specified condition.

[0066] Condition 5: Skip mode is not used to predict the current image block.

[0067] The first condition may be specifically represented by sps_triangle_enabled_flag=1, the second condition may be specifically represented by slice_type == B, and the third condition may be specifically represented by MaxNumTriangleMergeCand ≥ 2. MaxNumTriangleMergeCand indicates the maximum number of possible TPM modes supported by the fragment or group of fragments, in which the current image block is located.

[0068] In addition, for the current image block, the level-1 fusion mode and the level-2 fusion mode may include all optional fusion modes of the current image block, and for the current image block, it is necessary to determine the final target fusion mode from the level-1 fusion mode and level-2 fusion mode.

[0069] Optionally, the priority of the layer-1 merge mode is higher than the priority of the layer-2 merge mode.

[0070] That the priority of the layer-1 merge mode is higher than the priority of the layer-2 merge mode means that in the process of determining the target merge mode of the current image block, the target merge mode is preferably determined from the layer-1 merge mode. If there is no merge mode available in the level-1 merge mode, the target merge mode is then determined from the level-2 merge mode.

[0071] Optionally, the size of the current image block meets a predetermined condition includes: the current image block meets the following three conditions:

(cdWidth * cbHeight) ≥ 64;

cbWidth < 128; And

cbHeight < 128.

[0072] Optionally, determining whether a merge mode is used for the current image block includes: when a merge_flag corresponding to the current image block is 1, determining that the merge mode is used for the current image block; and when the merge_flag corresponding to the current image block is 0, determining that the merge mode is not used for the current image block.

[0073] It should be understood that when it is determined that the merge mode is not used for the current image block, a mode other than the merge mode may be used to predict the current image block. For example, when it is determined that the fusion mode is not used for the current image block, the extended motion vector AMVP mode may be used to predict the current image block.

[0074] Optionally, the layer-1 merging mode includes a normal merging mode, an MMVD mode, and a sub-block merging mode.

[0075] When it is determined whether the layer-1 merging mode is available, whether these modes are available can be sequentially determined in a sequence of the normal merging mode, the MMVD mode, and the sub-block merging mode. When all modes are unavailable, it is determined that layer-1 merge mode is unavailable.

[0076] In the present application, when the layer-1 merge mode is not available, based on some specified conditions, it can be determined whether the CIIP mode should be selected as the final merge mode, and the CIIP mode can be directly determined as the target merge mode under the condition that none of the specified conditions are met. This reduces the redundancy that occurs in the merge target determination process.

[0077] With respect to the second aspect, in some implementations of the second aspect, determining a target merge mode from a layer-2 merge mode includes: when any of condition 1 to condition 5 is not satisfied, setting a value of available status information that indicates the available status of the mode CIIP for the first value, wherein when the available status information value that indicates the available status of the CIIP mode is the first value, the CIIP mode is used to perform image prediction to the current image block.

[0078] It should be understood that setting the value of the available status information, which indicates the available status of the CIIP mode for the first value herein, is equivalent to specifying CIIP as the target merge mode.

[0079] Optionally, the available status information indicating the available status of the CIIP mode is ciip_flag.

[0080] Setting the value of the available status information that indicates the available status of the CIIP mode to a first value may, in particular, set ciip_flag to 1.

[0081] In addition, when the value of the available status information that indicates the available status of the CIIP mode is set to the second value, it may indicate that the CIIP mode is not used to perform image prediction for the current image block. For example, when the available status information that indicates the available status of the CIIP mode is ciip_flag and ciip_flag=0, the CIIP mode is not used to perform image prediction for the current image block.

[0082] With respect to the second aspect, in some implementations of the second aspect, determining a target merge mode from a layer-2 merge mode includes: when all conditions from condition 1 to condition 5 are satisfied, determining the target merge mode based on a high-level syntax element corresponding CIIP mode, and/or available status information that indicates the available status of the CIIP mode, wherein the available status information that indicates the available status of the CIIP mode is used to indicate whether the CIIP mode is used when the current picture block is predicted.

[0083] For example, available status information that indicates the available status of the CIIP mode is a ciip_flag value. When ciip_flag is 0, CIIP mode is not available for the current image block. When ciip_flag is 1, CIIP mode is enabled for the current image block.

[0084] In the present application, the target merge mode may be determined based on a high-level syntax element of the CIIP mode and/or available status information that indicates the available status of the CIIP mode only when five specified conditions are met. Compared to the conventional solution, more conditions must be satisfied before the target merge mode is further determined based on the high-level syntax element and information about the available CIIP mode status. Otherwise, CIIP mode can be directly defined as the target merge mode. This can reduce the number of redundant processes in the process of determining the target merge mode.

[0085] With respect to the second aspect, in some implementations of the second aspect, determining the target merge mode based on a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode includes: when the syntax element high level corresponding to the CIIP mode, and/or available status information that indicates the available status of the CIIP mode, indicates that the CIIP mode is prohibited from being used, defining the TPM mode as the target merge mode.

[0086] With respect to the second aspect, in some implementations of the second aspect, when a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, determining TPM mode as the target merge mode includes:

when the high-level syntax element corresponding to the CIIP mode and/or the available status information that indicates the available status of the CIIP mode indicates/indicates that the CIIP mode is prohibited from being used, setting the value of the available status information that indicates the available status of the TPM mode to the first value, wherein when the value of the available status information that indicates the available status of the TPM mode is the first value, the TPM mode is used to perform image prediction for the current image block.

[0087] It should be understood that setting the value of the available status information, which indicates the available status of the TPM mode, to the first value herein is equivalent to specifying the TPM as the target merge mode.

[0088] Optionally, the available status information indicating the available status of the TPM mode is MergeTriangleFlag.

[0089] Setting the value of the available status information, which indicates the available status of the TPM mode, to the first value, may, in particular, set the MergeTriangleFlag to 1.

[0090] With respect to the second aspect, in some implementations of the second aspect, before determining the target merge mode based on a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode, the method further includes:

determining that at least one of the following conditions is met:

[0091] The size of the current image block meets the specified condition; And

Skip mode is not used to predict the current image block.

[0092] cdWidth is the width of the current image block, and cbHeight is the height of the current image block.

[0093] According to a third aspect, an image prediction method is provided. The method includes: determining whether a merge mode is used for the current image block; when the merge mode is used for the current image block, determining whether the level-1 merge mode is available; and when the layer-1 merge mode is not available, determining the target merge mode from the layer-2 merge mode. Layer-2 merge mode includes TPM mode and CIIP mode. When CIIP mode is enabled and all of the following conditions are met (condition 1 - condition 3), the CIIP mode available status information is obtained by analyzing the bit stream, and the target merge mode is determined based on the CIIP mode available status information:

[0094] Condition 1: TPM mode is allowed.

[0095] Condition 2: The size of the current image block meets the specified condition.

[0096] Condition 3: Skip mode is not used to predict the current image block.

[0097] In a possible implementation form according to the third aspect, if the CIIP mode available status information obtained by analyzing the bit stream indicates that the CIIP mode is not available, the TPM is used as the target merge mode.

[0098] In the present application, when the layer-1 merge mode is not available, based on some specified conditions, it can be determined whether the CIIP mode should be selected as the final merge mode, and the CIIP mode can be directly determined as the target merge mode under the condition that none of the specified conditions are met. This reduces the redundancy that occurs in the process of determining the merge target.

[0099] According to a fourth aspect, an image prediction method is provided. The method includes: determining whether a merge mode is used for the current image block; when the merge mode is used for the current image block, determining whether the level-1 merge mode is available; and when the layer-1 merge mode is not available, determining the target merge mode from the layer-2 merge mode. Layer-2 merge mode includes TPM mode and CIIP mode. When CIIP mode is enabled and all of the following conditions are met (condition 1 to condition 5), the CIIP mode available status information is obtained by analyzing the bitstream, and the target merge mode is determined based on the CIIP mode available status information:

[00100] Condition 1: TPM mode is allowed.

[00101] Condition 2: The type of fragment or group of fragments in which the current image block is located is B.

[00102] Condition 3: The maximum number of possible TPM modes supported by the tile or group of tiles in which the current image block is located is determined to be greater than or equal to 2.

[00103] Condition 4: The size of the current image block meets the specified condition.

[00104] Condition 5: Skip mode is not used to predict the current image block.

[00105] In a possible implementation form according to the fourth aspect, if the CIIP mode available status information obtained by analyzing the bit stream indicates that the CIIP mode is not available, the TPM is used as the target merge mode.

[00106] In the present application, when the layer-1 merge mode is not available, based on some specified conditions, it can be determined whether the CIIP mode should be selected as the final merge mode, and the CIIP mode can be directly determined as the target merge mode under the condition that none of the specified conditions are met. This reduces the redundancy that occurs in the process of determining the merge target.

[00107] According to a fifth aspect, an image prediction method is provided. The method includes: determining whether a merge mode is used for the current image block; when the merge mode is used for the current image block, continuing to determine whether the layer-1 merge mode is available; when the level-1 merge mode is not available and the high-level syntax element corresponding to the first merge mode set indicates that the merge mode in the first merge mode set is prohibited from being used, determining a target merge mode applicable to the current image block from the second merge mode set; and predicting the current image block using the target fusion mode.

[00108] Both the first set of merge modes and the second set of merge modes belong to the layer-2 merge mode. In other words, the layer-2 merge mode includes a first set of merge modes and a second set of merge modes. In addition, for the current image block, the level-1 fusion mode and level-2 fusion mode already include all the optional fusion modes of the current image block, and for the current image block, the final target fusion mode needs to be determined from the level-1 fusion mode and the level-fusion mode. 2.

[00109] Optionally, the first set of merge modes includes at least one merge mode, and the second set of merge modes includes at least one merge mode.

[00110] It should be understood that the first set of merge modes and the second set of merge modes are merely concepts introduced for ease of description and are primarily used to distinguish between different merge modes. In the actual process of determining the final target merge mode, the first set of merge modes and the second set of merge modes may not exist.

[00111] In the present application, when the high-level syntax elements of some merge modes indicate that these merge modes are prohibited from being used, there is no need to analyze information about the available status of these merge modes. Instead, the final target merge mode can be directly determined from the remaining optional merge modes. This can reduce, as much as possible, the redundancy generated due to determining the target fusion mode in the image prediction process.

[00112] With respect to the fifth aspect, in some implementations of the fifth aspect, when the level-1 merge mode is not available and the high-level syntax element corresponding to the first merge mode set indicates that the merge mode in the first merge mode set is allowed to be used, the target merge mode determined based on a high-level syntax element corresponding to the second set of merge modes and/or information about the available status of the second set of merge modes.

[00113] The available status information of the second set of merge modes is used to indicate whether a merge mode is used in the second set of merge modes when the current image block is predicted.

[00114] For example, if the second merge mode set includes a CIIP mode, the available status information of the second merge mode set may be a ciip_flag value. When ciip_flag is 0, CIIP mode is not available for the current image block. When ciip_flag is 1, CIIP mode is enabled for the current image block.

[00115] With respect to the fifth aspect, in some implementations of the fifth aspect, the first set of fusion modes includes triangular division TPM modes, and the second set of fusion modes includes a combined intra- and inter-frame prediction CIIP mode.

[00116] Optionally, the first set of merge modes consists of the TPM mode, and the second set of modes consists of the CIIP mode.

[00117] When the first merge mode set and the second merge mode each include only one merge mode, if the merge mode in the first merge mode set is prohibited from being used, the merge mode in the second merge mode set can be determined as the target merge mode; and if the merge mode in the second merge mode set is prohibited from being used, the merge mode in the first merge mode set may be defined as the target merge mode.

[00118] When the first merge mode set and the second merge mode set each include only one merge mode, as long as the merge mode in one of the merge mode sets is prohibited from being used, the merge mode in the other merge mode set can be directly determined as last target merge mode.

[00119] With respect to the fifth aspect, in some implementations of the fifth aspect, when the level-1 merge mode is not available and the high-level syntax element corresponding to the first merge mode set indicates that the merge mode in the first merge mode set is prohibited from being used, determining the target mode the merge applied to the current image block from the second set of merge modes includes: when the layer-1 merge mode is not available and the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from being used, specifying the CIIP mode as the target merge mode .

[00120] According to a sixth aspect, an image prediction device is provided. The apparatus includes a module corresponding to a method according to any one of the first to fifth aspects, and the corresponding module may implement the steps of the method according to any one of the first to fifth aspects.

[00121] The image prediction apparatus in the sixth aspect may include one or more modules, and any of the one or more modules may include any of a circuit, a field programmable gate array FPGA, an application specific integrated circuit ASIC, and a general purpose processor.

[00122] The image predictor in the sixth aspect may be located in the encoding device or decoding device.

[00123] According to a seventh aspect, an image prediction apparatus is provided including a memory and a processor. The processor calls program code stored in memory to implement a method in accordance with any of the first, second and third aspects.

[00124] The image prediction device in the seventh aspect may be located in the image encoding device or the image decoding device.

[00125] According to an eighth aspect, an image encoding/decoding apparatus is provided. The apparatus includes a module corresponding to a method according to any one of the first to fifth aspects, and the corresponding module may implement the steps of the method according to any one of the first to fifth aspects.

[00126] According to a ninth aspect, an image encoding/decoding apparatus including a memory and a processor is provided. The processor calls program code stored in memory to implement the method in accordance with any of the first to fifth aspects.

[00127] Optionally, the memory is non-volatile.

[00128] Optionally, the memory and processor are coupled to each other.

[00129] According to a tenth aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores the instruction, and the instruction enables one or more processors to implement the method according to any of the first to fifth aspects.

[00130] Any one or more processors may include any of a circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and a general purpose processor.

[00131] According to an eleventh aspect, an embodiment of the present application provides a computer program product. When the computer program product is executed on a computer, the computer is configured to carry out some or all of the steps of the method in accordance with any of the first to fifth aspects.

BRIEF DESCRIPTION OF THE DRAWINGS

[00132] FIG. 1 is a schematic block diagram of an example of a video encoding system for implementing an embodiment of the present application;

[00133] FIG. 2 is a schematic block diagram of an example video encoder for implementing an embodiment of the present application;

[00134] FIG. 3 is a schematic block diagram of an example video decoder for implementing an embodiment of the present application;

[00135] FIG. 4 is a schematic block diagram of an example of a video encoding system for implementing an embodiment of the present application;

[00136] FIG. 5 is a schematic block diagram of an example of a video encoding apparatus for implementing an embodiment of the present application;

[00137] FIG. 6 is a schematic block diagram of an example of an encoding device or a decoding device for implementing an embodiment of the present application;

[00138] FIG. 7 is a schematic diagram of spatial and temporal motion information of a possible current coding unit;

[00139] FIG. 8 is a schematic diagram of an MMVD search point used to implement an embodiment of the present application;

[00140] FIG. 9 is another schematic diagram of an MMVD search point used to implement an embodiment of the present application;

[00141] FIG. 10 is a schematic diagram of a triangular division;

[00142] FIG. 11 is a schematic diagram of a triangular division prediction method;

[00143] FIG. 12 is a schematic block diagram of a video communication system according to an embodiment of the present application; And

[00144] FIG. 13 is a schematic flow diagram of an image prediction method according to an embodiment of the present application;

[00145] FIG. 14 is a schematic flow diagram of an image prediction method according to an embodiment of the present application;

[00146] FIG. 15 is a schematic flow diagram of an image prediction method according to an embodiment of the present application;

[00147] FIG. 16 is a schematic block diagram of an image prediction apparatus according to an embodiment of the present application;

[00148] FIG. 17 is a schematic block diagram of an image prediction apparatus according to an embodiment of the present application; and

[00149] FIG. 18 is a schematic block diagram of an image encoding/decoding apparatus according to an embodiment of the present application.

DESCRIPTION OF IMPLEMENTATION OPTIONS

[00150] The technical solutions of the present application are described below with reference to the accompanying drawings.

[00151] In the following description, reference is made to the accompanying drawings, which form a part of the present application and show by way of illustration specific aspects of embodiments of the present application or specific aspects in which embodiments of the present application may be used. It should be understood that embodiments of the present application may be further used in another aspect and may include structural or logical changes not depicted in the accompanying drawings. Accordingly, the following detailed description is not to be construed in a limiting sense, and the scope of this application is to be determined by the appended claims.

[00152] For example, it should be understood that the disclosure in connection with a described method may also be true of a corresponding device or system configured to implement the method, and vice versa.

[00153] As another example, if one or more specific method steps are described, the corresponding apparatus may include one or more blocks, such as functional blocks, for carrying out the one or more method steps described (e.g., one block performing one or more more steps; or a plurality of blocks, each of which performs one or more of a plurality of steps), even if such one or more blocks are not expressly described or illustrated in the accompanying drawings.

[00154] Additionally, if a particular device is described in terms of one or more blocks, such as functional blocks, the corresponding method may include one step used to perform a function of one or more blocks (e.g., one step used to perform function one or more blocks or a plurality of steps, each of which is used to perform the function of one or more of the plurality of blocks), even if such one or more steps are not expressly described or illustrated in the accompanying drawings. In addition, it should be understood that features of various exemplary embodiments and/or aspects described in this specification may be combined with each other unless specifically stated otherwise.

[00155] The technical solutions in embodiments of the present application can be applied to the H.266 standard and a future video encoding standard. The terms used in the implementations of this application are intended only to explain specific embodiments of this application and are not intended to limit this application. The following first briefly describes related concepts in embodiments of the present application.

[00156] Video encoding generally refers to the processing of a sequence of images making up a video or video sequence. In the field of video encoding, the terms "image", "frame" and "image" can be used interchangeably. Video coding used in this specification includes video encoding and video decoding. Video encoding is performed at the source side and typically involves processing (eg, by compressing) the source video image to reduce the amount of data to represent the video image for more efficient storage and/or transmission. Video decoding is performed on the destination side and typically involves reverse processing to the encoder to reconstruct the video image. "Encoding" of a video image in embodiments should be understood as "encoding" or "decoding" a video sequence. The combination of an encoding part and a decoding part is also called CODEC (encoding and decoding).

[00157] A video sequence includes a series of images (image), the image is further divided into fragments (slice), and the fragment is further divided into blocks (block). Video encoding is done in blocks. Some new video encoding standards have further expanded the concept of "block". For example, macroblock (MB) is included in the H.264 standard. The macroblock may further be divided into a plurality of prediction blocks (partition), which may be used for predictive coding. The high efficiency video coding (HEVC) standard has core concepts such as the coding unit (CU), prediction unit (PU), and transform unit (TU). . A set of block units is obtained through functional partitioning and described using a new tree structure. For example, a CU may be partitioned into smaller CUs based on a quadtree, and the smaller CU may be further partitioned to form a quadtree structure. CU is the basic unit for dividing and encoding the encoded image. PU and TU also have similar tree structures. The PU may correspond to a prediction block and is a basic block for prediction coding. The CU is further divided into multiple PUs based on the partitioning pattern. The TU may correspond to a transformation block and is the basic block for transforming the prediction residual. However, in essence, CUs, PUs and TUs are all conceptually blocks (or image units).

[00158] For example, in HEVC, a CTU is divided into multiple CUs using a quadtree structure, represented as an encoding tree. The decision of whether to encode an image region using inter-frame (temporal) or intra-frame (spatial) prediction is made at the CU level. Each CU can be further divided into one, two, or four PUs based on the PU partitioning pattern. Within a single PU, the same prediction process is applied and the corresponding information is passed to the PU-based decoder. After obtaining the residual block by applying a prediction process based on the PU partitioning pattern, the CU can be divided into transform units (TUs) based on a different quadtree structure similar to the encoding tree used for the CU. Recent developments in video compression technologies use a quad-tree and binary tree (QTBT) split frame to split the encoding block. In a block structure, the QTBT CU can be square or rectangular.

[00159] In this specification, for ease of description and understanding, the image block to be encoded in the current encoded image may be referred to as the current image block. For example, in encoding, the current image block is the block currently being encoded, and in decoding, the current image block is the block currently being decoded. The decoded image block in the reference picture used to predict the current image block is called a reference block. In other words, a reference block is a block that provides a reference signal for the current image block, and the reference signal represents the pixel value in the image block. A block that provides a prediction signal for a current image block in a reference picture may be called a prediction block, and the prediction signal represents a pixel value, a sample value, or a sample signal in the prediction block. For example, after passing through a plurality of reference blocks, an optimal reference block is found, and the optimal reference block provides prediction for the current image block, and this block is referred to as a prediction block.

[00160] In the case of lossless video encoding, the original video images can be restored, which means that the reconstructed video images are of the same quality as the original video images (provided that no transmission loss or other data loss occurs during storage or transmission). In the case of lossy video encoding, further compression is performed, such as through quantization, to reduce the amount of data required to represent the video images, and the video images cannot be fully reconstructed at the decoder side, which means the quality of the reconstructed video images is lower or worse than that of the original video images.

[00161] Several H.261 video coding standards are designed for "hybrid lossy video codecs" (specifically, spatial and temporal sampling domain prediction is combined with 2D transform coding to apply transform domain quantization). Each image of a video sequence is usually divided into a set of non-overlapping blocks, and coding is usually performed at the block level. In other words, on the encoder side, the video is usually processed, that is, encoded at the block (video block) level. For example, a prediction block is formed by spatial (intra-frame) prediction and temporal (inter-frame) prediction, the prediction block is subtracted from the current image block (the block that is currently being processed or about to be processed) to obtain a residual block, and the residual block is converted into regions transforms and is quantized to reduce the amount of data that must be transferred (compressed). On the decoder side, some reverse processing relative to the encoder is applied to the encoded or compressed block to recover the current image block for presentation. In addition, the encoder duplicates the decoder's processing loop so that the encoder and decoder generate the same prediction (eg, intra-frame prediction and inter-frame prediction) and/or reconstruction for processing, ie, encoding subsequent blocks.

[00162] The following describes the system architecture applicable to embodiments of the present invention. Fig. 1 is a schematic block diagram of an example video encoding and decoding system 10 applicable to embodiments of the present application. As shown in FIG. 1, the video encoding and decoding system 10 may include a source device 12 and a target device 14. The source device 12 generates encoded video data, and therefore the source device 12 may be referred to as a video encoding device. The target device 14 can decode encoded video data generated by the source device 12, and therefore the target device 14 may be referred to as a video decoding device. In various embodiments, source device 12, target device 14, or both source device 12 and target device 14 may include one or more processors and memory associated with the one or more processors. Memory may include, but is not limited to, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), flash memory, or any another medium that can be configured to store the necessary program code in the form of an instruction or data structure and that can be accessed from a computer as described herein. Source device 12 and target device 14 may include a variety of devices, including a desktop computer, a mobile computing device, a portable computer (e.g., a laptop), a tablet computer, a set-top box, a handset such as a so-called smart phone, a television, a camera, a display device, a digital media player, a game console, a vehicle-mounted computer, a wireless communication device, or the like.

[00163] Although FIG. 1 depicts the source device 12 and the target device 14 as separate devices, an embodiment of the device may alternatively include both the source device 12 and the target device 14, or the functionality of both the source device 12 and the target device 14, that is, the device - source 12 or corresponding function and target device 14 or corresponding function. In such an embodiment, source device 12 or related functionality and target device 14 or related functionality may be implemented using the same hardware and/or software, separate hardware and/or software, or any combination thereof.

[00164] The communication connection between the source device 12 and the target device 14 may be implemented through a communication channel 13. Target device 14 may receive encoded video data from source device 12 via communication channel 13. Communication channel 13 may include one or more media or devices capable of moving encoded video data from source device 12 to target device 14. In an example, communication channel 13 may include one or more communication media that enable source device 12 directly transmit the encoded video data to the target device 14 in real time. In this example, source device 12 may modulate encoded video data in accordance with a communication standard (e.g., a wireless communication protocol) and may transmit the modulated video data to target device 14. One or more communication media may include a wireless communication medium and/or a wired medium communications, such as radio frequency (RF) spectrum or one or more physical transmission cables. One or more communication media may be part of a packet-based network, and the packet-based network is, for example, a local area network, a wide area network, or a wide area network (eg, the Internet). The one or more communication media may include a router, switch, base station, or other device that provides communication from source device 12 to target device 14.

[00165] Source device 12 includes an encoder 20. Optionally, source device 12 may further include an image source 16, an image preprocessor 18, and a communications interface 22. In a particular implementation, encoder 20, image source 16, image preprocessor 18, and communication interface 22 may be hardware components in source device 12 or may be software programs in source device 12. Descriptions are provided separately as follows:

[00166] The image source 16 may include or be any type of image capturing device, configured, for example, to capture a real image; and/or any type of device for creating an image or annotation (for encoding the content of the screen, some text on the screen is also considered part of the image or image to be encoded), for example, a computer graphics processor configured to create a computer animated image; or a device of any type configured to receive and/or provide a real image or computer animated image (for example, screen content or a virtual reality (VR) image); and/or any combination thereof (for example, an augmented reality image , AR)).The image source 16 may be a camera configured to capture an image, or a memory configured to store an image. The image source 16 may further include any type of interface (internal or external) through which previously captured or generated an image is stored and/or an image is received or received. When the image source 16 is a camera, the image source 16 may be, for example, a local camera or an integrated camera built into the source device. When the image source 16 is a memory, the image source 16 may be a local memory or, for example, integrated memory integrated into the source device. When the image source 16 includes an interface, the interface may be, for example, an external interface for receiving an image from an external video source. The external video source is, for example, an external image capturing device such as a camera, an external memory, or an external imaging device. The external imaging device is, for example, an external computer graphics processor, a computer, or a server. The interface may be any type of interface, such as a wired or wireless interface or an optical interface, in accordance with any proprietary or standardized interface protocol.

[00167] An image can be thought of as a two-dimensional array or matrix of pixel elements (picture element, pixel). A pixel element in an array can also be called a sample. The number of samples in the horizontal and vertical directions (or axes) of an array or image determines the size and/or resolution of the image. Three color components are commonly used to represent color. More specifically, the image may be represented as or include three arrays of samples. For example, in RBG or color space, an image includes corresponding arrays of samples of red, green, and blue. However, when encoding video, each pixel is typically represented in a luminance/chrominance format or color space. For example, a YUV image includes a luma component, denoted Y (sometimes denoted alternatively L), and two chrominance components, denoted U and V. A luma Y component represents the brightness or intensity of the gray level (e.g., both are the same in a grayscale image), and the two chroma components U and V represent the chroma components or color information. Accordingly, a YUV image includes an array of luminance samples of luminance (Y) sample values and two chroma sample arrays of chroma (U and V) values. An RGB image can be converted or converted to YUV format and vice versa. This process is also called color conversion or conversion. If the image is monochrome, it may only include an array of luminance samples. In this embodiment of the present application, the image transmitted by the image source 16 to the image processor may also be referred to as raw image data 17.

[00168] The image preprocessor 18 is configured to receive raw image data 17 and perform preprocessing of the raw image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. For example, pre-processing performed by image preprocessor 18 may include cropping, color format conversion (eg, from RGB to YUV format), color correction, or noise reduction.

[00169] Encoder 20 (also referred to as video encoder 20) is configured to receive preprocessed image data 19 and process preprocessed image data 19 using an appropriate prediction mode (such as the prediction mode in each embodiment of this specification) to provide encoded data 21 images (structural details of the encoder 20 are further described below based on FIG. 2, FIG. 4, or FIG. 5). In some embodiments, the encoder 20 may be configured to implement each embodiment described below to implement, on the encoder side, the image prediction method described herein.

[00170] Communication interface 22 may be configured to receive encoded image data 21 and transmit encoded image data 21 to target device 14 or any other device (eg, memory) via communication link 13 for storage or direct recovery. The other device can be any device used for decoding or storage. The communication interface 22 may, for example, be configured to encapsulate the encoded image data 21 into an appropriate format, such as a data packet, for transmission over the communication channel 13.

[00171] Target device 14 includes a decoder 30. Optionally, target device 14 may further include a communications interface 28, an image post processor 32, and a display device 34. Descriptions are provided separately as follows:

[00172] Communication interface 28 may be configured to receive encoded image data 21 from source device 12 or any other source. Any other source is, for example, a storage device. The storage device is, for example, a storage device for encoded image data. The communication interface 28 may be configured to transmit or receive encoded image data 21 through a communication channel 13 between the source device 12 and the target device 14 or through any type of network. The communication channel 13 is, for example, a direct wired or wireless connection. Any type of network is, for example, a wired or wireless network or any combination thereof, or any type of private or public network, or any combination thereof. The communication interface 28 may, for example, be configured to decapsulate a data packet transmitted through the communication interface 22 to obtain encoded image data 21.

[00173] Both communication interface 28 and communication interface 22 may be configured as unidirectional communication interfaces or bidirectional communication interfaces, and may be configured to, for example, send and receive messages to establish a connection, as well as confirm and exchange any other information, relating to a communication channel and/or data transmission, such as the transmission of encoded image data.

[00174] The decoder 30 (also called decoder 30) is configured to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (structural details of the decoder 30 are further described below based on FIG. 3, FIG. 4, or FIG. 5) . In some embodiments, the decoder 30 may be configured to implement each embodiment described below to implement, on the decoder side, the image prediction method described herein.

[00175] The image post-processor 32 is configured to post-process the decoded image data 31 (also referred to as reconstructed image data) to obtain post-processed image data 33 . Post-processing performed by image post-processor 32 may include color format conversion (eg, from YUV format to RGB format), color correction, cropping, resampling, or any other processing. The image post-processor 32 may further be configured to transmit post-processed image data 33 to the display device 34.

[00176] The display device 34 is configured to receive post-processed image data 33 to display the image, for example, to a user or viewer. The display device 34 may be or include any type of display for presenting the reconstructed image, such as an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light-emitting diode (OLED), a plasma display, a projector, a micro-LED display, a liquid crystal on silicon , LCoS), digital light processor (DLP) or any other display.

[00177] Although FIG. 1 depicts source device 12 and target device 14 as separate devices, an embodiment of the device may alternatively include both source device 12 and target device 14, or the functionality of both source device 12 and target device 14, i.e. a source device 12 or related functionality and a target device 14 or related functionality. In such an embodiment, source device 12 or related functionality and target device 14 or related functionality may be implemented using the same hardware and/or software, separate hardware and/or software, or any combination thereof.

[00178] As will be apparent to one skilled in the art based on the descriptions, the existence and (precise) separation of functionality of the various blocks or functions of source device 12 and/or target device 14 shown in FIG. 1 may vary depending on actual device and application. Source device 12 and target device 14 may include any of a wide range of devices, including any type of portable or desktop device, such as a laptop or notebook computer, mobile phone, smartphone, tablet or tablet computer, video camera, desktop computer, set-top box , television, camera, vehicle-mounted device, display device, digital media player, game console, video streaming device (such as a content service server or content delivery server), broadcast receiver device, or broadcast transmitter device and may or may not use any type of operating system.

[00179] Encoder 20 and decoder 30 may each be implemented as any of various suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs). , field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technologies are implemented in part using software, the device may store the software instruction on a suitable non-volatile computer-readable storage medium and may implement the instruction using hardware, such as one or more processors, to implement the technologies of this specification. Any of the foregoing content (including hardware, software, combinations of hardware and software, etc.) may be considered one or more processors.

[00180] In some cases, the video encoding and decoding system 10 shown in FIG. 1 is merely an example, and the technologies of the present application may be applicable to video encoding settings (eg, video encoding or video decoding) that do not necessarily involve any communication between the encoding device and the decoding device. In another example, data may be retrieved from local memory, streamed over a network, or the like. A video encoding apparatus may encode data and store the data in memory, and/or a video decoding apparatus may retrieve data from memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data into memory and/or retrieve data from memory and decode the data.

[00181] FIG. 2 is a schematic/conceptual block diagram of an example encoder 20 configured to implement an embodiment of the present invention. In the example in FIG. 2, the encoder 20 includes a remainder calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a restoration unit 214, a buffer 216, a loop filter 220, a decoded picture buffer 230 , DPB), prediction processing unit 260 and entropy encoding unit 270. The prediction processing unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a mode selection unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown in the figure). The encoder 20 shown in FIG. 2 may also be called a hybrid video encoder or a hybrid video codec-based video encoder.

[00182] For example, the residual calculation block 204, the transform processing block 206, the quantization block 208, the prediction processing block 260, and the entropy encoding block 270 form the forward signal path of the encoder 20, while, for example, the inverse quantization block 210, the inverse transform processing block 212 , recovery unit 214, buffer 216, loop filter 220, decoded picture buffer (DPB) 230, and prediction processing unit 260 form the reverse encoder signal path. The return signal path of the encoder corresponds to the signal path of the decoder (see decoder 30 in FIG. 3).

[00183] The encoder 20 receives, for example, through an input 202, an image 201 or an image block 203 for an image 201, for example, an image in a sequence of images forming a video or video sequence. The image block 203 may also be referred to as the current image block or the image block to be encoded, and the image 201 may be referred to as the current image or the image to be encoded (particularly in video encoding, distinguishing the current image from other images, e.g. previously encoded and/or decoded images in the same video sequence, namely a video sequence that also includes the current image).

[00184] An embodiment of encoder 20 may include a divider (not shown in FIG. 2) configured to divide image 201 into a plurality of blocks, such as image block 203. The image 201 is typically divided into a plurality of non-overlapping blocks. The splitter may be configured to use the same block size for all images in a video sequence and a corresponding grid defining the block size, or to vary the block size between images, or subsets or groups of images, and divide each image into appropriate blocks.

[00185] In one example, the prediction processing unit 260 of the encoder 20 may be configured to implement any combination of the separation techniques described above.

[00186] Like image 201, image block 203 can also be thought of as a two-dimensional array or matrix of samples with sample values, although the size of image block 203 is smaller than the size of image 201. In other words, image block 203 can include, for example, one array samples (eg, a luma array in the case of a monochrome image 201), three arrays of samples (eg, one luma array and two chrominance arrays in the case of a color image), or any other number and/or type of arrays depending on the color format being used. The number of samples in the horizontal and vertical directions (or axes) of the image block 203 determines the size of the image block 203.

[00187] The encoder 20 shown in FIG. 2 is configured to encode the image 201 block by block, for example, to perform encoding and prediction for each image block 203.

[00188] The residual calculation block 204 is configured to calculate the residual block 205 based on the image block 203 and the prediction block 265 (details about the prediction block 265 are further provided below), for example, by subtracting the sample values of the prediction block 265 from the sample values of the image sample block 203 by sample (pixel by pixel) to obtain a residual block 205 in the sample area.

[00189] The transform processing unit 206 is configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values of the residual block 205 to obtain transform coefficients 207 in the transform domain. . Transformation coefficient 207 may also be referred to as a residual transform coefficient and represents a residual block 205 in the transform domain.

[00190] Transform processing block 206 may be configured to apply DCT/DST integer approximations, such as the transforms specified in HEVC/H.265. Compared to the orthogonal DCT transform, such integer approximations typically scale by a factor of several. To preserve the norm of the residual block that is processed using forward and inverse transforms, an additional scale factor is applied as part of the transform process. The scale factor is usually selected based on some constraints, for example, the scale factor is a power of two for the shift operation, the bit depth of the transform factor, and the optimal tradeoff between accuracy and implementation cost. For example, a particular scale factor is specified for the inverse transform, for example, by the inverse transform processing unit 212 on the decoder 30 side (and the corresponding inverse transform, for example, by the inverse transform processing unit 212 on the encoder 20 side), and accordingly, the corresponding scale factor may be specified for direct conversion by the conversion processing unit 206 on the side of the encoder 20.

[00191] The quantization unit 208 is configured to quantize the transform coefficients 207 to obtain quantized transform coefficients 209, for example, by applying scalar quantization or vector quantization. The quantized transform coefficient 209 may also be referred to as the quantized residual coefficient 209. The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit conversion factor may be rounded to an m-bit conversion factor during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, for scalar quantization, different scales can be used to achieve finer or coarser quantization. A smaller quantization step corresponds to a finer quantization, and a larger quantization step corresponds to a coarser quantization. The corresponding quantization step can be specified by the quantization parameter (QP). For example, the quantization parameter may be an index for a given set of suitable quantization steps. For example, a smaller quantization parameter may correspond to a finer quantization (smaller quantization step), and a larger quantization parameter may correspond to a coarser quantization (larger quantization step), or vice versa. Quantization may include division into a quantization step and corresponding quantization or inverse quantization, such as performed by inverse quantization unit 210, or may include multiplication by a quantization step. Embodiments according to some standards, such as HEVC, may use a quantization parameter to determine the quantization step. Typically, the quantization step can be calculated based on the quantization parameter using a fixed-point approximation to the equation involving division. Additional scaling factors can be introduced for quantization and dequantization to recover the residual block norm, where the residual block norm can be changed due to the scale used in the fixed-point approximation of the equation for the quantization step and the quantization parameter. In an exemplary implementation, the inverse transform scale may be combined with the dequantization scale. Alternatively, a customized quantization table can be used and the signal passed from the encoder to the decoder, for example in a bit stream. Quantization is a lossy operation, where a larger quantization step indicates larger losses.

[00192] The inverse quantization block 210 is configured to apply the inverse quantization of the quantization block 208 to the quantized coefficient to obtain the dequantized coefficient 211, for example, applied based on or using the same quantization step as the quantization block 208, the inverse of the quantization scheme applied quantization block 208. The dequantized coefficient 211 may also be referred to as the dequantized residual coefficient 211 and corresponds to the transform coefficient 207, although it is typically different from the transform coefficient due to losses caused by quantization.

[00193] The inverse transform processing unit 212 is configured to apply an inverse transform transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST) to obtain block 213 of inverse transformation in the sampling area. The inverse transform block 213 may also be referred to as the inverse transform dequantization block 213 or the residual inverse transform block 213 .

[00194] The reconstruction block 214 (e.g., adder 214) is configured to add an inverse transform block 213 (namely, the reconstructed residual block 213) to the prediction block 265, for example, by adding the sample values of the reconstructed residual block 213 and the sample values of the prediction block 265 to obtain the reconstructed block 215 in the sample region.

[00195] Optionally, a buffer block 216 ("buffer" 216 for short), e.g., line buffer 216, is configured to buffer or store the reconstructed block 215 and the corresponding sample value, for example, for intra-frame prediction. In another embodiment, the encoder may be configured to use the unfiltered reconstructed block and/or the corresponding sample value that is/are stored in buffer block 216 to perform any type of estimation and/or prediction, such as intra-frame prediction.

[00196] For example, in an embodiment, encoder 20 may be configured such that buffer block 216 is configured to store reconstructed block 215 that is not only used for intra-frame prediction 254, but also used for loop filter block 220 (not shown in FIG. 2), and/or so that, for example, the buffer block 216 and the decoded image buffer 230 form one buffer. In another embodiment, filtered block 221 and/or a block or sample (not shown in FIG. 2) from decoded image buffer 230 is/are used as input or basis for intra-prediction 254.

[00197] A contour filter block 220 (“contour filter” 220 for short) is configured to filter the reconstructed block 215 to produce a filtered block 221, smoothing pixel transitions, or improving video quality. Loop filter block 220 is configured to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or another filter, such as a two-way filter, an adaptive loop filter. (adaptive loop filter, ALF), sharpening or smoothing filter, or joint filter. Although loop filter block 220 is shown as an input loop filter in FIG. 2, in another implementation, loop filter block 220 may be implemented as a post-loop filter. The filtered block 221 may also be referred to as the filtered reconstructed block 221. The decoded image buffer 230 may store the reconstructed coded block after the loop filter block 220 performs a filtering operation on the reconstructed coded block.

[00198] In an embodiment, encoder 20 (respectively, loop filter block 220) may be configured to output a loop filter parameter (e.g., adaptive offset sample information), for example, directly or after entropy encoding performed by entropy encoding block 270 or any other entropy encoding block so that the decoder 30 can receive and apply the same loop filter parameter for decoding.

[00199] The decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use when encoding video data by the encoder 20. The DPB 230 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM). dynamic random access memory (DRAM) (including synchronous DRAM (synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive random access memory (RRAM)) or other type of storage device. DPB 230 and buffer 216 may be provided by the same storage device or by separate storage devices. In the example, the decoded picture buffer (DPB) 230 is configured to store a filtered block 221. The decoded picture buffer 230 may further be configured to store other previously filtered blocks, such as previously reconstructed and filtered blocks 221, of the same current image or different images, such as previously reconstructed images, and may provide fully previously reconstructed, i.e., decoded images (and corresponding reference blocks and samples) and/or partially reconstructed current image (and corresponding reference blocks and samples), for example, for interframe prediction. In one example, if reconstructed block 215 is reconstructed without in-loop filtering, decoded picture buffer (DPB) 230 is configured to store reconstructed block 215.

[00200] The prediction processing unit 260, also called the block prediction processing unit 260, is configured to receive or obtain an image block 203 (the current image block 203 of the current image 201) and reconstructed image data, such as reference samples of the same (current) images from buffer 216 and/or reference image data 231 of one or more previously decoded images from decoded image buffer 230, and processing such data for prediction, namely, to provide a prediction block 265, which may be an inter-frame prediction block 245 or an intra-frame prediction block 255 forecasting.

[00201] The mode selection block 262 may be configured to select a prediction mode (eg, intra- or inter-frame prediction mode) and/or a corresponding prediction block 245 or 255 to be used as the prediction block 265, to calculate the residual block 205, and to recovery of the restored block 215.

[00202] In an embodiment, the mode selection block 262 may be configured to select a prediction mode (eg, from among the prediction modes supported by the prediction processing block 260), wherein the prediction mode provides the best match, or in other words, the minimum remainder (minimum remainder means better compression for transmission or storage), or provides minimal signaling overhead (minimum signaling overhead means better compression for transmission or storage), or accounts for or balances both. Mode selector 262 may be configured to determine a prediction mode based on rate distortion optimization (RDO), namely, selecting a prediction mode that optimizes the minimum rate distortion, or selecting a prediction mode that has less rate distortion. least satisfies the criterion for choosing a forecasting mode.

[00203] The following describes in detail the prediction processing performed (for example, by the prediction processing unit 260) and the mode selection performed (for example, by the mode selection unit 262), using the encoder 20 as an example.

[00204] As described above, encoder 20 is configured to determine or select an optimal or most effective prediction mode from a set of (predetermined) prediction modes. The set of prediction modes may include, for example, an intra-prediction mode and/or an inter-frame prediction mode.

[00205] The set of intra-prediction modes may include 35 different intra-prediction modes, for example, omnidirectional modes such as constant (or average) mode and planar mode, or directed modes such as those defined in H.265, or may include 67 various intra-prediction modes, for example, omnidirectional modes such as constant current (or average) mode and planar mode, or directed modes such as those defined in H.266, which is under development.

[00206] In an exemplary implementation, the set of inter-prediction modes depends on available reference pictures (namely, for example, at least partially decoded pictures stored in DBP 230, as described above) and other inter-prediction parameters, for example, depends on whether the entire reference image or just part of the reference image, e.g. the search box area around the current image block area, is used to find the best matching reference block and/or, for example, depends on whether pixel interpolation is performed, e.g. half-pixel interpolation is applied and/ or a quarter pixel. A set of interframe prediction modes may include, for example, an Advanced Motion Vector Prediction (AMVP) mode and a merge mode. In a specific implementation, a set of inter-frame prediction modes may include an enhanced checkpoint-based AMVP mode and an enhanced checkpoint-based merge mode in embodiments of the present application. In an example, intra-frame prediction block 254 may be configured to perform any combination of inter-frame prediction technologies described below.

[00207] In addition to the above prediction modes, embodiments of the present application may also employ a skip mode and/or a direct mode.

[00208] The prediction processing unit 260 may further be configured to divide the image block 203 into smaller parts or sub-units, for example, by iteratively using quad-tree (QT) partitioning, binary-tree (BT) partitioning. , splitting a ternary tree (triple-tree, TT) or any combination thereof and performing, for example, prediction for each of the sections of the block or subblocks. Mode selection includes selecting a tree structure of the divided image block 203 and selecting a prediction mode to be applied to each of the block sections or subblocks.

[00209] Inter-frame prediction block 244 may include a motion estimation (ME) block (not shown in FIG. 2) and a motion compensation (MC) block (not shown in FIG. 2). The motion estimator is configured to receive or obtain an image image block 203 (current image block 203 of the current image 201) and a decoded image 231, or at least one or more previously reconstructed blocks, for example, one or more reconstructed blocks of other/different previously decoded ones. 231 images for motion estimation. For example, the video sequence may include the current image and previously decoded images 231, or in other words, the current image and previously decoded images 231 may be part of or form a sequence of images forming the video sequence.

[00210] For example, encoder 20 may be configured to select a reference block from a plurality of reference blocks of the same image or different images from a plurality of other images and provide a reference image and/or offset (spatial offset) between locations (X, Y coordinates ) of the reference block and the location of the current image block as inter-frame prediction parameters for the motion estimation block (not shown in FIG. 2). Displacement is also called motion vector (MV).

[00211] The motion compensation unit is configured to obtain an inter-prediction parameter and perform inter-prediction based on or using the inter-prediction parameter to obtain an inter-prediction unit 245 . The motion compensation performed by the motion compensation block (not shown in FIG. 2) may include sampling or generating a prediction block based on the motion vector/block determined by motion estimation (possibly performing sub-pixel-accurate interpolation). Interpolation filtering can generate additional pixel samples from known pixel samples, thereby potentially increasing the number of possible prediction blocks that can be used to encode an image block. After receiving the motion vector for the PU of the current image block, the motion compensation block 246 can find the prediction block pointed to by the motion vector in one of the reference image lists. Motion compensation block 246 may also generate a syntax element associated with a block and a video fragment, such that the decoder 30 uses the syntax element to decode a block of image in a video fragment.

[00212] In particular, interframe prediction block 244 may provide a syntax element to entropy encoding block 270. The syntax element includes an inter-prediction parameter (eg, information indicating the selection of an inter-prediction mode used to predict the current image block after traversing a plurality of inter-prediction modes). In a possible application scenario, if there is only one inter-prediction mode, the inter-prediction parameter may alternatively not be carried in the syntax element. In this case, the decoder side 30 may directly perform decoding using the default prediction mode. It can be understood that the inter-frame prediction unit 244 may be configured to implement any combination of inter-frame prediction technologies.

[00213] The intra-frame prediction unit 254 is configured to obtain, for example, receiving an image block 203 (the current image block) and one or more previously reconstructed blocks, such as reconstructed neighboring blocks, of the same image for intra-frame estimation. Encoder 20 may, for example, be configured to select an intra-prediction mode from a plurality of (predetermined) intra-prediction modes.

[00214] In an embodiment, an encoder 20 configured to select an intra-prediction mode according to an optimization criterion, for example, based on a minimum residual (e.g., an intra-prediction mode providing a prediction block 255 that is most similar to the current image block 203) or a minimum rate distortions.

[00215] The intra prediction block 254 is further configured to determine the intra prediction block 255 based on, for example, an intra prediction parameter in the selected intra prediction mode. In any case, after selecting an intra prediction mode for a block, the intra prediction block 254 is further configured to provide an intra prediction parameter, namely information indicating the selected intra prediction mode for the block, to the entropy encoding block 270. In an example, intra-prediction block 254 may be configured to perform any combination of intra-prediction technologies.

[00216] In particular, intra-frame prediction block 254 may provide a syntax element to entropy encoding block 270. The syntax element includes an intra-prediction parameter (eg, intra-prediction mode selection indication information used to predict the current image block after traversing a plurality of intra-prediction modes). In a possible application scenario, if there is only one intra-prediction mode, the intra-prediction parameter may alternatively not be carried in the syntax element. In this case, the decoder side 30 may directly perform decoding using the default prediction mode.

[00217] Entropy encoding block 270 is configured to use (or not use) an entropy encoding algorithm or scheme (e.g., variable length coding (VLC) scheme, context adaptive VLC (CAVLC) scheme, arithmetic coding, context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval division entropy coding (probability interval partitioning entropy (PIPE) or other entropy encoding methodology or technology) to one or all of the quantized residual coefficient 209, the inter-prediction parameter, the intra-prediction parameter, and/or the edge filter parameter to obtain encoded image data 21 that can be output 272, for example, in the form of an encoded bitstream 21. The encoded bitstream may be transmitted to video decoder 30 or archived for later transmission or retrieval by video decoder 30. Entropy encoding block 270 may further be configured to entropy encode another syntax element for the current portion of video being encoded. .

[00218] Another design of video encoder 20 may be used to encode a video stream. For example, non-transform encoder 20 may directly quantize the residual signal without transform processing block 206 for some blocks or frames. In another implementation, encoder 20 may have a quantization unit 208 and an inverse quantization unit 210 combined into a single unit.

[00219] In particular, in this embodiment of the present application, the encoder 20 may be configured to implement the video encoding process described in the following embodiments.

[00220] It should be understood that the video encoder in the present application may include only some modules in the video encoder 20. For example, the video encoder in the present application may include an image decoding unit and a division unit. The image decoding unit may include one or more of an entropy decoding unit, a prediction unit, an inverse transform unit, and an inverse quantization unit.

[00221] Additionally, another design of video encoder 20 may be used to encode the video stream. For example, for some image blocks or image frames, the video encoder 20 may directly quantize the residual signal, processing by the transform processing unit 206 is not required, and accordingly, processing by the inverse transform processing unit 212 is also not required. Alternatively, for some image blocks or image frames, video encoder 20 does not generate residual data, and accordingly, processing by transform processing unit 206, quantization unit 208, inverse quantization unit 210, and inverse transform processing unit 212 is not required. Alternatively, video encoder 20 may directly store the reconstructed image block as a reference block, and processing by filter 220 is not required. Alternatively, quantization unit 208 and inverse quantization unit 210 in video encoder 20 may be combined. The 220 loop filter is optional. Moreover, in the case of lossless compression encoding, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 are optional. It should be understood that in various application scenarios, inter-frame prediction unit 244 and intra-frame prediction unit 254 may be used selectively.

[00222] FIG. 3 is a schematic/conceptual block diagram of an example decoder 30 configured to implement an embodiment of the present invention. Video decoder 30 is configured to receive encoded image data 21 (e.g., a coded bitstream) encoded, for example, by encoder 20 to obtain a decoded image 231. During the decoding process, video decoder 30 receives video data from video encoder 20, for example, an encoded video bitstream that represents an image block of an encoded video fragment and its associated syntax element.

[00223] In the example of FIG. 3, the decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a restoration unit 314 (eg, an adder 314), a buffer 316, a loop filter 320, a decoded image buffer 330, and a prediction processing unit 360. The prediction processing unit 360 may include an inter-frame prediction unit 344, an intra-frame prediction unit 354, and a mode selection unit 362. In some examples, video decoder 30 may perform a decoding step that is generally the reverse of the encoding step described with reference to video encoder 20 in FIG. 2.

[00224] The entropy decoding unit 304 is configured to perform entropy decoding of the encoded image data 21 to obtain, for example, a quantized coefficient 309 and/or a decoded encoding parameter (not shown in FIG. 3), for example, any one or all of the interframe parameter prediction, intra-prediction parameter, edge filter parameter, and/or other syntax element (which is decoded). The entropy decoding unit 304 is further configured to forward an inter-prediction parameter, an intra-prediction parameter, and/or other syntax element to the prediction processing unit 360 . Video decoder 30 may receive syntax elements at the video fragment level and/or video block level.

[00225] The inverse quantization unit 310 may have the same function as the inverse quantization unit 110 . The inverse transform processing unit 312 may have the same function as the inverse transform processing unit 212. The recovery block 314 may have the same function as the recovery block 214. Buffer 316 may have the same function as buffer 216. Loop filter 320 may have the same function as loop filter 220. Decoded image buffer 330 may have the same function as decoded image buffer 230.

[00226] The prediction processing block 360 may include an inter-frame prediction block 344 and an intra-frame prediction block 354. The inter-frame prediction block 344 may resemble the inter-frame function prediction block 244 , and the intra-frame prediction block 354 may resemble the intra-function prediction block 254 . The intra-frame prediction processing unit 360 is typically configured to perform block prediction and/or obtain prediction block 365 from the encoded data 21, and receive or obtain (explicitly or implicitly) a prediction-related parameter and/or information about the selected prediction mode, e.g. , from the entropy decoding block 304.

[00227] When a video fragment is encoded into an intra-coded (I) fragment, an intra-frame prediction block 354 of a prediction processing block 360 is configured to generate a prediction block 365 for an image block of the current video fragment based on the signaled intra-frame prediction mode and data from a previously decoded block of the current frame or image . When a video frame is encoded into an inter-coded (namely, B or P) fragment, an inter-frame prediction block 344 (e.g., a motion compensation block) in the prediction processing block 360 is configured to generate a video block prediction block 365 in the current video fragment based on the motion vector and other element syntax that is received from entropy decoding block 304. For inter-picture prediction, a prediction block may be formed from one of the reference pictures in one reference picture list. Video decoder 30 may create reference frame lists, list 0 and list 1, using a default construction technology and based on reference pictures stored in DPB 330.

[00228] The prediction processing unit 360 is configured to determine prediction information for a video block of the current video fragment by analyzing the motion vector and other syntax elements and using the prediction information to generate a prediction block for the current video block to be decoded. In an example of the present application, prediction processing unit 360 determines, using certain received syntax elements, a prediction mode (eg, intra-frame or inter-frame prediction) for encoding a video block in a video fragment, the type of inter-frame prediction fragment (eg, B-tile, P-tile, or GPB-tile ), information about the construction of one or more reference picture lists for the fragment, the motion vector of each inter-coded video block for the fragment, the inter-frame prediction status of each inter-frame encoded video block in the fragment, and other information for decoding the video block in the current fragment. In another example of the present application, a syntax element received by video decoder 30 from the bit stream includes a syntax element in one or more of an adaptive parameter set (APS), a sequence parameter set (SPS), an image parameter set ( picture parameter set, PPS) or fragment title.

[00229] The inverse quantization block 310 may be configured to perform inverse quantization (namely, dequantization) of the quantized transform coefficient provided in the bitstream and decoded by the entropy decoding block 304. The inverse quantization process may include: using a quantization parameter calculated by video encoder 20 for each video block in a video fragment to determine the degree of quantization that should be applied, and likewise, the degree of inverse quantization that should be applied.

[00230] The inverse transform processing unit 312 is configured to apply an inverse transform (eg, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficient to generate a residual block in the pixel domain.

[00231] The reconstruction block 314 (e.g., adder 314) is configured to add an inverse transform block 313 (namely, the reconstructed residual block 313) to the prediction block 365, for example, by adding the sample values of the reconstructed residual block 313 and the sample values of the prediction block 365 to obtain the reconstructed block 315 in the sample region.

[00232] A loop filter block 320 (in the encoding loop or after the encoding loop) is configured to filter the reconstructed block 315 to produce a filtered block 321, smoothing pixel transitions, or improving video quality. In one example, loop filter block 320 may be configured to perform any combination of filtering technologies described below. Loop filter block 320 is configured to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or another filter, such as a two-way filter, an adaptive loop filter (ALF). ), a sharpening or smoothing filter, or a joint filter. Although loop filter block 320 is shown as an input loop filter in FIG. 3, in another implementation, loop filter block 320 may be implemented as a post-loop filter.

[00233] The decoded video block 321 of a given frame or image is then stored in a decoded image buffer 330, which stores a reference image used for subsequent motion compensation.

[00234] The decoder 30 is configured to, for example, output the decoded image 31 through an output 332 for presentation or viewing to a user.

[00235] Another embodiment of video decoder 30 may be used to decode the compressed bit stream. For example, decoder 30 may generate an output video stream without loop filter block 320. For example, the non-transform decoder 30 may directly perform inverse quantization of the residual signal without the inversion processing block 312 for some blocks or frames. In another implementation, video decoder 30 may have an inverse quantization block 310 and an inverse transform processing block 312 combined into one block.

[00236] Specifically, in this embodiment of the present application, the decoder 30 is configured to implement the video decoding method described in the following embodiments.

[00237] It should be understood that the video encoder in the present application may include only some modules in the video encoder 30. For example, the video encoder in the present application may include a division block and an image encoding block. The image encoding unit may include one or more of a prediction unit, a transform unit, a quantization unit, and an entropy encoding unit.

[00238] Additionally, another design of video decoder 30 may be used to decode the encoded video bitstream. For example, video decoder 30 may generate an output video stream without processing by filter 320. Alternatively, for some image blocks or image frames, entropy decoding block 304 of video decoder 30 does not obtain quantized coefficients through decoding, and accordingly there is no need for inverse quantization block 310 and block 312 inverse conversion processing to perform processing. The 320 loop filter is optional. Moreover, in the case of lossless compression, the inverse quantization unit 310 and the inverse transform processing unit 312 are also optional. It should be understood that in various application scenarios, the inter-frame prediction unit and the intra-frame prediction unit may be used selectively.

[00239] It should be understood that in the encoder 20 and decoder 30 in the present application, the processing result for a procedure may be output to the next procedure after further processing. For example, after a procedure such as interpolation filtering, motion vector inference, or contour filtering, an operation such as clip or shift is additionally performed on the result of the corresponding procedure.

[00240] For example, a motion vector that is a reference point of the current image block and which is derived based on the motion vector of an adjacent affine coding block (a coding block that is predicted using an affine motion model may be called an affine coding block) or a motion vector that is a subblock of the current image block and which is inferred based on the motion vector of the adjacent affine encoding block can be further processed. This is not expressly limited in this application. For example, the value of a motion vector is limited to a certain bit width range. It is assumed that the allowed bit depth of the motion vector is bitDepth, the value of the motion vector is in the range -2^(bitDepth - 1) to 2^(bitDepth - 1) - 1, where the symbol "^" represents exponentiation. If bitDepth is 16, the value is in the range -32768 to 32767. If bitDepth is 18, the value is in the range -131072 to 131071.

[00241] In another example, the value of a motion vector (e.g., the MV motion vectors of four 4x4 subblocks in one 8x8 image block) may be further limited such that the maximum difference between the entire MV portions of four 4x4 subblocks will not exceed N (e.g., N may be set to 1) pixels.

[00242] FIG. 4 is an illustrative diagram of an example video encoding system 40 including the encoder 20 of FIG. 2 and/or decoder 30 in FIG. 3 according to an exemplary embodiment. The video encoding system 40 may implement a combination of various technologies in the embodiments of the present application. In the illustrated implementation, the video encoding system 40 may include an imager 41, an encoder 20, a decoder 30 (and/or a video encoder/decoder implemented by logic 47 of the processing unit 46), an antenna 42, one or more processors 43, one or more storage devices 44 and/or display device 45.

[00243] As shown in FIG. 4, the imaging apparatus 41, antenna 42, processing unit 46, logic circuit 47, encoder 20, decoder 30, processor 43, memory 44, and/or display 45 may communicate with each other. As described, although the video encoding system 40 is illustrated with an encoder 20 and a decoder 30, the video encoding system 40 may include only the encoder 20 or only the decoder 30 in various examples.

[00244] In some examples, antenna 42 may be configured to transmit or receive an encoded video data bit stream. Additionally, in some examples, display device 45 may be configured to present video data. In some examples, logic circuit 47 may be implemented by processing unit 46. Processing unit 46 may include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, or the like. Video encoding system 40 may also include an optional processor 43. The optional processor 43 may likewise include application-specific integrated circuit (ASIC) logic, a graphics processor, a general purpose processor, or the like. In some examples, logic circuit 47 may be implemented in hardware, such as dedicated video encoding hardware. The processor 43 may be implemented by general purpose software, an operating system, or the like. In addition, memory 44 may be any type of memory, such as volatile memory (eg, static random access memory (SRAM) or dynamic random access memory (DRAM)) or non-volatile memory (eg, flash). memory). In a non-limiting example, memory 44 may be implemented as a cache memory. In some examples, logic circuit 47 may access memory 44 (eg, to implement an image buffer). In another example, logic circuit 47 and/or processing unit 46 may include memory (eg, cache) for implementing an image buffer or the like.

[00245] In some examples, encoder 20 implemented by logic may include an image buffer (eg, implemented by processing unit 46 or memory 44) and a graphics processor (eg, implemented by processing unit 46). The GPU may be in communicative communication with an image buffer. The graphics processor may include an encoder 20 implemented by logic 47 for implementing various modules, which are described with reference to FIG. 2 and/or any other encoder system or subsystem described in this specification. The logic circuit may be configured to perform the various operations described in this specification.

[00246] In some examples, decoder 30 may be implemented by logic 47 in a similar manner to implement various modules that are described with reference to decoder 30 in FIG. 3 and/or any other decoder system or subsystem described in this specification. In some examples, the decoder 30 implemented by logic may include an image buffer (eg, implemented by processing unit 2820 or memory 44) and a graphics processor (eg, implemented by processing unit 46). The GPU may be in communicative communication with an image buffer. The graphics processor may include a decoder 30, implemented by logic 47, for implementing various modules, which are described with reference to FIG. 3 and/or any other decoder system or subsystem described in this specification.

[00247] In some examples, antenna 42 may be configured to receive an encoded video data bitstream. As described, the encoded bitstream may include data, an indicator, an index value, mode selection data, and the like that relate to coding of video frames and that are described in this specification, for example, data related to coding division (for example, a coefficient transform or quantized transform coefficient, optional indicator (as described) and/or data defining the encoding division). The video encoding system 40 may further include a decoder 30 that is coupled to antenna 42 and configured to decode the encoded bit stream. The display device 45 is adapted to present a video frame.

[00248] It should be understood that in this embodiment of the present application, for the example described with reference to the encoder 20, the decoder 30 may be configured to perform the reverse process. With respect to syntax element signaling, decoder 30 may be configured to receive and parse such syntax element and accordingly decode associated video data. In some examples, encoder 20 may entropy encode a syntax element into an encoded video bitstream. In such examples, decoder 30 may parse the syntax element and accordingly decode the corresponding video data.

[00249] FIG. 5 is a schematic block diagram of a video encoding apparatus 400 (eg, a video encoding apparatus 400 or a video decoding apparatus 400) according to an embodiment of the present invention. Video encoding apparatus 400 is suitable for implementing the embodiments described in this specification. In an embodiment, video encoding device 400 may be a video decoder (eg, decoder 30 in FIG. 3) or a video encoder (eg, encoder 20 in FIG. 2). In another embodiment, video encoding device 400 may be one or more components of decoder 30 in FIG. 3 or encoder 20 in FIG. 2.

[00250] The video encoding device 400 includes: input ports 410 and a receiving unit (Rx) 420, which are configured to receive data; a processor, logical unit, or central processing unit (CPU) 430 that is configured to process data; a transmitter (Tx) unit 440 and output ports 450 that are configured to transmit data; and a memory 460 configured to store data. The video encoding device 400 may further include an optical-electrical component and an electrical-optical (EO) component connected to input ports 410, a receiver unit 420, a transmitter unit 440, and output ports 450 for outputting or inputting an optical or electrical signal.

[00251] The processor 430 is implemented in hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (eg, a multi-core processor), FPGA, ASIC, and DSP. Processor 430 communicates with input ports 410, receiver unit 420, transmitter unit 440, output ports 450, and memory 460. Processor 430 includes an encoding module 470 (eg, encoding module 470 or decoding module 470). The encoding/decoding module 470 implements the embodiments disclosed in this description to implement the image prediction method provided in the embodiments of the present invention. For example, encoding/decoding module 470 performs, processes, or provides various encoding operations. Therefore, the encoding/decoding unit 470 significantly improves the functions of the video encoding device 400 and affects the conversion of the video encoding device 400 to another status. Alternatively, encoding/decoding module 470 is implemented as an instruction stored in memory 460 and executed by processor 430.

[00252] Memory 460 includes one or more disks, tape drives, and solid state drives and can be used as an overflow storage device, for storing programs when such programs are selected for execution, and for storing instructions and data that are read during program execution. Memory 460 may be volatile and/or nonvolatile and may be read-only memory (ROM), random access memory (RAM), random access memory (ternary content-addressable memory TCAM), and/or static random access memory (SRAM). ).

[00253] FIG. 6 is a simplified block diagram of a device 500 that can be used as one or two of the source device 12 and the target device 14 in FIG. 1 according to an exemplary embodiment. The device 500 may implement the image prediction method in embodiments of the present application. In other words, FIG. 6 is a schematic block diagram of an implementation of an encoding device or a decoding device (referred to as encoding device 500 for short) according to an embodiment of the present application. The encoding device 500 may include a processor 510, a memory 530, and a bus system 550. The processor and memory are coupled through the bus system. The memory is configured to store instructions. The processor is configured to execute an instruction stored in memory. The program code is stored in the memory of the encoder. The processor may call program code stored in memory to implement various video encoding or decoding methods described herein, in particular, various new methods for dividing image blocks. To avoid repetition, details are again omitted here.

[00254] In this embodiment of the present application, the processor 510 may be a central processing unit (CPU). Alternatively, processor 510 may be another general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component and so on. The general purpose processor may be a microprocessor, or the processor may be any conventional processor or the like.

[00255] Memory 530 may include read-only memory (ROM) or random access memory (RAM). Alternatively 530, any other suitable type of storage device may be used. Memory 530 may include code and data 531 that is accessed by processor 510 via bus 550. Memory 530 may further include an operating system 533 and an application program 535. Application program 535 includes at least one program that allows processor 510 to implement the video encoding or decoding method described in this application. For example, application program 535 may include applications 1 through N and further include a video encoding or decoding application (referred to as a video encoding application for short) that implements the video encoding or decoding method described herein.

[00256] In addition to the data bus, bus system 550 may further include a power bus, a control bus, a status signal bus, or the like. However, for purposes of clear description, the various types of buses are referred to in the drawing as bus system 550.

[00257] Optionally, encoder 500 may further include one or more output devices, such as a display 570. In an example, display 570 may be a touch display that integrates a display and a touch unit that operatively recognizes touch input. Display 570 may be connected to processor 510 via bus 550.

[00258] To better understand the image prediction method in embodiments of the present application, the following first describes in detail some related concepts and the main content of inter-frame prediction.

[00259] Inter-frame prediction means searching the reconstructed image for a consistent reference block for the current image block in the current image and using the pixel value of a pixel element in the reference block as a predictor of the pixel value of a pixel element in the current image block (This process is called Motion estimation, ME)).

[00260] Motion estimation is to try many reference blocks in a reference image for the current image block, and then finally determine one or two reference blocks (two reference blocks are required for dual prediction) from the set of reference blocks using speed optimization distortion (rate-distortion optimization, RDO) or another method. The reference block is used to perform inter-frame prediction of the current image block.

[00261] The motion information of the current image block includes prediction direction indication information (which is usually forward prediction, backward prediction, or bidirectional prediction), one or two motion vectors (MVs) pointing to the reference block, image indication information , in which the reference block (which is usually represented by a reference index) is located.

[00262] Forward prediction means selecting a reference image from a set of forward reference images to obtain a reference block for the current image block. Inverse prediction means selecting a reference image from a set of inverse reference images to obtain a reference block for the current image block. Dual prediction means selecting a reference picture from a set of forward reference pictures and a reference picture from a set of inverse reference pictures to obtain a reference block. When the double prediction method is used, there are two reference blocks in the current encoding block. Each reference block must be specified by a motion vector and a reference index. A pixel value predictor of the pixel element in the current image block is then determined based on the pixel values of the pixel elements in the two reference blocks.

[00263] HEVC has two inter-frame prediction modes: AMVP mode and merge mode.

[00264] In AMVP mode, spatially or temporally adjacent coded blocks (denoted as adjacent blocks) of the current coding block are first scanned. The list of possible motion vectors is constructed based on information about the motion of neighboring blocks. The optimal motion vector is then determined from the list of candidate motion information based on the speed distortion cost, and the candidate motion information with minimal velocity distortion cost is used as a motion vector predictor (MVP) of the current encoding block.

[00265] The location and order of traversal of adjacent blocks is specified. The rate-distortion cost can be obtained by calculating using formula (1), where J is the rate-distortion cost, SAD is the sum of absolute differences (SAD) between the original pixel value and the predicted one pixel value, which is obtained by motion estimation performed using a candidate motion vector predictor, R is the bit rate, and λ is the Lagrange multiplier. The encoder side transmits to the decoder side the index value and the reference index value of the selected motion vector predictor in the list of possible motion vectors. In addition, the encoder side can perform a motion search in the neighborhood centered on the MVP to obtain the actual motion vector of the current encoding block, and then transmit the difference (motion vector difference) between the MVP and the actual motion vector to the decoder side.

J=SAD+λR (1)

[00266] In addition, from the point of view of different motion models, the AMVP mode can be divided into an AMVP mode based on a translational model and an AMVP mode based on a non-translational model.

[00267] In the merging mode, a list of possible motion information is first created based on the motion information of a spatially or temporally encoded unit of the current encoding unit. Then, the optimal motion information is determined from the list of possible motion information as the motion information for the current encoding unit based on the speed distortion cost. Finally, an index value (hereinafter referred to as a fusion index) of the location of the optimal motion information in the possible motion information list is transmitted to the decoder side.

[00268] In the fusion mode, spatial and temporal information about possible movement of the current coding unit can be shown in FIG. 7. Information about a possible candidate in space comes from five spatially adjacent blocks (A0, A1, B0, B1 and B2). If neighboring blocks are not available or the prediction mode is intra-frame prediction, neighboring blocks are not added to the possible motion information list.

[00269] Temporary possible motion information for the current coding unit can be obtained by scaling the MV block at an appropriate location in the reference frame based on the picture order count (POC) counts of the reference frame and the current frame. When a block at a corresponding location in the reference frame is received, it may first be determined whether the block at location T in the reference frame is available. If the block at location T is not available, the block at location C is selected.

[00270] When a translational model is used for prediction, the same motion information is used for all pixels in the encoding block, and then motion compensation is performed based on the motion information to obtain a predictor of the pixel in the encoding block. However, in the real world there are many movements. Many objects, such as a spinning object, roller coasters spinning in different directions, fireworks, and some movie stunts, are not in forward motion. If these moving objects, especially those in user generated content (UGC) scripts, are encoded using block motion compensation technology based on the translational motion model of the current encoding standard, it greatly impacts encoding efficiency. Therefore, prediction based on the non-translational motion model is provided to improve the encoding effect.

[00271] In non-translational motion model-based prediction, the same motion model is used on the encoder side and the decoder side to obtain the motion information of each auxiliary motion compensation block in the current encoding block, and then motion compensation is performed based on the motion information of the auxiliary block motion compensation to obtain a prediction sub-block of each sub-block to improve the prediction efficiency. Frequently used non-translational motion models include the 4-parameter affine motion model and the 6-parameter affine motion model.

[00272] In addition, skip mode is a special mode of the merge mode. The difference is that during skip mode transmission there is no remainder and only the possible merge index is transmitted. The merge index is used to indicate information about the best or target possible move in the list of possible merge move information.

[00273] When predicting an image, different modes can be used. These common modes are described in detail below.

[00274] Merging with motion vector difference mode:

[00275] In merge with motion vector difference (MMVD) mode, one or more candidate motion vectors are selected from a list of candidate motion vectors to merge, and then a motion vector expansion (MV) expression is performed based on the movement of the candidate vector. The MV expansion expression includes the MV starting point, the motion step, and the motion direction.

[00276] Typically, the type of possible motion vector selected in MMVD mode is the default merge type (eg, MRG_TYPE_DEFAULT_N). The selected possible motion vector is the starting point of the MV. In other words, the selected candidate motion vector is used to determine the initial location of the MV.

[00277] As shown in Table 1, the Base candidate IDX indicates which candidate motion vector is selected from the list of candidate motion vectors as the optimal candidate motion vector. If the list of possible fusion motion vectors includes one possible motion vector to select, the underlying candidate IDX may not be determined.

Table 1

Base Possible Index 0 1 2 3 ^Nth MVP ^1st MVP ^2nd MVP ^3rd MVP ^4th MVP

[00278] For example, if the candidate base index is 1, the selected candidate motion vector is the second motion vector in the list of candidate motion vectors for merging.

[00279] The distance index (Distance IDX) represents the displacement distance information of the motion vector. The distance index value represents the offset (eg, a predefined distance) from the original location. The distance here is represented by Pixel Distance. The distance between pixels can be briefly referred to as pel. The correspondence between the distance index and the distance between pixels can be shown in Table 2.

table 2

Distance index 0 1 2 3 4 5 6 7 Distance in pixels 1/4-pal 1/2-pal 1-pal 2-pal 4-pal 8-pal 16-pal 32-pal

[00280] The Direction IDX is used to represent the motion vector difference (MVD) direction based on the origin location. A total of four cases can be included in the direction indicator. The specific definition can be shown in Table 3.

Table 3

Direction index 00 01 10 eleven X axis + - N/A N/A Y axis N/A N/A + -

[00281] The process of determining the predicted pixel value of the current image block based on the MMVD method includes the following steps:

[00282] Step 1: Determine the starting point of the MV based on the base candidate IDX.

[00283] FIG. 8 is a schematic diagram of an MMVD search point according to an embodiment of the present application, and FIG. 9 is a schematic diagram of an MMVD search process according to this embodiment of the present application.

[00284] For example, the starting point MV is the hollow point at the center in FIG. 8, or the location corresponding to the solid line in FIG. 9.

[00285] Step 2: Determine, based on the IDX direction, the offset direction ab based on the starting point MV.

[00286] Step 3: Determine, based on the IDX distance, the number of pixel elements that are offset in the direction indicated by the IDX direction.

[00287] For example, direction IDX==00 and distance IDX=2 indicate that a motion vector that is shifted by one pixel element in the positive X direction is used as the motion vector of the current image block to predict or obtain the predicted pixel value of the current block Images. Combined intra-frame and inter-frame modes:

[00288] In a merge-mode encoded encoding block/CU, an indicator (eg, ciip_flag) may be transmitted to indicate whether combined inter and intra prediction (CIIP) mode is being used for the current block Images. When the CIIP mode is used, the intra prediction block may be generated based on the intra prediction mode selected from the intra candidate list according to the corresponding syntax element, and the inter prediction block is generated using a conventional inter prediction method. Finally, an adaptive weighting method is used to combine an intra prediction block and an inter prediction block to form a final prediction block.

[00289] For the luma block, the list of possible intra-frame encoded modes can be selected from four modes: constant current mode, flat mode, horizontal (horizontal) mode, and vertical (vertical) mode. The size of the list of possible intra-frame modes is selected based on the shape of the current coding block, and there may be three or four modes in the list of possible intra-frame modes. When the width of the current encoding block/CU is more than twice the height, the list of intra-frame possible modes does not include the horizontal mode. When the height of the current encoding block/CU is more than twice the width, the list of possible intra-frame modes does not include vertical mode.

[00290] In a weighting method that combines intra-frame coding and inter-frame coding, different weighting coefficients are used for different intra-frame prediction modes. In particular, when DC or planar mode is used for intra-frame encoding or when the length or width of the current encoding block is less than or equal to 4, the same weight value/weighting factor is used for the predictor obtained by intra-prediction and the predictor obtained by inter-prediction. Otherwise, the weight value/weighting factor may be determined based on the intra prediction mode used by the current block and/or the location of the prediction sample in the current block. For example, variable weighting is used when horizontal and vertical modes are used for intra-frame coding.

[00291] Triangle prediction block mode:

[00292] Triangle prediction unit mode (triangle PU) may also be referred to as triangle partition mode (TPM) or triangle merging mode. For ease of description, in the present application, the triangle prediction block mode or the triangle division mode is briefly referred to as TPM, which is also applicable hereinafter.

[00293] As shown in FIG. 11, the current image block is divided into two triangle prediction blocks, and a motion vector and a reference index are selected from a single prediction candidate list for each triangle prediction block. A predictor is then obtained for each of the two triangle prediction blocks, and the predictor is obtained by performing adaptive weighting on the pixel included in each hypotenuse region. The conversion and quantization processes are then carried out for the entire current image block. It should be noted that the triangle prediction block method is generally applied only in the skip mode or merge mode. Left side of fig. 10 shows the top left to bottom right division mode (in other words, top left to bottom right division), and the right side of FIG. 10 shows the top-right to bottom-left division mode (in other words, top-right-bottom-left division).

[00294] The possible single prediction list in the triangle prediction block mode may typically include five predicted possible motion vectors. These predicted possible motion vectors are obtained, for example, using seven peripheral neighboring blocks (five spatial neighboring blocks and two temporally co-located blocks) in FIG. 5. Information about the movement of seven neighboring blocks is sought, and the seven neighboring blocks are placed in a list of possible single predictions in the sequence. For example, the sequence may be a dual-prediction motion vector in L0, a dual-prediction motion vector in L1, and the average of the motion vectors in L0 and L1. If there are fewer than five possible ones, the null motion vector 0 is added to the list of possible single predictions. During encoding, a list of possible single predictions is obtained in the above manner. For example, in the candidate single prediction list, forward prediction motion information is used to predict a pixel predictor of one triangle PU, and backward motion information is used to predict a pixel predictor of another triangle PU. The encoder side selects the optimal motion vector by traversal. For example, you can use the following {m, i, j} method:

{ 0, 1, 0 }, { 1, 0, 1 }, { 1, 0, 2 }, { 0, 0, 1 }, { 0, 2, 0 }

{ 1, 0, 3 }, { 1, 0, 4 }, { 1, 1, 0 }, { 0, 3, 0 }, { 0, 4, 0 }

{ 0, 0, 2 }, { 0, 1, 2 }, { 1, 1, 2 }, { 0, 0, 4 }, { 0, 0, 3 }

{ 0, 1, 3 }, { 0, 1, 4 }, { 1, 1, 4 }, { 1, 1, 3 }, { 1, 2, 1 }

{ 1, 2, 0 }, { 0, 2, 1 }, { 0, 4, 3 }, { 1, 3, 0 }, { 1, 3, 2 }

{ 1, 3, 4 }, { 1, 4, 0 }, { 1, 3, 1 }, { 1, 2, 3 }, { 1, 4, 1 }

{ 0, 4, 1 }, { 0, 2, 3 }, { 1, 4, 2 }, { 0, 3, 2 }, { 1, 4, 3 }

{ 0, 3, 1 }, { 0, 2, 4 }, { 1, 2, 4 }, { 0, 4, 2 }, { 0, 3, 4 }

where in { m, i, j}, m in the first position represents the top-left-down-right split mode or the top-right-down-left split mode, the second location represents the forward motion information of the ^ith predicted possible motion vector used for the first triangle PU, and the third location represents backward motion information of the ^jth predicted possible motion vector used for the second triangle PU.

[00295] For an adaptive weighting process performed based on a pixel predictor included in the hypotenuse region, see FIG. 11. After the prediction for triangle prediction blocks P1 and P2 is completed, the adaptive weighting process is carried out for pixel included in the hypotenuse region to obtain the final predictor of the current image block.

[00296] For example, in the image on the left in FIG. 11, the predictor of the pixel at location 2 is $$\frac{2}{8}\times {\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="00000012.png"\; state="\{\{state\}\}">P</figure-callout>}}_1\text{+}\frac{6}{8}\times {\mathrm{<figure-callout\; id="2"\; label="P"\; filenames="00000012.png"\; state="\{\{state\}\}">P</figure-callout>}}_2$$ . $${\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="00000012.png"\; state="\{\{state\}\}">P</figure-callout>}}_1$$ represents the pixel predictor in the upper right area in FIG. 11, and is a pixel predictor in the lower left region of FIG. eleven.

[00297] Two sets of weighted parameters:

[00298] The first set of weighted parameters, {7/8, 6/8, 4/8, 2/8, 1/8} and {7/8, 4/8, 1/8}, are used for the luma and chrominance points , respectively.

[00299] Second set of weighted parameters, {7/8, 6/8, 5/8, 4/8, 3/8, 2/8, 1/8} and {6/8, 4/8, 2/8 } are used for luma and chrominance points, respectively.

[00300] One set of weighted parameters is used to encode and decode the current image block. When the reference images of two prediction blocks are different or the difference of motion vectors between two prediction blocks exceeds 16 pixels, a second set of weighted parameters is selected; otherwise, the first set of weighted parameters is used.

[00301] FIG. 12 is a schematic block diagram of a video communication system according to an embodiment of the present application.

[00302] The video communication system 500 shown in FIG. 12 includes a source device 600 and a target device 700. The source device 600 may encode the received video and transmit the encoded video bit stream to the receiving device 700. The target device 700 may analyze the received video bit stream to obtain a video image and display the video by a display device.

[00303] The image prediction method in embodiments of the present application may be performed by a source device 600 or a target device 700. Specifically, the image prediction method in embodiments of the present application may be performed by a video encoder 603 or a video decoder 702.

[00304] The video communication system 500 may also be referred to as a video encoding system. The source device 600 may also be referred to as a video encoding apparatus or video encoding apparatus. The target device 700 may also be referred to as a video decoding apparatus or a video decoding apparatus.

[00305] In FIG. 12, the source device 600 includes a video capture device 601, a video memory 602, a video encoder 603, and a transmitter 604. The video memory 602 may store video acquired by the video capture device 601. Video encoder 603 may encode video data from video memory 602 and video capture device 601. In some examples, source device 600 directly transmits encoded video data to target device 700 via transmitter 604. The encoded video data may further be stored on a storage medium or file server such that target device 700 retrieves the encoded video data later for decoding and/or playback.

[00306] In FIG. 12, target device 700 includes a receiver 701, a video decoder 702, and a display device 703. In some examples, receiver 701 may receive encoded video data via channel 800. Display device 703 may be integrated with target device 700 or may be located outside of target device 7000. Typically, display device 703 displays decoded video data. The display device 703 may include many types of display devices, such as a liquid crystal display, plasma display, organic light-emitting diode display, or other type of display device.

[00307] Specific implementations of source device 600 and target device 700 may be any of the following devices: desktop computer, mobile computing device, laptop (e.g., laptop), tablet computer, set-top box, smartphone, handset, television, camera, device display, digital media player, game console, vehicle-mounted computer or other similar device.

[00308] A target device 700 may receive encoded video data from a source device 600 on a channel 800. The channel 800 may include one or more media and/or devices that can move encoded video data from a source device 600 to a target device 700. In one example, channel 800 may include one or more transmission media that may allow source device 600 to directly transmit encoded video data to target device 700 in real time. In this example, source device 600 may modulate encoded video data in accordance with a communication standard (e.g., a wireless communication protocol) and may transmit the modulated video data to target device 700. One or more communication media may include wireless and/or wired communication media, for example, radio frequency (RF) spectrum or one or more physical transmission lines. One or more communication media may form part of a packet-based network (eg, a local area network, a wide area network, or a wide area network (eg, the Internet)). The one or more communication media may include a router, switch, base station, or other device that implements communication between source device 600 and target device 700.

[00309] In another example, channel 800 may include a storage medium on which encoded video data generated by source device 600 is stored. In this example, target device 700 may access the storage medium via disk access or card access. The storage medium may include a variety of locally available storage media, such as Blu-ray, digital video disc (DVD), compact disc read-only memory (CD-ROM), flash memory or other suitable digital storage medium configured to store encoded video data.

[00310] In another example, channel 800 may include a file server or other intermediate storage device that stores encoded video data generated by source device 600. In this example, target device 700 may access, by streaming or downloading, the encoded video data stored on a file server or other intermediate storage device. A file server may be a type of server that can store encoded video data and transmit the encoded video data to a target device 700. For example, the file server may include a World Wide Web (Web) server (for example, used for a website) , file transfer protocol (FTP) server, network attached storage (NAS) and local disk.

[00311] Target device 700 can access encoded video data over a standard data connection (eg, an Internet connection). An exemplary data connection type includes a wireless link, a wired connection (eg, a cable modem), or a combination thereof that is suitable for accessing encoded video data stored on a file server. The transfer of encoded video data from a file server can be streaming, downloading, or a combination of both.

[00312] The image prediction method in embodiments of the present application is described in detail below with reference to specific accompanying drawings.

[00313] FIG. 13 is a schematic flow diagram of an image prediction method according to an embodiment of the present application. The image prediction method shown in FIG. 13 may be performed by an image prediction device (the image prediction device may be located in an image encoding device (system) or an image decoding device (system). In particular, the method shown in FIG. 13 may be performed by an image encoding device or an image decoding device. The method shown in FIG. 13 may be performed on the encoder side or may be performed on the decoder side. The method shown in FIG. 13 includes steps 1001-1008. Below is a detailed description of these steps.

[00314] 1001: Start.

[00315] Step 1001 indicates that image prediction begins.

[00316] 1002: Determines whether merge mode is used for the current image block.

[00317] Optionally, the method shown in FIG. 13 further includes: obtaining the current image block before step 1002.

[00318] The current image block may be an image block in the current image being encoded or decoded.

[00319] It should be understood that in the present application, the current image block may be obtained during the process of determining the merge target mode of the current image block or after determining the merge target mode of the current image block.

[00320] For the decoder side, at step 1002, it can be determined whether the merge mode is being used for the current image block based on the CU level syntax element merge_flag[x0][y0].

[00321] If merge_flag[x0][y0]=1, it is determined that the merge mode is used to predict the current image block. If merge_flag[x0][y0]=1, it is determined that the merge mode is not used to predict the current image block. x0 and y0 represent the coordinates of the location of the luminance pixel element in the upper left corner of the current image block relative to the luminance pixel element in the upper left corner of the current image.

[00322] After determining, based on the CU level syntax element merge_flag[x0][y0], that the merge mode is being used for the current image block, the target merge mode that is ultimately used can be determined by parsing specific information into the level syntax element CU merge_flag[x0][y0].

[00323] When it is determined at step 1002 that the merge mode is not used for the current image block, a mode other than the merge mode may be used to predict the current image block. For example, when it is determined that the merge mode is not used for the current image block, the AMVP mode may be used to predict the current image block.

[00324] After it is determined at step 1002 that the merge mode is used for the current image block, step 1003 continues to determine the target merge mode applicable to the current image block.

[00325] 1003: Determine whether to use layer-1 merge mode.

[00326] Specifically, whether the layer-1 merge mode is available can be determined based on a high-level syntax element corresponding to the layer-1 merge mode and/or available status information corresponding to the layer-1 merge mode.

[00327] In an exemplary implementation of the present invention, it is assumed that the layer-1 merge mode includes only two merge modes: merge mode A and merge mode B. In this case, whether there is a layer-1 merge mode available is determined by the merge mode one by one. If there is a merge mode available, step 1005 occurs. If the merge mode is not available in the layer-1 merge mode, it is determined that the layer-1 merge mode is not available. In this case, the target merge mode must be determined from the layer-2 merge mode. In other words, step 1004 occurs.

[00328] 1004: Determine whether the high-level syntax element corresponding to the first merge mode indicates that use of the first merge mode is prohibited.

[00329] The first merging mode belongs to the layer-2 merging mode, and the layer-2 merging mode further includes a second merging mode.

[00330] When it is determined at step 1004 that the high-level syntax element corresponding to the first merge mode indicates that use of the first merge mode is prohibited, step 1006 is executed to determine the second merge mode as the target merge mode.

[00331] In the present application, when the high-level syntax element of the first merge mode indicates that the use of the first merge mode is prohibited, there is no need to analyze the available status information of the remaining second merge mode, and the second merge mode can be directly determined as the final target merge mode. This can reduce as much as possible the redundancy caused by determining the target fusion mode in the image prediction process.

[00332] When it is determined at step 1004 that the high-level syntax element corresponding to the first merge mode indicates that the first merge mode is enabled, step 1007 is executed to further determine the target merge mode.

[00333] 1005: Predict the current image block based on the layer-1 merge mode.

[00334] It should be understood that at step 1005, if merge mode A is available in layer-1 merge mode, the current image block is predicted based on merge mode A.

[00335] 1006: Determine the second merge mode as the target merge mode applicable to the current image block.

[00336] When the high-level syntax element corresponding to the first merge mode indicates that the first merge mode is prohibited from being used, there is no need to parse the high-level syntax element and/or available status information corresponding to the second merge mode, and the second merge mode can be directly defined as the target merge mode.

[00337] 1007: Determine the target merge mode based on a high-level syntax element corresponding to the second merge mode and/or information about the available status of the second merge mode.

[00338] Information about the available status of the second merge mode is used to indicate whether the second merge mode is used when the current image block is predicted.

[00339] For example, the second merge mode is the CIIP mode, and the available status information of the second merge mode is the ciip_flag value. When ciip_flag is 0, CIIP mode is not used for the current image block. When ciip_flag is 1, CIIP mode is used to predict the current image block.

[00340] It should be understood that for the CIIP mode, if the CIIP mode is to be selected as the target merge mode, the high-level syntax element corresponding to CIIP must indicate that the CIIP mode is enabled for use and available status information indicating the available status CIIP mode must indicate that CIIP is available.

[00341] For example, when sps_ciip_enabled_flag=1 and ciip_flag=1, the CIIP mode can be determined as the merge target mode of the current image block.

[00342] At step 1007, since the first merge mode is enabled, both the first merge mode and the second merge mode can be used as the merge target mode of the current image block. Therefore, the target merge mode can be determined from the layer-2 merge mode based on the high-level syntax and available status information that corresponds to one of the merge modes.

[00343] Optionally, the first merge mode is the TPM mode and the second merge mode is the CIIP mode.

[00344] The following describes in detail how to determine the target merge mode when the first merge mode is a TPM mode and the second merge mode is a CIIP mode.

[00345] Optionally, when a high-level syntax element corresponding to a TPM mode indicates that the TPM mode is prohibited from being used, the CIIP mode is defined as the target merge mode.

[00346] For example, when the sps_triangle_enabled_flag corresponding to the TPM mode is 0, use of the TPM mode is disabled. In this case, there is no need to analyze the specific ciip_flag value. Instead, the CIIP mode can be directly defined as the target merge mode. This way, you can reduce unnecessary parsing process and reduce solution redundancy.

[00347] In the present application, when a high-level syntax element corresponding to a TPM mode indicates that the TPM mode is prohibited from being used, it is not necessary to determine by analyzing the high-level syntax corresponding to the CIIP mode and/or the available status information indicating the available status CIIP mode and whether CIIP mode is available. Instead, the CIIP mode can be directly defined as the target merge mode. This can reduce redundancy in the process of determining the target merge mode.

[00348] Optionally, when the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is enabled for use, the target merge mode is determined based on the high-level syntax element corresponding to the CIIP mode and/or available status information indicating available CIIP mode status.

[00349] For example, when the sps_triangle_enabled_flag corresponding to the TPM mode is 1, the TPM mode can be used. In this case, both TPM mode and CIIP mode can be used as the target merge mode. Therefore, it is necessary to further determine whether to select TPM mode or CIIP mode as the target merge mode.

[00350] Optionally, when the high-level syntax element corresponding to the CIIP mode and/or the available status information that indicates the available status of the CIIP mode indicates/indicates that the CIIP mode is prohibited from being used, the TPM mode is determined to be the target merge mode.

[00351] That the high-level syntax element corresponding to the CIIP mode and/or the available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used includes cases 1 to 3:

[00352] Case 1: The high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is prohibited from being used, and the available status information, which indicates the available status of the CIIP mode, indicates that the CIIP mode is not available.

[00353] Case 2: The high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is enabled for use, and the available status information, which indicates the available status of the CIIP mode, indicates that the CIIP mode is not available.

[00354] Case 3: The available status information, which indicates the available status of the CIIP mode, indicates that the CIIP mode is not available.

[00355] Optionally, when a high-level syntax element corresponding to a CIIP mode indicates that the CIIP mode is enabled for use, and the available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is available, the CIIP mode is determined to be the target mode mergers.

[00356] 1008: Predict the current image block based on the target merge mode.

[00357] In the method shown in FIG. 13, before performing step 1007, the method shown in FIG. 13 further includes: determining that the type of the tile or group of tiles in which the current image block is located is B; and determining that the maximum number of possible TPM modes supported by the tile or group of tiles in which the current image block is located is greater than or equal to 2.

[00358] Optionally, before the target merge mode is determined based on a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode, the method further includes: determining that the fragment type or the group of fragments in which the current image block is located is B; and determining that the maximum number of possible TPM modes supported by the tile or group of tiles in which the current image block is located is greater than or equal to 2.

[00359] Optionally, in an embodiment, the method shown in FIG. 13 further includes: when the layer-1 merge mode is not available and the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is allowed to be used, but the current image block does not meet at least one of condition A and condition B, the mode CIIP is defined as the target fusion mode.

[00360] Condition A and condition B are as follows:

[00361] Condition A: The fragment type in which the current image block is located is B.

[00362] Condition B: The maximum number of possible TPM modes supported by the tile or group of tiles in which the current image block is located is greater than or equal to 2.

[00363] Condition A and condition B may be represented in certain specific ways. For example, condition A may be specifically represented by the parameter slice_type == B, and condition B may be specifically represented by parameter MaxNumTriangleMergeCand ≥ 2. MaxNumTriangleMergeCand specifies the maximum number of possible TPM modes supported by the slice or group of slices in which the current image block is located.

[00364] The TPM mode may be selected as the target fusion mode finally used to predict the current image block only when both condition A and condition B are satisfied.

[00365] If condition A or condition B is not satisfied, the CIIP mode is determined as the target merge mode.

[00366] When a high-level syntax element corresponding to a TPM mode specifies that the TPM mode is prohibited from being used unless condition A or condition B is satisfied, the CIIP mode is defined as the target merge mode.

[00367] When a high-level syntax element corresponding to a TPM mode indicates that the TPM mode is allowed to be used if condition A or condition B is not satisfied, the CIIP mode is determined to be the target merge mode.

[00368] In other words, the CIIP mode can be defined as the target merge mode provided that one of sps_trangle_enabled_flag=1, condition A and condition B is not satisfied.

[00369] Conversely, if sps_trangle_enabled_flag=1, condition A and condition B are all satisfied, the target merge mode must be determined based on the ciip_flag according to several conditions in the prior art.

[00370] Optionally, a high-level syntax element is a syntax element at at least one of a sequence level, an image level, a fragment level, and a fragment group level.

[00371] Optionally, the layer-1 merge mode includes a normal merge mode, an MMVD mode, and a subblock merge mode.

[00372] When it is determined whether the layer-1 merge mode is available, whether these modes are available can be sequentially determined in a sequence of the normal merge mode, the MMVD mode, and the subblock merge mode.

[00373] For example, it may first be determined whether normal merge mode is available. When normal merge mode is not available (if normal merge mode is available, normal merge mode can directly be used as the final target merge mode), it continues to determine whether MMVD mode is available. When MMVD mode is not available, it continues to determine whether subblock merging mode is available.

[00374] Optionally, the method shown in FIG. 13 can be applied to the encoder side to encode the current image block.

[00375] Optionally, the method shown in FIG. 13 can be applied to the decoder side to decode the current picture block.

[00376] In order to better understand the specific process of the image prediction method in the embodiments of the present application, the process of determining the image fusion mode in the image prediction method in the embodiments of the present application is described below in detail with reference to a specific example.

[00377] The following describes in detail the image prediction method in embodiments of the present application with reference to FIGS. 14 and table 4.

[00378] FIG. 14 shows a merge mode determination process according to an embodiment of the present application. The process shown in FIG. 14 includes steps 3001 through 3007. The following is a detailed description of these steps.

[00379] 3001: Start.

[00380] Step 3001 indicates that image prediction begins.

[00381] 3002: Determines whether merge mode is used for the current image block.

[00382] Specifically, when step 3002 is performed by the decoder side, in step 3002, whether the merge mode is used for the current image block can be determined based on the value of the CU level syntax element merge_flag[x0] [y0] corresponding to the current image block .

[00383] For example, as shown in Table 4, when merge_flag[x0][y0] = 0, the merge mode is not used for the current image block. In this case, the current image block can be predicted in a different way. For example, the current image block can be predicted in AMVP mode.

[00384] When merge_flag[x0][y0] = 1, merge mode is used for the current image block. It may then be further determined which fusion mode is used to predict the current image block.

[00385] It should be understood that when there is no merge_flag[x0][y0] in the bitstream, merge_flag[x0][y0] defaults to 0.

[00386](x0, y0) represents the coordinates of the location of the luminance pixel element in the upper left corner of the current image block relative to the luminance pixel element in the upper left corner of the current image. The meaning of (x0, y0) in the following syntax elements is the same as this, and the details are not described here.

[00387] Optionally, the method shown in FIG. 14 further includes: obtaining the current image block before step 3002.

[00388] The current image block may be an image block in the current image being encoded or decoded.

[00389] It should be understood that in the present application, the current image block may be obtained during the process of determining the merge target mode of the current image block or after determining the merge target mode of the current image block.

[00390] 3003: Determine whether normal merge mode is used for the current image block.

[00391] Specifically, at step 3003, whether the regular merge mode is used for the current image block can be determined by analyzing the value of the regular_merge_flag[x0][y0] syntax element.

[00392] When regular_merge_flag[x0][y0] = 1, it is determined that the current image block is in normal merge mode. In this case, step 3007 may be performed. To be specific, the current image block is predicted based on the normal merge mode.

[00393] When regular_merge_flag[x0][y0]=0, it is determined that the regular merge mode is not used for the current image block. In this case, it is necessary to continue with step 3004 to further determine the merge mode used for the current image block.

[00394] It should be understood that when there is no regular_merge_flag[x0][y0] in the bitstream, regular_merge_flag[x0][y0] defaults to 0.

[00395] 3004: Determine whether MMVD mode is used for the current image block.

[00396] Specifically, at step 3004, when the high-level syntax element corresponding to the MMVD mode indicates that MMVD may be allowed to be used, and the area of the current image block is not 32, whether MMVD is used for the current image block can be determined by analysis syntax element values mmvd_flag[x0][y0].

[00397] When mmvd_flag[x0][y0]=1, it is determined that the MMVD mode is used for the current image block. In this case, step 3007 may be performed. To be specific, the current image block is predicted based on the MMVD mode.

[00398] When mmvd_flag[x0][y0]=0, it is determined that the MMVD mode is not used for the current image block. In this case, it is necessary to continue with step 3005 to further determine the merge mode used for the current image block.

[00399] It should be understood that when there is no mmvd_flag[x0][y0] in the bitstream, mmvd_flag[x0][y0] defaults to 0.

[00400] 3005: Determine whether sub-block merging mode is used for the current image block.

[00401] At step 3004, whether the subblock merging mode is used for the current image block can be determined based on the value of the merge_subblock_flag[x0][y0] syntax element, which is obtained by analyzing the bitstream.

[00402] When merge_subblock_flag[x0][y0] = 1, it is determined that the subblock merge mode is used for the current image block. In this case, step 3007 may be performed. Specifically, the current image block is predicted based on the subblock merging mode.

[00403] When merge_subblock_flag[x0][y0] = 0, it is determined that the subblock merge mode is not used for the current image block. In this case, it is necessary to continue with step 3006 to further determine the merge mode used for the current image block.

[00404] It should be understood that when there is no merge_subblock_flag[x0][y0] in the bitstream, merge_subblock_flag[x0][y0] defaults to 0.

[00405] Additionally, at step 3004, the value of the merge_subblock_flag[x0][y0] syntax element can be parsed only when the maximum length of the candidate subblock merge list is greater than 0, and the width and height of the current image block are greater than or equal to 8, and step 3007 continues to execute when the resulting merge_subblock_flag[x0][y0] value is 0.

[00406] 3006: Determine the merge mode used for the current image block from the CIIP mode and the TPM mode.

[00407] Specifically, at step 3006, if all six conditions in the following conditions (1) - (6) are satisfied, ciip_flag[x0][y0] is parsed from the bit stream, the merge mode used for the current image block is determined based on on the value ciip_flag[x0][y0]. When ciip_flag[x0][y0] = 1, CIIP mode is used to predict the current image block.

[00408] In addition, when the following condition (1) is satisfied, if any of the following conditions (2) - (6) is not satisfied, the CIIP mode is used to predict the current image block.

(1) sps_ciip_enabled_flag = 1;

(2) sps_triangle_enabled_flag = 1;

(3) cu_skip_flag [x0] [y0] == 0;

(4) (cbWidth * cbHeight) ≥ 64;

(5) cbWidth < 128; And

(6) cbHeight < 128.

[00409] cbWidth and cbHeight are the width and height of the current image block, respectively.

[00410] Optionally, additional determination conditions may be further added when the merge mode used for the current block is determined at step 3006.

[00411] Based on the above conditions (1) - (6), conditions (7) and (8) can be additionally added:

(7) slice_type == B; And

(8) MaxNumTriangleMergeCand ≥ 2.

[00412] Optionally, at step 3006, if all eight conditions in the above conditions (1) - (8) are satisfied, ciip_flag[x0][y0] is parsed from the bit stream, the merge mode used for the current image block is determined based on values ciip_flag[x0][y0]. When ciip_flag[x0][y0] = 1, CIIP mode is used to predict the current image block.

[00413] In addition, when the above condition (1) is satisfied, if any of the following conditions (2) to (8) are not satisfied, the CIIP mode is used to predict the current image block.

[00414] 3007: Predict the current image block based on the merge mode used for the current image block.

[00415] When it is determined at step 3003 that the current image block is using the normal merge mode, at step 3007 the current image block is predicted based on the normal merge mode. When it is determined at step 3004 that the MMVD mode is used for the current image block, at step 3007 the current image block is predicted based on the MMVD mode. When it is determined at step 3005 that the sub-block merging mode is used for the current image block, at step 3007 the current image block is predicted based on the sub-block merging mode.

[00416] Table 4 shows how to determine, based on the corresponding syntax element, the merge mode used for the current image block when the merge mode is used. The definition of the merge mode of the current image block is described in detail below with reference to Table 4.

Table 4

merge_data( x0, y0, cbWidth, cbHeight ) {if ( CuPredMode[ x0 ][ y0 ] = = MODE_IBC ) {if( MaxNumMergeCand > 1 ) merge_idx[ x0 ][ y0 ] } else {… } else {… } else {if( MaxNumSubblockMergeCand > 0 && cbWidth >= 8 && cbHeight >= 8 ) merge_subblock_flag[ x0 ][ y0 ] if( merge_subblock_flag[ x0 ][ y0 ] == 1 ) {if( MaxNumSubblockMergeCand > 1 ) merge_subblock_idx[ x0 ][ y0 ] } else { if(sps_ciip_enabled_flag && sps_triangle_enabled_flag &&
cu_skip_flag[ x0 ][ y0 ] = = 0 && ( cbWidth * cbHeight ) ≥ 64 && cbWidth < 128 && cbHeight < 128 ) { ciip_flag[ x0 ][ y0 ] if( ciip_flag[ x0 ][ y0 ] && MaxNumMergeCand > 1 ) merge_idx[ x0 ][ y0 ] } if( MergeTriangleFlag[ x0 ][ y0 ] ) {merge_triangle_split_dir[ x0 ][ y0 ] merge_triangle_idx0[ x0 ][ y0 ] merge_triangle_idx1[ x0 ][ y0 ] } } } }

[00417] When regular_merge_flag[x0][y0] shown in Table 4 is 1, it is determined that the normal merge mode is used for the current image block. In this case, the normal merge mode parameter can be obtained by parsing the merge_idx[x0][y0] syntax element. When regular_merge_flag[x0][y0] shown in Table 4 is 0, it is determined that the regular merge mode is not used for the current image block, and it is further necessary to determine the merge mode used for the current image block.

[00418] When sps_mmvd_enabled_flag and cbWidth * cbHeight ! Table 4 shows 1 and 32 respectively, indicating that MMVD mode can be used for the current image block. In this case, the merge mode of the current image block can be determined based on the value of mmvd_flag[x0][y0]. If mmvd_flag[x0][y0] = 1, it is determined that the MMVD mode is used for the current image block, and the MMVD mode parameter can be determined by parsing the elements mmvd_merge_flag[x0][y0], mmvd_distance_idx[x0][y0], and mmvd_direction_idx [x0][y0]. If mmvd_flag[x0][y0]=0, it is additionally necessary to determine the merge mode used for the current image block.

[00419] When merge_subblock_flag[x0][y0] shown in Table 4 is 1, it is determined that the subblock merge mode is used for the current image block. When merge_subblock_flag[x0][y0] shown in Table 4 is 0, it is determined that the subblock merging mode is not used for the current image block, and it is further necessary to determine the merge mode used for the current image block.

[00420] When sps_ciip_enabled_flag shown in Table 4 is 0, it can be directly determined that the TPM mode is used for the current image block. However, when sps_ciip_enabled_flag and sps_ciip_enabled_flag shown in Table 4 are 1 and 0, respectively, it can be directly determined that the CIIP mode is used for the current image block.

[00421] As shown in Table 4, when all six conditions in the following conditions (1) - (6) are satisfied, information about the available status of the CIIP mode must be indicated, namely, the value ciip_flag[x0][y0] must be obtained from the bit flow. If ciip_flag[x0][y0] = 1, it is determined that CIIP mode is used for the current image block. If ciip_flag[x0][y0]=0, it is determined that the TPM mode is used for the current image block.

(1) sps_ciip_enabled_flag = 1;

(2) sps_triangle_enabled_flag = 1;

(3) cu_skip_flag[x0][y0] == 0;

(4) (cbWidth * cbHeight) ≥ 64;

(5) cbWidth < 128;

(6) cbHeight < 128.

[00422] Optional, if(sps_ciip_enabled_flag && sps_triangle_enabled_flag &&cu_skip_flag[x0][y0] == 0 && (cbWidth * cbHeight) ≥ 64&&cbWidth < 128 && cbHeight < 128) in table 4 can be replaced by if(sps_triangle_enabled_ flag && sps_ciip_enabled_flag && cu_skip_flag[x0 ][y0] = = 0 && (cbWidth * cbHeight) ≥ 64 && cbWidth < 128 && cbHeight < 128). In other words, the sequence sps_ciip_enabled_flag and sps_triangle_enabled_flag can be configured. The specific result can be shown in Table 5.

Table 5

merge_data( x0, y0, cbWidth, cbHeight ) {if ( CuPredMode[ x0 ][ y0 ] = = MODE_IBC ) {if( MaxNumMergeCand > 1 ) merge_idx[ x0 ][ y0 ] } else {… } else {… } else {if( MaxNumSubblockMergeCand > 0 && cbWidth >= 8 && cbHeight >= 8 ) merge_subblock_flag[ x0 ][ y0 ] if( merge_subblock_flag[ x0 ][ y0 ] == 1 ) {if( MaxNumSubblockMergeCand > 1 ) merge_subblock_idx[ x0 ][ y0 ] } else { if(sps_triangle_enabled_flag &&sps_ciip_enabled_flag &&
cu_skip_flag[ x0 ][ y0 ] = = 0 && ( cbWidth * cbHeight ) ≥ 64 && cbWidth < 128 && cbHeight < 128 ) { ciip_flag[ x0 ][ y0 ] if( ciip_flag[ x0 ][ y0 ] && MaxNumMergeCand > 1 ) merge_idx[ x0 ][ y0 ] } if( MergeTriangleFlag[ x0 ][ y0 ] ) {merge_triangle_split_dir[ x0 ][ y0 ] merge_triangle_idx0[ x0 ][ y0 ] merge_triangle_idx1[ x0 ][ y0 ] } } } }

[00423] It should be noted that in Table 4 and Table 5, the CIIP determination time sequence precedes the TPM determination time sequence. To be specific, the CIIP is determined first, and the prediction mode finally used for the current block is determined based on the CIIP state. If CIIP is correct, there is no need to further define TPM information. If CIIP is false, it means that only TPM is available. In this case, the last prediction mode of the current block can be set to TPM mode. The priority setting or time sequencing logic is just an example and can alternatively be adjusted as needed. For example, the TPM time sequence is made earlier than the CIIP time sequence. In this case, the condition for determining whether the TPM mode is applicable should also be adjusted as necessary.

Table 6

merge_data( x0, y0, cbWidth, cbHeight ) {if ( CuPredMode[ x0 ][ y0 ] = = MODE_IBC ) {if( MaxNumMergeCand > 1 ) merge_idx[ x0 ][ y0 ] } else {… } else {… } else {if( MaxNumSubblockMergeCand > 0 && cbWidth >= 8 && cbHeight >= 8 ) merge_subblock_flag[ x0 ][ y0 ] if( merge_subblock_flag[ x0 ][ y0 ] == 1 ) {if( MaxNumSubblockMergeCand > 1 ) merge_subblock_idx[ x0 ][ y0 ] } else { if(sps_ciip_enabled_flag && sps_triangle_enabled_flag &&
slice_type == B && MaxNumTriangleMergeCand ≥ 2 &&
cu_skip_flag[ x0 ][ y0 ] = = 0 && ( cbWidth * cbHeight ) ≥ 64 && cbWidth < 128 && cbHeight < 128) { ciip_flag[ x0 ][ y0 ] if( ciip_flag[ x0 ][ y0 ] && MaxNumMergeCand > 1 ) merge_idx[ x0 ][ y0 ] } if( MergeTriangleFlag[ x0 ][ y0 ] ) {merge_triangle_split_dir[ x0 ][ y0 ] merge_triangle_idx0[ x0 ][ y0 ] merge_triangle_idx1[ x0 ][ y0 ] } } } }

[00424] As shown in Table 6, when all eight conditions in the following conditions (1) - (8) are satisfied, the indication of CIIP mode available status information, namely, the value of ciip_flag[x0][y0], must be obtained from the bit flow. If ciip_flag[x0][y0] = 1, it is determined that CIIP mode is used for the current image block. If ciip_flag[x0][y0] = 0, it is determined that the TPM mode is used for the current image block.

(1) sps_ciip_enabled_flag = 1;

(2) sps_triangle_enabled_flag = 1;

(3) cu_skip_flag[x0][y0] == 0;

(4) (cbWidth * cbHeight) ≥ 64;

(5) cbWidth < 128;

(6) cbHeight < 128.

[00425] Optional, if(sps_ciip_enabled_flag && sps_triangle_enabled_flag &&slice_type == B && MaxNumTriangleMergeCand ≥ 2 && cu_skip_flag[x0][y0] = =0&& (cbWidth * cbHeight) ≥ 64&&cbWidth < 128 && cb Height < 128) in table 6 can alternatively be replaced by if(sps_triangle_enabled_flag && sps_ciip_enabled_flag &&slice_type == B && MaxNumTriangleMergeCand ≥ 2 && cu_skip_flag[x0][y0] = =0&& (cbWidth * cbHeight) ≥ 64&&cbWidth < 128 && cbHeight < 128). In other words, the sequence of sps_ciip_enabled_flag and sps_triangle_enabled_flag can be adjusted. The specific result can be shown in Table 7.

Table 7

merge_data( x0, y0, cbWidth, cbHeight ) {if ( CuPredMode[ x0 ][ y0 ] = = MODE_IBC ) {if( MaxNumMergeCand > 1 ) merge_idx[ x0 ][ y0 ] } else {… } else {… } else {if( MaxNumSubblockMergeCand > 0 && cbWidth >= 8 && cbHeight >= 8 ) merge_subblock_flag[ x0 ][ y0 ] if( merge_subblock_flag[ x0 ][ y0 ] == 1 ) {if( MaxNumSubblockMergeCand > 1 ) merge_subblock_idx[ x0 ][ y0 ] } else if(sps_triangle_enabled_flag && sps_ciip_enabled_flag &&
slice_type == B && MaxNumTriangleMergeCand ≥ 2 &&
cu_skip_flag[ x0 ][ y0 ] = = 0 && ( cbWidth * cbHeight ) ≥ 64 && cbWidth < 128 && cbHeight < 128) { ciip_flag[ x0 ][ y0 ] if( ciip_flag[ x0 ][ y0 ] && MaxNumMergeCand > 1 ) merge_idx[ x0 ][ y0 ] } if( MergeTriangleFlag[ x0 ][ y0 ] ) {merge_triangle_split_dir[ x0 ][ y0 ] merge_triangle_idx0[ x0 ][ y0 ] merge_triangle_idx1[ x0 ][ y0 ] } } } }

[00426] The foregoing describes in detail the image prediction method in embodiments of the present application with reference to FIGS. 13 and fig. 14. The image prediction method in embodiments of the present application will be described below with reference to FIG. 15.

[00427] FIG. 15 is a schematic flow diagram of an image prediction method according to an embodiment of the present application. The image prediction method shown in FIG. 15 may be performed by an image prediction device (the image prediction device may be located in an image decoding device (system) or an image encoding device (system). In particular, the method shown in FIG. 15 may be performed by an image encoding device or an image decoding device. The method shown in FIG. 15 may be performed on the encoder side or may be performed on the decoder side. The method shown in FIG. 15 includes steps 4001-4007. Below is a detailed description of these steps.

[00428] 4001: Start.

[00429] Step 4001 indicates that image prediction begins.

[00430] 4002: Determines whether merge mode is used for the current image block.

[00431] For the decoder side, in step 4002, whether the merge mode is used for the current image block can be determined based on the CU level syntax element merge_flag[x0][y0]. For a specific determination process, see the corresponding descriptions below step 1003.

[00432] When it is determined at step 4002 that the merge mode is not used for the current image block, a mode other than the merge mode may be used to predict the current image block. For example, when it is determined that the merge mode is not used for the current image block, the AMVP mode may be used to predict the current image block.

[00433] After it is determined at step 4002 that the merge mode is used for the current image block, step 4003 continues to be executed to determine the target merge mode applicable to the current image block.

[00434] Optionally, the method shown in FIG. 15 further includes: obtaining the current image block before step 4002.

[00435] The current image block may be an image block in the current image being encoded or decoded.

[00436] It should be understood that in the present application, the current image block may be obtained during the process of determining the merge target mode of the current image block or after determining the merge target mode of the current image block.

[00437] 4003: Determine whether to use layer-1 merge mode.

[00438] In particular, whether the layer-1 merge mode is available can be determined based on a high-level syntax element corresponding to the layer-1 merge mode and/or available status information corresponding to the layer-1 merge mode.

[00439] Optionally, the layer-1 merge mode at step 4003 includes a normal merge mode, an MMVD mode, and a subblock merge mode.

[00440] When it is determined whether the layer-1 merging mode is available, whether these modes are available can be sequentially determined in a sequence of the normal merging mode, the MMVD mode, and the subblock merging mode. When all modes are unavailable, it is determined that layer-1 merge mode is unavailable.

[00441] When it is determined at step 4003 that the layer-1 merge mode is not available, step 4004 may continue to determine the target merge mode from the layer-2 merge mode.

[00442] For the image prediction method shown in FIG. 15, the level-1 fusion mode and level-2 fusion mode may include all optional fusion modes of the current image block, and for the current image block, the final target fusion mode must be determined from the level-1 fusion mode and the level-2 fusion mode.

[00443] Optionally, the priority of the layer-1 merge mode is higher than the priority of the layer-2 merge mode.

[00444] That the priority of the layer-1 merge mode is higher than the priority of the layer-2 merge mode means that in the process of determining the target merge mode of the current image block, the target merge mode is preferably determined from the layer-1 merge mode. If there is no merge mode available in the level-1 merge mode, the target merge mode is then determined from the level-2 merge mode.

[00445] 4004: Determine whether conditions 1 to 5 are met.

[00446] Condition 1 - Condition 5 are as follows:

[00447] Condition 1: TPM mode is allowed.

[00448] Condition 2: The type of fragment or group of fragments in which the current image block is located is B.

[00449] Condition 3: The maximum number of possible TPM modes supported by the tile or group of tiles in which the current image block is located is determined to be greater than or equal to 2.

[00450] Condition 4: The size of the current image block meets the specified condition.

[00451] Condition 5: Skip mode is not used to predict the current image block.

[00452] Condition 1 may be specifically represented by sps_triangle_enabled_flag=1, condition 2 may be specifically represented by slice_type == B, and condition 3 may be specifically represented by MaxNumTriangleMergeCand ≥ 2. MaxNumTriangleMergeCand indicates the maximum number of possible TPM modes supported by a fragment or group of fragments, in which the current image block is located.

[00453] When it is determined at step 4004 that any of condition 1 to condition 5 is not satisfied, the CIIP mode can be directly determined as the target merge mode. In other words, step 4005 is carried out. When it is determined in step 4004 that the five conditions, namely condition 1 to condition 5, are satisfied, the target merge mode must be further determined based on the corresponding CIIP mode information. In other words, step 4006 is carried out.

[00454] 4005: Define CIIP mode as the target merge mode when CIIP mode is enabled.

[00455] In other words, at step 4005, when the CIIP mode is enabled and any of condition 1 to condition 5 is not satisfied, the CIIP mode is determined as the target merge mode.

[00456] Optionally, when any of condition 1 to condition 5 is not satisfied, the value of the available status information, which indicates the available status of the CIIP mode, is set to the first value. When the value of the available status information, which indicates the available status of the CIIP mode, is the first value, the CIIP mode is used to perform image prediction for the current image block.

[00457] It should be understood that the value of the available status information, which indicates the available status of the CIIP mode, is set to the first value herein, is equivalent to CIIP being defined as the target merge mode.

[00458] Optionally, the available status information indicating the available status of the CIIP mode is ciip_flag.

[00459] That the available status information value, which indicates the available status of the CIIP mode, is set to the first value, in particular, it may be that ciip_flag is set to 1.

[00460] In addition, when the value of the available status information that indicates the available status of the CIIP mode is set to the second value, it may indicate that the CIIP mode is not used to perform image prediction for the current image block. For example, when the available status information that indicates the available status of the CIIP mode is ciip_flag and ciip_flag=0, the CIIP mode is not used to perform image prediction for the current image block.

[00461] 4006: Determine the target merge mode based on a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode.

[00462] Available status information, which indicates the available status of the CIIP mode, is used to indicate whether the CIIP mode is used when the current picture block is predicted.

[00463] In other words, at step 4006, when all conditions from condition 1 to condition 5 are satisfied, the target merge mode must further be determined based on a high-level syntax element corresponding to the CIIP mode and/or available status information indicating the available status CIIP mode.

[00464] Available status information that indicates the available status of the CIIP mode may be a ciip_flag value. When ciip_flag is 0, CIIP mode is not available for the current image block. When ciip_flag is 1, CIIP mode is enabled for the current image block.

[00465] Optionally, the target merge mode is determined based on a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode includes: when the high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode, indicates that the CIIP mode is prohibited from being used, the TPM mode is determined to be the target merge mode.

[00466] That the high-level syntax element corresponding to the CIIP mode and/or the available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used includes cases 1 and 3:

[00467] Case 1: The high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is prohibited from being used, and the available status information, which indicates the available status of the CIIP mode, indicates that the CIIP mode is not available.

[00468] Case 2: The high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is enabled for use, and the available status information, which indicates the available status of the CIIP mode, indicates that the CIIP mode is not available.

[00469] Case 3: The available status information, which indicates the available status of the CIIP mode, indicates that the CIIP mode is not available.

[00470] It should be understood that when the high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is enabled for use, and the available status information, which indicates the available status of the CIIP mode, indicates that the CIIP mode is available, the CIIP mode is determined as the target merge mode.

[00471] Optionally, when a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, that the TPM mode is determined to be the target merge mode included in myself:

[00472] When the high-level syntax element corresponding to the CIIP mode and/or the available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, the value of the available status information that indicates the available status of the TPM mode , is set to the first value, where, when the value of the available status information that indicates the available status of the TPM mode is the first value, the TPM mode is used to perform image prediction for the current image block.

[00473] It should be understood that the value of the available status information, which indicates the available status of the TPM mode, is set to the first value herein is equivalent to the TPM being defined as the target merge mode.

[00474] Optionally, the available status information indicating the available status of the TPM mode is MergeTriangleFlag.

[00475] That the value of the available status information, which indicates the available status of the TPM mode, is set to the first value, in particular, it may be that the MergeTriangleFlag is set to 1.

[00476] In the present application, the target merge mode may be determined based on a high-level syntax element of the CIIP mode and/or available status information that indicates the available status of the CIIP mode only when five specified conditions are met. Compared to the conventional solution, more conditions must be satisfied before the target merge mode is further determined based on the high-level syntax element and information about the available CIIP mode status. Otherwise, CIIP mode can be directly defined as the target merge mode. This can reduce the number of redundant processes in the process of determining the target merge mode.

[00477] On the other hand, when the layer-1 merge mode is not available, based on some specified conditions, it can be determined whether the CIIP mode should be selected as the final merge mode, and the CIIP mode can be directly determined as the target merge mode under the condition that none of the specified conditions are met. This reduces the redundancy that occurs in the merge target determination process.

[00478] 4007: Predict the current image block based on the target merge mode.

[00479] Optionally, before the target merge mode is determined based on a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode, the method shown in FIG. 15, also includes:

determining that at least one of the following conditions is met:

[00480] The size of the current image block meets the specified condition; And

Skip mode is not used to predict the current image block.

[00481] Optionally, the size of the current image block meets a given condition includes: the current image block meets the following three conditions:

(cdWidth * cbHeight) ≥ 64;

cbWidth < 128; and

cbHeight < 128.

[00482] cdWidth is the width of the current image block, and cbHeight is the height of the current image block.

[00483] The foregoing describes in detail the image prediction method in embodiments of the present application with reference to the accompanying drawings. Below, an image prediction device in an embodiment of the present application will be described with reference to FIG. 16. It should be understood that the image prediction device shown in FIG. 16 may implement the steps of an image prediction method in embodiments of the present application. To avoid unnecessary repetition, the following will appropriately omit repetitive descriptions when describing the image prediction device in this embodiment of the present application.

[00484] FIG. 16 is a schematic block diagram of an image prediction apparatus according to an embodiment of the present application.

[00485] The image prediction device 5000 shown in FIG. 16 includes a determination block 5001 and a prediction block 5002.

[00486] The image prediction device 5000 shown in FIG. 16 is configured to implement the image prediction method in embodiments of the present application. Specifically, the determination unit 5001 in the image prediction device 5000 may be configured to carry out a target fusion mode determination process in the image prediction method shown in FIG. 13 - fig. 15. The prediction unit 5002 in the image prediction device 5000 is configured to carry out a process of performing image prediction for the current image block based on the target fusion mode in the image prediction method shown in FIG. 13 - fig. 15.

[00487] FIG. 17 is a schematic diagram of a hardware structure of an image prediction apparatus according to an embodiment of the present application. The image prediction device 6000 shown in FIG. 17 (device 6000 may in particular be a computer device) includes memory 6001, processor 6002, communications interface 6003, and bus 6004. Communication connections between memory 6001, processor 6002, and communications interface 6003 are implemented via bus 6004.

[00488] Memory 6001 may be read only memory (ROM), static memory, dynamic memory, or random access memory (RAM). The 6001 memory can store a program. When the program stored in the memory 6001 is executed by the processor 6002, the processor 6002 is configured to implement the steps of the image prediction method in the embodiments of the present application.

[00489] Processor 6002 may use a central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits, and configured to execute an associated program to implement the image detection method in the method embodiments of the present application.

[00490] The processor 6002 may be an integrated circuit and may have signal processing capabilities. In implementation, the steps of the image prediction method of the present application may be performed using a hardware integrated logic circuit or an instruction in the form of software in the processor 6002.

[00491] Alternatively, processor 6002 may be a general purpose processor, digital signal processing (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate logic, or transistor, or discrete hardware component. Processor 602 may implement or execute the methods, steps, and logic flowcharts that are disclosed in embodiments of this application. The general purpose processor may be a microprocessor, or the processor may be any conventional processor or the like. The steps of the method disclosed with reference to embodiments of the present application may be directly performed and completed by a hardware decoding processor, or may be performed and completed by a combination of hardware and software modules in a decoding processor. The software module may be located on a storage medium well known in the art, such as random access memory, flash memory, read only memory, programmable read only memory, electrically erasable programmable memory, or a register. The storage medium is located in the memory 6001. The processor 6002 reads the information in the memory 6001 and performs, in combination with the hardware of the processor 6002, functions to be performed by units included in the image prediction apparatus, or performs the image prediction method in the method embodiments of the present application.

[00492] Communications interface 6003 uses a transceiver device, such as, but not limited to, a transceiver, to implement communication between device 6000 and another device or communications network. For example, information about the neural network to be built and training data needed in the process of building the neural network can be obtained through the communication interface 6003.

[00493] Bus 6004 may include a path for transferring information between components (e.g., memory 6001, processor 6002, and communications interface 6003) of device 6000.

[00494] The determination unit 5001 and the prediction unit 5002 in the image prediction device 5000 are equivalent to the processor 6002 in the image prediction device 6000.

[00495] FIG. 18 is a schematic diagram of a hardware structure of an image encoding/decoding apparatus according to an embodiment of the present application. The image encoding/decoding apparatus 7000 shown in FIG. 18 (device 7000 may in particular be a computer device) includes memory 7001, processor 7002, communications interface 7003, and bus 7004. Communication connections between memory 7001, processor 7002, and communications interface 7003 are implemented via bus 7004.

[00496] The above limitations and explanations of the modules in the image prediction device 6000 also apply to the image encoding/decoding device 7000, and the details are again not described here.

[00497] The memory 7001 may be configured to store a program. The processor 7002 is configured to execute the program stored in the memory 7001. When the program stored in the memory 7001 is executed, the processor 7002 is configured to execute the steps of the image prediction method in the embodiments of the present application.

[00498] In addition, when encoding a video image, the image encoding/decoding apparatus 7000 may receive a video image through a communication interface, and then encode the received video image to obtain encoded video data. The encoded video data may be transmitted to the video decoding device via communication interface 7003.

[00499] When decoding a video image, the image encoding/decoding apparatus 7000 can obtain a video image through a communication interface, and then decode the received video image to obtain a video image to be displayed.

[00500] One skilled in the art may be aware that, in combination with the examples described in the embodiments disclosed herein, the blocks and steps of the algorithm may be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether functions are performed by hardware or software depends on the specific application and the design constraints of the technical solutions. One skilled in the art may employ various methods to implement the described functions for each particular application, but such implementation should not be considered beyond the scope of this application.

[00501] It may be clear to one skilled in the art that, for the purpose of convenient and concise description, the detailed working process of the above-mentioned system, apparatus and blocks refers to the corresponding process in the above-mentioned method embodiments, and the details are not given here again.

[00502] It should be understood that in several embodiments presented herein, the disclosed system, apparatus, and method may be implemented in another manner. For example, the described embodiment of the device is merely an example. For example, blocking is simply a division of logical functions and may be a different division in the actual implementation. For example, multiple blocks or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual connections or direct connections or communication connections displayed or discussed may be implemented through some interfaces. Indirect connections or communication connections between devices or units may be implemented in electronic form, mechanical form, or other form.

[00503] Blocks depicted as individual pieces may or may not be physically separate, and pieces depicted as blocks may or may not be physical units, may be located in one location, or may be distributed across multiple network blocks. Some or all of the blocks may be selected based on the actual requirement to achieve the objectives of the solutions of the embodiments.

[00504] In addition, the functional blocks in embodiments of the present application may be integrated into a single processing block, or each of the blocks may exist separately physically, or two or more blocks are integrated into a single block.

[00505] When functions are implemented in the form of a software function block and sold or used as an independent product, the functions may be stored on a computer-readable storage medium. Based on this understanding, the essence of the technical solutions in this application, or a certain part making a contribution to the prior art, or some technical solutions can be implemented in the form of a software product. The software product is stored in a storage medium and includes a number of instructions for directing a computing device (which may be a personal computer, a server, or a network device) to perform all or some of the steps of the methods described in the embodiments. of this application. The above storage medium includes: any medium that can store program code, such as a USB flash drive, removable hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disc.

[00506] The foregoing description represents only specific implementations of the present application, but is not intended to limit the scope of protection of the present application. Any change or substitution readily detectable by one skilled in the art within the technical scope disclosed in this application shall/shall fall within the scope of protection of this application. Therefore, the scope of protection of the present application should correspond to the scope of protection of the claims.

Claims (60)

1. A method for predicting an image, comprising: determining whether a merge mode is used for the current image block; when the merge mode is used for the current image block, determining whether the level-1 merge mode is available; when the level-1 merge mode is not available, and the high-level syntax element corresponding to the first merge mode in the level-2 merge mode indicates that the first merge mode is prohibited from being used, defining the second merge mode in the level-2 merge mode as the target merge mode, applicable to the current image block, wherein the layer-2 fusion mode comprises a first fusion mode and a second fusion mode, wherein the first fusion mode is a triangular division TPM mode, and the second fusion mode is a combined intra- and inter-frame prediction CIIP mode; And Predicting the current image block based on the target fusion mode. 2. The method according to claim 1, additionally containing: when the level-1 merge mode is not available and the high-level syntax element corresponding to the first merge mode in the level-2 merge mode indicates that the first merge mode is enabled for use, determining the target merge mode based on the high-level syntax element corresponding to the second merge mode in a layer-2 merge mode, and/or information about the available status of the second merge mode, wherein the information about the available status of the second merge mode is used to indicate whether the second merge mode is used when the current image block is predicted. 3. The method of claim 2, wherein determining the target merge mode based on a high-level syntax element corresponding to the second merge mode and/or information about the available status of the second merge mode comprises: when a high-level syntax element corresponding to the second merge mode and/or information about the available status of the second merge mode indicates that the second merge mode is prohibited from being used, determining the first merge mode as the target merge mode. 4. The method of claim 2 or 3, wherein before determining the target merge mode based on a high-level syntax element corresponding to the second merge mode and/or information about the available status of the second merge mode, the method further comprises: determining that the current image block satisfies at least one of the following conditions: the size of the current image block meets the specified condition; Skip mode is not used to predict the current image block. 5. The method according to claim 4, in which the size of the current image block meets a given condition, contains: The current image block satisfies the following three conditions: (cdWidth * cbHeight) ≥ 64; cbWidth < 128; And cbHeight < 128, while cdWidth is the width of the current image block, and cbHeight is the height of the current image block. 6. Method according to any one of paragraphs. 1-5, wherein the high-level syntax element is a syntax element at at least one of a sequence level, an image level, a fragment level, and a fragment group level. 7. Method according to any one of paragraphs. 1-6, in which the layer-1 fusion mode contains a normal fusion mode, a motion vector difference MMVD fusion mode, and a sub-block fusion mode. 8. The method according to claim 1, additionally containing: when the layer-1 merge mode is not available, determining the target merge mode from the layer-2 merge mode, wherein the layer-2 merge mode contains a TPM mode and a CIIP mode; And When CIIP mode is enabled and one of the following conditions is not met, specify CIIP mode as the target merge mode: Condition D: TPM mode enabled; condition E: skip mode is not used to predict the current image block; condition F: (cbWidth * cbHeight) ≥ 64; condition G: cbWidth < 128; And condition H: cbHeight < 128, while cbWidth of the current image block is the width of the current image block, and cbHeight is the height of the current image block. 9. Method according to any one of paragraphs. 1-8, in which the prediction method is applied to the encoder side to encode the current image block. 10. Method according to any one of paragraphs. 1-8, in which the prediction method is applied to the decoder side to decode the current image block. 11. A method for predicting an image, comprising: determining whether a merge mode is used for the current image block; when the merge mode is used for the current image block, determining whether the layer-1 merge mode is available; when the layer-1 merge mode is not available, determining the target merge mode from the layer-2 merge mode, wherein the layer-2 merge mode contains a TPM mode and a CIIP mode; And defining CIIP mode as the target merge mode when CIIP mode is enabled and one of the following conditions is not true: condition 1: TPM mode is allowed; condition 2: the type of the fragment or group of fragments in which the current image block is located is B; condition 4: the size of the current image block meets the specified condition; And Condition 5: Skip mode is not used to predict the current image block. 12. The method of claim 11, wherein determining the target merge mode from the layer-2 merge mode comprises: when any of condition 1, condition 2, condition 4 and condition 5 is not satisfied, setting the available status information value that indicates the available status of the CIIP mode to the first value, wherein when the available status information value that indicates the available status of the mode CIIP is the first value, CIIP mode is used to perform image prediction for the current image block. 13. The method of claim 11, wherein determining the target merge mode from the layer-2 merge mode comprises: when all the conditions from condition 1, condition 2, condition 4 and condition 5 are satisfied, determining the target merge mode based on a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode, wherein the information available status, which indicates the available status of the CIIP mode, is used to indicate whether the CIIP mode is used when the current picture block is predicted. 14. The method of claim 13, wherein determining the target merge mode based on a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode, comprises: when a high-level syntax element corresponding to the CIIP mode and/or available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, determining the TPM mode as the target merge mode. 15. The method of claim 14, wherein when the high-level syntax element corresponding to the CIIP mode and/or the available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, determining the TPM mode in contains as a merge target mode: when a high-level syntax element corresponding to the CIIP mode and/or the available status information that indicates the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, setting the available status information value that indicates the available status of the TPM mode to the first value, wherein when the available status information value that indicates the available status of the TPM mode is the first value, the TPM mode is used to perform image prediction for the current image block. 16. Method according to any one of paragraphs. 11-15, in which the prediction method is applied to the encoder side to encode the current image block. 17. Method according to any one of paragraphs. 11-15, in which the prediction method is applied to the decoder side to decode the current image block. 18. An image prediction device, wherein the image prediction device comprises a module configured to implement the method according to any one of claims. 1-10 or pp. 11-17. 19. An image prediction device comprising: a memory configured to store a program; And a processor configured to execute a program stored in the memory, wherein when the program stored in the memory is executed by the processor, the processor implements the method according to any one of claims. 1-10 or pp. 11-17. 20. An encoding device comprising an image prediction device according to claim 18 or 19. 21. A decoding device comprising an image prediction device according to claim 18 or 19. 22. A video encoding system, wherein the electronic device contains an encoding device according to claim 20 and/or a decoding device according to claim 21. 23. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program that can be executed by the processor, and when the computer program is executed by the processor, the processor implements the method of any one of claims. 1-10 or pp. 11-17. 24. An encoder containing a processing circuit for performing the method according to any one of claims. 1-10 or pp. 11-17.

Images