TW202209893A - Inter-frame prediction method, coder, decoder and computer storage medium characterized by reducing the prediction errors and promoting the coding performance to increase the coding/decoding efficiency - Google Patents

Abstract

Disclosed are an inter-frame prediction method, a coder, a decoder and a computer storage medium. The decoder parses a bit stream and obtains the prediction mode parameters of the current block. When the prediction mode parameters indicate to use the inter-frame prediction mode to determine the inter-frame prediction value of the current block, the first motion vector of the sub-block of the current block is determined. When the current block includes several sub-blocks, based on the first motion vector, the first prediction value of the sub-block and the first motion vector deviation between the each pixel position in the sub-block and the sub-block are determined. Based on the first motion vector deviation value, the center position corresponding to each pixel position and the second motion vector deviation are determined. Based on the first prediction value, the center position and the second motion vector deviation to perform the second prediction or the PROF process, the second prediction value of the sub-block is determined and the second prediction value is determined to be the inter-frame prediction value of the sub-block.

Description

Inter-frame prediction method, encoder, decoder and computer storage medium

The present application relates to the technical field of image coding and decoding, and in particular, to an inter-frame prediction method, an encoder, a decoder, and a computer storage medium.

In the field of video coding and decoding, in order to take into account the performance and cost, under normal circumstances, in the Versatile Video Coding (VVC) and Digital Audio Video coding Standard Workgroup of China (AVS) The affine prediction of is implemented based on sub-blocks. At present, on the one hand, a prediction refinement with optical flow (PROF) is proposed to use the optical flow principle to modify the sub-block-based affine prediction; The quadratic prediction achieves a more accurate prediction value, thus achieving an improvement over affine prediction. Specifically, PROF relies on the horizontal and vertical gradients of the reference position to correct the affine prediction results, while the secondary prediction uses a filter to predict a pixel position in a sub-block again.

However, when there is a large motion vector deviation in the center position of the filter that the secondary prediction or PROF needs to use, the accuracy of the prediction may be greatly reduced, that is, the existing PROF correction prediction method and secondary prediction In fact, the method is not rigorous. When improving affine prediction, it cannot be well applied to all scenarios, and the coding performance needs to be improved.

The present application proposes an inter-frame prediction method, an encoder, a decoder, and a computer storage medium, which can reduce prediction errors, greatly improve encoding performance, and thus improve encoding and decoding efficiency.

The technical solution of the present application is realized as follows:

In a first aspect, an embodiment of the present application provides an inter-frame prediction method, which is applied to a decoder, and the method includes:

Parse the bit stream to obtain the prediction mode parameters of the current block;

When the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, a first motion vector of a subblock of the current block is determined; wherein the current block includes a plurality of subblocks;

determining a first predicted value for the subblock based on the first motion vector, and a first motion vector offset between each pixel position in the subblock and the subblock;

Determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation;

Based on the first predicted value, secondary prediction or PROF processing is performed according to the center position and the deviation of the second motion vector, the second predicted value of the sub-block is determined, and the second predicted value is determined as the Inter-predicted value of the sub-block.

In a second aspect, an embodiment of the present application provides an inter-frame prediction method, which is applied to an encoder, and the method includes:

determine the prediction mode parameter of the current block;

When the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, a first motion vector of a subblock of the current block is determined; wherein the current block includes a plurality of subblocks;

determining a first predicted value for the subblock based on the first motion vector, and a first motion vector offset between each pixel position in the subblock and the subblock;

Determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation;

Based on the first predicted value, secondary prediction or PROF processing is performed according to the center position and the deviation of the second motion vector, the second predicted value of the sub-block is determined, and the second predicted value is determined as the Inter-predicted value of the sub-block.

In a third aspect, an embodiment of the present application provides a decoder, where the decoder includes a parsing unit and a first determining unit;

The parsing unit is used to parse the bit stream to obtain the prediction mode parameter of the current block;

The first determining unit is configured to determine a first motion vector of a sub-block of the current block when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode; wherein the the current block includes a plurality of sub-blocks; and determining a first predictor for the sub-block based on the first motion vector, and a first motion vector between each pixel position in the sub-block and the sub-block and determining a center position corresponding to each pixel position and a second motion vector deviation according to the first motion vector deviation; and based on the first predicted value, according to the center position and the second motion vector The deviation is subjected to secondary prediction or PROF processing, the second prediction value of the sub-block is determined, and the second prediction value is determined as the inter-frame prediction value of the sub-block.

In a fourth aspect, an embodiment of the present application provides a decoder. The decoder includes a first processor and a first memory storing an executable instruction of the first processor. When the instruction is executed, the When executed by the first processor, the above-mentioned inter-frame prediction method is implemented.

In a fifth aspect, an embodiment of the present application provides an encoder, where the encoder includes a second determining unit;

the second determining unit, configured to determine a prediction mode parameter of the current block; and when the prediction mode parameter indicates that an inter prediction value of the current block is determined using an inter prediction mode, determining a subblock of the current block The first motion vector of the a first motion vector deviation between sub-blocks; and determining a center position corresponding to each pixel position and a second motion vector deviation according to the first motion vector deviation; and based on the first predicted value, according to the Perform secondary prediction or PROF processing on the center position and the second motion vector deviation, determine the second prediction value of the sub-block, and determine the second prediction value as the inter-frame prediction value of the sub-block.

In a sixth aspect, an embodiment of the present application provides an encoder, the encoder includes a second processor, and a second memory storing an executable instruction of the second processor, when the instruction is executed, the When executed by the second processor, the inter-frame prediction method as described above is implemented.

In a seventh aspect, an embodiment of the present application provides a computer storage medium, the computer storage medium stores a computer program, and when the computer program is executed by the first processor and the second processor, the above-mentioned inter-frame prediction method is implemented .

An inter-frame prediction method, an encoder, a decoder, and a computer storage medium provided by the embodiments of the present application. The decoder parses the bit stream and obtains the prediction mode parameter of the current block; when the prediction mode parameter indicates that the inter-frame prediction mode is used to determine the current When the inter-frame prediction value of the block is used, the first motion vector of the sub-block of the current block is determined; wherein, the current block includes a plurality of sub-blocks; the first prediction value of the sub-block is determined based on the first motion vector, and each pixel in the sub-block The first motion vector deviation between the position and the sub-block; the center position and the second motion vector deviation corresponding to each pixel position are determined according to the first motion vector deviation; based on the first predicted value, according to the center position and the second motion vector deviation Secondary prediction or PROF processing is performed to determine the second prediction value of the sub-block, and the second prediction value is determined as the inter-frame prediction value of the sub-block. The encoder determines the prediction mode parameter of the current block; when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, the first motion vector of the subblock of the current block is determined; wherein, the current block includes a plurality of subblocks Determine the first predicted value of the sub-block based on the first motion vector, and the first motion vector deviation between each pixel position in the sub-block and the sub-block; Determine the center position corresponding to each pixel position according to the first motion vector deviation and the second motion vector deviation; based on the first predicted value, carry out secondary prediction or PROF processing according to the center position and the second motion vector deviation, determine the second predicted value of the sub-block, and determine the second predicted value as the frame of the sub-block inter-prediction value. That is to say, the inter-frame prediction method proposed in this application can, after the prediction based on the sub-block, for the pixel position with a large deviation of the first motion vector between the motion vector and the motion vector of the sub-block, it can be based on the first motion vector. The vector deviation re-determines the determination of the center position and the second motion vector deviation used for secondary prediction or PROF processing, so that the center position and the second motion vector deviation can be used on the basis of the first prediction value based on the sub-block. Based on the secondary prediction of the points, a second predicted value is obtained. It can be seen that the inter-frame prediction method proposed in this application can be well applied to all scenarios, can reduce the prediction error, greatly improve the coding performance, and thus improve the coding and decoding efficiency.

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the specific embodiments described herein are only used to explain the related application, but not to limit the application. In addition, it should be noted that, for the convenience of description, only the parts related to the relevant application are shown in the drawings.

Currently, common video coding standards are based on the use of block-based hybrid coding frameworks. Each frame in the video image is divided into square Largest Coding Units (LCUs) of the same size (such as 128×128, 64×64, etc.), and each LCU can also be divided into rectangles according to rules The coding unit (Coding Unit, CU); and the coding unit may be divided into smaller prediction units (Prediction Unit, PU). Specifically, the hybrid coding framework may include modules such as prediction, transform (Transform), quantization (Quantization), entropy coding (EntropyCoding), and loop filtering (In Loop Filter); wherein, the prediction module may include intraPrediction (intraPrediction) ) and inter-frame prediction (interPrediction), inter-frame prediction can include motion estimation (motion estimation) and motion compensation (motion compensation). Since there is a strong correlation between adjacent pixels in a frame of an image image, the use of intra-frame prediction in the image coding and decoding technology can eliminate the spatial redundancy between adjacent pixels; There is also a strong similarity between adjacent frames. In the image coding and decoding technology, the inter-frame prediction method is used to eliminate the temporal redundancy between adjacent frames, thereby improving the coding efficiency. The present application will be described in detail below in terms of inter prediction.

Inter-frame prediction is to use the already encoded/decoded frame to predict the part that needs to be encoded/decoded in the current frame. In the block-based encoding/decoding framework, the part that needs to be encoded/decoded is usually a coding unit or a prediction unit. Here, the coding unit or prediction unit that needs to be encoded/decoded is collectively referred to as the current block. Translation motion is a common and simple motion method in images, so translation prediction is also a traditional prediction method in image coding and decoding. The translational motion in the image can be understood as the change of a part of the content over time, moving from a certain position on one frame to a certain position on another frame. A simple unidirectional prediction of translation can be represented by a motion vector (MV) between a certain frame and the current frame. A certain frame mentioned here is a reference frame of the current frame. The current block can find a reference block on the reference frame with the same size as the current block through the motion information including the reference frame and motion vector, and use this reference block as the current block. The predicted block for the block. In an ideal translation motion, the content of the current block does not change from frame to frame, such as deformation, rotation, and brightness and color. However, the content in the image does not always conform to such an ideal situation. Bidirectional prediction can solve the above problems to a certain extent. The usual bidirectional prediction refers to bidirectional translation prediction. Bidirectional prediction is to use the motion information of two reference frames and motion vectors to find two reference blocks with the same size as the current block from two reference frames (the two reference frames may be the same reference frame), and use these two reference frames. The block generates a predicted block for the current block. Generation methods include averaging, weighted averaging, and some other calculations.

In this application, prediction can be considered as a part of motion compensation, and some documents will call the prediction in this application motion compensation, and as mentioned in this application, affine prediction, some documents will call it affine motion compensation.

Rotation, enlargement, reduction, distortion, deformation, etc. are also common changes in images. However, ordinary translation prediction cannot handle such changes well, so affine prediction models are used in image encoding and decoding. Such as affine in VVC and AVS, where the affine prediction models of VVC and AVS3 are similar. In changes such as rotation, enlargement, reduction, distortion, deformation, etc., it can be considered that not all points in the current block use the same MV, so the MV of each point needs to be exported. The affine prediction model uses a small number of parameters to export the MV of each point by calculating. Both VVC and AVS3 affine prediction models use 2-control point (4-parameter) and 3-control point (6-parameter) models. The 2 control point is the 2 control points in the upper left corner and the upper right corner of the current block, and the 3 control point is the 3 control points in the upper left corner, the upper right corner and the lower left corner of the current block. Exemplarily, FIG. 1 is a schematic diagram 1 of an affine model, and FIG. 2 is a schematic diagram 2 of an affine model, as shown in FIGS. 1 and 2 . Because each MV includes an x-component and a y-component, 2 control points have 4 parameters, and 3 control points have 6 parameters.

According to the affine prediction model, an MV can be exported for each pixel position, and each pixel position can be found in the reference frame. The value of the pixel position. The interpolation methods used in the current image coding and decoding standards are usually implemented by Finite Impulse Response (FIR) filters, and the complexity (cost) is very high to implement in this way. For example, in AVS3, an 8-tap interpolation filter is used for the luminance component, and the sub-pixel accuracy of normal mode is 1/4 pixel, and the sub-pixel accuracy of affine mode is 1/16 pixel. For each sub-pixel point that meets the 1/16 pixel precision, it needs to be obtained by using 8 integer pixels in the horizontal direction and 8 integer pixels in the vertical direction, that is, 64 integer pixels. Figure 3 is a schematic diagram of pixel interpolation. As shown in Figure 3, the circular pixel is the desired sub-pixel point, the dark square pixel is the position of the integer pixel corresponding to the sub-pixel, and the vector between them is The motion vector of the sub-pixels, the light-colored square pixels are the pixels that need to be used for the interpolation of the circular sub-pixel positions. To get the value of the sub-pixel position, the pixel values of these 8x8 light-colored square pixel areas need to be interpolated. , which also contains dark pixel locations.

In traditional translational prediction, the MV of each pixel location of the current block is the same. If the concept of sub-block is further introduced, the size of the sub-block is 4x4, 8x8, etc. FIG. 4 is a schematic diagram 1 of sub-block interpolation. The pixel area that needs to be used for 4×4 block interpolation is shown in FIG. 4 . FIG. 5 is a schematic diagram 2 of sub-block interpolation. The pixel area that needs to be used for 8×8 block interpolation is shown in FIG. 5 .

If the MV of each pixel location in a sub-block is the same. The pixel positions in a sub-block can then be interpolated together, sharing the bandwidth, using filters of the same phase, and sharing the intermediate values of the interpolation process. But if one MV is used per pixel, the bandwidth will increase, and filters of different phases may be used and intermediate values that cannot be shared in the interpolation process.

Point-based affine prediction is expensive. Therefore, in order to balance performance and cost, affine prediction in VVC and AVS3 is implemented based on sub-blocks. The sub-block size in AVS3 is 4x4 and 8x8, and the 4x4 sub-block size is used in VVC. Each sub-block has one MV, and the pixel positions within the sub-block share the same MV. In this way, all pixel positions inside the sub-block are uniformly interpolated. Through the above method, the motion compensation complexity of sub-block-based affine prediction is similar to that of other sub-block-based prediction methods.

It can be seen that in the sub-block-based affine prediction method, the pixel positions inside the sub-block share the same MV, and the method for determining this shared MV is to take the MV of the center of the current sub-block. For 4x4, 8x8 and other sub-blocks with at least one even number of pixels in the horizontal and vertical directions, the center of the sub-block actually falls on a non-integer pixel position. In the current standard, the position of an integer pixel is taken. For example, for a 4x4 sub-block, the position of the pixel from the upper left corner position is (2, 2). For an 8x8 subblock, take the pixel position (4, 4) from the upper left corner.

The affine prediction model can export the MV for each pixel location based on the control points (2 control points or 3 control points) used by the current block. In sub-block-based affine prediction, the MV of this position is calculated according to the pixel position in the previous segment as the MV of the sub-block. Figure 6 is a schematic diagram of the motion vector of each sub-block. As shown in Figure 6, in order to derive the motion vector of each sub-block, the motion vector sampled by the center of each sub-block is as shown in the figure, rounded to 1/16 precision, and then motion compensation.

With the development of technology, a method called prediction improvement technique PROF using optical flow was proposed. This technique can improve the prediction value of block-based affine prediction without increasing bandwidth. After the sub-block-based affine prediction is completed, the gradients in the horizontal and vertical directions are calculated for each pixel for which the sub-block-based affine prediction has been completed. In VVC, PROF uses a 3-tap filter [-1, 0, 1] when calculating the gradient, and its calculation method is the same as that of Bi-directional Optical flow (BDOF). Then, for each pixel position, calculate its motion vector deviation, which is the difference between the motion vector of the current pixel position and the MV used for the entire sub-block. These motion vector deviations can be calculated according to the formula of the affine prediction model. Due to the characteristics of the formula, the motion vector deviations at the same position of some sub-blocks are the same. For these sub-blocks, only a set of motion vector deviations need to be calculated, and other sub-blocks can directly reuse these values. For each pixel position, use the horizontal pixel vertical gradient of the point and the motion vector deviation (including the deviation in the horizontal direction and the deviation in the vertical direction) to calculate the corrected value of the predicted value of the pixel position, and then use the original predicted value, that is, based on The predicted value of the affine prediction of the sub-block is added to the modified value of the predicted value to obtain the modified predicted value.

When calculating the gradient in the horizontal and vertical directions, the [-1, 0, 1] filter is used, that is, for the current pixel position, the horizontal direction will use the prediction of the pixel position with a distance of 1 on the left and a pixel position with a distance of 1 on the right. value, the vertical direction will use the predicted value to the pixel position with a distance of 1 above and a pixel position with a distance of 1 below. If the current pixel position is the boundary position of the current block, some of the above-mentioned pixel positions will exceed the boundary position of the current block by a distance of one pixel. The predicted value of the boundary of the current block is used to fill the position of one pixel distance outward to satisfy the gradient calculation, so that there is no need to additionally increase the predicted value of one pixel distance beyond the boundary of the current block. Since the gradient calculation only needs to use the predicted value of the sub-block-based affine prediction, no additional bandwidth needs to be added.

In the VVC standard text, the MV of each sub-block of the current block and the motion vector deviation of each pixel position in the sub-block are derived from the MV of the control point. Each sub-block in VVC uses the same pixel position as the sub-block MV, so it is only necessary to export the motion vector deviation of a group of sub-blocks, and other sub-blocks can reuse the sub-block. Further, in the VVC standard text, the description of the PROF process, the calculation of the motion vector deviation by PROF is included in the above process.

AVS3's affine prediction has the same basic principle as VVC when calculating the MV of a sub-block, but AVS3 has special processing for the upper-left sub-block A, upper-right sub-block B and lower-left sub-block C of the current block.

The following is the AVS3 standard text description of the export of affine motion unit subblock motion vector arrays:

If there are 3 motion vectors in the affine control point motion vector group, then the motion vector group can be expressed as mvsAffine(mv0, mv1, mv2); if there are 2 motion vectors in the affine control point motion vector group, then the motion vector group Can be represented as mvsAffine(mv0, mv1). Then, the affine motion unit subblock motion vector array can be exported according to the following steps:

1. Calculate the variables dHorX, dVerX, dHorY and dVerY:

dHorX=(mv1_x-mv0_x)<<(7-Log(width));

dHorY=(mv1_y-mv0_y)<<(7-Log(width));

If the motion vector group is mvsAffine(mv0, mv1, mv2), then:

dVerX=(mv2_x-mv0_x)<<(7-Log(height));

dVerY=(mv2_y-mv0_y)<<(7-Log(height));

If the motion vector group is mvsAffine(mv0, mv1), then:

dVerX=−dHorY;

dVerY=dHorX;

It should be noted that Figure 7 is a schematic diagram of the sample position. As shown in Figure 7, (xE, yE) is the position of the upper left sample of the luminance prediction block of the current prediction unit in the luminance sample matrix of the current image, and the width of the current prediction unit and height are width and height respectively, the width and height of each sub-block are subwidth and subheight respectively, the sub-block where the upper-left corner sample of the luminance prediction block of the current prediction unit is located is A, the sub-block where the upper-right sample is located is B, and the lower-left corner is located. The subblock where the sample is located is C.

2.1. If the prediction reference mode of the current prediction unit is 'Pred_List01' or AffineSubblockSizeFlag is equal to 1 (AffineSubblockSizeFlag is used to indicate the size of the subblock size), then subwidth and subheight are both equal to 8, (x, y) is a subblock of size 8x8 The coordinates of the upper left corner position, then the motion vector mvE (mvE_x, mvE_y) of each 8x8 luminance sub-block can be calculated:

If the subblock is A, both xPos and yPos are equal to 0;

If the subblock is B, then xPos is equal to width and yPos is equal to 0;

If the sub-block is C and there are 3 motion vectors in mvsAffine, then xPos is equal to 0 and yPos is equal to height;

Otherwise, xPos is equal to (x-xE)+4, and yPos is equal to (y-yE)+4;

Therefore, the motion vector mvE of the current 8x8 subblock is:

mvE_x=Clip3(-131072,131071,Rounding((mv0_x<<7)+dHorX×xPos+dVerX×yPos,7));

mvE_y=Clip3(-131072,131071,Rounding((mv0_y<<7)+dHorY×xPos+dVerY×yPos,7));

2.2. If the prediction reference mode of the current prediction unit is 'Pred_List0' or 'Pred_List1', and AffineSubblockSizeFlag is equal to 0, then both subwidth and subheight are equal to 4, (x, y) is the coordinate of the upper left corner of the subblock with size 4x4, Compute the motion vector mvE(mvE_x, mvE_y) for each 4x4 luma subblock:

If the subblock is A, both xPos and yPos are equal to 0;

If the subblock is B, then xPos is equal to width and yPos is equal to 0;

If the sub-block is C and there are 3 motion vectors in mvAffine, xPos is equal to 0, yPos, etc. height;

Otherwise, xPos is equal to (x-xE)+2, and yPos is equal to (y-yE)+2;

Therefore, the motion vector mvE of the current 4x4 subblock is:

mvE_x=Clip3(-131072,131071,Rounding((mv0_x<<7)+dHorX×xPos+dVerX×yPos,7));

mvE_y=Clip3(-131072,131071,Rounding((mv0_y<<7)+dHorY×xPos+dVerY×yPos,7)).

The following is the description of the AVS3 text for the export of affine prediction samples and the interpolation of luma and chroma samples:

If the position of the upper left sample of the luma prediction block of the current prediction unit is (xE, yE) in the luma sample matrix of the current image.

If the prediction reference mode of the current prediction unit is 'PRED_List0' and the value of AffineSubblockSizeFlag is 0, mv0E0 is the LO motion vector of the 4x4 unit with the MvArrayL0 motion vector set at (xE+x, yE+y) position. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<< 4) The sample value of +mv0E0_x, (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chrominance prediction sample matrix predMatrixL0, which is in the reference image queue 0 The reference index is the sample value at position (((xE+2x)<<4)+MvC_x, (yE+2y)<<4)+MvC_y))) in the 1/32-precision chroma sample matrix of RefIdxL0. Among them, x1=((xE+2x)>>3)<<3, y1=((yE+2y)>>3)<<3, mv1E0 is the 4x4 unit of MvArrayL0 motion vector set at (x1, y1) position The LO motion vector of the , mv4E0 is the LO motion vector of the 4x4 unit of the MvArrayL0 motion vector set at the (x1+4, y1+4) position.

MvC_x=(mv1E0_x+mv2E0_x+mv3E0_x+mv4E0_x+2)＞＞2

MvC_y=(mv1E0_y+mv2E0_y+mv3E0_y+mv4E0_y+2)＞＞2

If the prediction reference mode of the current prediction unit is 'PRED_List0' and the value of AffineSubblockSizeFlag is 1, mv0E0 is the LO motion vector of the 8x8 unit with the MvArrayL0 motion vector set at (xE+x, yE+y) position. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<< 4) The sample value of +mv0E0_x, (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chroma prediction sample matrix predMatrixL0 is the reference in the reference image queue 0 The sample value at position (((xE+2x)<<4)+MvC_x, (yE+2y)<<4)+MvC_y))) in the 1/32-precision chroma sample matrix with index RefIdxL0. Among them, MvC_x is equal to mv0E0_x, and MvC_y is equal to mv0E0.

If the prediction reference mode of the current prediction unit is 'PRED_List1' and the value of AffineSubblockSizeFlag is 0, mv0E1 is the L1 motion vector of the 4x4 unit with the MvArrayL1 motion vector set at (xE+x, yE+y) position. The value of the element predMatrixL1[x][y] in the luma prediction sample matrix predMatrixL1 is the position in the 1/16-precision luma sample matrix whose reference index is RefIdxL1 in the reference image queue 1 is (((xE+x)<< 4) The sample value of +mv0E1_x, (yE+y)<<4)+mv0E1_y))), the value of the element predMatrixL1[x][y] in the chroma prediction sample matrix predMatrixL1 is the reference in the reference image queue 1 The index is the sample value at position (((xE+2x)<<4)+MvC_x, (yE+2y)<<4)+MvC_y))) in the 1/32-precision chroma sample matrix of RefIdxL1. Among them, x1=((xE+2x)>>3)<<3, y1=((yE+2y)>>3)<<3, mv1E1 is the 4x4 unit of MvArrayL1 motion vector set at (x1, y1) position , mv2E1 is the L1 motion vector of the 4x4 unit of the MvArrayL1 motion vector set at the (x1+4, y1) position, mv3E1 is the MvArrayL1 motion vector set at the (x1, y1+4) position of the 4x4 unit L1 motion vector of the unit , mv4E1 is the L1 motion vector of the 4x4 unit of the MvArrayL1 motion vector set at the (x1+4, y1+4) position.

MvC_x=(mv1E1_x+mv2E1_x+mv3E1_x+mv4E1_x+2)>>2

MvC_y=(mv1E1_y+mv2E1_y+mv3E1_y+mv4E1_y+2)＞＞2

If the prediction reference mode of the current prediction unit is 'PRED_List1' and the value of AffineSubblockSizeFlag is 1, mv0E1 is the L1 motion vector of the 8x8 unit with the MvArrayL1 motion vector set at (xE+x, yE+y) position. The value of the element predMatrixL1[x][y] in the luma prediction sample matrix predMatrixL1 is the position in the 1/16-precision luma sample matrix whose reference index is RefIdxL1 in the reference image queue 1 is (((xE+x)<< 4) The sample value of +mv0E1_x, (yE+y)<<4)+mv0E1_y))), the value of the element predMatrixL1[x][y] in the chroma prediction sample matrix predMatrixL1 is the reference in the reference image queue 1 The index is the sample value at position (((xE+2x)<<4)+MvC_x, (yE+2y)<<4)+MvC_y))) in the 1/32-precision chroma sample matrix of RefIdxL1. where MvC_x is equal to mv0E1_x and MvC_y is equal to mv0E1.

If the prediction mode of the current prediction unit is 'PRED_List01', mv0E0 is the L0 motion vector of the 8x8 unit with the MvArrayL0 motion vector set at the (xE+x, yE+y) position, and mv0E1 is the MvArrayL1 motion vector set at the (x, y) position , an L1 motion vector of 8x8 cells. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<<4 )+mv0E0_x, (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chroma prediction sample matrix predMatrixL0 is the reference index in the reference image queue 0 It is the sample value whose position is (((xE+2x)<<4)+MvC0_x, (yE+2y)<<4)+MvC0_y))) in the 1/32 precision chroma sample matrix of RefIdxL0, and the luminance prediction sample matrix The value of the element predMatrixL1[x][y] in predMatrixL1 is the position in the 1/16-precision luminance sample matrix whose reference index is RefIdxL1 in the reference image queue 1 is ((((xE+x)<<4)+mv0E1_x , (yE+y)<<4)+mv0E1_y)))), the value of the element predMatrixL1[x][y] in the chroma prediction sample matrix predMatrixL1 is the reference index in the reference image queue 1 RefIdxL1 The sample value at position (((xE+2x)<<4)+MvC1_x,(yE+2y)<<4)+MvC1_y))) in the 1/32 precision chroma sample matrix of . Where MvC0_x is equal to mv0E0_x, MvC0_y is equal to mv0E0_y, MvC1_x is equal to mv0E1_x, and MvC1_y is equal to mv0E1_y.

Among them, the element values of each position in the luminance 1/16 precision sample matrix and the chrominance 1/32 precision sample matrix of the reference image pass through the interpolation method defined by the following affine luminance sample interpolation process and affine chrominance sample interpolation process get. Integer samples outside the reference image should be replaced by the nearest integer samples (edge or corner samples) within the image to the sample, i.e. the motion vector can point to samples outside the reference image.

Specifically, the affine luminance sample interpolation process is as follows:

A, B, C, D are adjacent integer pixel samples, dx and dy are the horizontal and vertical distances between sub-pixel samples a(dx, dy) and A around integer pixel sample A, dx is equal to fx&15, dy is equal to fy&15, where ( fx, fy) are the coordinates of the sub-pixel sample in the 1/16-precision luminance sample matrix. There are 255 sub-pixel samples ax, y (dx, dy) around the integer pixel Ax, y.

Specifically, the sample position ax, 0 (x=1~15) is obtained by filtering the 8 integer values closest to the interpolation point in the horizontal direction, and the predicted value is obtained as follows:

ax,0=Clip1((fL[x][0]×A−3,0+fL[x][1]×A−2,0+fL[x][2]×A−1,0+fL [x][3]×A0, 0+fL[x][4]×A1, 0+fL[x][5]×A2, 0+fL[x][6]×A3, 0+fL[x ][7]×A4, 0+32) >> 6).

Specifically, the sample positions a0, y (y=1~15) are obtained by filtering the 8 integer values closest to the interpolation point in the vertical direction, and the predicted values are obtained as follows:

a0,y=Clip1((fL[y][0]×A0,−3+fL[y][1]×A−2,0+fL[y][2]×A−1,0+fL[ y][3]×A0, 0+fL[y][4]×A1, 0+fL[y][5]×A2, 0+fL[y][6]×A3, 0+fL[y] [7]×A−4, 0+32) >> 6).

Specifically, the acquisition method of the predicted value of the sample position ax, y (x=1～15, y=1～15) is as follows:

ax,y=Clip1((fL[y][0]×a'x,y-3+fL[y][1]×a'x,y-2+fL[y][2]×a'x , y-1+fL[y][3]×a'x, y+fL[y][4]×a'x, y+1+fL[y][5]×a'x, y+2 +fL[y][6]×a'x,y+3+fL[y][7]×a'x,y+4+(1＜＜(19-BitDepth)))＞＞(20-BitDepth) )).

in:

a'x,y=(fL[x][0]×A−3,y+fL[x][1]×A−2,y+fL[x][2]×A−1,y+fL [x][3]×A0, y+fL[x][4]×A1, y+fL[x][5]×A2, y+fL[x][6]×A3, y+fL[x ][7]×A4,y+((1<<(BitDepth-8))>>1))>>(BitDepth-8).

The luminance interpolation filter coefficients are shown in Table 1:

Table 1 sub-pixel position Luminance interpolation filter coefficients fL[p][0] fL[p][1] fL[p][2] fL[p][3] fL[p][4] fL[p][5] fL[p][6] fL[p][7] 1 0 1 −3 63 4 −2 1 0 2 −1 2 −5 62 8 −3 1 0 3 −1 3 −8 60 13 −4 1 0 4 −1 4 −10 58 17 −5 1 0 5 −1 4 −11 52 26 −8 3 −1 6 −1 3 −9 47 31 −10 4 −1 7 −1 4 −11 45 34 −10 4 −1 8 −1 4 −11 40 40 −11 4 −1 9 −1 4 −10 34 45 −11 4 −1 10 −1 4 −10 31 47 −9 3 −1 11 −1 3 −8 26 52 −11 4 −1 12 0 1 −5 17 58 −10 4 −1 13 0 1 −4 13 60 −8 3 −1 14 0 1 −3 8 62 −5 2 −1 15 0 1 −2 4 63 −3 1 0

Specifically, the affine chroma sample interpolation process is as follows:

A, B, C, D are adjacent integer pixel samples, dx and dy are the horizontal and vertical distances between the sub-pixel samples a(dx, dy) and A around the integer pixel sample A, dx is equal to fx&31, dy is equal to fy&31, where (fx, fy) are the coordinates of the sub-pixel sample in the 1/32-precision chroma sample matrix. There are 1023 sub-pixel samples ax,y(dx,dy) around the whole pixel Ax,y.

Specifically, for the sub-pixel points where dx is equal to 0 or dy is equal to 0, it can be directly obtained by chrominance integer pixel interpolation. Calculate by pixel: if(dx==0){ ax,y(0,dy)=Clip3(0,(1<<BitDepth)-1,(fC[dy][0]×Ax,y-1+fC[dy][1]×Ax,y+fC [dy][2]×Ax,y+1+fC[dy][3]×Ax,y+2+32)>>6) } else if(dy==0){ ax,y(dx,0)=Clip3(0,(1<<BitDepth)-1,(fC[dx][0]×Ax-1,y+fC[dx][1]×Ax,y+fC [dx][2]×Ax+1, y+fC[dx][3]×Ax+2, y+32) >> 6) } else{ ax,y(dx,dy)=Clip3(0,(1<<BitDepth)-1,(C[dy][0]×a'x,y-1(dx,0)+C[dy][1 ]×a'x,y(dx,0)+C[dy][2]×a'x,y+1(dx,0)+C[dy][3]×a'x,y+2( dx, 0)+(1<<(19-BitDepth)))>>(20-BitDepth)) }

where a'x,y(dx,0) is the temporary value of the sub-pixel on the whole pixel row, defined as: a'x,y(dx,0)=(fC[dx][0]×Ax-1 , y+fC[dx][1]×Ax, y+fC[dx][2]×Ax+1, y+fC[dx][3]×Ax+2, y+((1<<(BitDepth- 8))>>1))>>(BitDepth-8).

The chroma interpolation filter coefficients are shown in Table 2:

Table 2 sub-pixel position Interpolation filter coefficients fC[p][0] fC[p][1] fC[p][2] fC[p][3] 1 −1 63 2 0 2 −2 62 4 0 3 -2 60 7 −1 4 −2 58 10 −2 5 −3 57 12 −2 6 −4 56 14 −2 7 −4 55 15 −2 8 −4 54 16 −2 9 −5 53 18 −2 10 −6 52 20 −2 11 −6 49 twenty four −3 12 −6 46 28 −4 13 −5 44 29 −4 14 −4 42 30 −4 15 −4 39 33 −4 16 −4 36 36 −4 17 −4 33 39 −4 18 −4 30 42 −4 19 −4 29 44 −5 20 −4 28 46 −6 twenty one −3 twenty four 49 −6 twenty two −2 20 52 −6 twenty three −2 18 53 −5 twenty four −2 16 54 −4 25 −2 15 55 −4 26 −2 14 56 −4 27 −2 12 57 −3 28 −2 10 58 −2 29 −1 7 60 −2 30 0 4 62 −2 31 0 2 63 −1

Common affine prediction methods can include the following steps:

Step 101: Determine the motion vector of the control point.

Step 102: Determine the motion vector of the sub-block according to the motion vector of the control point.

Step 103: Predict the sub-block according to the motion vector of the sub-block.

At present, when improving the predicted value of block-based affine prediction through PROF, the following steps can be specifically included:

Step 101: Determine the motion vector of the control point.

Step 102: Determine the motion vector of the sub-block according to the motion vector of the control point.

Step 103: Predict the sub-block according to the motion vector of the sub-block.

Step 104: Determine the motion vector deviation between each position in the sub-block and the sub-block according to the motion vector of the control point and the motion vector of the sub-block.

Step 105: Determine the motion vector of the sub-block according to the motion vector of the control point.

Step 106: Use the sub-block-based prediction value to export each position to calculate the gradients in the horizontal and vertical directions.

Step 107: Calculate the deviation value of the predicted value of each position according to the motion vector deviation of each position and the gradients in the horizontal and vertical directions by using the principle of optical flow.

Step 108: Add the deviation value of the prediction value to the prediction value based on the sub-block for each position to obtain the revised prediction value.

At present, when improving the predicted value of block-based affine prediction through PROF, the following steps can be specifically included:

Step 101: Determine the motion vector of the control point.

Step 109: Determine the motion vector of the sub-block and the deviation between each position in the sub-block and the motion vector of the sub-block according to the motion vector of the control point.

Step 103: Predict the sub-block according to the motion vector of the sub-block.

Step 106: Use the sub-block-based prediction value to export each position to calculate the gradients in the horizontal and vertical directions.

Step 107: Calculate the deviation value of the predicted value of each position according to the motion vector deviation of each position and the gradients in the horizontal and vertical directions by using the principle of optical flow.

Step 108: Add the deviation value of the prediction value to the prediction value based on the sub-block for each position to obtain the revised prediction value.

PROF can revise sub-block-based affine prediction using the principle of optical flow, which improves the compression performance. However, the application of PROF is based on the case where the motion vector of the pixel position within the sub-block has a very small deviation from the sub-block motion vector, that is, the deviation of the motion vector of the pixel position within the sub-block from the sub-block motion vector is very small. The optical flow calculation method using PROF is effective in the case of The gradient does not truly reflect the gradient in the horizontal and vertical directions between the reference position and the actual position. Therefore, when the motion vector of the pixel position in the sub-block has a large deviation from the motion vector of the sub-block, this method will Not particularly effective anymore.

At this time, the method of quadratic prediction is proposed to overcome the shortcomings of PROF. The decoder's method for secondary prediction may include the following steps:

Step 201: Predict the sub-block according to the motion vector of the sub-block to obtain a predicted value.

Step 202: Determine the motion vector deviation between each position in the sub-block and the sub-block.

Step 203: According to the motion vector deviation of each position, use a two-dimensional filter to filter the predicted value to obtain the predicted value of secondary prediction.

That is to say, the secondary prediction method can perform point-based secondary prediction on the basis of the sub-block-based prediction for the pixel positions where the motion vector deviates from the motion vector of the sub-block after the sub-block-based prediction, and finally The correction of the predicted value is completed, and a new predicted value is obtained, that is, the predicted value of the secondary prediction.

Specifically, point-based secondary prediction uses a two-dimensional filter. A two-dimensional filter is a filter composed of adjacent points forming a preset shape. Adjacent points constituting the preset shape can be 9 points. For a pixel location, the result of the filter processing is the predicted value of the secondary prediction for that location. Among them, the filter coefficient of the two-dimensional filter is determined by the motion vector deviation of each position, the input of the two-dimensional filter is the predicted value, and the output is the predicted value of the secondary prediction.

Further, in this application, if the affine mode is used for the current block in this application, the motion vector of the control point needs to be determined first. The decoder's method for secondary prediction may include the following steps:

Step 204: Determine the motion vector of the control point.

Step 205: Determine the motion vector of the sub-block according to the motion vector of the control point.

Step 201: Predict the sub-block according to the motion vector of the sub-block to obtain a predicted value.

Step 202: Determine the motion vector deviation between each position in the sub-block and the sub-block.

Step 203: Use a two-dimensional filter to filter the predicted value according to the motion vector deviation of each position to obtain the predicted value of secondary prediction.

It can be seen that, in the embodiments of the present application, after the motion vector of the control point is determined, the motion vector of the control point can be used to perform prediction processing on the sub-block, and the pixel position in the sub-block and the sub-block can be determined after the pixel position in the sub-block is determined. After the motion vector deviation of the sub-block, the pixel position where the motion vector is deviated from the motion vector of the sub-block is subjected to the point-based secondary prediction based on the sub-block-based prediction, and finally the correction of the predicted value is completed, and a new prediction is obtained. value, which is the predicted value of the secondary prediction.

Further, in the present application, the two steps of determining the motion vector of the sub-block and determining the deviation of each position in the sub-block from the motion vector of the sub-block may also be performed simultaneously. The decoder's method for secondary prediction may include the following steps:

Step 204: Determine the motion vector of the control point.

Step 206: Determine the motion vector of the sub-block according to the motion vector of the control point, and the deviation of each position in the sub-block from the motion vector of the sub-block.

Step 201: Predict the sub-block according to the motion vector of the sub-block to obtain a predicted value.

Step 203: Use a two-dimensional filter to filter the predicted value according to the motion vector deviation of each position to obtain the predicted value of secondary prediction.

It can be seen that in the embodiments of the present application, after the motion vector of the control point is determined, the motion vector of the sub-block and the deviation of each position in the sub-block from the motion vector of the sub-block can be determined at the same time, and then the motion vector of the sub-block can be determined. The pixel position where the vector is deviated from the motion vector of the sub-block, the point-based secondary prediction is performed on the basis of the sub-block-based prediction, and finally the correction of the prediction value is completed, and a new prediction value is obtained, that is, the prediction of the secondary prediction value.

It should be noted that, because the affine model can explicitly calculate the motion vector of each pixel position, or the deviation between each pixel position in the sub-block and the motion vector of the sub-block, the secondary prediction method can be used for simulating Of course, secondary prediction can also be applied to improve other sub-block-based predictions. That is to say, the sub-block-based prediction proposed in this application includes, but is not limited to, affine sub-block-based prediction.

Further, the secondary prediction method may be based on the AVS3 standard, and may also be applied to the VVC standard, which is not specifically limited in this application.

For the pixel of each pixel position in the sub-block, when performing secondary prediction or PROF processing, the center point of the preset filter is the same pixel position as the pixel position in the prediction block based on the prediction of the sub-block. If the motion vector deviation is 0, then the predicted value of the pixel position is the value of the same pixel position as the pixel position in the predicted block based on the prediction of the sub-block. In general, the above method can be applied to the condition that the motion vector deviation between the pixel position and the sub-block is small. However, for the case where the motion vector deviation between the pixel position and the sub-block is not very small, such as in the horizontal or vertical direction, when the deviation exceeds half a pixel or a certain proportion, the prediction effect obtained by secondary prediction or PROF processing may be will be greatly reduced.

Figure 8 is a schematic diagram of the center position. As shown in Figure 8, the square represents the pixel position of the prediction block based on the prediction of the sub-block, and the circle represents the actual position of the pixel position to be predicted in the prediction block based on the prediction of the sub-block, The square 1 represents the pixel position in the prediction block based on the prediction of the sub-block that is the same as the pixel position to be predicted, that is, the (1, 1) position of the sub-block. The square 1 is the preset reference position of point-based prediction, that is, the center position of the filter, and the motion vector deviation thereof is recorded as (dmv_x0, dmv_y0).

Figure 9 is a schematic diagram of the center position. As shown in Figure 9, if both dmv_x0 and dmv_y0 are greater than one-half of a pixel, then in this sub-block, there is a large distance between the square 1 and the position of the pixel to be predicted , if the square 1 is used as the center position of the filter to perform secondary prediction or PROF processing, there will be a large error in the final prediction result.

It can be seen that when there is a large motion vector deviation in the center position of the filter that the secondary prediction or PROF needs to use, it may cause the problem of degraded prediction performance, that is, the existing PROF correction prediction method and secondary prediction method In fact, it is not rigorous. When improving affine prediction, it cannot be well applied to all scenarios, and the coding performance needs to be improved.

In order to solve the defects in the prior art, in the embodiments of the present application, after the prediction based on the sub-block, for the pixel position with a large deviation of the first motion vector between the motion vector and the motion vector of the sub-block, the The center position and the second motion vector offset used for secondary prediction or PROF processing are re-determined based on the first motion vector offset, so that the center position and the second The motion vector bias performs point-based secondary prediction to obtain a second predicted value. It can be seen that the inter-frame prediction method proposed in this application can be well applied to all scenarios, can reduce the prediction error, greatly improve the coding performance, and thus improve the coding and decoding efficiency.

It should be understood that an embodiment of the present application provides an image encoding system, and FIG. 10 is a schematic block diagram of the composition of an image encoding system provided by an embodiment of the present application. As shown in FIG. 10 , the image encoding system 11 may include: a transformation unit 111 , Quantization unit 112, mode selection and encoding control logic unit 113, intra prediction unit 114, inter prediction unit 115 (including: motion compensation and motion estimation), inverse quantization unit 116, inverse transform unit 117, in-loop filtering unit 118, encoding The unit 119 and the decoded image temporary storage unit 110; for the input original image signal, an image reconstruction block can be obtained through the division of the coding tree unit (Coding Tree Unit, CTU), and the coding mode is determined through the mode selection and coding control logic unit 113. , and then transform the image reconstruction block through the transform unit 111 and the quantization unit 112 for the residual pixel information obtained after intra-frame or inter-frame prediction, including transforming the residual information from the pixel domain to the transform domain, and The obtained transform coefficients are quantized to further reduce the bit rate; the intra-frame prediction unit 114 is used to perform intra-frame prediction on the image reconstruction block; wherein, the intra-frame prediction unit 114 is used to determine the optimal frame of the image reconstruction block Intra-prediction mode (ie, target prediction mode); the inter-prediction unit 115 is used to perform inter-prediction encoding of the received image reconstruction block relative to one or more blocks in one or more reference frames to provide temporal prediction information ; wherein, motion estimation is a process of generating a motion vector, and the motion vector can estimate the motion of the image reconstruction block, and then, motion compensation performs motion compensation based on the motion vector determined by motion estimation; after determining the inter-frame prediction mode, the inter-frame The prediction unit 115 is also used to provide the selected inter-frame prediction data to the coding unit 119, and also to send the calculated motion vector data to the coding unit 119; in addition, the inverse quantization unit 116 and the inverse transform unit 117 are used for The reconstruction of the image reconstruction block reconstructs a residual block in the pixel domain, the reconstructed residual block removes blocking artifacts through the in-loop filtering unit 118, and then adds the reconstructed residual block to the decoded image A predictive block in the frame of the temporary storage unit 110 is used to generate a reconstructed image reconstruction block; the encoding unit 119 is used to encode various coding parameters and quantized transform coefficients. The decoded image temporary storage unit 110 is used for storing reconstructed image reconstruction blocks for prediction reference. As the image encoding proceeds, new reconstructed image reconstruction blocks are continuously generated, and these reconstructed image reconstruction blocks are stored in the decoded image temporary storage unit 110 .

An embodiment of the present application also provides an image decoding system. FIG. 11 is a schematic block diagram of the composition of an image decoding system provided by an embodiment of the present application. As shown in FIG. 11 , the image decoding system 12 may include: a decoding unit 121 , an inverse transform The unit 127, together with the inverse quantization unit 122, the intra-frame prediction unit 123, the motion compensation unit 124, the loop filtering unit 125 and the decoded image temporary storage unit 126; The bit stream of the video signal; the bit stream is input into the video decoding system 12, and firstly passes through the decoding unit 121 to obtain the decoded transform coefficients; the transform coefficients are processed by the inverse transform unit 127 and the inverse quantization unit 122, to generate residual blocks in the pixel domain; intra-prediction unit 123 may be used to generate prediction data for the current image decoding block based on the determined intra-prediction direction and data from previously decoded blocks of the current frame or picture; motion compensation Unit 124 determines prediction information for the image decoding block by parsing the motion vector and other associated syntax elements, and uses the prediction information to generate the prediction block for the image decoding block being decoded; The residual block of inverse quantization unit 122 is summed with the corresponding predictive blocks generated by intra prediction unit 123 or motion compensation unit 124 to form a decoded image block; the decoded image signal is passed through in-loop filter unit 125 to remove regions Blocking artifacts can improve image quality; the decoded image blocks are then stored in the decoded image temporary storage unit 126, which stores a reference image for subsequent intra prediction or motion compensation, At the same time, it is also used for the output of the image signal to obtain the restored original image signal.

An inter-frame prediction method provided by this embodiment of the present application mainly acts on the inter-frame prediction unit 115 of the image coding system 11 and the inter-frame prediction unit of the image decoding system 12, that is, the motion compensation unit 124; 11. A better prediction effect can be obtained through the inter-frame prediction method provided in the embodiment of the present application, and correspondingly, the image decoding system 12 can also improve the image decoding and restoration quality.

Based on this, the technical solutions of the present application are further elaborated below with reference to the accompanying drawings and embodiments. Before going into detail, it should be noted that the "first", "second", "third", etc. mentioned throughout the specification are only for distinguishing different features, and do not have a limited priority or sequence. , size relationship and other functions.

It should be noted that this embodiment is exemplified based on the AVS3 standard, and the inter-frame prediction method proposed in this application can also be applied to other coding standard technologies such as VVC, which is not specifically limited in this application.

An embodiment of the present application provides an inter-frame prediction method, which is applied to an image decoding device, that is, a decoder. The function realized by the method can be realized by calling the computer program through the first processor in the decoder. Of course, the computer program can be stored in the first memory. It can be seen that the decoder includes at least the first processor and the first memory. .

Further, in the embodiment of the present application, FIG. 12 is a schematic diagram 1 of the implementation flow of the inter-frame prediction method. As shown in FIG. 12 , the method for performing the inter-frame prediction by the decoder may include the following steps:

Step S301 , parse the bit stream to obtain the prediction mode parameter of the current block.

In the embodiment of the present application, the decoder may first parse the binary bit stream to obtain the prediction mode parameter of the current block. The prediction mode parameter may be used to determine the prediction mode used by the current block.

It should be noted that the image to be decoded can be divided into multiple image blocks, and the current image block to be decoded can be called the current block (which can be represented by CU), and the image block adjacent to the current block can be called as Neighboring block; that is, in the image to be decoded, the current block and the neighboring block have an adjacent relationship. Here, each current block may include a first image component, a second image component, and a third image component, that is, the current block indicates that the first image component, the second image component and the second image component are currently to be decoded in the image to be decoded. or the predicted image block of the third image component.

Wherein, it is assumed that the current block performs the first image component prediction, and the first image component is a luminance component, that is, the image component to be predicted is a luminance component, then the current block can also be called a luminance block; Two image components are predicted, and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, then the current block may also be called a chrominance block.

Further, in this embodiment of the present application, the prediction mode parameter may not only indicate the prediction mode adopted by the current block, but may also indicate parameters related to the prediction mode.

It can be understood that, in the embodiment of the present application, the prediction mode may include an inter prediction mode, a traditional intra prediction mode, a non-traditional intra prediction mode, and the like.

That is to say, on the encoding side, the encoder can select the optimal prediction mode to pre-encode the current block. In the process, the prediction mode of the current block can be determined, and then the prediction mode parameters used to indicate the prediction mode can be determined. Thereby, the corresponding prediction mode parameters are written into the bit stream and transmitted from the encoder to the decoder.

Correspondingly, on the decoder side, the decoder can directly obtain the prediction mode parameters of the current block by parsing the bit stream, and determine the prediction mode used by the current block according to the prediction mode parameters obtained by parsing, and the corresponding prediction mode of the prediction mode. Related parameters.

Further, in the embodiment of the present application, after the decoder obtains the prediction mode parameter by parsing, it may determine whether the current block uses the inter prediction mode based on the prediction mode parameter.

Step S302 , when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, determine the first motion vector of the sub-block of the current block; wherein the current block includes a plurality of sub-blocks.

In the embodiment of the present application, after the decoder obtains the prediction mode parameter by parsing, if the prediction mode parameter obtained by parsing indicates that the current block uses the inter prediction mode to determine the inter prediction value of the current block, the decoder may first determine the current block. The first motion vector for each sub-block of the block. Wherein, one sub-block corresponds to one first motion vector.

It should be noted that, in the embodiments of the present application, the current block is the image block to be decoded in the current frame, the current frame is decoded in a certain order in the form of image blocks, and the current block is the image block in the current frame according to the The image block to be decoded at the next moment in order. The current block may have various sizes, such as 16x16, 32x32 or 32x16, where the numbers represent the number of rows and columns of pixels on the current block.

Further, in the embodiments of the present application, the current block may be divided into a plurality of sub-blocks, wherein the size of each sub-block is the same, and the sub-block is a set of pixel points of a smaller specification. The size of the sub-block can be 8x8 or 4x4.

Exemplarily, in this application, the size of the current block is 16×16, which can be divided into 4 sub-blocks each with a size of 8×8.

It can be understood that, in the embodiment of the present application, when the decoder parses the bit stream and obtains the prediction mode parameter indicating that the inter-frame prediction value of the current block is determined using the inter-frame prediction mode, the implementation of the present application can be continued. Example of the inter prediction method provided.

In the embodiment of the present application, further, when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, the method for the decoder to determine the first motion vector of the sub-block of the current block may include the following steps :

Step S302a: Parse the bit stream to obtain the affine mode parameter and prediction reference mode of the current block.

Step S302b, when the affine mode parameter indicates to use the affine mode, determine the control point mode and the sub-block size parameter.

Step S302c: Determine the first motion vector according to the prediction reference mode, the control point mode and the sub-block size parameter.

In the embodiment of the present application, after the decoder obtains the prediction mode parameter by parsing, if the prediction mode parameter obtained by parsing indicates that the current block uses the inter prediction mode to determine the inter prediction value of the current block, the decoder can parse the bit stream to obtain the affine mode parameters and the prediction reference mode.

It should be noted that, in the embodiments of the present application, the affine mode parameter is used to indicate whether to use the affine mode. Specifically, the affine mode parameter may be the affine motion compensation enable flag affine_enable_flag, and the decoder may further determine whether to use the affine mode by determining the value of the affine mode parameter.

That is, in this application, the affine mode parameter can be a binary variable. If the value of the affine mode parameter is 1, it indicates that the affine mode is used; if the value of the affine mode parameter is 0, it indicates that the affine mode is not used.

It can be understood that, in this application, the decoder parses the bit stream, and if the affine mode parameter is not obtained by parsing, it can also be understood as indicating that the affine mode is not used.

Exemplarily, in this application, the value of the affine mode parameter may be equal to the value of the affine motion compensation enable flag affine_enable_flag, if the value of affine_enable_flag is '1', it means that affine motion compensation can be used; if the value of affine_enable_flag is affine motion compensation '0', indicating that affine motion compensation should not be used.

Further, in the embodiment of the present application, if the affine mode parameter obtained by the decoder parsing the bit stream indicates that the affine mode is used, the decoder may obtain the control point mode and the sub-block size parameter.

It should be noted that, in the embodiments of the present application, the control point mode is used to determine the number of control points. In the affine model, a sub-block can have 2 control points or 3 control points, correspondingly, the control point pattern can be the control point pattern corresponding to 2 control points, or the control point pattern corresponding to 3 control points . That is, the control point mode can include a 4-parameter mode and a 6-parameter mode.

It can be understood that, in the embodiments of the present application, for the AVS3 standard, if the current block uses the affine mode, the decoder also needs to determine the number of control points in the affine mode of the current block to determine, so that it can be determined. Determine if you are using 4-parameter (2 control points) mode or 6-parameter (3 control points) mode.

Further, in the embodiment of the present application, if the affine mode parameter obtained by the decoder parsing the bit stream indicates that the affine mode is used, the decoder can further obtain the sub-block size parameter by parsing the bit stream.

Specifically, the sub-block size parameter can be determined by the affine prediction sub-block size flag affine_subblock_size_flag. The decoder obtains the sub-block size flag by parsing the bit stream, and determines the sub-block size of the current block according to the value of the sub-block flag . The size of the sub-block may be 8x8 or 4x4. Specifically, in this application, the sub-block size flag may be a binary variable. If the value of the subblock size flag is 1, it indicates that the subblock size parameter is 8x8; if the value of the subblock size flag is 0, it indicates that the subblock size parameter is 4x4.

Exemplarily, in this application, the value of the subblock size flag may be equal to the value of the affine prediction subblock size flag affine_subblock_size_flag, if the value of affine_subblock_size_flag is '1', the current block is divided into subblocks with a size of 8x8; If the value of affine_subblock_size_flag is '0', the current block is divided into sub-blocks of size 4x4.

It can be understood that, in this application, the decoder parses the bit stream, and if the sub-block size flag is not obtained by parsing, it can also be understood that the current block is divided into 4×4 sub-blocks. That is to say, if the affine_subblock_size_flag does not exist in the bit stream, the value of the subblock size flag can be directly set to 0.

Further, in the embodiment of the present application, after determining the control point mode and the sub-block size parameter, the decoder can further determine the size of the sub-block in the current block according to the prediction reference mode, the control point mode and the sub-block size parameter. The first motion vector.

Specifically, in the embodiment of the present application, the decoder may first determine the control point motion vector group according to the prediction reference mode; a motion vector.

It can be understood that, in the embodiment of the present application, the control point motion vector group may be used to determine the motion vector of the control point.

It should be noted that, in the embodiment of the present application, the decoder can traverse each sub-block in the current block according to the above method, and use the control point motion vector group, control point mode and sub-block size parameter of each sub-block , the first motion vector of each sub-block is determined, so that the motion vector set can be obtained by constructing and obtaining the first motion vector of each sub-block.

It can be understood that, in this embodiment of the present application, the motion vector set of the current block may include the first motion vector of each sub-block of the current block.

Further, in the embodiment of the present application, when determining the first motion vector according to the control point motion vector group, the control point mode and the sub-block size parameter, the decoder may first determine the first motion vector according to the control point motion vector group, the control point mode and the sub-block size parameter. The size parameter of the current block is used to determine the difference variable; then the sub-block position can be determined based on the prediction mode parameter and the sub-block size parameter; finally, the first motion vector of the sub-block can be determined by using the difference variable and the sub-block position, and then the sub-block can be determined. Obtain a set of motion vectors for multiple sub-blocks of the current block.

Exemplarily, in this application, the difference variable may include 4 variables, specifically dHorX, dVerX, dHorY and dVerY. When calculating the difference variable, the decoder needs to determine the control point motion vector group first, wherein the control point The point motion vector group can characterize the motion vector of the control point.

Specifically, if the control point mode is a 6-parameter mode, that is, there are 3 control points, then the control point motion vector group may be a motion vector group including 3 motion vectors, expressed as mvsAffine(mv0, mv1, mv2); if the control point If the point mode is a 4-parameter mode, that is, there are 2 control points, then the control point motion vector group may be a motion vector group including 2 motion vectors, which is expressed as mvsAffine(mv0, mv1).

Then, the decoder can use the control point motion vector group to calculate the difference variable:

dHorX=(mv1_x-mv0_x)<<(7-Log(width));

dHorY=(mv1_y-mv0_y)<<(7-Log(width));

If the motion vector group is mvsAffine(mv0, mv1, mv2), then:

dVerX=(mv2_x-mv0_x)<<(7-Log(height));

dVerY=(mv2_y-mv0_y)<<(7-Log(height));

If the motion vector group is mvsAffine(mv0, mv1), then:

dVerX=−dHorY;

dVerY=dHorX.

The width and height are respectively the width and height of the current block, that is, the size parameter of the current block. Specifically, the size parameter of the current block may be obtained by the decoder by parsing the bit stream.

Further, in the embodiment of the present application, after determining the difference variable, the decoder may then determine the sub-block position based on the prediction mode parameter and the sub-block size parameter. Specifically, the decoder can determine the size of the sub-block through the sub-block size flag, and can determine which prediction mode to use through the prediction mode parameter, and then can determine the sub-block according to the size of the sub-block and the prediction mode used Location.

Exemplarily, in this application, if the value of the prediction reference mode of the current block is 2, it is the third reference mode 'Pred_List01', or, the value of the subblock size flag is 1, that is, the width subwidth of the subblock. and height subheight are both equal to 8, then (x, y) is the coordinates of the upper left corner of the 8×8 sub-block, then the coordinates xPos and yPos of the sub-block position can be determined in the following ways:

If the sub-block is the control point in the upper left corner of the current block, both xPos and yPos are equal to 0;

If the subblock is the control point in the upper right corner of the current block, xPos equals width and yPos equals 0;

If the sub-block is the control point in the lower left corner of the current block, the control point motion vector group may be a motion vector group including 3 motion vectors, then xPos is equal to 0, and yPos is equal to height;

Otherwise, xPos equals (x-xE)+4 and yPos equals (y-yE)+4.

Exemplarily, in this application, if the prediction reference mode of the current block is 0 or 1, it is the first reference mode 'Pred_List0' or the second reference mode 'Pred_List1', and the value of the subblock size flag is 0, that is, the width subwidth and height subheight of the subblock are both equal to 4, (x, y) is the coordinate of the upper left corner of the 4×4 subblock, then the coordinates xPos and yPos of the subblock position can be determined by the following methods:

If the sub-block is the control point in the upper left corner of the current block, both xPos and yPos are equal to 0;

If the subblock is the control point in the upper right corner of the current block, xPos equals width and yPos equals 0;

If the sub-block is the control point in the lower left corner of the current block, the control point motion vector group may be a motion vector group including 3 motion vectors, then xPos is equal to 0, and yPos is equal to height;

Otherwise, xPos is equal to (x-xE)+2 and yPos is equal to (y-yE)+2.

Further, in the embodiment of the present application, after the decoder obtains the sub-block position by calculation, the first motion vector of the sub-block can be determined based on the sub-block position and the difference variable, and finally, each sub-block of the current block is traversed. block, and obtain the first motion vector of each sub-block, then a motion vector set for obtaining multiple sub-blocks of the current block can be constructed.

Exemplarily, in this application, after determining the sub-block positions xPos and yPos, the decoder may determine the first motion vector mvE of the sub-block in the following manner (mvE_x, mvE_y)

mvE_x=Clip3(-131072,131071,Rounding((mv0_x<<7)+dHorX×xPos+dVerX×yPos,7));

mvE_y=Clip3(-131072,131071,Rounding((mv0_y<<7)+dHorY×xPos+dVerY×yPos,7)).

It should be noted that, in this application, when determining the deviation between each position in the sub-block and the motion vector of the sub-block, if the current block uses an affine prediction model, it can be calculated according to the formula of the affine prediction model. The motion vector of each position within the sub-block is subtracted from the motion vector of the sub-block to obtain their offset. If the motion vectors of the sub-blocks all select the motion vector at the same position in the sub-block, for example, the 4x4 block uses the position from the upper left corner (2, 2), and the 8x8 block uses the position from the upper left corner (4, 4). The standard includes the affine model used in VVC and AVS3, and the motion vector bias at the same location of each sub-block is the same. But AVS is in the upper left corner, upper right corner, and lower left corner in the case of 3 control points (the A, B, C positions in the above AVS3 text, as shown in Figure 7) are different from the positions used by other blocks, correspondingly The calculation of the motion vector deviation of the upper left corner, the upper right corner, and the lower left sub-block in the case of three control points is also different from other blocks.

Step S303: Determine the first predicted value of the sub-block based on the first motion vector, and the first motion vector deviation between each pixel position in the sub-block and the sub-block.

In the embodiment of the present application, after determining the first motion vector of each sub-block of the current block, the decoder may separately determine the first predicted value of the sub-block and the sub-block based on the first motion vector of the sub-block. The first motion vector offset between each pixel position in the block and the sub-block, where one pixel position corresponds to one first motion vector offset.

It can be understood that, in the embodiment of the present application, step S303 may specifically include:

Step S303a: Determine the first predicted value of the sub-block based on the first motion vector.

Step S303b: Determine the first motion vector deviation between each pixel position in the sub-block and the sub-block based on the first motion vector.

The inter-frame prediction method proposed in this embodiment of the present application does not limit the order in which the decoder performs step S303a and step S303b, that is, in the present application, after determining the first motion of each sub-block of the current block After the vector, the decoder may first perform step S303a and then step S303b, or may first perform step S303b and then step S303a, or simultaneously perform step S303a and step S303b.

Further, in the embodiment of the present application, when determining the first predicted value of the subblock based on the first motion vector, the decoder may first determine a sample matrix; wherein, the sample matrix includes a luminance sample matrix and a chrominance sample matrix; then The first predictor may be determined from a prediction reference mode, a subblock size parameter, a sample matrix, and a set of motion vectors.

It should be noted that, in the embodiment of the present application, when the decoder determines the first prediction value according to the prediction reference mode, the sub-block size parameter, the sample matrix, and the motion vector set, it can first determine the first prediction value according to the prediction reference mode and the sub-block size parameter. , determine the target motion vector from the motion vector set; then the reference image queue corresponding to the prediction reference mode and the reference index sample matrix and the target motion vector can be used to determine the prediction sample matrix; wherein, the prediction sample matrix a predicted value.

Specifically, in the embodiment of the present application, the sample matrix may include a luma sample matrix and a chroma sample matrix, and correspondingly, the predicted sample matrix determined by the decoder may include a luma predicted sample matrix and a chrominance predicted sample matrix, wherein, The luma prediction sample matrix includes the first luma predicted values of multiple sub-blocks, the chrominance prediction sample matrix includes the first chrominance predicted values of the multiple sub-blocks, and the first luma predicted value and the first chrominance predicted value constitute the first luma predicted value of the sub-block. Predictive value.

Exemplarily, in this application, it is assumed that the position of the upper left sample of the current block in the luminance sample matrix of the current image is (xE, yE). If the prediction reference mode of the current block is 0, that is, the first reference mode 'PRED_List0' is used, and the sub-block size flag is 0, that is, the sub-block size parameter is 4×4, then the target motion vector mv0E0 is the current The motion vector of the block sets the first motion vector of the 4×4 sub-block at the (xE+x, yE+y) position. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<<4 )+mv0E0_x, (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chroma prediction sample matrix predMatrixL0 is the reference index in the reference image queue 0 It is the sample value at position (((xE+2×x)<<4)+MvC_x, (yE+2×y)<<4)+MvC_y))) in the 1/32 precision chroma sample matrix of RefIdxL0. Among them, x1=((xE+2×x)>>3)<<3, y1=((yE+2×y)>>3)<<3, mv1E0 is the motion vector set of the current block in (x1, y1) The first motion vector of the 4×4 unit at the position, mv2E0 is the first motion vector of the 4×4 unit at the position (x1+4, y1) of the motion vector set of the current block, and mv3E0 is the motion vector set of the current block. The first motion vector of the 4×4 unit at the (x1, y1+4) position, mv4E0 is the first motion vector of the 4×4 unit of the current block’s motion vector set at the (x1+4, y1+4) position.

Specifically, MvC_x and MvC_y can be determined in the following ways:

MvC_x=(mv1E0_x+mv2E0_x+mv3E0_x+mv4E0_x+2)＞＞2

MvC_y=(mv1E0_y+mv2E0_y+mv3E0_y+mv4E0_y+2)＞＞2

Exemplarily, in this application, it is assumed that the position of the upper left corner sample of the current block in the luminance sample matrix of the current image is (xE, yE), if the prediction reference mode of the current block is 0, that is, the first reference Mode 'PRED_List0', and the value of the sub-block size flag is 1, that is, the sub-block size parameter is 8 × 8, then the target motion vector mv0E0 is the motion vector set of the current block at (xE+x, yE+y) position The first motion vector of the 8x8 unit. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<<4 )+mv0E0_x, (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chroma prediction sample matrix predMatrixL0 is the reference index in the reference image queue 0 It is the sample value at position (((xE+2×x)<<4)+MvC_x, (yE+2×y)<<4)+MvC_y))) in the 1/32 precision chroma sample matrix of RefIdxL0. Among them, MvC_x is equal to mv0E0_x, and MvC_y is equal to mv0E0.

Exemplarily, in this application, it is assumed that the position of the upper left sample of the current block in the luminance sample matrix of the current image is (xE, yE). If the prediction reference mode of the current block is 1, that is, the second reference mode 'PRED_List1' is used, and the sub-block size flag is 0, that is, the sub-block size parameter is 4×4, then the target motion vector mv0E1 is the current The block's motion vector sets the first motion vector of the 4x4 unit at the (xE+x, yE+y) position. The value of the element predMatrixL1[x][y] in the luma prediction sample matrix predMatrixL1 is the position in the 1/16-precision luma sample matrix whose reference index is RefIdxL1 in the reference image queue 1 is (((xE+x)<<4 )+mv0E1_x, (yE+y)<<4)+mv0E1_y))), the value of the element predMatrixL1[x][y] in the chroma prediction sample matrix predMatrixL1 is the reference index in the reference image queue 1 It is the sample value at position (((xE+2×x)<<4)+MvC_x, (yE+2×y)<<4)+MvC_y))) in the 1/32 precision chroma sample matrix of RefIdxL1. Among them, x1=((xE+2×x)>>3)<<3, y1=((yE+2×y)>>3)<<3, mv1E1 is the first motion vector set of MvArray in (x1, y1) the first motion vector of the 4×4 unit at the position, mv2E1 is the first motion vector of the 4×4 unit of the MvArray first motion vector set at the (x1+4, y1) position, mv3E1 is the MvArray first motion vector set The first motion vector of the 4×4 element at the (x1, y1+4) position, mv4E1 is the first motion vector of the 4×4 element of the MvArray first motion vector set at the (x1+4, y1+4) position.

Specifically, MvC_x and MvC_y can be determined in the following ways:

MvC_x=(mv1E1_x+mv2E1_x+mv3E1_x+mv4E1_x+2)>>2

MvC_y=(mv1E1_y+mv2E1_y+mv3E1_y+mv4E1_y+2)＞＞2

Exemplarily, in this application, it is assumed that the position of the upper left sample of the current block in the luminance sample matrix of the current image is (xE, yE). If the prediction reference mode of the current block is 1, that is, the second reference mode 'PRED_List1' is used, and the sub-block size flag is 1, that is, the sub-block size parameter is 8×8, then the target motion vector mv0E1 is the current The block's motion vector sets the first motion vector of an 8x8 unit at (xE+x, yE+y) position. The value of the element predMatrixL1[x][y] in the luma prediction sample matrix predMatrixL1 is the position in the 1/16-precision luma sample matrix whose reference index is RefIdxL1 in the reference image queue 1 is (((xE+x)<<4 )+mv0E1_x, (yE+y)<<4)+mv0E1_y))), the value of the element predMatrixL1[x][y] in the chroma prediction sample matrix predMatrixL1 is the reference index in the reference image queue 1 It is the sample value at position (((xE+2×x)<<4)+MvC_x, (yE+2×y)<<4)+MvC_y))) in the 1/32 precision chroma sample matrix of RefIdxL1. where MvC_x is equal to mv0E1_x and MvC_y is equal to mv0E1.

Exemplarily, in this application, it is assumed that the position of the upper left sample of the current block in the luminance sample matrix of the current image is (xE, yE). If the prediction reference mode of the current block is 2, that is, the third reference mode 'PRED_List01' is used, then the target motion vector mv0E0 is an 8×8 unit where the motion vector set of the current block is located at (xE+x, yE+y) position The first motion vector of , the target motion vector mv0E1 is the first motion vector of the 8×8 unit of the motion vector set of the current block at the (x, y) position. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<<4 )+mv0E0_x, (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chroma prediction sample matrix predMatrixL0 is the reference index in the reference image queue 0 is the sample value whose position is (((xE+2×x)<<4)+MvC0_x, (yE+2×y)<<4)+MvC0_y))) in the 1/32 precision chroma sample matrix of RefIdxL0, The value of the element predMatrixL1[x][y] in the luma prediction sample matrix predMatrixL1 is the 1/16 precision luma sample matrix whose reference index is RefIdxL1 in the reference image queue 1. The position in the matrix is ((((xE+x)<< 4) The sample value of +mv0E1_x, (yE+y)<<4)+mv0E1_y)))), the value of the element predMatrixL1[x][y] in the chroma prediction sample matrix predMatrixL1 is the reference image queue 1 The sample whose reference index is 1/32 precision chroma sample matrix of RefIdxL1 is (((xE+2×x)<<4)+MvC1_x,(yE+2×y)<<4)+MvC1_y)))) value. Where MvC0_x is equal to mv0E0_x, MvC0_y is equal to mv0E0_y, MvC1_x is equal to mv0E1_x, and MvC1_y is equal to mv0E1_y.

It should be noted that, in the embodiment of the application, the luminance sample matrix in the sample matrix may be a 1/16 precision luminance sample matrix, and the chrominance sample matrix in the sample matrix may be a 1/32 precision chrominance sample matrix.

It can be understood that, in the embodiments of the present application, for different prediction reference modes, the reference picture queue and reference index obtained by the decoder by parsing the bit stream are different.

Further, in the embodiment of the present application, when the decoder determines the sample matrix, it can first obtain the luminance interpolation filter coefficients and the chrominance interpolation filter coefficients; then the luminance sample matrix can be determined based on the luminance interpolation filter coefficients, and at the same time, it can A matrix of chroma samples is determined based on the chroma interpolation filter coefficients.

Exemplarily, in the present application, when the decoder determines the luminance sample matrix, the obtained luminance interpolation filter coefficients are shown in Table 1, and then calculates and obtains the luminance sample matrix according to the pixel position and the sample position.

Specifically, the sample position ax, 0 (x=1~15) is obtained by filtering the 8 integer values closest to the interpolation point in the horizontal direction, and the predicted value is obtained as follows:

ax,0=Clip1((fL[x][0]×A−3,0+fL[x][1]×A−2,0+fL[x][2]×A−1,0+fL [x][3]×A0, 0+fL[x][4]×A1, 0+fL[x][5]×A2, 0+fL[x][6]×A3, 0+fL[x ][7]×A4, 0+32) >> 6).

Specifically, the sample positions a0, y (y=1~15) are obtained by filtering the 8 integer values closest to the interpolation point in the vertical direction, and the predicted values are obtained as follows:

a0,y=Clip1((fL[y][0]×A0,−3+fL[y][1]×A−2,0+fL[y][2]×A−1,0+fL[ y][3]×A0, 0+fL[y][4]×A1, 0+fL[y][5]×A2, 0+fL[y][6]×A3, 0+fL[y] [7]×A−4, 0+32) >> 6).

Specifically, the acquisition method of the predicted value of the sample position ax, y (x=1～15, y=1～15) is as follows:

ax,y=Clip1((fL[y][0]×a'x,y-3+fL[y][1]×a'x,y-2+fL[y][2]×a'x , y-1+fL[y][3]×a'x, y+fL[y][4]×a'x, y+1+fL[y][5]×a'x, y+2 +fL[y][6]×a'x,y+3+fL[y][7]×a'x,y+4+(1＜＜(19-BitDepth)))＞＞(20-BitDepth) )).

in:

a'x,y=(fL[x][0]×A−3,y+fL[x][1]×A−2,y+fL[x][2]×A−1,y+fL [x][3]×A0, y+fL[x][4]×A1, y+fL[x][5]×A2, y+fL[x][6]×A3, y+fL[x ][7]×A4,y+((1<<(BitDepth-8))>>1))>>(BitDepth-8).

Exemplarily, in this application, when the decoder determines the chroma sample matrix, it can first parse the bit stream to obtain the chroma interpolation filter coefficients as shown in Table 2 above, and then calculate and obtain the chroma according to the pixel position and the sample position. Degree sample matrix.

Specifically, for the sub-pixel points where dx is equal to 0 or dy is equal to 0, it can be obtained directly by chrominance integer pixel interpolation. Calculate by pixel: if(dx==0){ ax,y(0,dy)=Clip3(0,(1<<BitDepth)-1,(fC[dy][0]×Ax,y-1+fC[dy][1]×Ax,y+fC [dy][2]×Ax,y+1+fC[dy][3]×Ax,y+2+32)>>6) } else if(dy==0){ ax,y(dx,0)=Clip3(0,(1<<BitDepth)-1,(fC[dx][0]×Ax-1,y+fC[dx][1]×Ax,y+fC [dx][2]×Ax+1, y+fC[dx][3]×Ax+2, y+32) >> 6) } else{ ax,y(dx,dy)=Clip3(0,(1<<BitDepth)-1,(C[dy][0]×a'x,y-1(dx,0)+C[dy][1 ]×a'x,y(dx,0)+C[dy][2]×a'x,y+1(dx,0)+C[dy][3]×a'x,y+2( dx, 0)+(1<<(19-BitDepth)))>>(20-BitDepth)) } where a'x,y(dx,0) is the temporary value of the sub-pixel on the whole pixel row, defined as: a'x,y(dx,0)=(fC[dx][0]×Ax-1 , y+fC[dx][1]×Ax, y+fC[dx][2]×Ax+1, y+fC[dx][3]×Ax+2, y+((1<<(BitDepth- 8))>>1))>>(BitDepth-8).

Further, in the embodiment of the present application, when the decoder is based on the first motion vector deviation between the pixel position and the sub-block, it can first parse the bit stream to obtain the secondary prediction parameter; if the secondary prediction parameter indicates to use Secondary prediction, the decoder can then determine the first motion vector offset between the sub-block and each pixel location based on the value variable.

Specifically, in the embodiment of the present application, when the decoder determines the first motion vector deviation between the sub-block and each pixel position based on the difference variable, the decoder can follow the method proposed in the above step S302 according to the control point motion. The vector group, the control point mode, and the size parameters of the current block, determine the four difference variables of dHorX, dVerX, dHorY, and dVerY, and then use the difference variables to further determine the number of pixels corresponding to each pixel position in the sub-block. A motion vector bias.

Exemplarily, in this application, width and height are the width and height of the current block obtained by the decoder, respectively, and the width subwidth and height subheight of the subblock are determined by using the subblock size parameter. Assuming that (i, j) is the coordinate of any pixel inside the sub-block, where the value range of i is 0～(subwidth-1), and the value range of j is 0～(subheight-1), then, we can The first motion vector bias at each pixel (i, j) position inside the 4 different types of sub-blocks is calculated by the following method:

If the sub-block is the control point A in the upper left corner of the current block, then the first motion vector deviation dMvA[i][j] of the (i, j) pixel:

dMvA[i][j][0]=dHorX×i+dVerX×j

dMvA[i][j][1]=dHorY×i+dVerY×j;

If the sub-block is the control point B in the upper right corner of the current block, then the first motion vector deviation dMvB[i][j] of the (i, j) pixel:

dMvB[i][j][0]=dHorX×(i-subwidth)+dVerX×j

dMvB[i][j][1]=dHorY×(i-subwidth)+dVerY×j;

If the sub-block is the control point C in the lower left corner of the current block, the control point motion vector group may be a motion vector group including 3 motion vectors, then the first motion vector deviation of the (i, j) pixel dMvC[i][j ]:

dMvC[i][j][0]=dHorX×i+dVerX×(j-subheight)

dMvC[i][j][1]=dHorY×i+dVerY×(j-subheight);

Otherwise, the first motion vector offset dMvN[i][j] of the (i, j) pixel:

dMvN[i][j][0]=dHorX×(i–(subwidth>>1))+dVerX×(j–(subheight>>1))

dMvN[i][j][1]=dHorY×(i-(subwidth>>1))+dVerY×(j-(subheight>>1)).

Wherein, dMvX[i][j][0] represents the deviation value of the first motion vector deviation in the horizontal component, and dMvX[i][j][1] represents the deviation value of the first motion vector deviation in the vertical component. X is A, B, C or N.

It can be understood that, in the embodiment of the present application, after the decoder determines the first motion vector deviation between the sub-block and each pixel position based on the difference variable, it can use the corresponding data of all pixel positions in the sub-block. All the first motion vector deviations are used to construct a motion vector deviation matrix corresponding to the sub-block. It can be seen that the motion vector deviation matrix includes the motion vector deviation between the sub-block and any internal pixel point, that is, the first motion vector deviation.

Further, in the embodiment of the present application, if the secondary prediction parameter obtained by the decoder parsing the bit stream indicates that secondary prediction is not used, then the decoder may choose to directly The first predicted value of the block is used as the second predicted value of the sub-block, and the processing of steps S304 and S305 described below is not performed.

Specifically, in the embodiment of the present application, if the secondary prediction parameter indicates that secondary prediction is not used, the decoder may determine the second prediction value by using the prediction sample matrix. The prediction sample matrix includes the first prediction values of multiple sub-blocks, and the decoder may determine the first prediction value of the sub-block where the pixel position is located as its own second prediction value.

Exemplarily, in this application, if the prediction reference mode of the current block takes a value of 0 or 1, that is, the first reference mode 'PRED_List0' or the second reference mode 'PRED_List1' is used, it can be directly obtained from including a In the prediction sample matrix of the luminance prediction sample matrix and the two chrominance prediction sample matrices, the first prediction value of the sub-block where the pixel position is located is selected, and the first prediction value is determined as the inter-frame prediction value of the pixel position, that is, the first prediction value is the first prediction value of the pixel position. Two predicted values.

Exemplarily, in this application, if the prediction reference mode of the current block is 2, that is, the third reference mode 'PRED_List01' is used, then the two luma prediction sample matrices (2 A total of 4 chrominance prediction sample matrices in the group) are averaged to obtain 1 averaged luminance prediction sample (2 averaged chrominance prediction samples), and finally from the averaged luminance prediction sample (2 averaged In the chroma prediction sample), the first prediction value of the sub-block where the pixel position is located is selected, and the first prediction value is determined as the inter-frame prediction value of the pixel position, that is, the second prediction value.

Step S304: Determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation.

In the embodiment of the present application, after the decoder determines the first prediction value of the sub-block and the first motion vector deviation between each pixel position in the sub-block and the sub-block based on the first motion vector, the decoder can determine the first prediction value according to the first motion vector. The first motion vector offset is the center position corresponding to each pixel position and the second motion vector offset.

It should be noted that, in the embodiments of the present application, for the pixel at each pixel position in the sub-block, when performing secondary prediction or PROF processing, if the preset center position used by the filter is between the sub-block If the motion vector deviation is large, the obtained prediction results may have large errors. Therefore, after determining the first motion vector deviation between each pixel position and the sub-block, the decoder can Check whether the position is suitable as the center position of the filter used in the secondary prediction of a pixel or PROF processing. If it is not suitable, the center position can be adjusted again.

It can be understood that, in this application, the center position and the second motion vector deviation corresponding to a pixel position can be the center position and the motion vector deviation of the filter used when performing secondary prediction or PROF processing on the pixel position. . That is, the center position and the second motion vector deviation are used to perform secondary prediction or PROF processing on the pixel position.

Further, in the embodiment of the present application, the method for the decoder to determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation may include the following steps:

Step S304a, determine the horizontal deviation and vertical deviation of the first motion vector deviation;

Step S304b: Determine the center position and the second motion vector deviation according to the first absolute value of the horizontal deviation, the second absolute value of the vertical deviation, and a preset deviation threshold.

In the embodiment of the present application, the decoder may first determine the deviation values of the first motion vector deviation in different directions, that is, determine the horizontal deviation and the vertical deviation corresponding to the first motion vector deviation. Next, the decoder may further calculate the absolute value of the horizontal deviation, ie, the first absolute value, and the absolute value of the vertical deviation, ie, the second absolute value. Finally, the decoder can further determine the center position and the second motion vector deviation used for secondary prediction or PROF processing according to the first absolute value, the second absolute value and the preset deviation threshold.

It should be noted that, in the embodiment of the application, the decoder can compare the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, so that the center position and the first absolute value can be determined according to the comparison result. 2. Determination of motion vector bias. The preset deviation threshold may be preset and used to determine whether to adjust the deviation of the center position and the motion vector.

Exemplarily, in the present application, the preset deviation threshold may be in units of pixels, and specifically, the preset deviation threshold may be k pixels; where k is greater than 0.5 and less than or equal to 1. That is, the preset deviation threshold may be set in advance to be one-half pixel, three-quarter pixel, or one pixel.

Further, in the embodiment of the present application, when the decoder determines the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the first absolute value and the second absolute value are both smaller than the preset deviation If the deviation threshold is set, the first motion vector deviation may be determined as the second motion vector deviation, and each pixel position may be determined as the center position.

It can be understood that, in the embodiment of the application, after the decoder compares the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, if the first absolute value and the second absolute value are If the values are less than the preset deviation, it can be considered that the vector deviation between the pixel position and the sub-block is small, and the prediction result obtained by using the pixel position as the center position of the filter for secondary prediction or PROF processing is more accurate. Therefore, The first motion vector deviation corresponding to the pixel position can be directly determined as the second motion vector deviation used for secondary prediction or PROF processing, and at the same time, the pixel position can be directly determined as the filtering for secondary prediction or PROF processing. the center position used by the device.

Further, in the embodiment of the present application, when the decoder determines the deviation between the center position and the second motion vector according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the first absolute value is greater than or equal to the preset deviation threshold, and If the second absolute value is smaller than the preset deviation threshold, the first adjustment direction may be determined according to the horizontal deviation; then, the deviation between the center position and the second motion vector may be determined according to the first adjustment direction.

It should be noted that, in the embodiment of the present application, the first adjustment direction is used to adjust the first motion vector in the horizontal direction. Therefore, the first adjustment direction includes left and right sides.

It can be understood that, in the embodiment of the application, after the decoder compares the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, if the first absolute value is greater than or equal to the preset deviation threshold If the deviation threshold is set, and the second absolute value is less than the preset deviation threshold, it can be considered that the motion vector deviation in the horizontal direction between the pixel position and the sub-block is relatively large. If the pixel is used as the center position of the filter for secondary prediction Or the prediction results obtained by PROF processing may have errors. Therefore, it is necessary to further determine the first adjustment direction according to the horizontal deviation, and then adjust according to the first adjustment direction, and finally determine the second prediction or PROF processing. Use the center position and the second motion vector offset.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (0.75, 0), and the preset deviation threshold is 1/2 pixel, the first absolute value is 0.75, and the second absolute value is 0, It can be seen that the first absolute value is greater than the preset deviation threshold, and the second absolute value is smaller than the preset deviation threshold, that is, the motion vector deviation of the pixel position in the horizontal direction is relatively large, and based on the corresponding horizontal deviation of 0.75, it can be determined that it is closer to the right side , therefore, it can be determined that the first adjustment direction is the right side.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (-0.75, 0) and the preset deviation threshold is 1/2 pixel, the first absolute value is 0.75, and the second absolute value is 0 , it can be seen that the first absolute value is greater than the preset deviation threshold, and the second absolute value is less than the preset deviation threshold, that is, the motion vector deviation of the pixel position in the horizontal direction is relatively large, and based on the corresponding horizontal deviation -0.75 can be determined to be more It is close to the left side, therefore, it can be determined that the first adjustment direction is the left side.

Further, in the embodiment of the present application, when the decoder determines the deviation between the center position and the second motion vector according to the first adjustment direction, if the first adjustment direction is the left side, then the adjacent pixel position of any pixel can be adjusted. The left pixel position is taken as the center position, and then, (1, 0) can be added to the first motion vector deviation, so that the corresponding second motion vector deviation can be obtained.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (-0.75, 0) and the preset deviation threshold is 1/2 pixel, it is determined that the first adjustment direction is the left side, then the The pixel position adjacent to the pixel position and the left pixel position is used as the center position of the filter. Then, the corresponding first motion vector deviation (-0.75, 0) can be added to (1, 0), and finally the second motion vector can be obtained. The deviation is (0.25, 0).

Further, in the embodiment of the present application, the decoder determines the deviation between the center position and the second motion vector according to the first adjustment direction. If the first adjustment direction is the right side, then the adjacent right pixel of any pixel position can be adjusted. The position is taken as the center position, and then the first motion vector deviation can be subtracted by (1, 0), so that the second motion vector deviation can be obtained.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (0.75, 0) and the preset deviation threshold is 1/2 pixel, it is determined that the first adjustment direction is the right side, then the The pixel position of the adjacent and right side is used as the center position of the filter. Then, the corresponding first motion vector deviation (0.75, 0) can be subtracted by (1, 0), and the final obtained second motion vector deviation is (-0.25, 0).

Further, in the embodiment of the present application, when the decoder determines the deviation between the center position and the second motion vector according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the second absolute value is greater than or equal to the preset deviation threshold, and If the first absolute value is smaller than the preset deviation threshold, the second adjustment direction may be determined according to the vertical deviation; then the center position and the second motion vector deviation may be determined according to the second adjustment direction.

It should be noted that, in the embodiment of the present application, the second adjustment direction is used to adjust the first motion vector in the vertical direction. Therefore, the second adjustment direction includes an upper side and a lower side.

It can be understood that, in the embodiment of the application, after the decoder compares the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, if the second absolute value is greater than or equal to the preset deviation threshold If the deviation threshold is set, and the first absolute value is smaller than the preset deviation threshold, it can be considered that the deviation of the motion vector in the vertical direction between the pixel position and the sub-block is relatively large. If the pixel is used as the center position of the filter for secondary prediction Or the prediction results obtained by PROF processing may have errors. Therefore, it is necessary to further determine the second adjustment direction according to the vertical deviation, and then adjust according to the second adjustment direction. Finally, it can be determined that the second adjustment direction is used for secondary prediction or PROF processing. Use the center position and the second motion vector offset.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (0, 0.75), and the preset deviation threshold is 1/2 pixel, the second absolute value is 0.75, and the first absolute value is 0, It can be seen that the second absolute value is greater than the preset deviation threshold, and the first absolute value is smaller than the preset deviation threshold, that is, the motion vector deviation of the pixel position in the vertical direction is relatively large, and based on the corresponding vertical deviation of 0.75, it can be determined that it is closer to the bottom. Therefore, it can be determined that the second adjustment direction is the lower side.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (0, -0.75), and the preset deviation threshold is 1/2 pixel, the second absolute value is 0.75, and the first absolute value is 0 , it can be seen that the second absolute value is greater than the preset deviation threshold, and the first absolute value is smaller than the preset deviation threshold, that is, the motion vector deviation of the pixel position in the vertical direction is relatively large, and based on the corresponding vertical deviation -0.75, it can be determined that its more Close to the upper side, therefore, it can be determined that the second adjustment direction is the upper side.

Further, in the embodiment of the present application, when the decoder determines the deviation between the center position and the second motion vector according to the second adjustment direction, if the second adjustment direction is the upper side, then the adjacent pixel position of any pixel can be adjusted. The upper pixel position is taken as the center position, and then (0, 1) can be added to the first motion vector deviation, so that the corresponding second motion vector deviation can be obtained.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (0, -0.75) and the preset deviation threshold is 1/2 pixel, it is determined that the second adjustment direction is the upper side, then the The pixel position on the upper side adjacent to the pixel position is used as the center position of the filter. Then, the corresponding first motion vector deviation (0, -0.75) can be added to (0, 1), and finally the second motion vector is obtained. The deviation is (0, 0.25).

Further, in the embodiment of the present application, the decoder determines the deviation between the center position and the second motion vector according to the second adjustment direction. If the second adjustment direction is the lower side, then the adjacent lower side of any pixel position can be adjusted. The side pixel position is taken as the center position, and then, the first motion vector offset can be subtracted by (0, 1), so that the second motion vector offset can be obtained.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (0, 0.75) and the preset deviation threshold is 1/2 pixel, it is determined that the second adjustment direction is the lower side, then the The pixel position adjacent to the pixel position and the lower side pixel position is used as the center position of the filter. Then, the corresponding first motion vector deviation (0, 0.75) can be subtracted by (0, 1), and finally the second motion vector can be obtained. The deviation is (0, 0.25).

Further, in the embodiment of the present application, when the decoder determines the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the first absolute value and the second absolute value are both greater than or equal to If the deviation threshold is preset, the third adjustment direction may be determined according to the horizontal deviation and the vertical deviation; and then the center position and the deviation of the second motion vector may be determined according to the third adjustment direction.

It should be noted that, in the embodiment of the present application, the third adjustment direction is used to adjust the first motion vector in the horizontal direction and the vertical direction at the same time. Therefore, the third adjustment direction includes the upper right side, the lower right side, the upper left side side and lower left.

It can be understood that, in the embodiment of the application, after the decoder compares the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, if the first absolute value and the second absolute value are If the value is greater than or equal to the preset deviation threshold, it can be considered that the deviation of the motion vector between the pixel position and the sub-block in the horizontal and vertical directions is large. If the pixel is used as the center position of the filter for secondary prediction or There may be errors in the prediction results obtained by PROF processing. Therefore, it is necessary to further determine the third adjustment direction according to the horizontal deviation and vertical deviation, and then adjust according to the third adjustment direction. Finally, it can be determined to perform secondary prediction or PROF processing. The center position and the second motion vector offset used when .

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (0.75, 0.75), and the preset deviation threshold is 1/2 pixel, the first absolute value is 0.75, and the second absolute value is 0.75, It can be seen that both the first absolute value and the second absolute value are greater than the preset deviation threshold, that is, the motion vector deviation of the pixel position in the horizontal direction and the vertical direction is relatively large, and based on the corresponding horizontal deviation of 0.75, it can be determined that it is closer to the right side, Meanwhile, based on the corresponding vertical deviation of 0.75, it can be determined that it is closer to the lower side, therefore, it can be determined that the third adjustment direction is the lower right side.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (-0.75, -0.75), and the preset deviation threshold is 1/2 pixel, the first absolute value is -0.75, and the second absolute value is -0.75. is -0.75, it can be seen that the first absolute value and the second absolute value are both greater than the preset deviation threshold, that is, the motion vector deviation of the pixel position in the horizontal and vertical directions is relatively large, and can be determined based on the corresponding horizontal deviation -0.75 It is closer to the left side, and at the same time, it can be determined that it is closer to the upper side based on the corresponding vertical deviation -0.75, therefore, it can be determined that the third adjustment direction is the upper left side.

Further, in the embodiment of the present application, when the decoder determines the deviation between the center position and the second motion vector according to the third adjustment direction, if the third adjustment direction is the upper left side, then the phase difference of any pixel position can be determined. The pixel position adjacent to the upper left side is taken as the center position, and then, (1, 1) can be added to the first motion vector deviation, so that the corresponding second motion vector deviation can be obtained.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (-0.75, -0.75), and the preset deviation threshold is 1/2 pixel, it is determined that the third adjustment direction is the upper left side, then the The pixel position on the upper left side adjacent to the pixel position is used as the center position of the filter, and then the corresponding first motion vector deviation (-0.75, -0.75) can be added (1, 1), and finally obtained The second motion vector bias is (0.25, 0.25).

Further, in the embodiment of the present application, the decoder determines the deviation between the center position and the second motion vector according to the third adjustment direction. If the third adjustment direction is the lower right side, then the adjacent pixel position of any pixel The pixel position on the lower right side is taken as the center position, and then, the first motion vector deviation can be subtracted by (1, 1), so that the second motion vector deviation can be obtained.

Exemplarily, in this application, when the first motion vector deviation of the pixel position is (0.75, 0.75), and the preset deviation threshold is 1/2 pixel, it is determined that the third adjustment direction is the lower right side, then it can be combined with The pixel position adjacent to the pixel position on the lower right side is used as the center position of the filter. Then, the corresponding first motion vector deviation (0.75, 0.75) can be subtracted by (1, 1), and finally the second The motion vector bias is (-0.25, -0.25).

It can be understood that, in the embodiment of the present application, the deviation values of the second motion vector deviation in the horizontal direction and the vertical direction are both smaller than the preset deviation threshold.

That is, in the present application, after the decoder determines the deviation of the motion vector of each pixel position in the sub-block from the motion vector of the sub-block, and before filtering the prediction block based on the prediction of the sub-block, The center position and the second motion vector deviation may be determined first, wherein the center position and the second motion vector are used for secondary prediction or PROF processing.

Specifically, in this application, for each pixel position in a sub-block, the magnitude of the first motion vector deviation between it and the sub-block can be determined first, including the horizontal and vertical directions of the first motion vector deviation horizontal and vertical deviations. If the absolute value of the horizontal direction and/or the vertical direction of the motion vector deviation of a certain pixel position is greater than the preset deviation threshold, it can be determined that the center position of the filter corresponding to the pixel position is no longer the pixel position itself, and further The center position and the second motion vector offset are determined.

Exemplarily, in this application, it is assumed that the previous pixel position is CURRENT, correspondingly, the pixel position of the current pixel position on the left side of the horizontal direction is LEFT, then LEFT is the coordinate of CURRENT plus (-1, 0); the current pixel position The pixel position on the right side of the horizontal direction is RIGHT, then RIGHT is the coordinate of CURRENT plus (1, 0); the pixel position of the current pixel position on the upper side of the vertical direction is UP, then UP is the coordinate of CURRENT plus (0, -1) ); the pixel position of the current pixel position on the lower side of the vertical direction is DOWN, then DOWN is the coordinate of CURRENT plus (0, 1); the pixel position of the upper left of the current pixel position is UPLEFT, then UPLEFT is the coordinate of CURRENT plus ( -1, -1); the pixel position on the upper right side of the current pixel position is UPRIGHT, then UPRIGHT is the coordinate of CURRENT plus (1, -1); the pixel position on the lower left side of the current pixel position is DOWNLEFT, then DOWNLEFT is the coordinate of CURRENT Add (-1, 1); the pixel position on the lower right side of the current pixel position is DOWNRIGHT, then DOWNRIGHT is the coordinate of CURRENT plus (1, 1).

Further, in the embodiment of the present application, before performing secondary prediction or PROF processing on a pixel position in the sub-block, if it is determined that the first motion vector deviation between the pixel position and the sub-block is in the horizontal direction and \ Or the absolute value of the deviation component in the vertical direction is greater than or equal to the preset deviation threshold (such as 1/2 pixel, 3/4 pixel or 1 pixel), then the starting point used for secondary prediction or PROF processing needs to be adjusted. , that is, the center position of the two-dimensional filter used in secondary prediction or PROF processing is adjusted.

It can be understood that, in this application, when adjusting the center position and the motion vector deviation, it can be selected in the current block that the motion vector deviation from the pixel position in the horizontal direction and the vertical direction is less than or equal to the preset deviation. Another pixel location for the threshold.

Exemplarily, in this application, if the first motion vector deviation between the motion vector of the current pixel position in the sub-block and the motion vector of the sub-block is (0.75, 0), and the preset deviation threshold is 1/2 pixel, then The pixel position to the right of the current pixel position CURRENT, that is, RIGHT, can be used as the starting position of the secondary prediction of the current pixel position, that is, the center position of the two-dimensional filter. Correspondingly, after adjusting the starting position of the secondary prediction, the deviation of the first motion vector needs to be adjusted accordingly. Specifically, the second motion can be determined according to the deviation between the motion vector at the current pixel position and the motion vector at the center position. Vector bias. For example, since the starting position is adjusted to RIGHT from CURRENT, the first motion vector offset needs to be subtracted by (1, 0), then the adjusted second motion vector offset is (-0.25, 0).

Exemplarily, in this application, if the first motion vector deviation between the motion vector of the current pixel position in the sub-block and the motion vector of the sub-block is (0.75, 0.75), and the preset deviation threshold is 1/2 pixel, then The pixel position on the lower right side of the current pixel position CURRENT, that is, DOWNRIGHT, can be used as the starting position of PROF processing of the current pixel position, that is, the center position of the one-dimensional filter. Correspondingly, after adjusting the starting position of the secondary prediction, the deviation of the first motion vector needs to be adjusted accordingly. Specifically, the second motion can be determined according to the deviation between the motion vector at the current pixel position and the motion vector at the center position. Vector bias. For example, since the starting position is adjusted from CURRENT to DOWNRIGHT, the first motion vector deviation needs to be subtracted by (1, 1), then the adjusted second motion vector deviation is (-0.25, -0.25).

Based on the above Figure 9, Figure 13 is a schematic diagram of the center position. As shown in Figure 13, both dmv_x0 and dmv_y0 are greater than one-half of a pixel. It can be seen that the closest to the circle is the square 2 with the coordinates (2, 2). If the reference position, that is, the center position of the filter, is set at this position, it can be considered that the obtained prediction result is more accurate than the original one. The adjusted motion vector deviation is (dmv_x, dmv_y), the absolute value of dmv_x is smaller than dmv_x0, and the absolute value of dmv_y is smaller than dmv_y0. The dotted box in the figure represents the pixel position used by the filter before adjustment, and the solid line box in the figure represents the pixel position used by the filter after adjustment.

In the embodiment of the present application, further, FIG. 14 is a schematic diagram 2 of the implementation flow of the inter-frame prediction method. As shown in FIG. 14 , the center position and the second motion corresponding to each pixel position are determined according to the first motion vector deviation. Before the vector deviation, that is, before step S304, the method for performing inter-frame prediction by the decoder may further include the following steps:

Step S306 , performing limiting processing on the deviation of the first motion vector according to a preset deviation range, wherein the preset deviation range includes a deviation lower limit value and a deviation upper limit value.

In the embodiment of the present application, before determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, the decoder may first limit the first motion vector. Specifically, the decoder may limit the deviation of the first motion vector according to a preset deviation range, wherein the preset deviation range includes a lower deviation value and an upper deviation value.

Further, in the embodiment of the present application, when the decoder performs limiting processing on the first motion vector deviation according to the preset deviation range, if the horizontal deviation and/or the vertical deviation are less than the lower limit value of the deviation, then the horizontal deviation and \ Or vertical deviation is set as the lower deviation value; if the horizontal deviation and\or vertical deviation is greater than the upper deviation value, then the horizontal deviation and\or vertical deviation can be set as the upper deviation value.

It should be noted that, in this application, the decoder may not limit the first motion vector deviation, that is, without performing step S306, but directly adjust the center position to any suitable pixel position according to the first motion vector deviation. It is also possible to first execute the limitation process on the first motion vector deviation according to step S306, or to limit the adjustable range and the motion vector deviation.

Further, in the embodiment of the present application, the preset deviation range may be composed of a lower deviation limit min and an upper deviation upper limit max, that is, the preset deviation range may be expressed as (min, max), and when the first motion vector When the deviation is limited, if the deviation value of the horizontal direction and/or vertical direction of the first motion vector deviation is less than the deviation lower limit value min, then the deviation value can be directly set as min; if the horizontal direction of the first motion vector deviation And \ or the deviation value in the vertical direction is greater than max, then you can directly set the deviation value to max.

Exemplarily, in this application, it is assumed that the upper limit value max is 1, that is, the first motion vector deviation is limited to 1 pixel. If the first absolute value and \ or the second absolute value) are greater than 1, the absolute value of the horizontal direction and \ or vertical direction of the deviation of the first motion vector is set to 1 pixel.

In the embodiment of the present application, further, FIG. 15 is a schematic diagram 3 of the implementation flow of the inter-frame prediction method. As shown in FIG. 15 , the center position and the second motion corresponding to each pixel position are determined according to the first motion vector deviation. After the vector deviation, that is, after step S304, the method for performing inter-frame prediction by the decoder may further include the following steps:

Step S307, if the center position does not belong to the current block, re-determine the center position according to the pixel position in the current block.

In the embodiment of the present application, after determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, the decoder may first determine whether the center position exceeds the current block, if the center position If it exceeds the range of the current block, that is, the center position does not belong to the current block, the decoder needs to re-determine the center position according to the pixel position in the current block.

In the embodiment of the present application, further, FIG. 16 is a schematic diagram of the implementation flow of the inter-frame prediction method. As shown in FIG. 16 , the center position and the second motion corresponding to each pixel position are determined according to the first motion vector deviation. After the vector deviation, that is, after step S304, the method for performing inter-frame prediction by the decoder may further include the following steps:

Step S308: If the center position does not belong to the current block, directly determine each pixel position as the center position corresponding to each pixel position.

In the embodiment of the present application, after determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, the decoder may first determine whether the center position exceeds the current block, if the center position If it exceeds the range of the current block, that is, the center position does not belong to the current block, the decoder can directly determine the corresponding, original pixel position as the center position. That is, if the center position determined based on the first motion vector deviation corresponding to each pixel position in the sub-block exceeds the range of the current block, the decoder may choose to still use the position of each pixel as the center position.

That is to say, in the embodiment of the present application, for the starting position of the adjusted secondary prediction, that is, the center position used by the filter determined based on the first motion vector deviation, the decoder can limit the center position to no more than The range of the current block, that is, will not exceed the range of the starting positions of all pixel positions in the current block before the adjustment of the secondary prediction.

Step S305 , based on the first predicted value, perform secondary prediction or PROF processing according to the center position and the second motion vector deviation, determine the second predicted value of the subblock, and determine the second predicted value as the inter-frame predicted value of the subblock.

In the embodiment of the present application, after the decoder determines the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, the decoder can determine the center position and the second motion vector deviation based on the first predicted value. The deviation is subjected to secondary prediction or PROF processing, so that the predicted value of the pixel at each pixel position can be obtained.

Further, in the embodiment of the present application, after traversing each pixel position in the sub-block and obtaining the predicted value of the pixel at each pixel position, the decoder can determine the predicted value of the pixel at each pixel position according to the predicted value of the pixel at each pixel position. the second predicted value of the sub-block, so that the second predicted value can be determined as the inter-frame predicted value of the sub-block.

It should be noted that, in the embodiments of the present application, since the center position and the second motion vector deviation can be used to perform secondary prediction or PROF processing on the pixel points of each pixel position in the sub-block, therefore, after obtaining the center After the position and the second motion vector deviation, the decoder can use the center position and the second motion vector deviation to perform secondary prediction or PROF processing on the pixels of each pixel position based on the first predicted value, and finally obtain the corresponding sub-block. the second predicted value, so that the second predicted value can be determined as the inter-frame predicted value of the sub-block.

Further, in the embodiments of the present application, the decoder performs secondary prediction or PROF processing according to the center position and the second motion vector deviation based on the first predicted value, determines the second predicted value of the sub-block, and converts the second predicted value to the second predicted value. The method of determining an inter-predicted value for a sub-block may include the following steps:

Step S305a, parsing the bit stream to obtain PROF parameters;

Step S305b, when the PROF parameter indicates that PROF processing is performed, the pixel horizontal gradient and the pixel vertical gradient corresponding to the center position are determined based on the first predicted value;

Step S305c, calculate the deviation value corresponding to each pixel position according to the pixel horizontal gradient, the pixel vertical gradient and the second motion vector deviation;

Step S305d, based on the first predicted value and the deviation value, obtain the predicted value of the pixel of each pixel position;

Step S305e, using the predicted value of the pixel at each pixel position to determine the second predicted value.

In the embodiment of the present application, the decoder may first parse the bit stream to obtain the PROF parameter, and if the PROF parameter indicates to perform PROF processing, the decoder may determine the pixel horizontal gradient and the pixel vertical gradient corresponding to the center position based on the first predicted value ; Among them, the pixel horizontal gradient is the gradient value between the pixel value corresponding to the center position and the pixel value corresponding to the adjacent pixel position in the horizontal direction; the pixel vertical gradient is the pixel value corresponding to the center position and the vertical direction. , the gradient value between the pixel values corresponding to the adjacent pixel positions.

Further, in the embodiment of the present application, the decoder can calculate and obtain the deviation value corresponding to each pixel position according to the pixel horizontal gradient, the pixel vertical gradient and the second motion vector deviation of the center position corresponding to each pixel position. Wherein, the deviation value can be used to correct the predicted value of the pixel value of each pixel position.

It should be noted that, in the embodiment of the present application, the decoder can further obtain the corrected predicted value corresponding to any pixel position according to the first predicted value and the deviation value, and traverse each pixel in the current sub-block After obtaining the modified predicted value corresponding to each pixel position, the second predicted value corresponding to the current sub-block is determined by using the modified predicted value corresponding to all pixel positions, thereby determining the corresponding inter-frame predicted value. Specifically, in the present application, after the prediction based on the sub-block is completed, the first prediction value of the current sub-block is used as the prediction value of each pixel position, and then the deviation corresponding to the first prediction value and each pixel position is used The value is added, that is, the correction processing of the predicted value of each pixel position can be completed, and the corrected predicted value can be obtained, so that the second predicted value of the current sub-block can be further obtained, and the second predicted value can be used as the corresponding value of the current sub-block. Interframe prediction value.

Further, in the embodiments of the present application, the decoder performs secondary prediction or PROF processing according to the center position and the second motion vector deviation based on the first predicted value, determines the second predicted value of the sub-block, and converts the second predicted value to the second predicted value. The method of determining an inter-predicted value for a sub-block may include the following steps:

Step S305f, parsing the bit stream to obtain secondary prediction parameters;

Step S305g, when the secondary prediction parameter indicates the use of secondary prediction, determine the filter coefficient of the two-dimensional filter according to the second motion vector deviation; wherein, the two-dimensional filter is used to perform secondary prediction processing according to a preset shape;

Step S305h, based on the filter coefficient and the first predicted value, determine the predicted value of the pixel at each pixel position;

Step S305i, using the predicted value of the pixel at each pixel position to determine the second predicted value.

In this embodiment of the present application, the decoder may first parse the bit stream to obtain secondary prediction parameters. If the secondary prediction parameters indicate to use secondary prediction, the decoder may determine the size of the two-dimensional filter according to the second motion vector deviation. Filter coefficient; wherein, the two-dimensional filter is used to perform secondary prediction processing according to the preset shape.

It should be noted that, in the embodiment of the present application, the filter coefficient of the two-dimensional filter is related to the deviation of the second motion vector corresponding to the target pixel position. That is to say, for different target pixel positions, if the corresponding second motion vector deviations are different, the filter coefficients of the used two-dimensional filters are also different.

It can be understood that, in the embodiment of the present application, a two-dimensional filter is used to perform secondary prediction by using a plurality of adjacent pixel positions that form a preset shape. The preset shape is a rectangle, a rhombus or any symmetrical shape.

That is to say, in the present application, the two-dimensional filter used for secondary prediction is a filter formed by adjacent points forming a preset shape. The adjacent dots constituting the preset shape may include a plurality of dots, for example, composed of 9 dots. The predetermined shape may be a symmetrical shape, for example, the predetermined shape may include a rectangle, a rhombus, or any other symmetrical shape.

Exemplarily, in this application, the two-dimensional filter is a rectangular filter, and specifically, the two-dimensional filter is a filter composed of 9 adjacent pixel positions forming a rectangle. Among the 9 pixel positions, the pixel position located in the center is the pixel position of the pixel currently requiring secondary prediction, that is, the current pixel position.

Further, in the embodiment of the present application, when the decoder determines the filter coefficient of the two-dimensional filter according to the second motion vector deviation, it can first parse the bit stream to obtain the scale parameter, and then can obtain the scale parameter according to the scale parameter and the second motion vector. Vector bias, which determines the filter coefficients corresponding to the pixel positions.

It should be noted that, in the embodiment of the present application, the scale parameter may include at least one scale value, and the second motion vector deviation includes a horizontal deviation and a vertical deviation; wherein, at least one scale value is a non-zero real number.

Specifically, in this application, when the two-dimensional filter uses 9 adjacent pixel positions that form a rectangle to perform secondary prediction, the pixel position located in the center of the rectangle is the position to be predicted, that is, the current pixel position, and the other 8 The target pixel positions are sequentially located in the eight directions of the upper left, upper, upper right, right, lower right, lower, lower left, and left of the current pixel position.

Correspondingly, in the present application, the decoder may calculate and obtain 9 filter coefficients corresponding to 9 adjacent pixel positions based on at least one scale value and the deviation of the second motion vector of the position to be predicted according to a preset calculation rule.

It should be noted that, in this application, the preset calculation rule may include various calculation methods, such as addition operation, subtraction operation, multiplication operation, and the like. Among them, for different pixel positions, different calculation methods can be used to calculate the filter coefficients.

It can be understood that, in this application, when the decoder calculates and obtains multiple filter coefficients corresponding to multiple pixel positions according to different calculation methods in the preset calculation rules, some filter coefficients may be the difference of the second motion vector deviation. The linear function, that is, the two are in a linear relationship, may also be a quadratic function or a higher-order function of the deviation of the second motion vector, that is, the two are in a non-linear relationship.

That is to say, in the present application, any one of the multiple filter coefficients corresponding to multiple adjacent pixel positions may be a linear function, a quadratic function, or a higher-order function of the second motion vector deviation.

Exemplarily, in this application, it is assumed that the second motion vector deviation of the pixel position is (dmv_x, dmv_y), wherein, if the coordinates of the target pixel position are (i, j), then dmv_x can be expressed as dMvX[i][ j][0], that is, the deviation value of the second motion vector deviation in the horizontal component, and dmv_y can be expressed as dMvX[i][j][1], that is, the deviation value of the second motion vector deviation in the vertical component.

Correspondingly, Table 3 shows the filter coefficients obtained based on the second motion vector deviation (dmv_x, dmv_y). As shown in Table 3, for a two-dimensional filter, the second motion vector deviation according to the pixel position (the horizontal deviation is dmv_x , the vertical deviation is dmv_y) and different scale parameters, such as m and n, 9 filter coefficients corresponding to 9 adjacent pixel positions can be obtained, where the decoder can directly convert the filter coefficients of the current pixel position in the center Set to 1.

table 3 pixel position filter coefficients upper left (-dmv_x-dmv_y)×m left -dmv_x×n lower left (-dmv_x+dmv_y)×m superior -dmv_y×n center 1 Down dmv_y×n top right (dmv_x-dmv_y)×m right dmv_x×n lower right (dmv_x+dmv_y)×m

Among them, the scale parameters m and n are generally decimals or fractions, and a possible situation is that both m and n are powers of 2, such as 1/2, 1/4, 1/8 and so on. Here, dmv_x and dmv_y are their actual sizes, that is, 1 of dmv_x and dmv_y represents the distance of 1 pixel, and dmv_x and dmv_y are decimals or fractions.

It should be noted that, in the embodiment of the present application, compared with the existing 8-tap filter, the motion vectors of the integer pixel position and the sub-pixel position corresponding to the current common 8-tap filter are in the horizontal and vertical directions. All are non-negative, and the size is between 0 pixel and 1 pixel, that is, dmv_x and dmv_y cannot be negative. In this application, the motion vectors of the integer pixel position and the sub-pixel position corresponding to the filter may be negative in both the horizontal and vertical directions, that is, dmv_x and dmv_y may be negative.

Exemplarily, in the embodiment of the present application, if the proportional parameter m is 1/16 and n is 1/2, then the above Table 3 can be expressed as the following Table 4:

Table 4 pixel position filter coefficients upper left (-dmv_x-dmv_y)/16 left -dmv_x/2 lower left (-dmv_x+dmv_y)/16 superior -dmv_y/2 center 1 Down dmv_y/2 top right (dmv_x-dmv_y)/16 right dmv_x/2 lower right (dmv_x+dmv_y)/16

It can be understood that, in the embodiments of the present application, in the video coding and decoding technology and standards, a magnification is usually used to avoid decimals, floating-point number operations, and then the calculated result is reduced by an appropriate multiple to obtain a correct result. Left shift is usually used when zooming in, and right shift is usually used when zooming out. Therefore, when performing secondary prediction through a two-dimensional filter, it will be written in the following form in practical application:

Assuming that the second motion vector deviation of the pixel position is (dmv_x, dmv_y), which is obtained by shifting left by shift1 (dmv_x', dmv_y'), based on the above Table 4, the coefficients of the two-dimensional filter can be expressed as Table 5 below:

table 5 pixel position filter coefficients upper left -dmv_x'-dmv_y' left -dmv_x'×8 lower left -dmv_x'+dmv_y' superior -dmv_y'×8 center 16<<shift1 Down dmv_y'×8 top right dmv_x'-dmv_y' right dmv_x×8 lower right dmv_x'+dmv_y'

Fig. 17 is a schematic diagram 1 of a two-dimensional filter. As shown in Fig. 17, the sub-block-based prediction result is used as the basis for the secondary prediction, and the light-colored square is the integer pixel position of the filter, that is, the sub-block-based Predicted location. The circle is the sub-pixel position that needs to be predicted twice, that is, the position of the pixel position, and the dark square is the integer pixel position corresponding to the sub-pixel position. When the sub-pixel position is obtained by interpolation, 9 integer pixels as shown in the figure are required. Location.

Fig. 18 is a schematic diagram 2 of a two-dimensional filter. As shown in Fig. 18, the result of the sub-block-based prediction is used as the basis for the secondary prediction, and the light-colored square is the integer pixel position of the filter, that is, the sub-block-based Predicted location. The circle is the sub-pixel position that needs to be predicted twice, that is, the position of the pixel position, and the dark square is the integer pixel position corresponding to the sub-pixel position. When the sub-pixel position is obtained by interpolation, 13 integer pixels are required as shown in the figure. Location.

Further, in the embodiment of the present application, after determining the filter coefficient of the two-dimensional filter according to the second motion vector deviation, the decoder can determine the second prediction of the current sub-block based on the filter coefficient and the first prediction value. value, so that the secondary prediction of the current sub-block can be realized.

It can be understood that, in the embodiment of the present application, the decoder determines the filter coefficient by using the second motion vector deviation corresponding to the pixel position, so that the first predicted value can be processed through the two-dimensional filter according to the filter coefficient. Correction to obtain the corrected second predicted value of the current sub-block. It can be seen that the second predicted value is a modified value based on the first predicted value.

Further, in the embodiment of the present application, when determining the second predicted value of the current subblock based on the filter coefficient and the first predicted value, the decoder may first perform a multiplication operation on the filter coefficient and the first predicted value to obtain: Product result, after traversing all pixel positions in the current sub-block, add the product results of all pixel positions in the current sub-block to obtain the summation result, and finally normalize the addition result, and finally obtain The modified second predicted value of the current sub-block.

It should be noted that, in the embodiments of the present application, before the secondary prediction is performed, the first prediction value of the current sub-block where the pixel position is located is generally used as the prediction value before the correction of the pixel position. Therefore, When filtering through a two-dimensional filter, the filter coefficient can be multiplied by the predicted value of the corresponding pixel position, that is, the first predicted value, and the multiplication results corresponding to each pixel position can be accumulated and then normalized.

It can be understood that in this application, the decoder can perform normalization processing in various ways. For example, the accumulated result can be multiplied by the filter coefficient and the predicted value of the corresponding pixel position, and shifted by 4+shift1 bits to the right. . Alternatively, it is also possible to multiply the filter coefficient by the predicted value of the corresponding pixel position and then add (1<<(3+shift1)) to the accumulated result, and then shift to the right by 4+shift1 bits.

It can be seen that in this application, after obtaining the second motion vector deviation corresponding to the pixel position inside the current sub-block, for each sub-block and each pixel position in each sub-block, the second motion vector deviation can be obtained according to the second motion vector deviation. , based on the first prediction value of the motion compensation of the current sub-block, use a two-dimensional filter to perform filtering, complete the secondary prediction of the current sub-block, and obtain a new second prediction value.

Further, in the embodiments of the present application, the two-dimensional filter can be understood as performing secondary prediction by using a plurality of adjacent pixel positions that form a preset shape. The preset shape may be a rectangle, a rhombus or any symmetrical shape.

Specifically, in the embodiment of the present application, when the two-dimensional filter uses 9 adjacent pixel positions that form a rectangle to perform secondary prediction, it can first determine the prediction sample matrix of the current block, and the current sub-frame of the current block. The motion vector deviation matrix of the block; wherein, the motion vector deviation matrix includes the second motion vector deviation corresponding to all pixel positions; then based on 9 adjacent pixel positions that form a rectangle, using the predicted sample matrix and the motion vector deviation matrix, determine The sub-predicted sample matrix for the current block.

Exemplarily, in this application, if the width and height of the current block are width and height, respectively, the width and height of each sub-block are subwidth and subheight, respectively. As shown in Figure 7, the sub-block where the upper-left sample of the luminance prediction sample matrix of the current block is located is A, the sub-block where the upper-right sample is located is B, the sub-block where the lower-left sample is located is C, and the sub-blocks where the other positions are located are other sub-blocks.

For each sub-block in the current block, the motion vector deviation matrix of the sub-block can be denoted as dMv, then:

1. If the sub-block is A, dMv is equal to dMvA;

2. If the sub-block is B, dMv is equal to dMvB;

3. If the sub-block is C, and there are 3 motion vectors in the control point motion vector group mvAffine of the current sub-block, dMv is equal to dMvC;

4. If the sub-block is other than A, B, and C, dMv is equal to dMvN.

Further, it is assumed that (x, y) is the coordinate of the upper left corner of the current sub-block, (i, j) is the coordinate of the pixel inside the luminance sub-block, the value range of i is 0～(subwidth-1), and the value of j The value range is 0～(subheight-1), the prediction sample matrix based on sub-block is PredMatrixSb, the prediction sample matrix of secondary prediction is PredMatrixS, dCenterX, dCenterY are the offsets of the secondary prediction starting position in the horizontal and vertical directions Quantity matrix, the predicted sample PredMatrixS[x+i][y+j] of the secondary prediction of (x+i, y+j) can be calculated as follows: dCenterX[i][j]=(dMv[i][j][0]＜0?-1:1)*((abs(dMv[i][j][0])+(1＜＜10) )/(1＜＜11)) dCenterX[i][j]=CLIP3(-i, width-1-i) dCenterY[i][j]=(dMv[i][j][1]＜0?-1:1)*((abs(dMv[i][j][1])+(1＜＜10) )/(1＜＜11)) dCenterY[i][j]=CLIP3(-j, height-1-j) dMv[i][j][0]=dMv[i][j][0]–dCenterX * (1＜＜11) dMv[i][j][1]=dMv[i][j][1]–dCenterY * (1＜＜11) PredMatrixS[x+i][y+j]= (UPLEFT(x+i,y+j)×(-dMv[i][j][0]-dMv[i][j][1])+ UP(x+i,y+j)×((-dMv[i][j][1])<<3)+ UPRIGHT(x+i,y+j)×(dMv[i][j][0]-dMv[i][j][1])+ LEFT(x+i,y+j)×((-dMv[i][j][0])<<3)+ CENTER(x+i,y+j)×(1<<15)+ RIGHT(x+i,y+j)×(dMv[i][j][0]<<3)+ DOWNLEFT(x+i,y+j)×(-dMv[i][j][0]+dMv[i][j][1])+ DOWN(x+i, y+j)×(dMv[i][j][1]＜＜3)+ DOWNRIGHT(x+i,y+j)×(dMv[i][j][0]+dMv[i][j][1])+ (1<<14))>>15 PredMatrixS[x+i][y+j]=Clip3(0, (1<<BitDepth)-1, PredMatrixS[x+i][y+j]).

Where, UPLEFT(x+i,y+j)=PredMatrixSb[max(0,x+i+dCenterX[i][j]-1)][max(0,y+j+dCenterY[i][j] -1)] UP(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]][max(0,y+j+dCenterY[i][j]-1)] UPRIGHT(x+i,y+j)=PredMatrixSb[min(width-1,x+i+dCenterX[i][j]+1)][max(0,y+j+dCenterY[i][j] -1)] LEFT(x+i,y+j)=PredMatrixSb[max(0,x+i+dCenterX[i][j]-1)][y+j+dCenterY[i][j]] CENTER(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]][y+j+dCenterY[i][j]] RIGHT(x+i,y+j)=PredMatrixSb[min(width-1,x+i+dCenterX[i][j]+1)][y+j+dCenterY[i][j]] DOWNLEFT(x+i,y+j)=PredMatrixSb[max(0,x+i+dCenterX[i][j]-1)][min(height-1,y+j+dCenterY[i][j] +1)] DOWN(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]][max(0,x+i+dCenterX[i][j]-1)][min(height- 1, y+j+dCenterY[i][j]+1)] DOWNRIGHT(x+i,y+j)=PredMatrixSb[min(width-1,x+i+dCenterX[i][j]+1)][max(0,x+i+dCenterX[i][j] -1)][min(height-1,y+j+dCenterY[i][j]+1)]

Among them, max(a, b) can be understood as taking the larger value of a and b, and min(a, b) can be understood as taking the smaller value of a and b.

Further, in the present application, if the method of extending one row and one column to the prediction block based on the prediction of the sub-block is adopted, that is, the rows of -1 and height and the -1 and width columns of the predMatrix are expanded, and the previous paragraph can be expressed as: UPLEFT(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]-1][y+j+dCenterY[i][j]-1] UP(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]][y+j+dCenterY[i][j]-1] UPRIGHT(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]+1][y+j+dCenterY[i][j]-1] LEFT(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]-1][y+j+dCenterY[i][j]] CENTER(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]][y+j+dCenterY[i][j]] RIGHT(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]+1][y+j+dCenterY[i][j]] DOWNLEFT(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]-1][y+j+dCenterY[i][j]+1] DOWN(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]][y+j+dCenterY[i][j]+1] DOWNRIGHT(x+i,y+j)=PredMatrixSb[x+i+dCenterX[i][j]+1][y+j+dCenterY[i][j]+1].

Further, in this application, dCenterX and dCenterY can also be written in the form of temporary variables instead of in the form of matrices.

It can be understood that, in the embodiment of the present application, the CENTER (x+i, y+j) pixel position may be the center position of the above-mentioned 9 adjacent pixel positions that form a rectangle, and then based on the (x +i, y+j) pixel position, and the other 8 pixel positions adjacent to it, perform secondary prediction processing. Specifically, the other 8 pixel positions are UP (upper), UPRIGHT (upper right), LEFT (left), RIGHT (right), DOWNLEFT (lower left), DOWN (lower), DOWNRIGHT (lower right), UPLEFT (upper left) .

It should be noted that, in this application, the precision of the calculation formula of PredMatrixS[x+i][y+j] may use a lower precision. For example, the item on the right side of each multiplication is shifted to the right. For example, dMv[i][j][0] and dMv[i][j][1] are shifted to the right by 3 bits. Correspondingly, 1<<15 becomes 1 <<(15-shift3), …+(1<<10))>>11 becomes …+(1<<(10-shift3)))>>(11-shift3).

Exemplarily, the size of the motion vector can be limited to a reasonable range. For example, the positive and negative values of the motion vector used above in the horizontal and vertical directions do not exceed 1 pixel, 1/2 pixel, or 1/4 pixel.

It can be understood that, if the prediction reference mode of the current block is 'Pred_List01', the decoder averages the multiple prediction sample matrices of each component to obtain the final prediction sample matrix of the component. For example, two luminance prediction sample matrices are averaged to obtain a new luminance prediction sample matrix.

Further, in the embodiment of the present application, after obtaining the prediction sample matrix of the current block, if the current block has no transform coefficients, then the prediction matrix is used as the decoding result of the current block, and if the current block has transform coefficients, then First decode the transform coefficients, and obtain the residual matrix through inverse transformation and inverse quantization, and add the residual matrix to the prediction matrix to obtain the decoding result.

It can be understood that the inter-frame prediction method proposed in this application can first adjust the initial position of the secondary prediction based on the first motion vector deviation corresponding to a pixel position in the sub-block, that is, adjust the secondary prediction or PROF processing. The center position of the filter used by the filter, and then the first motion vector can be further adjusted based on the center position determined after the adjustment to obtain the second motion vector deviation, and finally the center position and the second motion vector deviation can be used for this Point-based prediction is performed on the pixel position, thereby obtaining the second prediction value of the sub-block.

It should be noted that the inter-frame prediction method proposed in this application can be applied to any image component. In this embodiment, a secondary prediction scheme is exemplarily used for the luminance component, but it can also be used for the chrominance component. or any component in other formats. The inter-frame prediction method proposed in this application can also be applied to any image format, including but not limited to the YUV format, including but not limited to the luminance component of the YUV format.

This embodiment provides an inter-frame prediction method. After prediction based on sub-blocks, for pixel positions with a large deviation of the first motion vector between the motion vector and the motion vector of the sub-block, the deviation of the first motion vector can be determined based on the first motion vector deviation. Redetermine the determination of the center position and the second motion vector offset used for secondary prediction or PROF processing, so that the center position and the second motion vector offset can be used for point-based point-based The secondary prediction of , obtains the second prediction value. It can be seen that the inter-frame prediction method proposed in this application can be well applied to all scenarios, can reduce the prediction error, greatly improve the coding performance, and thus improve the coding and decoding efficiency.

An embodiment of the present application provides an inter-frame prediction method, which is applied to an image encoding device, that is, an encoder. The functions implemented by the method can be implemented by calling a computer program through the second processor in the encoder. Of course, the computer program can be stored in the second memory. It can be seen that the encoder includes at least a second processor and a second memory. .

FIG. 19 is a schematic diagram 5 of the implementation flow of the inter-frame prediction method. As shown in FIG. 19 , the method for performing the inter-frame prediction by the encoder may include the following steps:

Step S401: Determine the prediction mode parameter of the current block.

In the embodiment of the present application, the encoder may first determine the prediction mode parameter of the current block. Specifically, the encoder may first determine the prediction mode used by the current block, and then determine the corresponding prediction mode parameter based on the prediction mode. The prediction mode parameter may be used to determine the prediction mode used by the current block.

It should be noted that, in the embodiments of the present application, the image to be encoded may be divided into multiple image blocks, the image block to be encoded currently may be referred to as the current block, and the image block adjacent to the current block may be referred to as the current block. is an adjacent block; that is, in the to-be-coded image, the current block and the adjacent block have an adjacent relationship. Here, each current block may include a first image component, a second image component, and a third image component; that is, the current block is the first image component, the second image component, and the current to-be-encoded image. or the predicted image block of the third image component.

Wherein, it is assumed that the current block performs the first image component prediction, and the first image component is a luminance component, that is, the image component to be predicted is a luminance component, then the current block can also be called a luminance block; Two image components are predicted, and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, then the current block may also be called a chrominance block.

It should be noted that, in this embodiment of the present application, the prediction mode parameter indicates the prediction mode adopted by the current block and parameters related to the prediction mode. Here, for the determination of the prediction mode parameters, a simple decision-making strategy can be adopted, such as determining according to the size of the distortion value; or a complex decision-making strategy can be adopted, such as conducting according to the result of Rate Distortion Optimization (RDO) It is confirmed that the embodiments of the present application do not make any limitation. Generally speaking, the RDO method can be used to determine the prediction mode parameter of the current block.

Specifically, in some embodiments, when the encoder determines the prediction mode parameter of the current block, it may first perform precoding processing on the current block by using multiple prediction modes to obtain the bit rate distortion cost value corresponding to each prediction mode; then A minimum rate-distortion cost value is selected from the obtained multiple rate-distortion cost values, and a prediction mode parameter of the current block is determined according to the prediction mode corresponding to the minimum rate-distortion cost value.

That is to say, on the encoder side, multiple prediction modes may be used for the current block to perform precoding processing on the current block respectively. Here, the multiple prediction modes usually include an inter prediction mode, a traditional intra prediction mode, and a non-traditional intra prediction mode; wherein, the traditional intra prediction mode may include a direct current (DC) mode, a plane (PLANAR) mode and angle mode, etc., non-traditional intra prediction modes can include matrix-based intra prediction (Matrix-based IntraPrediction, MIP) mode, cross-component linear model prediction (Cross-component Linear model prediction, CCLM) mode, intra block copy ( Intra Block Copy, IBC) mode and PLT (Palette) mode, etc., and the inter-frame prediction mode may include ordinary inter-frame prediction mode, GPM mode, and AWP mode.

In this way, after precoding the current block with multiple prediction modes, the corresponding bit rate distortion cost value of each prediction mode can be obtained; then the minimum bit rate is selected from the obtained multiple rate distortion cost values. The distortion cost value is determined, and the prediction mode corresponding to the minimum bit rate distortion cost value is determined as the prediction mode parameter of the current block. In addition, it is also possible to obtain the distortion value corresponding to each prediction mode after precoding the current block with multiple prediction modes; then select the minimum distortion value from the obtained multiple distortion values, and then select the minimum distortion value from the obtained distortion values. The prediction mode corresponding to the distortion value is determined as the prediction mode used by the current block, and corresponding prediction mode parameters are set according to the prediction mode. In this way, the current block is finally encoded using the determined prediction mode parameters, and in this prediction mode, the prediction residual can be made smaller, and the encoding efficiency can be improved.

That is to say, on the encoding side, the encoder can select the optimal prediction mode to pre-encode the current block. In the process, the prediction mode of the current block can be determined, and then the prediction mode parameters used to indicate the prediction mode can be determined. Thereby, the corresponding prediction mode parameters are written into the bit stream and transmitted from the encoder to the decoder.

Correspondingly, on the decoder side, the decoder can directly obtain the prediction mode parameters of the current block by parsing the bit stream, and determine the prediction mode used by the current block according to the prediction mode parameters obtained by parsing, and the corresponding prediction mode of the prediction mode. Related parameters.

Step S402 , when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, determine the first motion vector of the sub-block of the current block; wherein the current block includes a plurality of sub-blocks.

In the embodiment of the present application, if the prediction mode parameter indicates that the current block uses the inter prediction mode to determine the inter prediction value of the current block, the encoder may first determine the first motion vector of each subblock of the current block. Wherein, one sub-block corresponds to one first motion vector.

It should be noted that, in the embodiments of the present application, the current block is the image block to be encoded in the current frame, the current frame is encoded in a certain order in the form of image blocks, and the current block is the image block in the current frame according to the The image block to be encoded at the next moment in order. The current block may have various sizes, such as 16x16, 32x32 or 32x16, where the numbers represent the number of rows and columns of pixels on the current block.

Further, in the embodiments of the present application, the current block may be divided into a plurality of sub-blocks, wherein the size of each sub-block is the same, and the sub-block is a set of pixel points of a smaller specification. The size of the sub-block can be 8x8 or 4x4.

Exemplarily, in this application, the size of the current block is 16×16, which can be divided into 4 sub-blocks each with a size of 8×8.

It can be understood that, in the embodiments of the present application, when the encoder determines that the prediction mode parameter indicates that the inter-frame prediction mode is used to determine the inter-frame prediction value of the current block, the frame provided by the embodiments of the present application can continue to be used. inter-prediction method.

In the embodiment of the present application, further, when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, and the encoder determines the first motion vector of the sub-block of the current block, the current block can be determined. The affine mode parameters and the prediction reference mode. When the affine mode parameter indicates to use the affine mode, the control point mode and subblock size parameters are determined. Finally, the first motion vector may be determined according to the prediction reference mode, the control point mode and the sub-block size parameter.

In the embodiment of the present application, after the encoder determines the prediction mode parameter, if the prediction mode parameter indicates that the current block uses the inter prediction mode to determine the inter prediction value of the current block, the encoder may determine the affine mode parameter and the prediction reference model.

It should be noted that, in the embodiments of the present application, the affine mode parameter is used to indicate whether to use the affine mode. Specifically, the affine mode parameter may be the affine motion compensation enable flag affine_enable_flag, and the encoder may further determine whether to use the affine mode by determining the value of the affine mode parameter.

That is, in this application, the affine mode parameter can be a binary variable. If the value of the affine mode parameter is 1, it indicates that the affine mode is used; if the value of the affine mode parameter is 0, it indicates that the affine mode is not used.

Exemplarily, in this application, the value of the affine mode parameter may be equal to the value of the affine motion compensation enable flag affine_enable_flag, if the value of affine_enable_flag is '1', it means that affine motion compensation can be used; if the value of affine_enable_flag is affine motion compensation '0', indicating that affine motion compensation should not be used.

Further, in the embodiment of the present application, if the affine mode parameter determined by the encoder indicates that the affine mode is used, the encoder may acquire the control point mode and the sub-block size parameter.

It should be noted that, in the embodiments of the present application, the control point mode is used to determine the number of control points. In the affine model, a sub-block can have 2 control points or 3 control points, correspondingly, the control point pattern can be the control point pattern corresponding to 2 control points, or the control point pattern corresponding to 3 control points . That is, the control point mode can include a 4-parameter mode and a 6-parameter mode.

It can be understood that, in the embodiments of the present application, for the AVS3 standard, if the current block uses the affine mode, the encoder also needs to determine the number of control points in the affine mode of the current block to determine, so that the affine mode can be determined. Determine if you are using 4-parameter (2 control points) mode or 6-parameter (3 control points) mode.

Further, in the embodiment of the present application, if the affine mode parameter determined by the encoder indicates that the affine mode is used, the encoder may further determine the sub-block size parameter.

Specifically, the subblock size parameter can be represented by the affine prediction subblock size flag affine_subblock_size_flag, and the encoder can set the value of the subblock size flag to indicate the subblock size parameter, that is, the size of the subblock of the current block. The size of the sub-block may be 8x8 or 4x4. Specifically, in this application, the sub-block size flag may be a binary variable. If the value of the subblock size flag is 1, it indicates that the subblock size parameter is 8x8; if the value of the subblock size flag is 0, it indicates that the subblock size parameter is 4x4.

Exemplarily, in this application, the value of the subblock size flag may be equal to the value of the affine prediction subblock size flag affine_subblock_size_flag, if the value of affine_subblock_size_flag is '1', the current block is divided into subblocks with a size of 8x8; If the value of affine_subblock_size_flag is '0', the current block is divided into sub-blocks of size 4x4.

Further, in the embodiment of the present application, after determining the control point mode and the sub-block size parameter, the encoder can further determine the sub-block in the current block according to the prediction reference mode, the control point mode and the sub-block size parameter. The first motion vector.

Specifically, in the embodiment of the present application, the encoder may first determine the control point motion vector group according to the prediction reference mode; a motion vector.

It can be understood that, in the embodiment of the present application, the control point motion vector group may be used to determine the motion vector of the control point.

It should be noted that, in the embodiment of the present application, the encoder can traverse each sub-block in the current block according to the above method, and use the control point motion vector group, control point mode and sub-block size parameter of each sub-block , the first motion vector of each sub-block is determined, so that the motion vector set can be obtained by constructing and obtaining the first motion vector of each sub-block.

It can be understood that, in this embodiment of the present application, the motion vector set of the current block may include the first motion vector of each sub-block of the current block.

Further, in the embodiment of the present application, when the encoder determines the first motion vector according to the control point motion vector group, the control point mode and the sub-block size parameter, it may first determine the first motion vector according to the control point motion vector group, the control point mode and the The size parameter of the current block is used to determine the difference variable; then the sub-block position can be determined based on the prediction mode parameter and the sub-block size parameter; finally, the first motion vector of the sub-block can be determined by using the difference variable and the sub-block position, and then the sub-block can be determined. Obtain a set of motion vectors for multiple sub-blocks of the current block.

It should be noted that, in this application, when determining the deviation between each position in the sub-block and the motion vector of the sub-block, if the current block uses an affine prediction model, it can be calculated according to the formula of the affine prediction model. The motion vector of each position within the sub-block is subtracted from the motion vector of the sub-block to obtain their offset. If the motion vectors of the sub-blocks all select the motion vector at the same position in the sub-block, for example, the 4x4 block uses the position from the upper left corner (2, 2), and the 8x8 block uses the position from the upper left corner (4, 4). The standard includes the affine model used in VVC and AVS3, and the motion vector bias at the same location of each sub-block is the same. But AVS is in the upper left corner, upper right corner, and lower left corner in the case of 3 control points (the A, B, C positions in the above AVS3 text, as shown in Figure 7) are different from the positions used by other blocks, correspondingly in The calculation of the motion vector deviation of the upper left corner, the upper right corner, and the lower left sub-block in the case of three control points is also different from that of other blocks. Specific examples are as follows.

Step S403: Determine the first predicted value of the sub-block based on the first motion vector, and the first motion vector deviation between each pixel position in the sub-block and the sub-block.

In the embodiment of the present application, after determining the first motion vector of each sub-block of the current block, the encoder may separately determine the first predicted value of the sub-block and the sub-block based on the first motion vector of the sub-block. The first motion vector offset between each pixel position of and the sub-block.

It can be understood that, in the embodiment of the present application, step S403 may specifically include:

Step S403a: Determine the first predicted value of the sub-block based on the first motion vector.

Step S403b: Determine the first motion vector deviation between each pixel position in the sub-block and the sub-block based on the first motion vector.

The inter-frame prediction method proposed in this embodiment of the present application does not limit the order in which the encoder performs step S403a and step S403b, that is, in the present application, after determining the first motion of each sub-block of the current block After the vector, the encoder may first perform step S403a and then step S403b, or may first perform step S403b and then step S403a, or simultaneously perform step S403a and step S403b.

Further, in the embodiment of the present application, when the encoder determines the first predicted value of the sub-block based on the first motion vector, it may first determine a sample matrix; wherein, the sample matrix includes a luminance sample matrix and a chrominance sample matrix; then The first predictor may be determined from a prediction reference mode, a subblock size parameter, a sample matrix, and a set of motion vectors.

It should be noted that, in the embodiments of the present application, when the encoder determines the first prediction value according to the prediction reference mode, the sub-block size parameter, the sample matrix, and the motion vector set, the encoder may first determine the first prediction value according to the prediction reference mode and the sub-block size parameter. , determine the target motion vector from the motion vector set; then the reference image queue corresponding to the prediction reference mode and the reference index sample matrix and the target motion vector can be used to determine the prediction sample matrix; wherein, the prediction sample matrix a predicted value.

Specifically, in the embodiment of the present application, the sample matrix may include a luma sample matrix and a chroma sample matrix, and accordingly, the predicted sample matrix determined by the encoder may include a luma predicted sample matrix and a chrominance predicted sample matrix, wherein, The luma prediction sample matrix includes the first luma predicted values of multiple sub-blocks, the chrominance prediction sample matrix includes the first chrominance predicted values of the multiple sub-blocks, and the first luma predicted value and the first chrominance predicted value constitute the first luma predicted value of the sub-block. Predictive value.

It should be noted that, in the embodiment of the application, the luminance sample matrix in the sample matrix may be a 1/16 precision luminance sample matrix, and the chrominance sample matrix in the sample matrix may be a 1/32 precision chrominance sample matrix.

It can be understood that, in the embodiment of the present application, for different prediction reference modes, the reference picture queue and reference index obtained by the encoder are different.

Further, in the embodiment of the present application, when the encoder determines the sample matrix, it may first obtain the luminance interpolation filter coefficients and the chrominance interpolation filter coefficients; then the luminance sample matrix may be determined based on the luminance interpolation filter coefficients, and at the same time, A matrix of chroma samples is determined based on the chroma interpolation filter coefficients.

Further, in the embodiment of the present application, when the encoder determines the first motion vector deviation between each pixel position in the sub-block and the sub-block, it can determine the secondary prediction parameter; if the secondary prediction parameter indicates to use Secondary prediction, the encoder can then determine the first motion vector offset between the sub-block and each pixel location based on the difference variable.

It can be understood that, in the embodiment of the present application, after the encoder determines the first motion vector deviation between the sub-block and each pixel position based on the difference variable, the encoder can use the corresponding data of all pixel positions in the sub-block. All the first motion vector deviations are used to construct a motion vector deviation matrix corresponding to the sub-block. It can be seen that the motion vector deviation matrix includes the motion vector deviation between the sub-block and any internal pixel point, that is, the first motion vector deviation.

Further, in the embodiment of the present application, if the secondary prediction parameter determined by the encoder indicates that secondary prediction is not used, then the encoder may choose to directly use the first sub-block of the current block obtained in the above step S403a. The predicted value is used as the second predicted value of the sub-block, and the processes of steps S404 and S405 described below are not performed.

Specifically, in the embodiment of the present application, if the secondary prediction parameter indicates that secondary prediction is not used, the encoder may determine the second prediction value by using the prediction sample matrix. The prediction sample matrix includes the first prediction values of multiple sub-blocks, and the encoder may determine the first prediction value of the sub-block where the pixel position is located as its own second prediction value.

Exemplarily, in this application, if the prediction reference mode of the current block takes a value of 0 or 1, that is, the first reference mode 'PRED_List0' or the second reference mode 'PRED_List1' is used, then it can be directly obtained from including 1 In the prediction sample matrix of the luma prediction sample matrix (two chrominance prediction sample matrices), the first prediction value of the sub-block where the pixel position is located is selected, and the first prediction value is determined as the inter frame prediction value of the pixel position, That is, the second predicted value.

Exemplarily, in this application, if the prediction reference mode of the current block is 2, that is, the third reference mode 'PRED_List01' is used, then the two luma prediction sample matrices (2 A total of 4 chrominance prediction sample matrices in the group) are averaged to obtain 1 averaged luminance prediction sample (2 averaged chrominance prediction samples), and finally from the averaged luminance prediction sample (2 averaged In the chroma prediction sample), the first prediction value of the sub-block where the pixel position is located is selected, and the first prediction value is determined as the inter-frame prediction value of the pixel position, that is, the second prediction value.

Step S404: Determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation.

In the embodiment of the present application, after the encoder determines the first prediction value of the sub-block and the first motion vector deviation between each pixel position in the sub-block and the sub-block based on the first motion vector, the encoder can The first motion vector offset is the center position corresponding to each pixel position and the second motion vector offset.

It should be noted that, in the embodiments of the present application, for the pixel at each pixel position in the sub-block, when performing secondary prediction or PROF processing, if the preset center position used by the filter is between the sub-block If the deviation of the motion vector is large, the obtained prediction result may have a large error. Therefore, after determining the first motion vector deviation between each pixel position and the sub-block, the encoder can Check whether the position is suitable as the center position of the filter used in the secondary prediction of a pixel or PROF processing. If it is not suitable, the center position can be adjusted again.

It can be understood that, in this application, the center position and the second motion vector deviation corresponding to a pixel position can be the center position and the motion vector deviation of the filter used when performing secondary prediction or PROF processing on the pixel position. . That is, the center position and the second motion vector deviation are used to perform secondary prediction or PROF processing on the pixel position.

Further, in the embodiment of the present application, the method for the encoder to determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation may include the following steps:

Step S404a, determine the horizontal deviation and vertical deviation of the first motion vector deviation;

Step S404b: Determine the center position and the second motion vector deviation according to the first absolute value of the horizontal deviation, the second absolute value of the vertical deviation, and a preset deviation threshold.

In the embodiment of the present application, the encoder may first determine the deviation values of the first motion vector deviation in different directions, that is, determine the horizontal deviation and the vertical deviation corresponding to the first motion vector deviation. Next, the encoder may further calculate the absolute value of the horizontal deviation, that is, the first absolute value, and the absolute value of the vertical deviation, that is, the second absolute value. Finally, the encoder can further determine the center position and the second motion vector deviation used in secondary prediction or PROF processing according to the first absolute value, the second absolute value and the preset deviation threshold.

It should be noted that, in the embodiment of the application, the encoder can compare the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, so that the center position and the first absolute value can be compared according to the comparison result. 2. Determination of motion vector bias. The preset deviation threshold may be preset and used to determine whether to adjust the deviation of the center position and the motion vector.

Further, in the embodiment of the present application, when the encoder determines the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the first absolute value and the second absolute value are both smaller than the preset deviation If the deviation threshold is set, the first motion vector deviation may be determined as the second motion vector deviation, and each pixel position may be determined as the center position.

It can be understood that, in the embodiment of the application, after the encoder compares the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, if the first absolute value and the second absolute value are If the values are less than the preset deviation, it can be considered that the vector deviation between the pixel position and the sub-block is small, and the prediction result obtained by using the pixel position as the center position of the filter for secondary prediction or PROF processing is more accurate. Therefore, The first motion vector deviation corresponding to the pixel position can be directly determined as the second motion vector deviation used for secondary prediction or PROF processing, and at the same time, the pixel position can be directly determined as the filtering for secondary prediction or PROF processing. the center position used by the device.

Further, in the embodiment of the present application, when the encoder determines the deviation between the center position and the second motion vector according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the first absolute value is greater than or equal to the preset deviation threshold, and If the second absolute value is smaller than the preset deviation threshold, the first adjustment direction may be determined according to the horizontal deviation; then, the deviation between the center position and the second motion vector may be determined according to the first adjustment direction.

It should be noted that, in the embodiment of the present application, the first adjustment direction is used to adjust the first motion vector in the horizontal direction. Therefore, the first adjustment direction includes left and right sides.

It can be understood that, in the embodiment of the application, after the encoder compares the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, if the first absolute value is greater than or equal to the preset deviation threshold If the deviation threshold is set, and the second absolute value is less than the preset deviation threshold, it can be considered that the motion vector deviation in the horizontal direction between the pixel position and the sub-block is relatively large. If the pixel is used as the center position of the filter for secondary prediction Or the prediction results obtained by PROF processing may have errors. Therefore, it is necessary to further determine the first adjustment direction according to the horizontal deviation, and then adjust according to the first adjustment direction, and finally determine the second prediction or PROF processing. Use the center position and the second motion vector offset.

Further, in the embodiment of the present application, when the encoder determines the deviation between the center position and the second motion vector according to the first adjustment direction, if the first adjustment direction is the left side, then the adjacent pixel position of the encoder can be adjusted. The left pixel position is taken as the center position, and then, (1, 0) can be added to the first motion vector deviation, so that the corresponding second motion vector deviation can be obtained.

Further, in the embodiment of the present application, the encoder determines the deviation between the center position and the second motion vector according to the first adjustment direction. If the first adjustment direction is the right side, then the adjacent right pixels of any pixel position can be adjusted. The position is taken as the center position, and then the first motion vector deviation can be subtracted by (1, 0), so that the second motion vector deviation can be obtained.

Further, in the embodiment of the present application, when the encoder determines the deviation between the center position and the second motion vector according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the second absolute value is greater than or equal to the preset deviation threshold, and If the first absolute value is smaller than the preset deviation threshold, the second adjustment direction may be determined according to the vertical deviation; then the center position and the second motion vector deviation may be determined according to the second adjustment direction.

It should be noted that, in the embodiment of the present application, the second adjustment direction is used to adjust the first motion vector in the vertical direction. Therefore, the second adjustment direction includes an upper side and a lower side.

It can be understood that, in the embodiment of the application, after the encoder compares the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, if the second absolute value is greater than or equal to the preset deviation threshold If the deviation threshold is set, and the first absolute value is smaller than the preset deviation threshold, it can be considered that the deviation of the motion vector in the vertical direction between the pixel position and the sub-block is relatively large. If the pixel is used as the center position of the filter for secondary prediction Or the prediction results obtained by PROF processing may have errors. Therefore, it is necessary to further determine the second adjustment direction according to the vertical deviation, and then adjust according to the second adjustment direction. Finally, it can be determined that the second adjustment direction is used for secondary prediction or PROF processing. Use the center position and the second motion vector offset.

Further, in the embodiment of the present application, when the encoder determines the deviation between the center position and the second motion vector according to the second adjustment direction, if the second adjustment direction is the upper side, then the adjacent position of any pixel can be adjusted. The upper pixel position is taken as the center position, and then (0, 1) can be added to the first motion vector deviation, so that the corresponding second motion vector deviation can be obtained.

Further, in the embodiment of the present application, the encoder determines the deviation between the center position and the second motion vector according to the second adjustment direction, and if the second adjustment direction is the lower side, then the adjacent lower side of any pixel position can be adjusted. The side pixel position is taken as the center position, and then, the first motion vector offset can be subtracted by (0, 1), so that the second motion vector offset can be obtained.

Further, in the embodiment of the present application, when the encoder determines the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the first absolute value and the second absolute value are both greater than or equal to If the deviation threshold is preset, the third adjustment direction may be determined according to the horizontal deviation and the vertical deviation; and then the center position and the deviation of the second motion vector may be determined according to the third adjustment direction.

It should be noted that, in the embodiment of the present application, the third adjustment direction is used to adjust the first motion vector in the horizontal direction and the vertical direction at the same time. Therefore, the third adjustment direction includes the upper right side, the lower right side, the upper left side side and lower left.

It can be understood that, in the embodiment of the application, after the encoder compares the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, if the first absolute value and the second absolute value are If the value is greater than or equal to the preset deviation threshold, it can be considered that the deviation of the motion vector between the pixel position and the sub-block in the horizontal and vertical directions is large. If the pixel is used as the center position of the filter for secondary prediction or There may be errors in the prediction results obtained by PROF processing. Therefore, it is necessary to further determine the third adjustment direction according to the horizontal deviation and vertical deviation, and then adjust according to the third adjustment direction. Finally, it can be determined to perform secondary prediction or PROF processing. The center position and the second motion vector offset used when .

Further, in the embodiment of the present application, when the encoder determines the deviation between the center position and the second motion vector according to the third adjustment direction, if the third adjustment direction is the upper left side, then the phase difference of any pixel position can be determined. The pixel position adjacent to the upper left side is taken as the center position, and then, (1, 1) can be added to the first motion vector deviation, so that the corresponding second motion vector deviation can be obtained.

Further, in the embodiment of the present application, the encoder determines the deviation between the center position and the second motion vector according to the third adjustment direction. If the third adjustment direction is the lower right side, then the adjacent pixel position of any pixel The pixel position on the lower right side is taken as the center position, and then, the first motion vector deviation can be subtracted by (1, 1), so that the second motion vector deviation can be obtained.

It can be understood that, in the embodiment of the present application, the deviation values of the second motion vector deviation in the horizontal direction and the vertical direction are both smaller than the preset deviation threshold.

That is, in the present application, after the encoder determines the deviation of the motion vector of each pixel position in the sub-block from the motion vector of the sub-block, and before filtering the prediction block based on the prediction of the sub-block, The center position and the second motion vector deviation may be determined first, wherein the center position and the second motion vector are used for secondary prediction or PROF processing.

Specifically, in this application, for each pixel position in a sub-block, the magnitude of the first motion vector deviation between it and the sub-block can be determined first, including the horizontal and vertical directions of the first motion vector deviation horizontal and vertical deviations. If the absolute value of the horizontal direction and/or the vertical direction of the motion vector deviation of a certain pixel position is greater than the preset deviation threshold, it can be determined that the center position of the filter corresponding to the pixel position is no longer the pixel position itself, and further The center position and the second motion vector offset are determined.

Further, in the embodiment of the present application, before performing secondary prediction or PROF processing on a pixel position in the sub-block, if it is determined that the first motion vector deviation between the pixel position and the sub-block is in the horizontal direction and \ Or the absolute value of the deviation component in the vertical direction is greater than or equal to the preset deviation threshold (such as 1/2 pixel, 3/4 pixel or 1 pixel), then the starting point used for secondary prediction or PROF processing needs to be adjusted. , that is, the center position of the two-dimensional filter used in secondary prediction or PROF processing is adjusted.

It can be understood that, in this application, when adjusting the center position and the motion vector deviation, it can be selected in the current block that the motion vector deviation from the pixel position in the horizontal direction and the vertical direction is less than or equal to the preset deviation. Another pixel location for the threshold.

In the embodiment of the present application, further, before determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, that is, before step S404, the method for performing inter-frame prediction by the encoder may also be Include the following steps:

Step 406: Limit the deviation of the first motion vector according to a preset deviation range, wherein the preset deviation range includes a deviation lower limit value and a deviation upper limit value.

In the embodiment of the present application, before determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, the encoder may first limit the first motion vector. Specifically, the encoder may limit the deviation of the first motion vector according to a preset deviation range, wherein the preset deviation range includes a deviation lower limit value and a deviation upper limit value.

Further, in the embodiment of the present application, when the encoder performs limiting processing on the first motion vector deviation according to the preset deviation range, if the horizontal deviation and\or the vertical deviation are less than the deviation lower limit value, then the horizontal deviation and\ Or vertical deviation is set as the lower deviation value; if the horizontal deviation and\or vertical deviation is greater than the upper deviation value, then the horizontal deviation and\or vertical deviation can be set as the upper deviation value.

It should be noted that, in this application, the encoder may not limit the first motion vector deviation, that is, without performing step 407, but directly adjust the center position to any suitable pixel position according to the first motion vector deviation. It is also possible to first perform the limitation processing on the first motion vector deviation according to step 407, or to limit the adjustable range and the motion vector deviation.

Further, in the embodiment of the present application, the preset deviation range may be composed of a lower deviation limit min and an upper deviation upper limit max, that is, the preset deviation range may be expressed as (min, max), and when the first motion vector When the deviation is limited, if the deviation value of the horizontal direction and/or vertical direction of the first motion vector deviation is less than the deviation lower limit value min, then the deviation value can be directly set as min; if the horizontal direction of the first motion vector deviation And \ or the deviation value in the vertical direction is greater than max, then you can directly set the deviation value to max.

In the embodiment of the present application, further, after determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, that is, after step S404, the method for performing inter-frame prediction by the encoder may also Include the following steps:

Step 407: If the center position does not belong to the current block, re-determine the center position according to the pixel position in the current block.

In the embodiment of the present application, after determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, the encoder may first determine whether the center position exceeds the current block, if the center position If it exceeds the range of the current block, that is, the center position does not belong to the current block, the encoder needs to re-determine the center position according to the pixel position in the current block.

In the embodiment of the present application, further, after determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, that is, after step S404, the method for performing inter-frame prediction by the encoder may also Include the following steps:

Step 408: If the center position does not belong to the current block, directly determine each pixel position as the center position corresponding to each pixel position.

In the embodiment of the present application, after determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, the encoder may first determine whether the center position exceeds the current block, if the center position If it exceeds the range of the current block, that is, the center position does not belong to the current block, the encoder can directly determine the corresponding, original pixel position as the center position. That is, if the center position determined based on the first motion vector deviation corresponding to each pixel position in the sub-block is beyond the range of the current block, the encoder may choose to still use each pixel position as the center position.

That is to say, in the embodiment of the present application, for the starting position of the adjusted secondary prediction, that is, the center position used by the filter determined based on the first motion vector deviation, the encoder can limit the center position to no more than The range of the current block, that is, will not exceed the range of the starting positions of all pixel positions in the current block before the adjustment of the secondary prediction.

Step S405 , based on the first predicted value, perform secondary prediction or PROF processing according to the center position and the second motion vector deviation, determine the second predicted value of the subblock, and determine the second predicted value as the inter-frame predicted value of the subblock.

In the embodiment of the present application, after the encoder determines the center position and the second motion vector deviation corresponding to each pixel position according to the first motion vector deviation, the encoder can determine the center position and the second motion vector deviation based on the first predicted value The deviation is subjected to secondary prediction or PROF processing, so that the predicted value of the pixel at each pixel position can be obtained.

Further, in the embodiment of the present application, after the encoder traverses each pixel position in the sub-block and obtains the predicted value of the pixel at each pixel position, it can determine the predicted value according to the predicted value of the pixel at each pixel position. the second predicted value of the sub-block, so that the second predicted value can be determined as the inter-frame predicted value of the sub-block.

It should be noted that, in the embodiments of the present application, since the center position and the second motion vector deviation can be used to perform secondary prediction or PROF processing on the pixel points of each pixel position in the sub-block, therefore, after obtaining the center After the position and the second motion vector deviation, the encoder can use the center position and the second motion vector deviation to perform secondary prediction or PROF processing on the pixels of each pixel position based on the first predicted value, and finally obtain the corresponding sub-block. the second predicted value, so that the second predicted value can be determined as the inter-frame predicted value of the sub-block.

Further, in the embodiment of the present application, the encoder performs secondary prediction or PROF processing according to the center position and the second motion vector deviation based on the first predicted value, determines the second predicted value of the sub-block, and uses the second predicted value The method of determining an inter-predicted value for a sub-block may include the following steps:

Step S405a, determine PROF parameters;

Step S405b, when the PROF parameter indicates that PROF processing is performed, the pixel horizontal gradient and the pixel vertical gradient corresponding to the center position are determined based on the first predicted value;

Step S405c, according to the pixel horizontal gradient, the pixel vertical gradient and the second motion vector deviation, calculate the deviation value corresponding to each pixel position;

Step S405d, based on the first predicted value and the deviation value, obtain the predicted value of the pixel at each pixel position;

Step S405e, using the predicted value of the pixel at each pixel position to determine the second predicted value.

In the embodiment of the present application, the encoder may first determine the PROF parameter, and if the PROF parameter indicates that PROF processing is performed, the encoder may determine the pixel horizontal gradient and the pixel vertical gradient corresponding to the center position based on the first predicted value; The gradient is the gradient value between the pixel value corresponding to the center position and the pixel value corresponding to the adjacent pixel position in the horizontal direction; the vertical gradient of the pixel is the pixel value corresponding to the center position and the adjacent pixel position in the vertical direction. The gradient value between corresponding pixel values.

Further, in the embodiment of the present application, the encoder can calculate and obtain the deviation value corresponding to each pixel position according to the pixel horizontal gradient, the pixel vertical gradient and the second motion vector deviation of the center position corresponding to each pixel position. Wherein, the deviation value can be used to correct the predicted value of the pixel value of each pixel position.

It should be noted that, in the embodiment of the present application, the encoder can further obtain the corrected predicted value corresponding to any pixel position according to the first predicted value and the deviation value, and traverse each pixel in the current sub-block After obtaining the modified predicted value corresponding to each pixel position, the second predicted value corresponding to the current sub-block is determined by using the modified predicted value corresponding to all pixel positions, thereby determining the corresponding inter-frame predicted value. Specifically, in this application, after the sub-block-based prediction is completed, the first prediction value of the current sub-block is used as the prediction value of each pixel position, and then the deviation corresponding to the first prediction value and each pixel position is used The value is added, that is, the correction processing of the predicted value of each pixel position can be completed, and the corrected predicted value can be obtained, so that the second predicted value of the current sub-block can be further obtained, and the second predicted value can be used as the corresponding value of the current sub-block. Interframe prediction value.

Further, in the embodiment of the present application, the encoder performs secondary prediction or PROF processing according to the center position and the second motion vector deviation based on the first predicted value, determines the second predicted value of the sub-block, and uses the second predicted value The method of determining an inter-predicted value for a sub-block may include the following steps:

Step S405f, determine secondary prediction parameters;

Step S405g, when the secondary prediction parameter indicates the use of secondary prediction, determine the filter coefficient of the two-dimensional filter according to the second motion vector deviation; wherein, the two-dimensional filter is used to perform secondary prediction processing according to a preset shape;

Step S405h, based on the filter coefficient and the first predicted value, determine the predicted value of the pixel at each pixel position;

Step S405i, using the predicted value of the pixel at each pixel position to determine the second predicted value.

In the embodiment of the present application, the encoder may first determine the secondary prediction parameter, and if the secondary prediction parameter indicates to use the secondary prediction, the encoder may determine the filter coefficient of the two-dimensional filter according to the second motion vector deviation; wherein, The two-dimensional filter is used for secondary prediction processing according to the preset shape.

It should be noted that, in the embodiment of the present application, the filter coefficient of the two-dimensional filter is related to the deviation of the second motion vector corresponding to the target pixel position. That is to say, for different target pixel positions, if the corresponding second motion vector deviations are different, the filter coefficients of the used two-dimensional filters are also different.

It can be understood that, in the embodiment of the present application, a two-dimensional filter is used to perform secondary prediction by using a plurality of adjacent pixel positions that form a preset shape. The preset shape is a rectangle, a rhombus or any symmetrical shape.

That is to say, in the present application, the two-dimensional filter used for secondary prediction is a filter formed by adjacent points forming a preset shape. The adjacent dots constituting the preset shape may include a plurality of dots, for example, composed of 9 dots. The predetermined shape may be a symmetrical shape, for example, the predetermined shape may include a rectangle, a rhombus, or any other symmetrical shape.

Exemplarily, in this application, the two-dimensional filter is a rectangular filter, and specifically, the two-dimensional filter is a filter composed of 9 adjacent pixel positions forming a rectangle. Among the 9 pixel positions, the pixel position located in the center is the pixel position of the pixel currently requiring secondary prediction, that is, the current pixel position.

Further, in the embodiment of the present application, when the encoder determines the filter coefficient of the two-dimensional filter according to the second motion vector deviation, it can first determine the scale parameter, and then can determine the pixel according to the scale parameter and the second motion vector deviation. The filter coefficients corresponding to the positions.

It should be noted that, in the embodiment of the present application, the scale parameter may include at least one scale value, and the second motion vector deviation includes a horizontal deviation and a vertical deviation; wherein, at least one scale value is a non-zero real number.

Specifically, in this application, when the two-dimensional filter uses 9 adjacent pixel positions that form a rectangle to perform secondary prediction, the pixel position located in the center of the rectangle is the position to be predicted, that is, the current pixel position, and the other 8 The target pixel positions are sequentially located in the eight directions of the upper left, upper, upper right, right, lower right, lower, lower left, and left of the current pixel position.

Correspondingly, in the present application, the encoder can calculate and obtain 9 filter coefficients corresponding to 9 adjacent pixel positions based on at least one scale value and the deviation of the second motion vector of the position to be predicted according to a preset calculation rule.

It should be noted that, in this application, the preset calculation rule may include various calculation methods, such as addition operation, subtraction operation, multiplication operation, and the like. Among them, for different pixel positions, different calculation methods can be used to calculate the filter coefficients.

It can be understood that, in this application, when the encoder calculates and obtains multiple filter coefficients corresponding to multiple pixel positions according to different calculation methods in the preset calculation rules, some filter coefficients may be the difference of the second motion vector deviation. The linear function, that is, the two are in a linear relationship, may also be a quadratic function or a higher-order function of the deviation of the second motion vector, that is, the two are in a non-linear relationship.

That is to say, in the present application, any one of the multiple filter coefficients corresponding to multiple adjacent pixel positions may be a linear function, a quadratic function, or a higher-order function of the second motion vector deviation.

It can be understood that, in this application, the encoder can write prediction mode parameters, affine mode parameters, and prediction reference modes into the bitstream. PROF parameters, secondary prediction parameters can also be written to the bitstream.

This embodiment provides an inter-frame prediction method. After prediction based on sub-blocks, for pixel positions with a large deviation of the first motion vector between the motion vector and the motion vector of the sub-block, the deviation of the first motion vector can be determined based on the first motion vector deviation. Redetermine the determination of the center position and the second motion vector offset used for secondary prediction or PROF processing, so that the center position and the second motion vector offset can be used for point-based point-based The secondary prediction of , obtains the second prediction value. It can be seen that the inter-frame prediction method proposed in the present application can be well applied to all scenarios, can reduce the prediction error, greatly improve the coding performance, and thus improve the coding efficiency.

Based on the above embodiment, in yet another embodiment of the present application, FIG. 20 is a schematic diagram of the composition and structure of a decoder. As shown in FIG. 20 , the decoder 300 proposed in this embodiment of the present application may include a parsing unit 301 and a first determining unit 302;

The parsing unit 301 is used for parsing the bit stream to obtain the prediction mode parameter of the current block;

The first determining unit 302 is configured to determine the first motion vector of the sub-block of the current block when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode; wherein, The current block includes a plurality of sub-blocks; a first predictor of the sub-block is determined based on the first motion vector, and a first motion vector between each pixel position in the sub-block and the sub-block deviation; determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation; based on the first predicted value, perform according to the center position and the second motion vector deviation Secondary prediction or PROF processing, determining a second prediction value of the sub-block, and determining the second prediction value as an inter-frame prediction value of the sub-block.

FIG. 21 is a second schematic diagram of the structure of the decoder. As shown in FIG. 21 , the decoder 300 proposed in this embodiment of the present application may further include a first processor 303 and a first memory storing instructions executable by the first processor 303 304 , a first communication interface 305 , and a first bus bar 306 for connecting the first processor 303 , the first memory 304 and the first communication interface 305 .

Further, in the embodiment of the present application, the above-mentioned first processor 303 is configured to parse the bit stream to obtain the prediction mode parameter of the current block; when the prediction mode parameter indicates that the current block is determined using the inter prediction mode when the inter-frame prediction value of and the first motion vector deviation between each pixel position in the sub-block and the sub-block; determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation Based on the first predicted value, carry out secondary prediction or PROF processing according to the center position and the second motion vector deviation, determine the second predicted value of the sub-block, and determine the second predicted value as the inter-predicted value of the sub-block.

FIG. 22 is a schematic diagram 1 of the composition structure of the encoder. As shown in FIG. 22 , the encoder 400 proposed in this embodiment of the present application may include a second determining unit 401;

The second determining unit 401 is configured to determine a prediction mode parameter of the current block; when the prediction mode parameter indicates that an inter prediction value of the current block is determined using an inter prediction mode, determine a subblock of the current block The first motion vector of the first motion vector deviation between blocks; determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation; based on the first predicted value, according to the center position and The second motion vector deviation is subjected to secondary prediction or PROF processing, a second prediction value of the sub-block is determined, and the second prediction value is determined as an inter-frame prediction value of the sub-block.

FIG. 23 is a second schematic diagram of the composition and structure of the encoder. As shown in FIG. 23 , the encoder 400 proposed in this embodiment of the present application may further include a second processor 402 and a second memory storing instructions executable by the second processor 402 403 , a second communication interface 404 , and a second bus bar 405 for connecting the second processor 402 , the second memory 403 and the second communication interface 404 .

Further, in the embodiment of the present application, the above-mentioned second processor 402 is configured to determine the prediction mode parameter of the current block; when the prediction mode parameter indicates that the inter prediction mode is used to determine the inter prediction value of the current block When , determine the first motion vector of the sub-block of the current block; wherein, the current block includes a plurality of sub-blocks; determine the first predicted value of the sub-block based on the first motion vector, and the sub-block The first motion vector deviation between each pixel position in the sub-block and the sub-block; the center position corresponding to each pixel position and the second motion vector deviation are determined according to the first motion vector deviation; a predicted value, perform secondary prediction or PROF processing according to the center position and the deviation of the second motion vector, determine the second predicted value of the sub-block, and determine the second predicted value as the second predicted value of the sub-block Interframe prediction value.

The embodiments of the present application provide a decoder and an encoder, which can, after prediction based on a sub-block, for a pixel position with a large deviation of the first motion vector between the motion vector and the motion vector of the sub-block, based on the first motion The vector deviation re-determines the determination of the center position and the second motion vector deviation used for secondary prediction or PROF processing, so that the center position and the second motion vector deviation can be used on the basis of the first prediction value based on the sub-block. Based on the secondary prediction of the points, a second predicted value is obtained. It can be seen that the inter-frame prediction method proposed in the present application can be well applied to all scenarios, can reduce the prediction error, greatly improve the coding performance, and thus improve the coding efficiency.

Embodiments of the present application provide a computer-readable storage medium and a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, the method described in the above-mentioned embodiments is implemented.

Specifically, a program instruction corresponding to an inter-frame prediction method in this embodiment may be stored on a storage medium such as an optical disc, a hard disk, a pen drive, etc. When the program instruction corresponding to an inter-frame prediction method in the storage medium is When an electronic device reads or is executed, it includes the following steps:

Parse the bit stream to obtain the prediction mode parameters of the current block;

When the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, a first motion vector of a subblock of the current block is determined; wherein the current block includes a plurality of subblocks;

determining a first predicted value for the subblock based on the first motion vector, and a first motion vector offset between each pixel position in the subblock and the subblock;

Determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation;

Based on the first predicted value, secondary prediction or PROF processing is performed according to the center position and the deviation of the second motion vector, the second predicted value of the sub-block is determined, and the second predicted value is determined as the Inter-predicted value of the sub-block.

Specifically, a program instruction corresponding to an inter-frame prediction method in this embodiment may be stored on a storage medium such as an optical disc, a hard disk, a pen drive, etc. When the program instruction corresponding to an inter-frame prediction method in the storage medium is When an electronic device reads or is executed, it includes the following steps:

determine the prediction mode parameter of the current block;

When the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, a first motion vector of a subblock of the current block is determined; wherein the current block includes a plurality of subblocks;

determining a first predicted value for the subblock based on the first motion vector, and a first motion vector offset between each pixel position in the subblock and the subblock;

Determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation;

Based on the first predicted value, secondary prediction or PROF processing is performed according to the center position and the deviation of the second motion vector, the second predicted value of the sub-block is determined, and the second predicted value is determined as the Inter-predicted value of the sub-block.

It should be apparent to those skilled in the art that the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk memory, optical memory, and the like) having computer-usable program code embodied therein.

The present application is described with reference to schematic flowcharts and/or block diagrams of implementations of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It will be understood that each process and/or block in the schematic flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the schematic flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device generate a Means for implementing the functions specified in a process or processes and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in computer-readable memory capable of directing a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, The instruction means implements the functions specified in the flow or blocks of the implementation flow diagram and/or the block or blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, thereby executing a computer-implemented process on the computer or other programmable device. The instructions provide steps for implementing the functions specified in the flow or blocks of the implementation flow diagram and/or the block or blocks of the block diagram.

The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined under the condition of no conflict to obtain new method embodiments.

The features disclosed in the several product embodiments provided in this application can be combined arbitrarily without conflict to obtain a new product embodiment.

The features disclosed in several method or device embodiments provided in this application can be combined arbitrarily without conflict to obtain new method embodiments or device embodiments.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claimed item.

11: Video coding system 110: Decoding image temporary storage unit 111: Transform unit 112: Quantization unit 113: Mode selection and coding control logic unit 114: Intra prediction unit 115: Inter prediction unit 116: Inverse quantization unit 117: Inverse transformation unit 118: Loop filter unit 119: coding unit 12: Video decoding system 121: decoding unit 122: Inverse quantization unit 123: Intra prediction unit 124: Motion compensation unit 125: Loop filter unit 126: Decoding image temporary storage unit 127: Inverse transform unit 300: decoder 301: parsing unit 302: The first determination unit 303: first processor 304: first memory 305: The first communication interface 306: First busbar 400: Encoder 401: Second determination unit 402: second processor 403: second memory 404: Second communication interface 405: Second busbar S301~S308: Steps S401~S405: Steps

Fig. 1 is the schematic diagram 1 of the affine model;

Fig. 2 is the schematic diagram two of affine model;

Fig. 3 is the interpolation schematic diagram of the pixel;

4 is a schematic diagram 1 of sub-block interpolation;

5 is a schematic diagram 2 of sub-block interpolation;

Fig. 6 is the motion vector schematic diagram of each sub-block;

Figure 7 is a schematic diagram of the sample location;

Fig. 8 is the schematic diagram one of the center position;

Fig. 9 is the schematic diagram two of the center position;

FIG. 10 is a schematic block diagram of the composition of an image coding system according to an embodiment of the present application;

FIG. 11 is a schematic block diagram of the composition of an image decoding system according to an embodiment of the present application;

FIG. 12 is a schematic diagram 1 of the implementation flow of the inter-frame prediction method;

Figure 13 is a schematic diagram three of the center position;

14 is a schematic diagram 2 of the implementation flow of the inter-frame prediction method;

15 is a schematic diagram 3 of the implementation flow of the inter-frame prediction method;

FIG. 16 is a schematic diagram 4 of the implementation flow of the inter-frame prediction method;

17 is a schematic diagram two of a two-dimensional filter;

18 is a schematic diagram 3 of the implementation flow of the inter-frame prediction method;

FIG. 19 is a schematic diagram four of the implementation flow of the inter-frame prediction method;

Figure 20 is a schematic diagram of the composition structure of a decoder;

Figure 21 is a schematic diagram of the composition structure of the decoder 2;

Figure 22 is a schematic diagram of the composition structure of the encoder one;

FIG. 23 is a second schematic diagram of the composition and structure of the encoder.

S301~S305: Steps

Claims (65)

An inter-frame prediction method, applied to a decoder, the method comprising: Parse the bit stream to obtain the prediction mode parameters of the current block; When the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, a first motion vector of a subblock of the current block is determined; wherein the current block includes a plurality of subblocks; determining a first predicted value for the subblock based on the first motion vector, and a first motion vector offset between each pixel position in the subblock and the subblock; Determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation; Based on the first predicted value, perform secondary prediction according to the deviation of the center position and the second motion vector, or perform prediction correction PROF processing using the principle of optical flow, determine the second predicted value of the sub-block, and use the The second predicted value is determined to be the inter predicted value of the sub-block. The method according to claim 1, wherein the determining the first motion vector of the sub-block of the current block comprises: Parsing the bit stream to obtain the affine mode parameter and the prediction reference mode of the current block; When the affine mode parameter indicates to use the affine mode, determining a control point mode and a sub-block size parameter; The first motion vector is determined according to the prediction reference mode, the control point mode, and the subblock size parameter. The method according to claim 2, wherein the determining the first motion vector according to the prediction reference mode, the control point mode and the sub-block size parameter includes: determining a control point motion vector group according to the prediction reference mode; The first motion vector is determined according to the control point motion vector group, the control point pattern and the sub-block size parameter. The method according to claim 3, wherein the determining the first motion vector according to the control point motion vector group, the control point mode and the sub-block size parameter includes: determining a difference variable according to the control point motion vector group, the control point mode and the size parameter of the current block; determining a sub-block position based on the prediction mode parameter and the sub-block size parameter; Using the difference variable and the sub-block position, the first motion vector of the current sub-block is determined. The method according to any one of claim 2 to 4, further comprising: Traverse each sub-block of the current block, and construct a motion vector set according to the first motion vector of each sub-block. The method according to claim 5, wherein the determining the first motion vector according to the control point motion vector group, the control point mode and the sub-block size parameter includes: determining a difference variable according to the control point motion vector group, the control point mode and the size parameter of the current block; determining a sub-block position based on the prediction mode parameter and the sub-block size parameter; Using the difference variable and the sub-block position, the first motion vector of the sub-block is determined. The method according to claim 5, wherein the determining the first predicted value of the sub-block based on the first motion vector comprises: determining a sample matrix; wherein the sample matrix includes a luminance sample matrix and a chrominance sample matrix; The first predictor is determined from the prediction reference mode, the subblock size parameter, the sample matrix, and the set of motion vectors. The method according to claim 7, wherein the determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation comprises: determining a horizontal offset and a vertical offset of the first motion vector offset; The center position and the second motion vector deviation are determined according to the first absolute value of the horizontal deviation, the second absolute value of the vertical deviation, and a preset deviation threshold. The method according to claim 8, wherein the determining the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and a preset deviation threshold comprises: If both the first absolute value and the second absolute value are smaller than the preset deviation threshold, the first motion vector deviation is determined as the second motion vector deviation, and the position of each pixel is determined for the central location. The method according to claim 8, wherein the determining the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and a preset deviation threshold comprises: If the first absolute value is greater than or equal to the preset deviation threshold, and the second absolute value is less than the preset deviation threshold, a first adjustment direction is determined according to the horizontal deviation; wherein the first adjustment direction is The adjustment direction includes left and right; According to the first adjustment direction, the deviation of the center position and the second motion vector is determined. The method according to claim 10, wherein the determining the deviation between the center position and the second motion vector according to the first adjustment direction comprises: If the first adjustment direction is the left side, the adjacent left pixel position of any pixel position is used as the center position; The second motion vector offset is obtained by adding (1, 0) to the first motion vector offset. The method according to claim 10, wherein the determining the deviation between the center position and the second motion vector according to the first adjustment direction comprises: If the first adjustment direction is the right side, the adjacent right pixel position of any pixel position is used as the center position; The second motion vector offset is obtained by subtracting (1, 0) from the first motion vector offset. The method according to claim 8, wherein the determining the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and a preset deviation threshold comprises: If the second absolute value is greater than or equal to the preset deviation threshold, and the first absolute value is less than the preset deviation threshold, a second adjustment direction is determined according to the vertical deviation; wherein the second The adjustment direction includes the upper side and the lower side; According to the second adjustment direction, the center position and the second motion vector deviation are determined. The method according to claim 13, wherein the determining the deviation between the center position and the second motion vector according to the second adjustment direction comprises: If the second adjustment direction is the upper side, the adjacent upper pixel position of any pixel position is used as the center position; Add (0, 1) to the first motion vector offset to obtain the second motion vector offset. The method according to claim 13, wherein the determining the deviation between the center position and the second motion vector according to the second adjustment direction comprises: If the second adjustment direction is the lower side, use the adjacent lower pixel position of any pixel position as the center position; The second motion vector offset is obtained by subtracting (0, 1) from the first motion vector offset. The method according to claim 8, wherein the determining the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and a preset deviation threshold comprises: If both the first absolute value and the second absolute value are greater than or equal to the preset deviation threshold, a third adjustment direction is determined according to the horizontal deviation and the vertical deviation; wherein, the third adjustment direction Including the upper right side, the lower right side, the upper left side and the lower left side; According to the third adjustment direction, the center position and the second motion vector deviation are determined. The method according to any one of claims 8 to 16, wherein the preset deviation threshold is k pixels; wherein k is greater than 0.5 and less than or equal to 1. The method according to claim 8, wherein, before determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation, the method further comprises: The first motion vector deviation is limited according to a preset deviation range; wherein, the preset deviation range includes a deviation lower limit value and a deviation upper limit value. The method according to claim 18, wherein the performing limiting processing on the first motion vector deviation according to a preset deviation range includes: If the horizontal deviation and\or the vertical deviation are less than the deviation lower limit value, then the horizontal deviation and\or the vertical deviation are set as the deviation lower limit value; If the horizontal deviation and\or the vertical deviation are greater than the deviation upper limit value, set the horizontal deviation and\or the vertical deviation as the deviation upper limit value. The method according to claim 1, wherein after the center position corresponding to each pixel position and the second motion vector deviation are determined according to the first motion vector deviation, the method further includes: If the center position does not belong to the current block, the center position is re-determined according to the pixel positions in the current block. The method according to claim 1, wherein after the center position corresponding to each pixel position and the second motion vector deviation are determined according to the first motion vector deviation, the method further includes: If the center position does not belong to the current block, the position of each pixel is directly determined as the center position corresponding to the position of each pixel. The method according to claim 1, wherein the second prediction or PROF processing is performed according to the center position and the second motion vector deviation based on the first prediction value, and the second prediction value of the sub-block is determined. The predicted value, the second predicted value is determined as the inter-frame predicted value of the sub-block, including: Parse the bit stream to obtain PROF parameters; When the PROF parameter indicates that PROF processing is performed, the pixel horizontal gradient and the pixel vertical gradient corresponding to the center position are determined based on the first predicted value; Calculate the deviation value corresponding to each pixel position according to the pixel horizontal gradient, the pixel vertical gradient and the second motion vector deviation; Based on the first predicted value and the deviation value, obtain the predicted value of the pixel at each pixel position; The second predicted value is determined using the predicted value of the pixel at each pixel location. The method according to claim 1, wherein the second prediction or PROF processing is performed according to the center position and the second motion vector deviation based on the first prediction value, and the second prediction value of the sub-block is determined. The predicted value, the second predicted value is determined as the inter-frame predicted value of the sub-block, including: Parse the bit stream to obtain secondary prediction parameters; When the secondary prediction parameter indicates the use of secondary prediction, the filter coefficient of the two-dimensional filter is determined according to the second motion vector deviation; wherein, the two-dimensional filter is used to perform secondary prediction processing according to a preset shape; based on the filter coefficient and the first predicted value, determining a predicted value of the pixel at each pixel position; The second predicted value is determined using the predicted value of the pixel at each pixel location. The method of claim 23, wherein the two-dimensional filter is used to perform secondary prediction using a plurality of adjacent pixel positions constituting the preset shape. The method according to claim 24, wherein the preset shape is a rectangle, a rhombus, or any symmetrical shape. The method of claim 25, wherein, If the value of the affine mode parameter is 1, it indicates that the affine mode is used; If the value of the affine mode parameter is 0, or the affine mode parameter is not obtained through analysis, it indicates that the affine mode is not used. The method according to claim 2, wherein the determining the sub-block size parameter includes: parsing the bit stream to obtain a sub-block size flag; If the value of the sub-block size flag is 1, determine that the sub-block size parameter is 8×8; If the value of the sub-block size flag is 0, or the sub-block size flag is not obtained by parsing, the sub-block size parameter is determined to be 4×4. The method of claim 2, wherein the control point mode includes a 4-parameter mode and a 6-parameter mode. An inter-frame prediction method, applied to an encoder, the method comprising: determine the prediction mode parameter of the current block; When the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, a first motion vector of a subblock of the current block is determined; wherein the current block includes a plurality of subblocks; determining a first predicted value for the subblock based on the first motion vector, and a first motion vector offset between each pixel position in the subblock and the subblock; Determine the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation; Based on the first predicted value, secondary prediction or PROF processing is performed according to the center position and the deviation of the second motion vector, the second predicted value of the sub-block is determined, and the second predicted value is determined as the Inter-predicted value of the sub-block. The method according to claim 29, wherein the determining the prediction mode parameter of the current block comprises: Use multiple prediction modes to perform precoding processing on the current block to obtain a rate-distortion cost value corresponding to each prediction mode; A minimum rate-distortion cost value is selected from the obtained multiple rate-distortion cost values, and a prediction mode parameter of the current block is determined according to the prediction mode corresponding to the minimum rate-distortion cost value. The method according to claim 29, wherein the determining the first motion vector of the sub-block of the current block comprises: determining an affine mode parameter and a prediction reference mode for the current block; When the affine mode parameter indicates to use the affine mode, determining a control point mode and a sub-block size parameter; The first motion vector is determined according to the prediction reference mode, the control point mode, and the subblock size parameter. The method according to claim 31, wherein the determining the first motion vector according to the prediction reference mode, the control point mode and the sub-block size parameter comprises: determining a control point motion vector group according to the prediction reference mode; The first motion vector is determined according to the control point motion vector group, the control point pattern and the sub-block size parameter. The method according to claim 32, wherein the determining the first motion vector according to the control point motion vector group, the control point mode and the sub-block size parameter comprises: determining a difference variable according to the control point motion vector group, the control point mode and the size parameter of the current block; determining a sub-block position based on the prediction mode parameter and the sub-block size parameter; Using the difference variable and the sub-block position, the first motion vector of the current sub-block is determined. The method according to any one of claims 31 to 33, further comprising: Traverse each sub-block of the current block, and construct a motion vector set according to the first motion vector of each sub-block. The method according to claim 34, wherein the determining the first motion vector according to the control point motion vector group, the control point mode and the sub-block size parameter comprises: determining a difference variable according to the control point motion vector group, the control point mode and the size parameter of the current block; determining a sub-block position based on the prediction mode parameter and the sub-block size parameter; Using the difference variable and the sub-block position, the first motion vector of the sub-block is determined. The method of claim 34, wherein the determining the first predicted value of the sub-block based on the first motion vector comprises: determining a sample matrix; wherein the sample matrix includes a luminance sample matrix and a chrominance sample matrix; The first predictor is determined from the prediction reference mode, the subblock size parameter, the sample matrix, and the set of motion vectors. The method according to claim 36, wherein the determining the center position corresponding to each pixel position and the second motion vector deviation according to the first motion vector deviation comprises: determining a horizontal offset and a vertical offset of the first motion vector offset; The center position and the second motion vector deviation are determined according to the first absolute value of the horizontal deviation, the second absolute value of the vertical deviation, and a preset deviation threshold. The method according to claim 37, wherein the determining the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and a preset deviation threshold comprises: If both the first absolute value and the second absolute value are smaller than the preset deviation threshold, the first motion vector deviation is determined as the second motion vector deviation, and the position of each pixel is determined for the central location. The method according to claim 37, wherein the determining the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and a preset deviation threshold comprises: If the first absolute value is greater than or equal to the preset deviation threshold, and the second absolute value is less than the preset deviation threshold, a first adjustment direction is determined according to the horizontal deviation; wherein the first adjustment direction is The adjustment direction includes left and right; According to the first adjustment direction, the deviation of the center position and the second motion vector is determined. The method according to claim 39, wherein the determining the deviation between the center position and the second motion vector according to the first adjustment direction comprises: If the first adjustment direction is the left side, the adjacent left pixel position of any pixel position is used as the center position; The second motion vector offset is obtained by adding (1, 0) to the first motion vector offset. The method according to claim 39, wherein the determining the deviation between the center position and the second motion vector according to the first adjustment direction comprises: If the first adjustment direction is the right side, the adjacent right pixel position of any pixel position is used as the center position; The second motion vector offset is obtained by subtracting (1, 0) from the first motion vector offset. The method according to claim 37, wherein the determining the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and a preset deviation threshold comprises: If the second absolute value is greater than or equal to the preset deviation threshold, and the first absolute value is less than the preset deviation threshold, a second adjustment direction is determined according to the vertical deviation; wherein the second The adjustment direction includes the upper side and the lower side; According to the second adjustment direction, the center position and the second motion vector deviation are determined. The method according to claim 42, wherein the determining the deviation between the center position and the second motion vector according to the second adjustment direction comprises: If the second adjustment direction is the upper side, the adjacent upper pixel position of any pixel position is used as the center position; Add (0, 1) to the first motion vector offset to obtain the second motion vector offset. The method according to claim 42, wherein the determining the deviation between the center position and the second motion vector according to the second adjustment direction comprises: If the second adjustment direction is the lower side, use the adjacent lower pixel position of any pixel position as the center position; The second motion vector offset is obtained by subtracting (0, 1) from the first motion vector offset. The method according to claim 37, wherein the determining the center position and the second motion vector deviation according to the horizontal deviation, the vertical deviation and a preset deviation threshold comprises: If both the first absolute value and the second absolute value are greater than or equal to the preset deviation threshold, a third adjustment direction is determined according to the horizontal deviation and the vertical deviation; wherein, the third adjustment direction Including the upper right side, the lower right side, the upper left side and the lower left side; According to the third adjustment direction, the center position and the second motion vector deviation are determined. The method according to any one of claims 37 to 45, wherein the preset deviation threshold is k pixels; wherein k is greater than 0.5 and less than or equal to 1. The method according to claim 37, wherein, before the center position corresponding to each pixel position and the second motion vector deviation are determined according to the first motion vector deviation, the method further comprises: The first motion vector deviation is limited according to a preset deviation range; wherein, the preset deviation range includes a deviation lower limit value and a deviation upper limit value. The method according to claim 47, wherein the performing limiting processing on the deviation of the first motion vector according to a preset deviation range includes: If the horizontal deviation and\or the vertical deviation are less than the deviation lower limit value, then the horizontal deviation and\or the vertical deviation are set as the deviation lower limit value; If the horizontal deviation and\or the vertical deviation are greater than the deviation upper limit value, set the horizontal deviation and\or the vertical deviation as the deviation upper limit value. The method according to claim 29, wherein after the center position corresponding to each pixel position and the second motion vector deviation are determined according to the first motion vector deviation, the method further comprises: If the center position does not belong to the current block, the center position is re-determined according to the pixel positions in the current block. The method according to claim 29, wherein after the center position corresponding to each pixel position and the second motion vector deviation are determined according to the first motion vector deviation, the method further comprises: If the center position does not belong to the current block, the position of each pixel is directly determined as the center position corresponding to the position of each pixel. The method according to claim 29, wherein the second prediction or PROF processing is performed according to the center position and the second motion vector deviation based on the first prediction value, and the second prediction of the sub-block is determined. The predicted value, the second predicted value is determined as the inter-frame predicted value of the sub-block, including: Determine PROF parameters; When the PROF parameter indicates that PROF processing is performed, the pixel horizontal gradient and the pixel vertical gradient corresponding to the center position are determined based on the first predicted value; Calculate the deviation value corresponding to each pixel position according to the pixel horizontal gradient, the pixel vertical gradient and the second motion vector deviation; Based on the first predicted value and the deviation value, obtain the predicted value of the pixel at each pixel position; The second predicted value is determined using the predicted value of the pixel at each pixel location. The method according to claim 29, wherein the second prediction or PROF processing is performed according to the center position and the second motion vector deviation based on the first prediction value, and the second prediction of the sub-block is determined. The predicted value, the second predicted value is determined as the inter-frame predicted value of the sub-block, including: Determine the quadratic prediction parameters; When the secondary prediction parameter indicates the use of secondary prediction, the filter coefficient of the two-dimensional filter is determined according to the second motion vector deviation; wherein, the two-dimensional filter is used to perform secondary prediction processing according to a preset shape; based on the filter coefficient and the first predicted value, determining a predicted value of the pixel at each pixel position; The second predicted value is determined using the predicted value of the pixel at each pixel location. The method of claim 52, wherein the two-dimensional filter is used to perform secondary prediction using a plurality of adjacent pixel positions forming the preset shape. The method according to claim 53, wherein the preset shape is a rectangle, a rhombus, or any symmetrical shape. The method of claim 54, wherein, If the value of the affine mode parameter is 1, it indicates that the affine mode is used; If the value of the affine mode parameter is 0, or the affine mode parameter is not obtained through analysis, it indicates that the affine mode is not used. The method of claim 31, wherein, If the sub-block size parameter is 8×8, then the sub-block size flag is set to 1, and the sub-block size flag is written into the bit stream; If the sub-block size parameter is 4×4, the sub-block size flag is set to 0, and the sub-block size flag is written into the bit stream. The method of claim 31, wherein the control point modes include a 4-parameter mode and a 6-parameter mode. The method of claim 31, wherein, The prediction mode parameter, the affine mode parameter, and the prediction reference mode are written into a bitstream. The method of claim 51, wherein, Write the PROF parameters to the bitstream. The method of claim 52, wherein, The secondary prediction parameters are written to the bitstream. A decoder, the decoder includes a parsing unit and a first determining unit; The parsing unit is used to parse the bit stream to obtain the prediction mode parameter of the current block; the first determining unit, configured to determine the first motion vector of the sub-block of the current block when the prediction mode parameter indicates that the inter prediction value of the current block is determined by using the inter prediction mode; wherein, the the current block includes a plurality of sub-blocks; and determining a first predictor for the sub-block based on the first motion vector, and a first motion vector between each pixel position in the sub-block and the sub-block and determining a center position corresponding to each pixel position and a second motion vector deviation according to the first motion vector deviation; and based on the first predicted value, according to the center position and the second motion vector The deviation is subjected to secondary prediction or PROF processing, the second prediction value of the sub-block is determined, and the second prediction value is determined as the inter-frame prediction value of the sub-block. A decoder, the decoder comprising a first processor and a first memory storing executable instructions of the first processor, when the instructions are executed, the first processor implements as requested The method of any one of items 1-28. An encoder comprising a second determination unit; the second determination unit, configured to determine a prediction mode parameter of the current block; and when the prediction mode parameter indicates that an inter prediction value of the current block is determined using an inter prediction mode, determining a subblock of the current block The first motion vector of the a first motion vector deviation between sub-blocks; and determining a center position corresponding to each pixel position and a second motion vector deviation according to the first motion vector deviation; and based on the first predicted value, according to the Perform secondary prediction or PROF processing on the center position and the second motion vector deviation, determine the second prediction value of the sub-block, and determine the second prediction value as the inter-frame prediction value of the sub-block. An encoder comprising a second processor and a second memory storing instructions executable by the second processor, when the instructions are executed, the second processor implements as requested The method of any of items 29-60. A computer storage medium, the computer storage medium stores a computer program, and when the computer program is executed by a first processor, the method as described in any one of claim 1-28 is implemented, or, when executed by a second processor Implement the method of any of claims 29-60.

Images