CN116711304A

CN116711304A - Prediction method, encoder, decoder, and storage medium

Info

Publication number: CN116711304A
Application number: CN202080108133.4A
Authority: CN
Inventors: 王凡
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2020-12-28
Filing date: 2020-12-28
Publication date: 2023-09-05
Also published as: WO2022140905A1

Abstract

The embodiment of the application discloses a prediction method, an encoder, a decoder and a storage medium, wherein the decoder analyzes a code stream and determines a prediction mode parameter of a current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode. The encoder determines a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.

Description

Prediction method, encoder, decoder, and storage medium

Technical Field

The embodiment of the application relates to the technical field of image processing, in particular to a prediction method, an encoder, a decoder and a storage medium.

Background

Current intra prediction modes can only handle simple texture prediction. For complex textures, the complexity of the processing is high. The inter prediction sometimes cannot find a reference block with high matching degree from the reference frame, or can only find a part of the reference block with high matching degree, so that a good inter prediction effect cannot be achieved.

In view of the above, in the multi-function video coding (Versatile Video Coding, VVC), a joint intra-frame inter-frame prediction (combined inter and intra prediction, CIIP) technique is proposed in which prediction processing is performed based on one inter-frame predicted prediction block and one intra-frame predicted prediction block of a current block.

Although the CIIP technique can improve the prediction quality by a weighted average of the intra-prediction block and the inter-prediction block, if there is a case where a part of the intra-prediction block or the inter-prediction block is bad in prediction quality, a good prediction effect is still not obtained finally.

Disclosure of Invention

The embodiment of the application provides a prediction method, an encoder, a decoder and a storage medium, which can improve the prediction quality and the compression performance.

The technical scheme of the embodiment of the application can be realized as follows:

in a first aspect, an embodiment of the present application provides a prediction method, applied to a decoder, the method including:

analyzing the code stream and determining the prediction mode parameter of the current block;

determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode;

a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.

In a second aspect, an embodiment of the present application provides a prediction method, applied to an encoder, including:

determining a prediction mode parameter of the current block;

In a third aspect, an embodiment of the present application provides an encoder, including: the first determining section may determine, based on the first determining section,

the first determining part is configured to determine a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.

In a fourth aspect, embodiments of the present application provide an encoder comprising a first processor, a first memory storing instructions executable by the first processor, the first processor implementing a prediction method as described above when executed when the instructions are executed.

In a fifth aspect, an embodiment of the present application provides a decoder, including: a decoding section, a second determining section

The decoding part is configured to parse the code stream;

the second determining part is configured to determine a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.

In a sixth aspect, embodiments of the present application provide a decoder comprising a second processor, a second memory storing instructions executable by the second processor, the second processor implementing a prediction method as described above when executed when the instructions are executed.

In a seventh aspect, embodiments of the present application provide a computer storage medium storing a computer program which, when executed by a first processor and a second processor, implements a prediction method as described above.

The embodiment of the application provides a prediction method, an encoder, a decoder and a storage medium, wherein the decoder analyzes a code stream and determines a prediction mode parameter of a current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode. The encoder determines a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode. That is, in the embodiment of the present application, the codec may determine two different prediction blocks of the current block through two different prediction modes, and then may combine the two different prediction blocks according to the weight matrix, so as to finally obtain a more complex prediction block, thereby improving the accuracy of prediction, that is, the prediction method provided by the present application may improve the prediction quality and improve the compression performance.

Drawings

FIG. 1 is a diagram of a first weight distribution;

FIG. 2 is a second diagram of weight distribution;

FIG. 3 is a schematic diagram of intra prediction;

FIG. 4 is a second schematic diagram of intra prediction;

FIGS. 5A-5I are diagrams III of intra prediction;

FIG. 6 is a schematic diagram of an intra prediction mode I;

FIG. 7 is a second schematic diagram of an intra prediction mode;

FIG. 8 is a third schematic diagram of an intra prediction mode;

FIG. 9 is a schematic diagram of the position of the current block;

FIG. 10 is a schematic diagram of a conventional prediction method;

FIG. 11 is a block diagram of a video encoding system;

FIG. 12 is a block diagram of a video decoding system;

FIG. 13 is a schematic diagram of a prediction method implementation;

FIG. 14 is a second flow chart of the implementation of the prediction method;

FIG. 15 is a schematic diagram of a prediction method;

FIG. 16 is a third flow chart of an implementation of the prediction method;

FIG. 17 is a fourth flow chart of an implementation of the prediction method;

FIG. 18 is a schematic diagram showing a structure of an encoder according to an embodiment of the present application;

FIG. 19 is a schematic diagram showing a second structure of an encoder according to an embodiment of the present application;

FIG. 20 is a schematic diagram of a decoder according to an embodiment of the present application;

fig. 21 is a schematic diagram of a decoder according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It is to be understood that the specific embodiments described herein are merely illustrative of the application and not limiting of the application. It should be noted that, for convenience of description, only a portion related to the related application is shown in the drawings.

Currently, the common video codec standard is based on a hybrid coding framework that is all block-based. Each frame in the video image is partitioned into square maximum Coding units (Largest Coding Unit, LCU) of the same size (e.g., 128×128, 64×64, etc.), each maximum Coding Unit may also be partitioned into rectangular Coding Units (CU) according to rules; but the coding Unit may also be divided into smaller Prediction Units (PU). Specifically, the hybrid coding framework may include modules such as prediction, transform (Transform), quantization (Quantization), entropy coding (EntropyCoding), loop filtering (In Loop Filter), and the like; among other things, the prediction module may include intra prediction (inter prediction) and inter prediction (inter prediction), and the inter prediction may include motion estimation (motion estimation) and motion compensation (motion compensation). Because of the strong correlation between adjacent pixels in one frame of the video image, the use of intra-frame prediction modes in video encoding and decoding techniques can eliminate spatial redundancy between adjacent pixels; however, since there is also a strong similarity between adjacent frames in the video image, temporal redundancy between adjacent frames is eliminated using an inter prediction mode in the video encoding and decoding technique, so that encoding and decoding efficiency can be improved.

The basic flow of the video codec is as follows: at the encoding end, dividing a frame of image into blocks, generating a prediction block of the current block by using intra-frame prediction or inter-frame prediction on the current block, subtracting the prediction block from an original block of the current block to obtain a residual block, transforming and quantizing the residual block to obtain a quantization coefficient matrix, and entropy-encoding the quantization coefficient matrix and outputting the quantization coefficient matrix into a code stream. At the decoding end, the current block is predicted by using intra-frame prediction or inter-frame prediction to generate a predicted block of the current block, on the other hand, the code stream is analyzed to obtain a quantized coefficient matrix, the quantized coefficient matrix is subjected to inverse quantization and inverse transformation to obtain a residual block, and the predicted block and the residual block are added to obtain a reconstructed block. The reconstructed blocks form a reconstructed image, and the reconstructed image is subjected to loop filtering based on the image or the blocks to obtain a decoded image. The encoding side also needs to obtain a decoded image in a similar operation to the decoding side. The decoded image may be a subsequent frame as a reference frame for inter prediction. The block division information determined by the encoding end, prediction, transformation, quantization, entropy coding, loop filtering and other mode information or parameter information are output to the code stream if necessary. The decoding end analyzes and analyzes the existing information to determine the same block division information as the encoding end, and predicts, transforms, quantizes, entropy codes, loop filters and other mode information or parameter information, so that the decoded image obtained by the encoding end is ensured to be the same as the decoded image obtained by the decoding end. The decoded image obtained at the encoding end is also commonly called reconstructed image. The current block may be divided into prediction units at the time of prediction, and may be divided into transform units at the time of transformation, and the division of the prediction units and the transform units may be different. The foregoing is a basic flow of a video codec under a block-based hybrid coding framework, and as technology advances, some modules or steps of the framework or flow may be optimized.

The current block (current block) may be a current Coding Unit (CU) or a current Prediction Unit (PU), etc.

Inter prediction predicts the current block using information of the decoded or reconstructed image or called reference frame. Inter prediction uses motion information to find a reference block from a reference frame, from which a prediction block is generated. The motion information includes a reference frame list in which the reference frame is located, a reference frame index, and a motion vector. The motion vector may be integer or sub-pixel, and if the motion vector is sub-pixel, interpolation filtering is required in the reference frame to make the required sub-pixel block, where the integer or sub-pixel block in the reference frame found from the motion vector is called the reference block. Some techniques may use the reference block directly as a prediction block, and some techniques may reprocess the reference block to generate a prediction block. Reprocessing a prediction block on the basis of a reference block is also understood to mean that the reference block is taken as a prediction block and then a new prediction block is processed on the basis of the prediction block.

Among the currently established multi-function video coding (Versatile Video Coding, VVC) video codec standards is an inter prediction mode called the geometric partition prediction mode (GeometricpartitioningMode, GPM). Among the video codec standards (Audio Video coding Standard, AVS) currently under development is an inter prediction mode called the angle weighted prediction mode (Angular Weightedprediction, AWP). These two modes are common in principle, though they are different in name and specific in implementation.

It should be noted that, the conventional unidirectional prediction only searches for a reference block having the same size as the current block, while the conventional bidirectional prediction uses two reference blocks having the same size as the current block, and the pixel value of each point in the prediction block is an average value of the corresponding positions of the two reference blocks, that is, all points of each reference block account for 50%. The bi-directional weighted prediction allows the ratio of the two reference blocks to be different, e.g. 75% for all points in the first reference block and 25% for all points in the second reference block. But the proportions of all points in the same reference block are the same. Other optimization methods, such as decoding-side motion vector correction (Decoder sideMotion Vector Refinement, DMVR) techniques, bi-directional optical flow (Bi-directional Optical Flow, BIO), etc., may cause some changes to reference pixels or predicted pixels, and GPM or AWP may also use two reference blocks of the same size as the current block, but some pixel positions 100% use the pixel value of the first reference block corresponding position, some pixel positions 100% use the pixel value of the second reference block corresponding position, and in the border region, use the pixel values of the two reference block corresponding positions in proportion. The specific how these weights are assigned is determined by the prediction mode of the GPM or AWP, or it can also be considered that the GPM or AWP uses two reference blocks of different sizes from the current block, i.e. each takes a part of the required reference block. I.e. the part with the weight not being 0 is taken as a reference block, and the part with the weight being 0 is eliminated.

For example, fig. 1 is a schematic diagram of weight distribution, as shown in fig. 1, which shows a schematic diagram of weight distribution of multiple partitioning modes of a GPM on a 64×64 current block according to an embodiment of the present application, where there are 64 partitioning modes of the GPM. Fig. 2 is a second weight distribution diagram, as shown in fig. 2, which shows a weight distribution diagram of multiple partition modes of AWP on a 64×64 current block according to an embodiment of the present application, where there are 56 partition modes of AWP. In either of the division modes shown in fig. 2 and 3, the weight value of the corresponding position of the first reference block is 0% in the area 2, the weight value of the corresponding position of the first reference block is 100% in the area 1, the weight value of the corresponding position of the first reference block is greater than 0% and less than 100% in the gray area according to the difference of the color shades, and the weight value of the corresponding position of the second reference block is 100% minus the weight value of the corresponding position of the first reference block.

The weight derivation methods of GPM and AWP are different. The GPM determines the angle and the offset according to each mode, and then calculates the weight matrix of each mode. AWP first makes one-dimensional weighted lines and then uses a method similar to intra angle prediction to spread the one-dimensional weighted lines across the entire matrix.

It should be appreciated that in early codec techniques, only rectangular partitioning exists, whether CU, PU or Transform Unit (TU) partitioning. While either GPM or AWP achieves the predictive non-rectangular partitioning effect without partitioning. GPM and AWP use masks (masks) of the weights of the two reference blocks, i.e., the weight map described above. This mask determines the weights of the two reference blocks when generating the prediction block, or can be simply understood that part of the position of the prediction block is from the first reference block and part of the position is from the second reference block, and the transition area (blending area) is weighted with the corresponding positions of the two reference blocks, thereby smoothing the transition. GPM and AWP do not divide the current block into two CUs or PUs by a division line, and thus transform, quantization, inverse transform, inverse quantization, etc. of the residual after prediction are also all handled as a whole.

The motion information used by the current block may be saved. The blocks of the subsequent codec of the current frame may use motion information of previously-encoded blocks, such as neighboring blocks, according to neighboring positional relationships. This exploits the spatial correlation, so that such encoded and decoded motion information is called spatial motion information. The motion information used by each block of the current frame may be saved. The subsequent encoded frames may use motion information of the previously encoded frames according to a reference relationship. This exploits the temporal correlation, so the motion information of such a encoded frame is called temporal motion information. The method of storing motion information used for each block of the current frame typically uses a fixed size matrix, such as a 4 x 4 matrix, as a minimum unit, each of which stores a set of motion information separately. So that every time a block is encoded, its position corresponds to the smallest unit that can store the motion information of the block. Thus, when the motion information in the space domain or the motion information in the time domain is used, the motion information corresponding to the position can be directly found according to the position. If a 16 x 16 block uses conventional unidirectional prediction, then all 4 x 4 minimum units corresponding to the block store the motion information of the unidirectional prediction. If a block uses GPM or AWP, all the minimum units corresponding to the block determine motion information stored in each minimum unit according to the mode of GPM or AWP, the first motion information, and the second motion information and the position of each minimum unit. One method is that if all the 4 x 4 pixels corresponding to a minimum unit are from the first motion information, the minimum unit stores the first motion information, and if all the 4 x 4 pixels corresponding to a minimum unit are from the second motion information, the minimum unit stores the second motion information. If a 4 x 4 pixel corresponding to a minimum unit is from both the first motion information and the second motion information, then the AWP selects one of the motion information for storage; GPM is to combine two motion information into a bi-directional motion information store if they point to different reference frame lists, otherwise only store the second motion information.

It should be noted that, in the embodiment of the present application, the GPM or the AWP belongs to an inter-frame prediction technique, and the GPM or the AWP needs to transmit a flag (flag) indicating whether the GPM or the AWP is used in the code stream, where the flag may indicate whether the current block uses the GPM or the AWP. If GPM or AWP is used, the encoder needs to transmit a specific used mode in the code stream, namely one of 64 division modes of GPM or one of 56 division modes of AWP; and index values of two unidirectional motion information. That is, for the current block, the decoder may obtain information about whether the GPM or the AWP is used by parsing the code stream, and if it is determined that the GPM or the AWP is used, the decoder may parse the prediction mode parameter of the GPM or the AWP and two motion information index values, for example, the current block may be divided into two partitions, and may parse the first index value corresponding to the first partition and the second index value corresponding to the second partition.

Specifically, for the GPM mode, if the GPM is used, prediction mode parameters under the GPM, such as a GPM specific partition mode, are transmitted in the code stream; typically, the GPM includes 64 partitioning modes. For the AWP mode, if AWP is used, prediction mode parameters under AWP, such as specific partition mode of AWP, are transmitted in the code stream; typically, AWP includes 56 partitioning modes.

In inter prediction modes, such as GPM and AWP, two unidirectional motion information needs to be used to find two reference blocks. The current implementation method is that a unidirectional motion information candidate list is constructed by utilizing the related information of the coded/decoded part before the current block at the encoder side, unidirectional motion information is selected from the unidirectional motion information candidate list, and index values (index) of the two unidirectional motion information in the unidirectional motion information candidate list are written into a code stream. The same way is used on the decoder side, i.e. a unidirectional motion information candidate list is constructed using the information about the previously decoded part of the current block, which unidirectional motion information candidate list must be identical to the one constructed on the encoder side. In this way, index values of two unidirectional motion information are analyzed from the code stream, and then the two unidirectional motion information are searched from the unidirectional motion information candidate list, namely the two unidirectional motion information needed to be used by the current block.

That is, the unidirectional motion information described in the present application may include: motion vector information, i.e. the value of (x, y), and corresponding reference frame information, i.e. the reference frame list and the reference frame index values in the reference frame list. One representation is to record the reference frame index values of two reference frame lists, wherein one reference frame list corresponds to a valid reference frame index value, such as 0,1,2, etc.; the reference frame index value corresponding to the other reference frame list is invalid, i.e., -1. The reference frame list with the effective reference frame index value is the reference frame list used by the motion information of the current block, and the corresponding reference frame can be found from the reference frame list according to the reference frame index value. Each reference frame list has a corresponding motion vector, the motion vector corresponding to the active reference frame list is active, and the motion vector corresponding to the inactive reference frame list is inactive. The decoder can find the required reference frame through the reference frame information in the unidirectional motion information, and can find the reference block in the reference frame according to the position of the current block and the value of the motion vector (x, y), so as to further determine the inter prediction value of the current block.

The intra prediction method predicts the current block using reconstructed pixels, which have been encoded and decoded around the current block, as reference pixels. Fig. 3 is a schematic diagram of intra prediction, as shown in fig. 3, in which the current block has a size of 4×4, and pixels in a left row and a top column of the current block are reference pixels of the current block, and the intra prediction predicts the current block using the reference pixels. These reference pixels may already be fully available, i.e. fully already encoded. There may also be some unavailability, such as the current block is the leftmost side of the entire frame, and then the reference pixel to the left of the current block is not available. Or when the current block is encoded and decoded, the lower left part of the current block is not encoded and decoded yet, and then the lower left reference pixel is not available. For the case where reference pixels are not available, the available reference pixels or some value or some method may be used for filling or not.

Fig. 4 is a second schematic diagram of intra prediction, as shown in fig. 4, the multi-reference-line intra prediction method (Multiple reference line, MRL) can use more reference pixels to improve the coding efficiency, for example, 4 reference lines/columns are used as the reference pixels of the current block.

Further, there are multiple prediction modes for intra prediction, and fig. 5A-5I are three schematic diagrams of intra prediction, and as shown in fig. 5A-5I, intra prediction of a 4×4 block in h.264 may mainly include 9 modes. Wherein, the pattern 0 shown in fig. 5A copies the pixel above the current block to the current block in the vertical direction as a prediction value, the pattern 1 shown in fig. 5B copies the left reference pixel to the current block in the horizontal direction as a prediction value, the pattern 2 DC shown in fig. 5C copies the average value of the 8 points a to D and I to L as a prediction value of all points, and the patterns 3 to 8 shown in fig. 5D to 5I copy the reference pixel to the corresponding position of the current block at a certain angle, respectively, because some positions of the current block cannot exactly correspond to the reference pixel, it may be necessary to use the weighted average value of the reference pixel, or the sub-pixel of the interpolated reference pixel.

In addition to this, planar modes and the like are also available, and as technology advances and blocks expand, more and more angular prediction modes are available. Fig. 6 is a schematic diagram of intra prediction modes, as shown in fig. 6, the intra prediction modes used in HEVC include Planar, DC, and 33 angular modes for a total of 35 prediction modes. Fig. 7 is a second schematic diagram of intra prediction modes, and as shown in fig. 7, VVC uses intra modes having Planar, DC, and 65 angle modes for 67 prediction modes. Fig. 8 is a third schematic diagram of intra prediction modes, as shown in fig. 8, AVS3 uses a total of 66 prediction modes of DC, planar, bilinear and 63 angle modes.

Still other techniques improve the prediction, such as improving the fractional interpolation of reference pixels, filtering the predicted pixels, etc. A multi-combination intra prediction filter (multipleintraprediction filter, MIPF) as in AVS3 generates predictors for different block sizes using different filters. For pixels at different positions within the same block, a pixel closer to the reference pixel uses one filter to generate a predicted value, and a pixel farther from the reference pixel uses another filter to generate a predicted value. Techniques for filtering the predicted pixels such as intra-prediction filtering (intraprediction filter, IPF) in AVS3 may filter the predicted values using reference pixels.

In addition, in intra prediction, information such as intra prediction mode may be stored for reference of a block of a subsequent codec, as compared with inter prediction. Specifically, the block of the subsequent codec of the current frame may use the intra prediction mode of the previously encoded block, such as the neighboring block, according to the neighboring positional relationship. The chroma block (coding unit) may use an intra prediction mode of a previously encoded luma block (coding unit) according to a position. Note that these pieces of information stored here are referred to for blocks of subsequent codec, and since the coding mode information in the same block (coding unit) is directly available, but the coding mode information in a different block (coding unit) is not directly available, it is necessary to store them. The blocks of the subsequent codec read this information based on the location. The method of storing the intra prediction mode used for each block of the current frame generally uses a fixed size matrix, such as a 4×4 matrix, as a minimum unit, each of which stores one intra prediction mode separately. So that every block is coded, its position corresponds to the smallest unit that stores the intra prediction mode of the block. For example, if a 16×16 block uses intra prediction mode 5, then all the intra prediction modes of the 4×4 minimum units corresponding to the block are stored 5. For YUV formats, only intra-prediction modes of luminance are typically stored, including intra-prediction modes of luminance for blocks that contain both luminance and chrominance components, and intra-prediction modes of luminance for blocks that contain only luminance components.

Although current intra predictions have modes such as DC, planar, bilinear, they can only deal with simple texture predictions; even though the angle patterns are increasing, their predictions can only follow a straight line at an angle. It can be seen that current intra prediction modes can only handle simple texture prediction. For complex textures, it is necessary to divide into smaller blocks at the time of prediction, or to encode more residuals, or to cause greater distortion. Inter prediction sometimes cannot find a block with high matching degree from a reference frame, or can find only a part of blocks with high matching degree, and in this case, inter prediction cannot achieve a good effect.

In view of the above problems, a joint intra inter prediction (combined inter and intra prediction, CIIP) technique is currently proposed, and the CIIP technique performs weighted average on an inter predicted prediction block and an intra predicted prediction block of a current block, to finally obtain a new prediction block of the current block. In VVC, inter prediction block P of CIIP _inter Derived from a common merge mode, intra-prediction block P of CIIP _intra Derived from Planar modes of normal intra prediction. Then P is determined according to the conditions of the left block and the upper block of the current block _inter And P _intra Is a weight value of (a).

Specifically, a new predicted block P of the current block _CIIP The derivation can be achieved by the following formula:

P _CIIP ＝((4-wt)×P _inter +wt×P _intra +2)＞＞2 (1)

wherein, wt is a weight parameter, and fig. 9 is a schematic diagram of the position of the current block, as shown in fig. 9, if the left block left and the upper block top of the current block both use intra prediction, then wt can be set to 3; if the left block left and the upper block top of the current block use intra prediction and inter prediction, respectively, wt may be set to 2, and if the left block left and the upper block top of the current block use inter prediction, wt may be set to 1.

As can be seen, fig. 10 is a schematic diagram of a common prediction method, as shown in fig. 10, when intra-predicting a current block, an intra-prediction block of the current block may be determined using an intra-prediction mode, an inter-prediction block of the current block may be determined using an inter-prediction mode, and then weighted average processing may be performed on the intra-prediction block and the inter-prediction block using an intra-prediction block weight and an inter-prediction block weight, respectively, to finally obtain a new prediction block.

However, for some current blocks, while CIIP techniques may improve the prediction quality by a weighted average of intra-and inter-prediction blocks. However, CIIP uses the same weight for all pixels of the whole block, and can also be regarded as treating an intra prediction block and an inter prediction block as a whole, respectively. This is not flexible enough to take into account that an intra-frame prediction block or an inter-frame prediction block has a part of good prediction quality and a part of bad prediction quality.

In view of the above problems, in the embodiment of the present application, the codec may determine two different prediction blocks of the current block through two different prediction modes, and then may combine the two different prediction blocks according to the weight matrix to finally obtain a more complex prediction block, thereby improving the accuracy of prediction, that is, the prediction method provided by the present application may improve the prediction quality and improve the compression performance.

Referring to fig. 11, a block diagram illustrating an example of a video coding system according to an embodiment of the present application is shown; as shown in fig. 11, the video coding system 10 includes a transform and quantization unit 101, an intra estimation unit 102, an intra prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, a coding unit 109, a decoding image buffer unit 110, and the like, wherein the filtering unit 108 can implement deblocking filtering and Sample Adaptive 0 offset (sao) filtering, and the coding unit 109 can implement header information coding and Context-based Adaptive binary arithmetic coding (Context-based Adaptive Binary Arithmatic Coding, CABAC). For an input original video signal, a video Coding block can be obtained through Coding Tree Unit (CTU) division, and then residual pixel information obtained after intra-frame or inter-frame prediction is transformed by a transformation and quantization Unit 101, including transforming the residual information from a pixel domain to a transformation domain, and quantizing the obtained transformation coefficient to further reduce a bit rate; the intra estimation unit 102 and the intra prediction unit 103 are used for intra prediction of the video coding block; in particular, the intra-estimation unit 102 and the intra-prediction unit 103 are used to determine an intra-prediction mode to be used to encode the video encoding block; the motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-prediction encoding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by the motion estimation unit 105 is a process of generating a motion vector that can estimate the motion of the video encoding block, and then the motion compensation unit 104 performs motion compensation based on the motion vector determined by the motion estimation unit 105; after determining the intra prediction mode, the intra prediction unit 103 is further configured to provide the selected intra prediction data to the encoding unit 109, and the motion estimation unit 105 also transmits the calculated determined motion vector data to the encoding unit 109; in addition, the inverse transform and inverse quantization unit 106 is used for reconstructing the video coding block, reconstructing a residual block in the pixel domain, removing blocking artifacts by the filter control analysis unit 107 and the filtering unit 108, and adding the reconstructed residual block to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video coding block; the encoding unit 109 is configured to encode various encoding parameters and quantized transform coefficients, and in a CABAC-based encoding algorithm, context content may be based on neighboring encoding blocks, and may be configured to encode information indicating the determined intra prediction mode, and output a code stream of the video signal; and the decoded picture buffer unit 110 is for storing reconstructed video coding blocks for prediction reference. As video image encoding proceeds, new reconstructed video encoding blocks are generated, and the reconstructed video encoding blocks are stored in the decoded image buffer unit 110.

Referring to fig. 12, there is shown an example of a block diagram of a video decoding system according to an embodiment of the present application; as shown in fig. 12, the video decoding system 20 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra prediction unit 203, a motion compensation unit 204, a filtering unit 205, a decoded image buffer unit 206, and the like, wherein the decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement deblocking filtering and SAO filtering. After the input video signal is subjected to the encoding process of fig. 9, a code stream of the video signal is output; the code stream is input into the video decoding system 20, and first passes through the decoding unit 201 to obtain decoded transform coefficients; processing by the inverse transform and inverse quantization unit 202 for the transform coefficients to generate a residual block in the pixel domain; the intra prediction unit 203 may be used to generate prediction data for a current video decoded block based on the determined intra prediction mode and data from a previously decoded block of a current frame or picture; the motion compensation unit 204 is a unit that determines prediction information for a video decoding block by parsing the motion vector and other associated syntax elements and uses the prediction information to generate a predictive block of the video decoding block being decoded; forming a decoded video block by summing the residual block from the inverse transform and inverse quantization unit 202 with a corresponding predictive block generated by the intra prediction unit 203 or the motion compensation unit 204; the decoded video signal is passed through a filtering unit 205 to remove blocking artifacts, which may improve video quality; the decoded video blocks are then stored in a decoded image buffer unit 206, and the decoded image buffer unit 206 stores reference images for subsequent intra prediction or motion compensation, and is also used for output of video signals, i.e. the restored original video signals.

The prediction method in the embodiment of the application can be applied to a video coding system, a video decoding system and even a video coding system and a video decoding system at the same time, but the embodiment of the application is not particularly limited. It should be further noted that, when the prediction method is applied to a video coding system, the "current block" specifically refers to the current coding block; when the prediction method is applied to a video decoding system, "current block" specifically refers to a current decoded block.

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

An embodiment of the present application provides a prediction method applied to a decoder, and fig. 13 is a schematic diagram of an implementation flow of the prediction method, and as shown in fig. 13, the method for performing prediction by the decoder may include the following steps:

and 101, analyzing the code stream and determining the prediction mode parameter of the current block.

In an embodiment of the application, the decoder parses the bitstream to determine the prediction mode parameters for the current block.

It should be noted that, in the implementation of the present application, the prediction mode parameter may indicate whether the current block may use different prediction modes for determining the prediction value, i.e., the prediction mode parameter may indicate whether the current block may use two different prediction modes for performing the prediction processing.

It is understood that in the embodiment of the present application, the prediction mode parameter may be understood as a flag bit indicating whether a different prediction mode is used. Specifically, the decoder parses the code stream, and may determine a variable as a prediction mode parameter, so that the determination of the prediction mode parameter may be implemented by the value of the variable.

It should be noted that, in an embodiment of the present application, if the prediction mode parameter indicates that the current block may use two different prediction modes for determining the prediction value, the two different prediction modes include, but are not limited to, an intra prediction mode, an inter prediction mode, an intra block copy (IntraBlockCopy, IBC) mode, a Palette mode Palette, a string prediction mode, and the like.

Illustratively, in the present application, the decoder may perform prediction processing on the current block using an inter prediction mode and an intra prediction mode, respectively, e.g., the decoder determines the predicted block of the current block using a Plane mode and a translation prediction mode, respectively. It is because the inter prediction mode and the intra prediction mode are used to respectively predict the current block to obtain the predicted block of the current block, so that the prediction that the intra prediction mode can only process simple textures can be better optimized, and meanwhile, the reference block with high matching degree, which sometimes cannot be obtained from the reference frame, can be better optimized by the inter prediction mode, and finally, a better prediction effect is obtained.

For example, in the present application, the decoder may also perform prediction processing on the current block using an intra prediction mode and an intra block copy mode, respectively, or perform prediction processing on the current block using a palette mode and an inter prediction mode, respectively. That is, in the present application, if the prediction mode parameter indicates that the current block can perform the determination of the prediction value using two different prediction modes, the combination of the two different prediction modes may be various, and the present application is not particularly limited thereto.

Further, in an embodiment of the present application, when two different prediction modes are applied to predict a current block, the size of the current block may be limited.

It can be understood that, since the prediction method provided by the embodiment of the present application needs to generate two different prediction blocks by using two different prediction modes respectively, and then weighting is performed according to a weight matrix to obtain a new prediction block, in order to reduce the complexity of prediction, and meanwhile consider the trade-off of compression performance and complexity, in the embodiment of the present application, it is possible to limit that two different prediction modes are not used for some prediction blocks with different sizes. Thus, in the present application, the decoder may determine the size parameter of the current block first and then determine whether the current block uses two different prediction modes according to the size parameter.

It should be noted that, in the embodiment of the present application, the size parameter of the current block may include a height and a width of the current block, and thus, the decoder may limit the use of two different prediction modes, that is, limit the size of a prediction block capable of using two different prediction modes, using the height and the width of the current block.

Illustratively, in the present application, if the width is greater than the first threshold and the height is greater than the second threshold, it is determined that the current block uses two different prediction modes, i.e., it is determined that the current block uses different prediction modes to determine the prediction value of the current block. It can be seen that one possible limitation is to use two different prediction modes only if the width of the prediction block is greater than (or equal to) a first threshold and the height of the prediction block is greater than (or equal to) a second threshold. Wherein the values of the first threshold and the second threshold may be 8, 16, 32, etc., and the first threshold may be equal to the second threshold.

Illustratively, in the present application, if the width is smaller than the third threshold and the height is smaller than the fourth threshold, it is determined that the current block uses two different prediction modes, i.e., it is determined that the current block uses different prediction modes to determine the prediction value of the current block. It can be seen that one possible limitation is to use two different prediction modes only if the width of the prediction block is less than (or equal to) the third threshold and the height of the prediction block is less than (or equal to) the fourth threshold. Wherein the third and fourth thresholds may have values of 8, 16, 32, etc., and the third threshold may be equal to the fourth threshold.

Further, in an embodiment of the present application, the decoder may further determine an aspect ratio of the current block according to the width and length of the current block after determining the height and width of the current block; and then limited by the aspect ratio of the current block.

Specifically, in the present application, if the aspect ratio is smaller than the fifth threshold, determining that the current block uses different prediction modes to determine the prediction value of the current block; alternatively, if the inverse aspect ratio is less than the fifth threshold, then the current block is determined to use a different prediction mode to determine the predicted value of the current block.

That is, as technology advances, block partitioning becomes more flexible, and current partitioning methods can support aspect ratio 1 in addition to square blocks: 2,1:4,1:8,2:1,4:1,8:1, etc. Thus, in an embodiment of the application, for blocks of certain aspect ratios, such as 1:4 or 4: block 1 and 1:8 or 8:1, or for some size aspect ratio blocks, such as 8× 32,8 ×64, 32×8, 64×8, the compression performance achieved by the prediction method proposed by the present application may not be good or obvious, then the aspect ratio limit may be set, such as length to width is less than (or equal to) a fifth threshold, then the prediction value of the current block is determined using different prediction modes, or width to length is less than (or equal to) the fifth threshold, then the prediction value of the current block is determined using different prediction modes. Wherein the fifth threshold may be 4,8, etc.

It should be noted that, in the embodiment of the present application, the current block size limiting method and the aspect ratio limiting method may be used simultaneously. For example, the decoder may be set to perform prediction processing using two different prediction modes when the height of the current block is 8 or more and the width of the current block is 8 or more and the height-to-width (or width-to-width) ratio of the current block is 4 or less, or else, the prediction processing is performed by default of not two different prediction modes.

That is, in the present application, the current block can be predicted using two different prediction modes only if the size parameter of the current block satisfies the size requirement.

Further, in an embodiment of the present application, the decoder may also implement restriction of prediction blocks capable of using two different prediction modes by restriction of the total number of pixels.

Illustratively, in the present application, the decoder may determine the total number of pixels of the current block first, and then further determine whether the current block can determine the prediction of the current block using different prediction modes according to the total number of pixels and the sixth threshold. It can be seen that one possible limitation is to use two different prediction modes only if the number of pixels of the prediction block is greater than (or equal to) the sixth threshold. The sixth threshold may be 8, 16, 32, etc.

Alternatively, there is a possible restriction that two different prediction modes are used only in the case where the number of pixels of the prediction block is smaller (or smaller or equal) than the seventh threshold value. Wherein the seventh threshold may be equal to the sixth threshold, the seventh threshold may be 8, 16, 32, etc.

Illustratively, in the present application, there may be a frame-level flag to determine whether the current frame to be decoded uses the prediction method proposed by the present application. For example, intra frames (e.g., I frames) may be configured to use the prediction method proposed by the present application, and inter frames (e.g., B frames, P frames) do not use the method. Alternatively, intra frames may be configured without using the prediction method proposed by the present application, and inter frames may use the method. Alternatively, some inter frames may be configured to use the prediction method proposed by the present application, and some inter frames do not use the method. Since inter frames may also use intra prediction, inter frames are also possible to use the prediction method proposed by the present application.

Further, there may be a flag below a frame level and above a CU level (e.g., tile, slice, patch, LCU, etc.) to determine whether this region uses the prediction method proposed by the present application.

Optionally, in the present application, at the CU level, or the PU level, or the block level, whether the flag of the prediction method proposed by the present application is used is the same level as the judgment of the prediction modes of the intra prediction and the inter prediction. That is, the current block determines whether to use the intra prediction mode or the inter prediction mode or the prediction mode of the present application.

Alternatively, in the present application, whether or not the prediction method proposed by the present application is used may be regarded as the following prediction mode of an inter prediction mode. That is, if the current block determines to use the inter prediction mode, it is further determined whether to use the prediction method proposed by the present application.

Illustratively, in the present application, the decoder may also parse the code stream to determine the intra-prediction mode parameters of the current block before determining the prediction mode parameters of the current block; determining a prediction mode parameter of the current block when the intra prediction mode parameter indicates that the current block uses the intra prediction mode to determine a prediction value of the current block; determining a first prediction mode and a second prediction mode when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode or the second prediction mode is an intra prediction mode; finally, a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.

Illustratively, in the present application, the decoder may further parse the code stream to determine the inter prediction mode parameter of the current block before determining the prediction mode parameter of the current block; determining a prediction mode parameter of the current block when the inter prediction mode parameter indicates that the current block uses the inter prediction mode to determine a prediction value of the current block; determining a first prediction mode and a second prediction mode when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode or the second prediction mode is an inter prediction mode; and finally, determining the predicted value of the current block based on the first predicted mode and the second predicted mode.

It should be noted that, in the embodiment of the present application, the video image may be divided into a plurality of image blocks, and the current Block is each image Block to be decoded currently, which may be referred to as a decoding Block (CB). Here, each decoding block may include a first image component, a second image component, and a third image component. Specifically, in the present application, assuming that prediction processing is performed and that the first image component is a luminance component, that is, the image component to be predicted is a luminance component, a decoded block to be predicted may be referred to as a luminance block; alternatively, assuming that the prediction process is performed and the second image component is a chrominance component, i.e., the image component to be predicted is a chrominance component, the decoded block to be predicted may be referred to as a chrominance block.

Step 102, when the prediction mode parameter indicates that the current block uses different prediction modes to determine the prediction value of the current block, determining a first prediction mode and a second prediction mode of the current block, wherein the first prediction mode and the second prediction mode are any two modes of the following multiple prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode.

In an embodiment of the present application, after determining the prediction mode parameter of the current block, if the prediction mode parameter indicates that the current block determines a prediction value of the current block using two different prediction modes, the decoder may further determine a first prediction mode and a second prediction mode that the current block needs to use.

It should be noted that, in the embodiment of the present application, if the prediction mode parameter indicates that the current block may use two different prediction modes for determining the prediction value, the decoder may determine the two different prediction modes, i.e., the first prediction mode and the second prediction mode. The first prediction mode and the second prediction mode may include, but are not limited to, an intra prediction mode, an inter prediction mode, an intra block copy mode, a Palette mode Palette, a string prediction mode, and the like.

Illustratively, in the present application, the first prediction mode and the second prediction mode are an intra prediction mode and an inter prediction mode, respectively, for example, the first prediction mode may be an inter prediction mode and the second prediction mode may be an intra prediction mode.

Illustratively, in the present application, the first prediction mode and the second prediction mode are any two modes of the following plurality of prediction modes, respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode. For example, the first prediction mode may be an inter prediction mode and the second prediction mode may be a palette mode. Or the first prediction mode and the second prediction mode are an inter prediction mode and an intra prediction mode, respectively, for example, the first prediction mode may be an affine prediction mode, and the second prediction mode may be a Bilinear mode.

That is, in the present application, if the prediction mode parameter indicates that the current block can perform the determination of the prediction value using two different prediction modes, the combination of the first prediction mode and the second prediction mode may be various, and the present application is not particularly limited thereto.

It should be noted that, in the present application, the intra prediction mode used by the current block includes, but is not limited to, any of the intra prediction modes commonly used at present, including, but not limited to, an intra prediction mode such as DC, planar, bilinear and an angle prediction mode.

Alternatively, in the present application, the broader understanding is that the intra prediction mode further includes techniques for improving prediction, such as improving sub-pixel interpolation of reference pixels, filtering of prediction pixels, etc., such as MIPF, IPF, etc.

Accordingly, in the present application, the intra prediction modes such as DC, planar, bilinear and the angle prediction modes may be referred to as a base intra prediction mode, and techniques for improving prediction such as MIPF and IPF may be referred to as an improved intra prediction mode. The basic intra prediction mode is an intra prediction mode in which a prediction block can be independently generated independent of other intra prediction modes, i.e., a reference pixel and a basic intra prediction mode are determined, and a prediction block can be determined. While the improved intra prediction modes do not allow for independent generation of prediction blocks, they need to rely on the basic intra prediction mode to determine the prediction block. For example, a certain angular prediction mode may determine to generate a prediction block based on a reference pixel, and the MIPF may generate or determine a prediction block using different filters for pixels at different locations based on this angular prediction mode.

Illustratively, in the present application, if the first prediction mode or the second prediction mode is an intra prediction mode, the intra prediction mode may be a basic intra prediction mode or a combination of the basic intra prediction mode and the modified intra prediction mode. It is also understood that the intra prediction block is determined based on a basic intra prediction mode therein, but the process of determining the intra prediction block also uses an improved intra prediction mode.

Further, in the present application, inter prediction modes include, but are not limited to, unidirectional prediction, bidirectional prediction, and the like. Inter prediction modes include, but are not limited to, sub-block based predictions, such as AFFINE prediction (AFFINE), sub-block based temporal motion vector prediction (SbTMVP), and the like. Inter prediction modes include, but are not limited to, translational prediction (AFFINE), decoding-side motion vector optimization (DMVR), bi-directional optical flow (BIO), and the like.

Specifically, in the present application, the motion information derivation method of the inter prediction mode includes, but is not limited to, directly deriving motion information from related motion information, constructing a motion information candidate list, selecting motion information therefrom, and adding a motion vector difference value, a motion vector offset, and the like on the basis of the foregoing method.

Further, in an embodiment of the present application, the decoder may determine the first mode flag and the second mode flag by parsing the bitstream when determining the first prediction mode and the second prediction mode of the current block; and then determining the prediction mode indicated by the first mode mark as a first prediction mode, and determining the prediction mode indicated by the second mode mark as a second prediction mode.

That is, in the present application, the decoder may determine the corresponding first prediction mode and second prediction mode directly based on the first mode flag and the second mode flag, i.e., the first mode flag and the second mode flag may directly indicate the prediction mode

Illustratively, in an embodiment of the present application, the decoder parses the bitstream, determines that the first mode flag indicates that the first prediction mode is an intra prediction mode, and determines that the second mode flag indicates that the second prediction mode is an inter prediction mode, and then the decoder may perform prediction processing on the current block using the intra prediction mode and the inter prediction mode.

Further, in the embodiment of the present application, when determining the first prediction mode and the second prediction mode of the current block, the decoder may further parse the code stream to determine a first mode flag and a second mode flag; then, determining a first prediction mode in the candidate modes according to the first mode mark; and finally, determining a second prediction mode in other modes of the candidate modes except the first prediction mode according to the second mode mark.

That is, in the present application, the decoder may determine the first prediction mode from among the candidate modes according to the first mode flag, and then determine the second prediction mode from among the candidate modes excluding the first prediction mode according to the second mode flag, i.e., the first mode flag and the second mode flag may be used to select the prediction mode based on the candidate modes.

Illustratively, in an embodiment of the present application, the candidate modes include an inter prediction mode, an intra prediction mode, a string prediction mode, an intra block copy prediction mode, and the like, the decoder parses the bitstream, determines a first mode flag indicating that one of the candidate modes is the first prediction mode, and determines a second mode flag indicating that the other of the candidate modes is the second prediction mode.

As can be seen from the above, in the present application, the decoder analyzes the code stream, and can determine flag information indicating the first prediction mode and the second prediction mode in addition to determining the prediction mode parameter indicating that prediction processing can be performed using two different prediction modes.

Alternatively, in the present application, for prediction modes using only two determinations, such as intra prediction and inter prediction, the decoder may perform the determination of the first prediction mode and the second prediction mode directly through the first mode flag and the second mode flag obtained by decoding, without selecting the prediction modes.

Alternatively, in the present application, if two prediction modes need to be selected among a plurality of prediction modes, for example, in intra SCC (screen content coding) coding, the prediction modes that can be used include a (normal) intra prediction mode, an intra block copy mode, and a string prediction SP, then the first mode flag and the second mode flag obtained by the decoder decoding may indicate which prediction mode the first prediction mode is, and which prediction mode the second prediction mode is. For example, 1 to two bits may be used to indicate which prediction mode the first prediction mode is, e.g. 10 for intra prediction, 00 for IBC, and 01 for SP, and only two choices for the second prediction mode are needed, only 1 bit flag being needed.

In addition to the control flag described above, the code stream analyzed by the decoder is transmitted with various flags such as information indicating the first pattern index, the second pattern index, and the weight matrix.

Illustratively, in the present application, the decoder parses the code stream, and may determine the first mode index and the second mode index. The first mode index may be used to indicate which prediction mode the first prediction mode is, and the second mode index may be used to indicate which prediction mode the second prediction mode is, for example, the first mode index may indicate that the first prediction mode is a DC mode in the intra prediction mode and the second mode index may indicate that the second prediction mode is an affine mode in the inter prediction mode.

Illustratively, in the present application, the decoder parses the code stream, and may determine a first weight flag and a second weight flag; then, according to the first weight mark, a first weight corresponding to the first predicted value can be determined in the weight matrix; and determining a first weight corresponding to the first predicted value in the weight matrix according to the first weight mark.

That is, in an embodiment of the present application, if the current block uses two determined prediction modes, such as intra prediction and inter prediction. The decoded code stream may also determine mode information of the inter prediction mode, and may also determine information such as motion vector deviation of the inter prediction mode. Accordingly, decoding the bitstream may also determine mode information of the intra prediction mode.

Further, decoding the code stream may also determine information of the weight matrix. In particular, it may also be determined which mode is the first prediction mode for the weight matrix and which mode is the second prediction mode for the weight matrix. Alternatively, it is determined which prediction block is the first prediction block for the weight matrix and which prediction block is the second prediction block for the weight matrix.

Step 103, determining a prediction value of the current block based on the first prediction mode and the second prediction mode.

In an embodiment of the present application, if the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes, the decoder may further determine the prediction value of the current block based on the first prediction mode and the second prediction mode after determining the first prediction mode and the second prediction mode of the current block.

Specifically, in an embodiment of the present application, when determining a prediction value of a current block based on a first prediction mode and a second prediction mode, a decoder may determine a first prediction value corresponding to the current block based on the first prediction mode; determining a second prediction value corresponding to the current block based on the second prediction mode; finally, a prediction value of the current block may be further determined based on the first prediction value and the second prediction value.

In the embodiment of the present application, the conventional technique improves the prediction quality by weighted averaging of the intra-prediction block and the inter-prediction block of the current block, and the same weight is used for the entire intra-prediction block and the entire inter-prediction block, so that the final prediction effect cannot be effectively improved when the intra-prediction block or the inter-prediction block has a part of poor prediction quality. The prediction method provided by the application can realize the improvement of the prediction effect by using different weights for different pixels in the intra-frame prediction block and the inter-frame prediction block.

It should be understood that, in the embodiment of the present application, fig. 14 is a second flowchart illustrating an implementation of the prediction method, as shown in fig. 14, the method for performing prediction processing by the decoder after determining the prediction mode parameter of the current block, that is, after step 101, may further include the following steps:

step 104, when the prediction mode parameter indicates that the current block uses different prediction modes to determine the prediction value of the current block, determining a weight matrix derivation mode of the current block.

Step 105, determining the weight matrix of the current block according to the weight matrix export mode.

In an embodiment of the present application, after the decoder parses the code stream to determine the prediction mode parameter of the current block, if the prediction mode parameter indicates that the current block uses a different prediction mode to determine the prediction value of the current block, the decoder may further determine the weight matrix derivation mode of the current block, and may further determine the weight matrix of the current block according to the weight matrix derivation mode.

Further, in an embodiment of the present application, when determining the prediction value of the current block based on the first prediction mode and the second prediction mode, the decoder may finally determine the prediction value of the current block based on the first prediction mode, the second prediction mode, and the weight matrix.

It should be noted that, in the present application, the weight matrix derivation mode is used to determine the weight matrix used by the current block. Specifically, the weight matrix derivation mode may be a mode of deriving a weight matrix. For a prediction block of a given length and width, each weight matrix derivation mode may derive a weight matrix; the weight matrices derived by different weight matrix derivation modes are different for equally sized prediction blocks.

Further, in an embodiment of the present application, not all points of each weight matrix are weighted the same among all possible weight matrices. In other words, at least two different weight values are included in at least one possible weight matrix.

Optionally, in the present application, at least two different weight values are contained in all possible weight matrices.

Alternatively, in the present application, at least two different weight values are included in the partial weight matrix, for example, in the present application, if the minimum weight value is 0, the maximum weight value is 8, there are points in one weight matrix that are 0, and there are points that are 8. Accordingly, only one weight value is included in a partial weight matrix, for example, in the present application, all points in one weight matrix are 4.

It should be noted that, in the embodiment of the application, if the weight matrix includes at least one weight value, the at least one weight value is greater than the lower weight threshold and less than the upper weight threshold.

Specifically, when the lower weight threshold value may be set to 0, the upper weight threshold value is set to 1.

For example, in the present application, for some or some weight matrix containing at least two different weight values, a limit of a minimum weight value and a maximum weight value may be set so that each point in the current block is weighted by the prediction values derived from two different prediction modes. For example, the weight is set to 5 steps, including 0,1/4,1/2,3/4,1, i.e. 0-1 is divided into 5 steps, wherein 0 indicates that the point is completely obtained from the predicted value derived from one prediction mode, and 1 indicates that the point is completely obtained from the predicted value derived from another prediction mode. At this time, the minimum weight value may be set to 1/4, and the maximum weight value may be set to 3/4, so that all points of the weight matrix need to be weighted by the predicted values derived by the two prediction modes, but the weights of all points are not the same.

Accordingly, in the present application, for a certain or some weight matrix including one weight value, the one weight may also be limited by a minimum weight value and a maximum weight value, that is, the one weight is also any value greater than the lower weight threshold and less than the upper weight threshold. Thus, the weight matrix can be more diversified, more various prediction blocks can be made, and more scenes can be covered.

It should be noted that, in the embodiment of the application, if the weight matrix includes two weight values, the two weight values are a weight lower threshold and a weight upper threshold respectively.

Optionally, in the present application, there are only two weights in one weight matrix, where one weight indicates that the predicted value of the corresponding point is completely from the value of the corresponding point of the first predicted block, and the other weight indicates that the predicted value of the corresponding point is completely from the value of the corresponding point of the second predicted block. For example, in the present application, only two weights in one weight matrix are 0 and 1, respectively.

Alternatively, in the present application, there may be a plurality of weights in one weight matrix, wherein a maximum value (weight upper limit threshold) and a minimum value (weight lower limit threshold) respectively indicate that the predicted value of the corresponding point is entirely from the value of the corresponding point of the first predicted block or the value of the corresponding point of the second predicted block. The weight value that is not the maximum or minimum represents a weighted average of values of the prediction values of the corresponding points from the corresponding points of the first prediction block and the second prediction block. The region in which the weight value of this maximum value or minimum value is composed may be called a transition region (blending area).

In the present application, when the weight matrix has only two weights, the positions where the weights change form a straight line, or when the weight matrix has a plurality of weights, the positions where the weights are the same in the transition region form a straight line (line segment). The straight line may or may not be all horizontal and vertical.

Alternatively, in the present application, in the case where the weight matrix has only two weights, the position where the weight changes constitutes a curve (curve segment), or in the case where the weight matrix has a plurality of weights, the position where the weights are the same in the transition region constitutes a curve (curve segment).

Further, in embodiments of the present application, the decoder may employ methods similar to GPM or AWP to effect the determination of the weight matrix. In particular, if GPM or AWP is used in the same codec standard or codec, the method can be used to determine the weight matrix, so that a partially identical logic can be multiplexed. If AVS3 inter prediction uses AWP, then the method of AWP may be used in AVS3 to determine the weight matrix. Of course, different methods than GPM or AWP in the same codec standard or codec may be used, such as using different numbers of modes, or different transition region algorithms, or using different parameters, etc.

In the exemplary method, the AWP mode is used in inter prediction in AVS3, the number of weight matrices of AWP is 56, in the present application, the number of weight matrices of AWP is 64, and the number of weight matrices of AWP is 56, for example, the number of weight matrices of the first 56 is the same as the number of weight matrices of AWP, and each of the remaining 8 weight matrices has only one weight value, and the weight values are 1,2, … …,7 and 8 respectively. For these 8 weight matrices, a total weight of 16, i.e. a weight of 1 represents 1:15, weight value 2 represents 2:14 weight. Thus, when the mode numbers of the 64 weight matrices are binarized, codewords of 6 bits can be used.

Because inter prediction exploits correlation in the temporal domain, it uses reconstructed images in reference frames as reference blocks. While intra prediction exploits spatial correlation, it uses pixels reconstructed around the current block as reference pixels. The closer the distance in space domain, the stronger the correlation, and the farther the distance, the worse the correlation. So if some weight matrix is such that the resulting pixel locations used by a prediction block are all far from the reference pixels, then this part may not make more suitable predictions than in the prior art, and such a weight matrix may not be used, but may be used in inter prediction.

Illustratively, in the present application, the AWP of AVS3 has 56 weight matrix derivation modes and the GPM of VVC has 64 weight matrix derivation modes.

Further, in the present application, when determining the predicted value of the current block based on the first prediction mode, the second prediction mode, and the weight matrix, the decoder may determine the first predicted value of the current block according to the first prediction mode; determining a second predicted value of the current block according to the second prediction mode; and then, carrying out weighted average calculation on the first predicted value and the second predicted value by using a weight matrix, and finally obtaining the predicted value of the current block.

Alternatively, in the present application, the decoder may determine two different prediction blocks (a first prediction value and a second prediction value) for the current block, determine one weight matrix, and then combine the two different prediction blocks according to the weight matrix to obtain a new prediction block.

Alternatively, in the present application, the decoder may determine a prediction block (first prediction value) according to the first prediction mode, determine a prediction block (second prediction value) according to the second prediction mode, determine a weight matrix, and combine two different prediction blocks according to the weight matrix to obtain a new prediction block.

It will be appreciated that in the present application, two different prediction blocks (a first prediction value and a second prediction value) are respectively determined by two different prediction modes, namely, the first prediction value is determined by a first prediction mode and the second prediction value is determined by a second prediction mode.

In the present application, if the first prediction mode is an intra prediction mode and the second prediction mode is an inter prediction mode, two different prediction blocks are determined for the current block using the intra prediction mode and the inter prediction mode, respectively, a weight matrix of the current block is determined, and the two different prediction blocks are combined according to the weight matrix to obtain a new prediction block.

For example, in the present application, when a prediction process is performed on a current block, it may be determined that a first prediction mode is an intra prediction mode, and a prediction block is determined according to the intra prediction mode. And determining the second prediction mode as an inter prediction mode, and determining a prediction block according to the inter prediction mode. And determining a weight matrix of the current block, and combining two different prediction blocks according to the weight matrix to obtain a new prediction block.

Illustratively, in the present application, if the first prediction mode is an intra prediction mode and the second prediction mode is an intra block copy mode, two different prediction blocks are determined for the current block using intra prediction and intra block copy modes, respectively. And determining a weight matrix of the current block, and combining two different prediction blocks according to the weight matrix to obtain a new prediction block.

For example, in the present application, when a current block is predicted, a first prediction mode may be an intra prediction mode, and a prediction block may be determined according to the intra prediction mode. And determining the second prediction mode as an intra block copy mode, and determining a prediction block according to the intra block copy mode. And determining a weight matrix of the current block, and combining two different prediction blocks according to the weight matrix to obtain a new prediction block.

It can be seen that, fig. 15 is a schematic diagram of a prediction method, and as shown in fig. 15, when intra-prediction is performed on a current block, a first prediction block of the current block may be determined using a first prediction mode, a second prediction block of the current block may be determined using a second prediction mode, and then the first prediction block and the second prediction block may be combined using a weight matrix, to finally obtain a new prediction block. That is, based on the first prediction mode and the second prediction mode, two prediction blocks which are not identical to each other may be determined for the current block, a weight matrix may be determined, and the two different prediction blocks may be combined according to the weight matrix to obtain a new prediction block.

Further, in the embodiment of the present application, fig. 16 is a schematic diagram of a third implementation flow chart of the prediction method, and as shown in fig. 16, the method for performing prediction by the decoder may further include the following steps:

step 201, analyzing the code stream, and determining the prediction mode parameter of the current block.

Step 202, when the prediction mode parameter indicates that the current block uses different prediction modes to determine the prediction value of the current block, determining a first prediction mode of the current block, and determining the first prediction value of the current block according to the first prediction mode.

Step 203, determining a second prediction mode of the current block, and determining a second prediction value of the current block according to the second prediction mode.

Step 204, determining a predicted value of the current block based on the first predicted value and the second predicted value.

It is understood that in the present application, after determining that the current block uses a different prediction mode to determine the prediction value of the current block, the decoder uses the prediction mode decoded earlier as the first prediction mode, the prediction block of the prediction mode decoded earlier as the first prediction block (i.e., the first prediction value), the prediction mode decoded later as the second prediction mode, and the prediction block of the prediction mode decoded later as the second prediction block (i.e., the second prediction value).

Illustratively, in the present application, if the two prediction modes of the current block are an inter prediction mode and an intra prediction mode, whether the intra prediction mode is decoded first, then the inter prediction mode is decoded first, or the inter prediction mode is decoded first, then the intra prediction mode is decoded, the decoder may directly take the prediction block of the prediction mode decoded first as the first prediction block, and the prediction block of the prediction mode encoded later as the second prediction block.

In the embodiment of the present application, further, since the prediction method provided by the present application needs to use two different prediction modes and one weight matrix to determine the prediction value of the current block, in order to reduce the overhead, in the present application, the number of bits may be reduced by limiting the two prediction modes of the current block, and at the same time, the number of bits may also be reduced by limiting the weight matrix derivation mode of the current block.

Illustratively, in the present application, since the first 33 prediction modes of AVS3 already cover almost the entire angular range, only the first 33 prediction modes are used in AVS3 to ensure better performance, and only the EIPM needs to be adjusted to be not used in decoding. Furthermore, the number of intra-frame prediction modes possibly used by the current block can be further compressed and limited, and decoding is finally realized by using only 5 bits, so that the cost is further reduced, and better performance is achieved.

Alternatively, in the present application, the overhead of the weight matrix derivation mode can be reduced by reducing the number of weight matrix derivation modes that may be used by the current block. For example, in AVS3, if 56 weight matrix derivation modes are used, a maximum of 6 bits are required to decode one weight matrix derivation mode, and if only 3 weight matrix derivation modes are used, a maximum of 5 bits are required to decode one weight matrix derivation mode. Furthermore, the number of the weight matrix deriving modes which can be possibly used by the current block can be further compressed and limited, for example, only 16 weight matrix deriving modes are used, and decoding is finally realized by using only 4 bits, so that the cost is further reduced, and better performance is achieved.

In the embodiment of the present application, when the number of weight matrix derivation modes is limited, the number of weight matrix derivation modes of all the prediction blocks may be directly limited, or different limiting methods may be used for different sizes of current blocks with reference to the size parameters of the current blocks.

For example, in the present application, for a prediction block with a larger size parameter, such as a prediction block with a size of 64×64 or 32×32, the determination of the weight matrix derivation mode of the current block may be performed using the full weight matrix derivation mode, while for a prediction block with a smaller size parameter, such as a prediction block with a size of 8×8, the determination of the weight matrix derivation mode of the current block may be performed using the partial weight matrix derivation mode, that is, the number of weight matrix derivation modes that may be used by the current block is limited, because the effect caused by the fine angle difference is not obvious for a prediction block with a smaller size parameter.

It can be appreciated that in the present application, a size threshold, such as a first size threshold, may be set first, and if the size parameter of the current block is smaller than the first size threshold, then it may be considered that the subtle angle difference does not cause a larger influence, so that the number of weight matrix deriving modes that can be selected by the current block may be limited. Specifically, the limitation of the number of weight matrix derivation modes may be achieved by limiting the index number of the weight matrix derivation modes at the current block.

Illustratively, in the present application, if the size parameter of the current block is smaller than the first size threshold, the weight matrix derived mode may be determined according to the first mode index range; the first mode index range is used for limiting the index sequence number of the weight matrix derived mode. Specifically, the first mode index range may include a first lower threshold and a first upper threshold, the index sequence number of the weight matrix-everywhere mode may be compared with the first lower threshold and the first upper threshold, respectively, and if the index sequence number of the weight matrix-deriving mode is smaller than the first lower threshold, the index sequence number of the weight matrix-deriving mode may be set to the first lower threshold; if the index number of the weight matrix derivation mode is greater than the first upper threshold, the index number of the weight matrix derivation mode may be set to the first upper threshold.

That is, in the present application, the index number of the weight matrix derivation mode of the current block may be limited by the first mode index range, thereby completing the limitation of the number of weight matrix derivation modes.

For example, in AVS3, the first pattern index range may be 0-32, i.e., the index number of the weight matrix derivation pattern may be limited to be within 0-32 by the first pattern index range, so the current block may use the first 33 weight matrix derivation patterns to make the determination of the weight matrix derivation pattern of the current block.

Alternatively, in the present application, the overhead of decoding both prediction modes can be reduced by reducing the number of intra prediction modes that may be used by the current block. For example, in AVS3, if 66 intra prediction modes are used, at most 7 bits are required to decode one intra prediction mode, whereas if only 33 intra prediction modes are used and PCM mode is not used, at most 6 bits are required to decode one intra prediction mode.

In the embodiment of the present application, when limiting the number of intra prediction modes, the limiting process may be directly performed on the number of intra prediction modes of all prediction blocks, or different limiting methods may be used for different sizes of current blocks with reference to the size parameter of the current block.

Illustratively, in the present application, for a prediction block having a large size parameter, such as a prediction block having a size of 64×64 or 32×32, the determination of the intra prediction mode of the current block may be performed using all intra prediction modes, while for a prediction block having a small size parameter, such as a prediction block having a size of 8×8, the determination of the intra prediction mode of the current block may be performed using a partial intra prediction mode, i.e., the number of intra prediction modes that may be used for the current block is limited, because the effect of the fine angle difference is not significant for a prediction block having a small size parameter.

It is understood that in the present application, a size threshold, such as a second size threshold, may be set first, and if the size parameter of the current block is smaller than the second size threshold, it is considered that the subtle angle difference does not have a large influence, so that the number of intra prediction modes that can be selected by the current block may be limited. Specifically, the limitation of the number of intra prediction modes may be achieved by limiting the index number of intra prediction modes at the current block.

Illustratively, in the present application, if the size parameter of the current block is less than the second size threshold, the intra prediction mode may be determined according to the second mode index range; the second mode index range is used for limiting the index sequence number of the intra-frame prediction mode. Specifically, the second mode index range may include a second lower threshold and a second upper threshold, and an index number of the intra prediction mode may be determined first; then setting the index sequence number larger than a second lower threshold; the index sequence number is set to be smaller than the second upper threshold.

That is, in the present application, the index number of the intra prediction mode of the current block may be limited by the second mode index range, thereby accomplishing the limitation of the number of intra prediction modes.

For example, in AVS3, the second mode index range may be 0-32, i.e., the index number of the first prediction mode may be limited to be within 0-32 by the second mode index range, so the current block may use the first 33 prediction modes for intra prediction mode determination.

It is understood that in the present application, the first size threshold may be the same as or different from the second size threshold; the first pattern index range may be the same as or different from the second pattern index range.

Further, in the present application, for the inter prediction mode, the overhead of the present application encoding the inter prediction mode can be reduced by reducing the inter prediction mode used. For example, in the present application, it is possible to select to use only skip or merge or direct modes, instead of using the normal inter mode that needs to encode mvd.

Specifically, in the present application, since the influence of the fine angle difference is not significant in the case of a small block size, for a large block size such as 64×64, 32×32, a normal inter mode that needs to encode mvd is selected, and for a small block size such as 8×8, a normal inter mode that needs to encode mvd is selected. For example, setting a block SIZE prediction inter_mode_size_thr, such as 16, if both the block length and the block width are equal to or greater than inter_mode_size_thr, the normal INTER MODE that needs to encode mvd may be used, otherwise, the normal INTER MODE that needs to encode mvd may not be used.

That is, in the embodiment of the present application, when limiting the number of inter prediction modes, the limiting process may be directly performed on the number of inter prediction modes of all prediction blocks, or different limiting modes may be used for different sizes of current blocks with reference to the size parameter of the current block.

For example, in the present application, for a prediction block with a larger size parameter, such as a prediction block with a size of 64×64 or 32×32, the determination of the inter prediction mode of the current block may be performed using candidate inter prediction modes, wherein the candidate inter prediction modes may include a general inter mode that needs to encode mvd, while for a prediction block with a smaller size parameter, such as a prediction block with a size of 8×8, other inter prediction modes than the candidate inter prediction mode may be selected to perform the determination of the inter prediction mode of the current block, i.e., the number of inter prediction modes that may be used by the current block is limited, because the effect of the fine angle difference is not significant for a prediction block with a smaller size parameter.

It is to be understood that, in the present application, a size threshold, such as a third size threshold, may be set first, if the size parameter of the current block is smaller than the third size threshold, then it is considered that the subtle angle difference does not cause a larger influence, so that inter prediction modes other than the candidate inter prediction modes may be selected to determine the inter prediction mode of the current block, and when the size parameter is greater than the third size threshold, it is necessary to limit the inter prediction modes that can be selected by the current block through the candidate inter prediction modes.

It is understood that in the present application, the third size threshold may be the same as or different from the second size threshold.

Further, in the embodiment of the present application, after determining the prediction mode used by the current block, information such as the prediction mode of the current block needs to be saved for reference by a block of a subsequent codec, where the block of the subsequent codec includes, but is not limited to, a prediction block adjacent to the current block in the spatial domain and a prediction block that uses information of the current block in the time domain. In particular, if the current block uses two different prediction modes, the saving of the prediction mode of the current block may be performed according to various methods.

It will be appreciated that in embodiments of the present application, if the current block uses two different prediction modes, the first prediction mode and/or the second prediction mode is set to the prediction mode of the current block when the current block is used to predict other blocks to be predicted.

Illustratively, in the present application, if the current block uses two different prediction modes, i.e., an inter prediction mode and an intra prediction mode, a portion of the minimum units of the current block select to save the intra prediction mode and a portion of the minimum units select to save the inter prediction mode. In particular, a method similar to GPM or AWP may be used. If either the GPM or AWP is used in the same codec standard or codec, logic similar to this approach may be used. This allows for multiplexing of partly identical logic. For example, AVS3 inter prediction uses AWP, then in AVS3, two different prediction modes may be saved using logic similar to that used by AWP to save two different motion information, i.e., if only the prediction block determined by the intra prediction mode is used at the location corresponding to a minimum unit, then that minimum unit saves that intra prediction mode; if a minimum unit corresponds to a position where only a prediction block determined by an inter prediction mode is used, the minimum unit stores the inter prediction mode; if a position corresponding to a minimum unit uses both the prediction block determined by the intra prediction mode and the prediction block determined by the inter prediction mode, one of the intra prediction mode and the inter prediction mode is selected for saving.

For example, in the present application, if the current block uses two different prediction modes, i.e., an inter prediction mode and an intra prediction mode, only one prediction mode may be selected for saving for all the minimum units corresponding to the entire current block.

It is to be understood that, in the present application, although the intra prediction mode and the inter motion information are stored for reference to other blocks, the intra prediction mode and the inter motion information are encoded differently, and they are distinguished. The use of intra prediction modes or inter prediction modes for each minimum unit selection therefore introduces a certain complexity. Taking AVS3 as an example, the intra-prediction of the current AVS3 has 66 modes in total, and 2 most probable modes MPMs (most probable mode, MPM) are constructed before each current block is predicted, if the current block uses MPM, one of the 2 MPMs is selected, otherwise, one prediction mode needs to be selected from the 64 modes. For inter prediction, the current block before prediction will construct a motion vector predictor (motion vector prediction/predictor, MVP), and motion vector bias (motion vector difference, MVD) needs to be encoded by selection both when MVP is used and when MVP is not directly used. It is considered that the closer the MVP is to the coding efficiency is, the higher the MPM has no effect basically if it is not selected.

It can be seen that the complexity is lower because only one prediction mode is selected for storage for all the minimum units corresponding to the whole current block.

Alternatively, in the present application, all weight matrix derived modes of the current block all select intra prediction modes.

Alternatively, in the present application, all weight matrix derivation modes of the current block all select the inter prediction mode.

Optionally, in the present application, all the smallest units of some weight matrix derivation modes of the current block select the intra prediction mode, and all the smallest units of some weight matrix derivation modes select the inter prediction mode. I.e. determining whether all the smallest units of the current block hold intra prediction mode or inter prediction mode based on the weight matrix derivation mode.

Alternatively, in the present application, the mode number calculation of the mode is derived using the weight matrix of the current block.

Alternatively, in the present application, a pattern number lookup table of the pattern is derived using the weight matrix of the current block. For example, if the same weight matrix derivation mode as AWP is used, the intra prediction mode is selected if the mode number of the weight matrix derivation mode corresponds to 0, and the inter prediction mode is selected if the mode number of the weight matrix derivation mode corresponds to 1. Accordingly, in the present application, if a prediction mode held by a minimum unit of a current block is an intra prediction mode, the minimum unit is simultaneously marked as using intra prediction. If a minimum unit of the current block holds a prediction mode that is an inter prediction mode, then this minimum unit is marked as using inter prediction at the same time.

The embodiment of the application provides a prediction method, wherein a decoder analyzes a code stream and determines a prediction mode parameter of a current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode. That is, in the embodiment of the present application, the codec may determine two different prediction blocks of the current block through two different prediction modes, and then may combine the two different prediction blocks according to the weight matrix, so as to finally obtain a more complex prediction block, thereby improving the accuracy of prediction, that is, the prediction method provided by the present application may improve the prediction quality and improve the compression performance.

An embodiment of the present application provides a prediction method applied to an encoder, and fig. 17 is a schematic diagram showing an implementation flow of the prediction method, and as shown in fig. 17, the method for performing prediction by the encoder may include the following steps:

Step 301, determining a prediction mode parameter of the current block.

In an embodiment of the present application, an encoder may first determine a prediction mode parameter of a current block.

It is understood that in the embodiment of the present application, the prediction mode parameter may be understood as a flag bit indicating whether a different prediction mode is used. Specifically, the encoder may determine a variable as a prediction mode parameter, so that the determination of the prediction mode parameter may be achieved by the value of the variable.

It should be noted that, in an embodiment of the present application, if the prediction mode parameter indicates that the current block may use two different prediction modes for determining the prediction value, the two different prediction modes include, but are not limited to, an intra prediction mode, an inter prediction mode, an intra block copy mode, a Palette mode Palette, a string prediction mode, and the like.

Illustratively, in the present application, an encoder may perform prediction processing on a current block using an inter prediction mode and an intra prediction mode, respectively; the encoder may also predict the current block using an intra prediction mode and an intra block copy mode, respectively.

That is, in the present application, if the prediction mode parameter indicates that the current block can perform the determination of the prediction value using two different prediction modes, the combination of the two different prediction modes may be various, and the present application is not particularly limited thereto.

It can be understood that, since the prediction method provided by the embodiment of the present application needs to generate two different prediction blocks by using two different prediction modes respectively, and then weighting is performed according to a weight matrix to obtain a new prediction block, in order to reduce the complexity of prediction, and meanwhile consider the trade-off of compression performance and complexity, in the embodiment of the present application, it is possible to limit that two different prediction modes are not used for some prediction blocks with different sizes. Thus, in the present application, an encoder may determine a size parameter of a current block first, and then determine whether the current block uses two different prediction modes according to the size parameter.

It should be noted that, in the embodiment of the present application, the size parameter of the current block may include a height and a width of the current block, and thus, the encoder may limit the use of two different prediction modes, i.e., limit the size of a prediction block that can use the two different prediction modes, using the height and the width of the current block.

Further, in an embodiment of the present application, the encoder may further determine an aspect ratio of the current block according to the width and length of the current block after determining the height and width of the current block; and then limited by the aspect ratio of the current block.

It should be noted that, in the embodiment of the present application, the current block size limiting method and the aspect ratio limiting method may be used simultaneously. For example, the encoder may be set to perform prediction processing using two different prediction modes when the height of the current block is 8 or more and the width of the current block is 8 or more and the height-to-width (or width-to-width) ratio of the current block is 4 or less, or else, the prediction processing is performed by default of not two different prediction modes.

Further, in an embodiment of the present application, the encoder may also implement restriction of prediction blocks that can use two different prediction modes through restriction of the total number of pixels.

Illustratively, in the present application, the encoder may determine the total number of pixels of the current block, and then further determine whether the current block can determine the prediction of the current block using different prediction modes according to the total number of pixels and the sixth threshold. It can be seen that one possible limitation is to use two different prediction modes only if the number of pixels of the prediction block is greater than (or equal to) the sixth threshold. The sixth threshold may be 8, 16, 32, etc.

Illustratively, in the present application, the encoder may also determine the intra prediction mode parameter of the current block before determining the prediction mode parameter of the current block; determining a prediction mode parameter of the current block when the intra prediction mode parameter indicates that the current block uses the intra prediction mode to determine a prediction value of the current block; determining a first prediction mode and a second prediction mode when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode or the second prediction mode is an intra prediction mode; finally, a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.

Illustratively, in the present application, the encoder may also determine inter prediction mode parameters of the current block before determining the prediction mode parameters of the current block; determining a prediction mode parameter of the current block when the inter prediction mode parameter indicates that the current block uses the inter prediction mode to determine a prediction value of the current block; determining a first prediction mode and a second prediction mode when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode or the second prediction mode is an inter prediction mode; and finally, determining the predicted value of the current block based on the first predicted mode and the second predicted mode.

It should be noted that, in the embodiment of the present application, the video image may be divided into a plurality of image blocks, and the current block is each image block to be currently encoded, which may be referred to as an encoding block CB. Here, each of the encoding blocks may include a first image component, a second image component, and a third image component. Specifically, in the present application, assuming that prediction processing is performed and that the first image component is a luminance component, that is, the image component to be predicted is a luminance component, a coded block to be predicted may be referred to as a luminance block; alternatively, assuming that the prediction process is performed and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, the encoded block to be predicted may be referred to as a chrominance block.

Step 302, when the prediction mode parameter indicates that the current block uses different prediction modes to determine the prediction value of the current block, determining a first prediction mode and a second prediction mode of the current block; the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode.

In an embodiment of the present application, after determining the prediction mode parameter of the current block, if the prediction mode parameter indicates that the current block determines a prediction value of the current block using two different prediction modes, the encoder may further determine a first prediction mode and a second prediction mode that the current block needs to use.

It should be noted that, in the embodiment of the present application, if the prediction mode parameter indicates that the current block may use two different prediction modes for determining the prediction value, the encoder may determine the two different prediction modes, i.e., the first prediction mode and the second prediction mode. The first prediction mode and the second prediction mode may include, but are not limited to, an intra prediction mode, an inter prediction mode, an intra block copy mode, a Palette mode Palette, a string prediction mode, and the like.

Illustratively, in the present application, the first prediction mode and the second prediction mode are any two modes of the following plurality of prediction modes, respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode. For example, the first prediction mode may be an inter prediction mode and the second prediction mode may be a palette mode.

Further, in an embodiment of the present application, the encoder may determine a first mode flag and a second mode flag when determining a first prediction mode and a second prediction mode of the current block; and then determining the prediction mode indicated by the first mode mark as a first prediction mode, and determining the prediction mode indicated by the second mode mark as a second prediction mode.

That is, in the present application, the encoder may determine the corresponding first prediction mode and second prediction mode directly from the first mode flag and the second mode flag, i.e., the first mode flag and the second mode flag may directly indicate the prediction mode

Illustratively, in an embodiment of the present application, the first mode flag determined by the encoder indicates that the first prediction mode is an intra prediction mode, and the second mode flag determined indicates that the second prediction mode is an inter prediction mode, and then the encoder may perform prediction processing on the current block using the intra prediction mode and the inter prediction mode.

Further, in an embodiment of the present application, the encoder may further determine a first mode flag and a second mode flag when determining the first prediction mode and the second prediction mode of the current block; then, determining a first prediction mode in the candidate modes according to the first mode mark; and finally, determining a second prediction mode in other modes of the candidate modes except the first prediction mode according to the second mode mark.

That is, in the present application, the encoder may determine the first prediction mode from among the candidate modes according to the first mode flag, and then determine the second prediction mode from among the candidate modes excluding the first prediction mode according to the second mode flag, i.e., the first mode flag and the second mode flag may be used to select the prediction mode based on the candidate modes.

Illustratively, in an embodiment of the present application, the candidate modes include inter prediction mode, intra prediction mode, string prediction mode, intra block copy prediction mode, and the like, the first mode flag determined by the encoder indicates that one of the candidate modes is the first prediction mode, and the second mode flag determined indicates that the other of the candidate modes is the second prediction mode.

As can be seen from the above, in the present application, the encoder can determine flag information indicating the first prediction mode and the second prediction mode in addition to determining prediction mode parameters indicating that prediction processing can be performed using two different prediction modes.

Alternatively, in the present application, for prediction modes using only two determinations, such as intra prediction and inter prediction, the encoder may directly determine the first mode flag and the second mode flag to determine the first prediction mode and the second prediction mode without selecting the prediction modes.

Alternatively, in the present application, if two prediction modes need to be selected among a plurality of prediction modes, for example, in intra SCC encoding, the prediction modes that can be used include (normal) intra prediction, intra block copy mode, string prediction SP, then the first mode flag and the second mode flag determined by the encoder may indicate which prediction mode the first prediction mode is, and which prediction mode the second prediction mode is. For example, 1 to two bits may be used to indicate which prediction mode the first prediction mode is, e.g. 10 for intra prediction, 00 for IBC, and 01 for SP, and only two choices for the second prediction mode are needed, only 1 bit flag being needed.

In addition to the control flag, the encoder needs to transmit various flags such as information indicating the first mode index, the second mode index, and the weight matrix.

Illustratively, in the present application, an encoder may determine a first mode index and a second mode index and write the first mode index and the second mode index to a bitstream. The first mode index may be used to indicate which prediction mode the first prediction mode is, and the second mode index may be used to indicate which prediction mode the second prediction mode is, for example, the first mode index may indicate that the first prediction mode is a DC mode in the intra prediction mode and the second mode index may indicate that the second prediction mode is an affine mode in the inter prediction mode.

Illustratively, in the present application, an encoder may determine a first weight flag and a second weight flag, and write the first weight flag and the second weight flag to a code stream; then, according to the first weight mark, a first weight corresponding to the first predicted value can be determined in the weight matrix; and determining a first weight corresponding to the first predicted value in the weight matrix according to the first weight mark.

That is, in an embodiment of the present application, if the current block uses two determined prediction modes, such as intra prediction and inter prediction. The encoder may also determine mode information of the inter prediction mode and may also determine information such as motion vector deviation of the inter prediction mode. Accordingly, the encoder may also determine mode information of the intra prediction mode.

Further, the encoder may also determine information of the weight matrix. In particular, it may also be determined which mode is the first prediction mode for the weight matrix and which mode is the second prediction mode for the weight matrix. Alternatively, it is determined which prediction block is the first prediction block for the weight matrix and which prediction block is the second prediction block for the weight matrix.

That is, in an embodiment of the present application, if the current block uses two determined prediction modes, such as intra prediction and inter prediction. In addition to the flag of whether two different prediction modes are used or not, if the current block uses the prediction method proposed by the present application, prediction mode information of inter prediction needs to be transmitted, and if it is necessary, information such as motion vector deviation needs to be transmitted. Prediction mode information of intra prediction needs to be transmitted. Information determining the weight matrix needs to be transmitted. It is also necessary to determine which mode is the first prediction mode for the weight matrix and which mode is the second prediction mode for the weight matrix. Or determining which prediction block is the first prediction block for the weight matrix and which prediction block is the second prediction block for the weight matrix.

Step 303, determining a prediction value of the current block based on the first prediction mode and the second prediction mode.

In an embodiment of the present application, if the prediction mode parameter indicates that the current block determines the prediction value of the current block using different prediction modes, the encoder may further determine the prediction value of the current block based on the first prediction mode and the second prediction mode after determining the first prediction mode and the second prediction mode of the current block.

Specifically, in an embodiment of the present application, when determining a prediction value of a current block based on a first prediction mode and a second prediction mode, an encoder may determine a first prediction value corresponding to the current block based on the first prediction mode; determining a second prediction value corresponding to the current block based on the second prediction mode; finally, a prediction value of the current block may be further determined based on the first prediction value and the second prediction value.

It will be appreciated that, in an embodiment of the present application, after determining the prediction mode parameter of the current block, i.e. after step 301, the method for performing prediction processing by the encoder may further include the steps of:

step 304, when the prediction mode parameter indicates that the current block uses different prediction modes to determine the prediction value of the current block, determining the weight matrix derivation mode of the current block.

Step 305, determining the weight matrix of the current block according to the weight matrix export mode.

In an embodiment of the present application, after determining the prediction mode parameter of the current block, if the prediction mode parameter indicates that the current block determines the prediction value of the current block using a different prediction mode, the encoder may further determine a weight matrix derivation mode of the current block, and may further determine the weight matrix of the current block according to the weight matrix derivation mode.

Further, in an embodiment of the present application, when determining the prediction value of the current block based on the first prediction mode and the second prediction mode, the encoder may finally determine the prediction value of the current block based on the first prediction mode, the second prediction mode, and the weight matrix.

Further, in embodiments of the present application, the encoder may employ methods similar to GPM or AWP to effect the determination of the weight matrix. In particular, if GPM or AWP is used in the same codec standard or coder, the method can be used to determine the weight matrix, which can do part of the same logic. If AVS3 inter prediction uses AWP, then the method of AWP may be used in AVS3 to determine the weight matrix. Of course, different methods than GPM or AWP in the same codec standard or coder, such as using different numbers of modes, or different transition region algorithms, or using different parameters, etc. may also be used.

In the exemplary method, the AWP mode is used in inter prediction in AVS3, the number of weight matrices of AWP is 56, in the present invention, the number of weight matrices of AWP is 64, and the number of weight matrices of AWP is 56, for example, the number of weight matrices of the first 56 is the same as the number of weight matrices of AWP, and each of the remaining 8 weight matrices has only one weight value, and the weight values are 1,2, … …,7 and 8 respectively. For these 8 weight matrices, a total weight of 16, i.e. a weight of 1 represents 1:15, weight value 2 represents 2:14 weight. Thus, when the mode numbers of the 64 weight matrices are binarized, codewords of 6 bits can be used.

Further, in the present application, when determining the prediction value of the current block based on the first prediction mode, the second prediction mode, and the weight matrix, the encoder may determine the first prediction value of the current block according to the first prediction mode; determining a second predicted value of the current block according to the second prediction mode; and then, carrying out weighted average calculation on the first predicted value and the second predicted value by using a weight matrix, and finally obtaining the predicted value of the current block.

Alternatively, in the present application, the encoder may determine two different prediction blocks (a first prediction value and a second prediction value) for the current block, respectively, determine one weight matrix, and then may combine the two different prediction blocks according to the weight matrix to obtain a new prediction block.

Alternatively, in the present application, the encoder may determine a prediction block (first prediction value) according to the first prediction mode, determine a prediction block (second prediction value) according to the second prediction mode, determine a weight matrix, and combine two different prediction blocks according to the weight matrix to obtain a new prediction block.

It can be seen that when intra-prediction is performed on the current block, a first prediction block of the current block may be determined using a first prediction mode, a second prediction block of the current block may be determined using a second prediction mode, and then the first prediction block and the second prediction block may be combined using a weight matrix, to finally obtain a new prediction block. That is, based on the first prediction mode and the second prediction mode, two prediction blocks which are not identical to each other may be determined for the current block, a weight matrix may be determined, and the two different prediction blocks may be combined according to the weight matrix to obtain a new prediction block.

In the following, two different prediction modes, i.e. an intra prediction mode and an inter prediction mode, are taken as examples to illustrate the prediction method according to the present application, wherein predMatrix _new [i][j]predMatrix for a new predictor of point (i, j) in the current block _intra [i][j]As intra prediction value of point (i, j) in current block, predMatrix _inter For inter prediction value of point (i, j) in the current block, weight matrix is weight matrix of the current block, weight matrix [ i ]][j]Weight value for point (i, j) in the current block:

alternatively, in the present application, the encoder may encode the intra prediction mode first and then the inter prediction mode. Or encoding inter prediction modes before intra prediction modes. For the weight matrix, the intra-prediction block is a first prediction block and the inter-prediction block is a second prediction block:

due to weight matrix [ i ]][j]May be fractional or fractional, and weight matrix i may be used to avoid increasing the computational complexity of the fractional or fractional][j]Amplification 2 ⁿ Namely, the rewriting is:

alternatively, the intra-predicted block is the second predicted block for the weight matrix, and the inter-predicted block is the first predicted block:

alternatively, in the present application, the encoder may encode the intra prediction mode first and then the inter prediction mode. Or encoding inter prediction modes before intra prediction modes. A flag is then encoded to indicate which prediction block is the first prediction block, wherein the flag may also be encoded first. Specifically, if this flag is 0, it indicates that the intra prediction block is the first prediction block predMatrix ₀ [i][j]The inter prediction block is the second prediction block predMatrix ₁ [i][j]：

alternatively, in the present application, the encoder may encode a flag to indicate whether the intra prediction mode is encoded first and then the inter prediction mode is encoded first or whether the inter prediction mode is encoded first and then the intra prediction mode is encoded first, wherein a prediction block of the prediction mode encoded first is used as a first prediction block and a prediction block of the prediction mode encoded later is used as a second prediction block.

Further, in an embodiment of the present application, the method for predicting by the encoder may further include the steps of:

step 401, determining a prediction mode parameter of the current block.

Step 402, when the prediction mode parameter indicates that the current block uses different prediction modes to determine the prediction value of the current block, determining a first prediction mode of the current block, and determining the first prediction value of the current block according to the first prediction mode.

Step 403, determining a second prediction mode of the current block, and determining a second prediction value of the current block according to the second prediction mode.

Step 404, determining a predicted value of the current block based on the first predicted value and the second predicted value.

It will be appreciated that, in the present application, after determining that the current block uses a different prediction mode to determine the prediction value of the current block, the encoder uses the prediction mode encoded earlier as the first prediction mode, the prediction block of the prediction mode encoded earlier as the first prediction block (i.e., the first prediction value), the prediction mode encoded later as the second prediction mode, and the prediction block of the prediction mode encoded later as the second prediction block (i.e., the second prediction value).

Illustratively, in the present application, if the two prediction modes of the current block are an inter prediction mode and an intra prediction mode, whether the intra prediction mode is encoded first and then the inter prediction mode is encoded first or the inter prediction mode is encoded first and then the intra prediction mode is encoded, the encoder may directly take the prediction block of the prediction mode encoded first as the first prediction block and the prediction block of the prediction mode encoded later as the second prediction block.

Correspondingly, in the embodiment of the application, decoding can be performed at the decoding end according to the rule corresponding to the encoding end.

Illustratively, in the present application, if the current block uses two different prediction modes, i.e., an inter prediction mode and an intra prediction mode, a portion of the minimum units of the current block select to save the intra prediction mode and a portion of the minimum units select to save the inter prediction mode. In particular, a method similar to GPM or AWP may be used. If either the GPM or AWP is used in the same codec standard or coder, logic similar to this approach may be used. This allows for multiplexing of partly identical logic. For example, AVS3 inter prediction uses AWP, then in AVS3, two different prediction modes may be saved using logic similar to that used by AWP to save two different motion information, i.e., if only the prediction block determined by the intra prediction mode is used at the location corresponding to a minimum unit, then that minimum unit saves that intra prediction mode; if a minimum unit corresponds to a position where only a prediction block determined by an inter prediction mode is used, the minimum unit stores the inter prediction mode; if a position corresponding to a minimum unit uses both the prediction block determined by the intra prediction mode and the prediction block determined by the inter prediction mode, one of the intra prediction mode and the inter prediction mode is selected for saving.

Further, in the embodiment of the present application, the effect of the superposition of the prediction method proposed by the present application and some improved prediction modes may not be obvious, so that the mutually exclusive relationship between the prediction method proposed by the present application and a certain or some prediction modes may be set. That is, if the current block uses the prediction method proposed by the present application, it can be determined that the mutually exclusive prediction mode is not used, so that it is not necessary to transmit a flag of whether the mutually exclusive prediction mode is used in the code stream. Alternatively, if the current block uses the mutually exclusive prediction mode, it can be determined that the current block does not use the prediction method according to the present application, so that it is not necessary to transmit a flag in the code stream as to whether to use the prediction method according to the present application. Thereby saving the transmission of unnecessary flags in the code stream, which is better in overall compression performance.

Optionally, in the present application, in AVS3, the prediction method proposed by the present application is mutually exclusive with IPF.

Optionally, in the present application, in AVS3, the prediction method proposed by the present application is mutually exclusive with the Derived Tree (DT). Wherein DT techniques may divide the current CU into rectangular PUs.

Optionally, in the present application, the prediction method proposed by the present application is mutually exclusive with the bi-directional prediction.

Optionally, in the present application, the prediction method proposed by the present application is mutually exclusive to affine prediction.

Further, in the present application, there are some prediction modes, such as BIO and DMVR, which do not need to be used as a separate flag for the current block, but are automatically turned on if the condition is met. If the prediction method proposed by the present application is mutually exclusive with such a prediction mode, such mutually exclusive prediction modes are not used any more if the current block uses the prediction method proposed by the present application. For example, inter prediction, which may set the prediction method proposed by the present application, may use bi-directional prediction, but does not use BIO and DMVR.

The embodiment of the application provides a prediction method, wherein an encoder determines a prediction mode parameter of a current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode. That is, in the embodiment of the present application, the codec may determine two different prediction blocks of the current block through two different prediction modes, and then may combine the two different prediction blocks according to the weight matrix, so as to finally obtain a more complex prediction block, thereby improving the accuracy of prediction, that is, the prediction method provided by the present application may improve the prediction quality and improve the compression performance.

Based on the above-described embodiments, in still another embodiment of the present application, fig. 18 is a schematic diagram of the composition structure of an encoder according to an embodiment of the present application, and as shown in fig. 18, an encoder 300 according to an embodiment of the present application may include a first determining portion 301,

the first determining part 301 is configured to determine a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.

Fig. 19 is a schematic diagram of a second constituent structure of an encoder according to an embodiment of the present application, as shown in fig. 19, the encoder 300 according to an embodiment of the present application may further include a first processor 302, a first memory 303 storing executable instructions of the first processor 302, a first communication interface 304, and a first bus 305 for connecting the first processor 302, the first memory 303, and the first communication interface 304.

Further, in an embodiment of the present application, the first processor 302 is configured to determine a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.

Fig. 20 is a schematic diagram of the composition structure of a decoder according to an embodiment of the present application, as shown in fig. 20, a decoder 400 according to an embodiment of the present application may include a decoding portion 401, a second determining portion 402,

the decoding part 401 is configured to parse the code stream;

the second determining part 402 is configured to determine a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.

Fig. 21 is a schematic diagram of a second constituent structure of the decoder according to the embodiment of the present application, as shown in fig. 21, the decoder 400 according to the embodiment of the present application may further include a second processor 403, a second memory 404 storing executable instructions of the second processor 403, a second communication interface 405, and a second bus 406 for connecting the second processor 403, the second memory 404, and the second communication interface 405.

Further, in the embodiment of the present application, the second processor 403 is configured to parse the code stream to determine a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.

In addition, each functional module in the present embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional modules.

The integrated units, if implemented in the form of software functional modules, may be stored in a computer-readable storage medium, if not sold or used as separate products, and based on this understanding, the technical solution of the present embodiment may be embodied essentially or partly in the form of a software product, or all or part of the technical solution may be embodied in a storage medium, which includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or processor (processor) to perform all or part of the steps of the method of the present embodiment. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The embodiment of the application provides a coder-decoder, which analyzes a code stream and determines a prediction mode parameter of a current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode. The encoder determines a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode. That is, in the embodiment of the present application, the codec may determine two different prediction blocks of the current block through two different prediction modes, and then may combine the two different prediction blocks according to the weight matrix, so as to finally obtain a more complex prediction block, thereby improving the accuracy of prediction, that is, the prediction method provided by the present application may improve the prediction quality and improve the compression performance.

Embodiments of the present application provide a computer-readable storage medium and a computer-readable storage medium having a program stored thereon, which when executed by a processor, implements the method as described in the above embodiments.

Specifically, the program instruction corresponding to one prediction method in the present embodiment may be stored on a storage medium such as an optical disc, a hard disc, or a usb disk, and when the program instruction corresponding to one prediction method in the storage medium is read or executed by an electronic device, the method includes the following steps:

The method also comprises the following steps:

determining a prediction mode parameter of the current block;

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of implementations of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block and/or flow of the flowchart illustrations and/or block diagrams, and combinations of blocks and/or flow diagrams in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart block or blocks and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart block or blocks and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks and/or block diagram block or blocks.

The foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the present application.

Industrial applicability

Claims

A prediction method applied to a decoder, the method comprising:

analyzing the code stream and determining the prediction mode parameter of the current block;

determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode;

a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.
The method of claim 1, wherein the first prediction mode and the second prediction mode are an intra prediction mode and an inter prediction mode, respectively.
The method of claim 1, wherein the determining the prediction value of the current block based on the first prediction mode and the second prediction mode comprises:

determining a first prediction value corresponding to the current block based on the first prediction mode;

determining a second prediction value corresponding to the current block based on the second prediction mode;

A predictor of the current block is determined based on the first predictor and the second predictor.
The method of claim 1, wherein the method further comprises:

determining a weight matrix derivation mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using a different prediction mode;

and determining the weight matrix of the current block according to the weight matrix deriving mode.
The method of claim 1, wherein the determining the prediction value of the current block based on the first prediction mode and the second prediction mode comprises:

a prediction value of the current block is determined based on the first prediction mode, the second prediction mode, and the weight matrix.
The method of claim 5, wherein the determining the prediction value of the current block based on the first prediction mode, the second prediction mode, and the weight matrix comprises:

determining a first prediction value of the current block according to the first prediction mode; determining a second predicted value of the current block according to the second prediction mode;

and carrying out weighted average calculation on the first predicted value and the second predicted value by using the weight matrix to obtain the predicted value of the current block.
The method of claim 1 or 2, wherein the determining the first prediction mode and the second prediction mode of the current block comprises:

analyzing the code stream, and determining a first mode mark and a second mode mark;

and determining the prediction mode indicated by the first mode mark as the first prediction mode, and determining the prediction mode indicated by the second mode mark as the second prediction mode.
The method of claim 1 or 2, wherein the determining the first prediction mode and the second prediction mode of the current block comprises:

analyzing the code stream, and determining a first mode mark and a second mode mark;

determining the first prediction mode in candidate modes according to the first mode mark;

and determining the second prediction mode in other modes of the candidate modes, which are other than the first prediction mode, according to the second mode mark.
The method of claim 4, wherein,

if the weight matrix includes at least one weight value, the at least one weight value is greater than a lower weight threshold and less than an upper weight threshold.
The method of claim 4, wherein,

If the weight matrix comprises two weight values, the two weight values are respectively a weight lower limit threshold value and a weight upper limit threshold value.
The method according to claim 9 or 10, wherein,

and setting the lower weight threshold value as 0 and the upper weight threshold value as 1.
The method of claim 1, wherein the method further comprises:

determining a size parameter of the current block;

and determining the prediction mode parameter of the current block according to the size parameter.
The method of claim 12, wherein the dimensional parameters include width and height; the method further comprises the steps of:

if the width is greater than a first threshold and the height is greater than a second threshold, determining that the current block uses a different prediction mode to determine a prediction value of the current block; or alternatively, the process may be performed,

if the width is less than a third threshold and the height is less than a fourth threshold, determining that the current block uses a different prediction mode to determine a prediction value for the current block.
The method of claim 13, wherein,

determining an aspect ratio of the current block according to the width and the length;

if the aspect ratio is smaller than a fifth threshold value, determining that the current block uses different prediction modes to determine a predicted value of the current block; or alternatively, the process may be performed,

And if the reciprocal of the aspect ratio is smaller than the fifth threshold, determining that the current block uses different prediction modes to determine the prediction value of the current block.
The method of claim 1, wherein the method further comprises:

determining a total number of pixels of the current block;

and judging whether the current block uses different prediction modes to determine the prediction value of the current block according to the total number of pixels and a sixth threshold.
The method of claim 2, wherein the method further comprises:

determining the weight matrix deriving mode according to a first mode index range; the first mode index range is used for limiting the index sequence number of the weight matrix derived mode.
The method of claim 4, wherein the method further comprises:

if the size parameter of the current block is smaller than a first size threshold, determining the weight matrix deriving mode according to a first mode index range; the first mode index range is used for limiting the index sequence number of the weight matrix derived mode.
The method of claim 16 or 17, wherein the first pattern index range includes a first lower threshold and a first upper threshold, the method further comprising:

If the index sequence number of the weight matrix deriving mode is smaller than the first lower threshold, setting the index sequence number of the weight matrix deriving mode as the first lower threshold;

and if the index sequence number of the weight matrix deriving mode is larger than the first upper threshold, setting the index sequence number of the weight matrix deriving mode as the first upper threshold.
The method of claim 2, wherein the method further comprises:

determining the intra-prediction mode according to a second mode index range; the second mode index range is used for limiting the index sequence number of the intra-frame prediction mode.
The method of claim 2, wherein the method further comprises:

if the size parameter of the current block is smaller than a second size threshold, determining the intra-frame prediction mode according to a second mode index range; the second mode index range is used for limiting the index sequence number of the intra-frame prediction mode.
The method of claim 2, wherein the method further comprises:

and determining the inter prediction mode according to the candidate inter prediction mode.
The method of claim 2, wherein the method further comprises:

And if the size parameter of the current block is larger than a third size threshold, determining the inter prediction mode according to the candidate inter prediction mode.
The method of claim 1, wherein the method further comprises:

and when the current block is used for predicting other blocks to be predicted, setting the first prediction mode and/or the second prediction mode as the prediction mode of the current block.
The method of claim 1, wherein the method further comprises:

analyzing the code stream and determining the prediction mode parameters of the current block;

when the prediction mode parameter indicates that the current block uses different prediction modes to determine a prediction value of the current block, determining a first prediction mode of the current block, and determining a first prediction value of the current block according to the first prediction mode;

determining a second prediction mode of the current block, and determining a second prediction value of the current block according to the second prediction mode;

a predictor of the current block is determined based on the first predictor and the second predictor.
The method according to claim 1 or 2, wherein the method further comprises:

and analyzing the code stream and determining the first mode index and the second mode index.
The method of claim 5, wherein the method further comprises:

analyzing the code stream, and determining a first weight mark and a second weight mark;

determining a first weight corresponding to the first predicted value in the weight matrix according to the first weight mark; and determining a first weight corresponding to the first predicted value in the weight matrix according to the first weight mark.
A prediction method applied to an encoder, the method comprising:

determining a prediction mode parameter of the current block;

determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode;

a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.
The method of claim 27, wherein the first and second prediction modes are an intra prediction mode and an inter prediction mode, respectively.
The method of claim 27, wherein the determining the prediction value of the current block based on the first prediction mode and the second prediction mode comprises:

determining a first prediction value corresponding to the current block based on the first prediction mode;

determining a second prediction value corresponding to the current block based on the second prediction mode;

a predictor of the current block is determined based on the first predictor and the second predictor.
The method of claim 27, wherein the method further comprises:

determining a weight matrix derivation mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using a different prediction mode;

and determining the weight matrix of the current block according to the weight matrix deriving mode.
The method of claim 27, wherein the determining the prediction value of the current block based on the first prediction mode and the second prediction mode comprises:

a prediction value of the current block is determined based on the first prediction mode, the second prediction mode, and the weight matrix.
The method of claim 31, wherein the determining the prediction value of the current block based on the first prediction mode, the second prediction mode, and the weight matrix comprises:

Determining a first prediction value of the current block according to the first prediction mode; determining a second predicted value of the current block according to the second prediction mode;

and carrying out weighted average calculation on the first predicted value and the second predicted value by using the weight matrix to obtain the predicted value of the current block.
The method of claim 28, wherein the determining the first prediction mode and the second prediction mode of the current block comprises:

determining a first mode flag and a second mode flag;

and determining the prediction mode indicated by the first mode mark as the first prediction mode, and determining the prediction mode indicated by the second mode mark as the second prediction mode.
The method of claim 27, wherein the determining the first prediction mode and the second prediction mode of the current block comprises:

determining a first mode flag and a second mode flag;

determining the first prediction mode in candidate modes according to the first mode mark;

and determining the second prediction mode in other modes of the candidate modes, which are other than the first prediction mode, according to the second mode mark.
The method of claim 30, wherein,

if the weight matrix includes at least one weight value, the at least one weight value is greater than a lower weight threshold and less than an upper weight threshold.
The method of claim 30, wherein,

if the weight matrix comprises two weight values, the two weight values are respectively a weight lower limit threshold value and a weight upper limit threshold value.
The method of claim 35 or 36, wherein,

and setting the lower weight threshold value as 0 and the upper weight threshold value as 1.
The method of claim 27, wherein the method further comprises:

determining a size parameter of the current block;

and determining the prediction mode parameter of the current block according to the size parameter.
The method of claim 38, wherein the dimensional parameters include width and height; the method further comprises the steps of:

if the width is greater than a first threshold and the height is greater than a second threshold, determining that the prediction mode parameter indicates the current block to determine a prediction value for the current block using a different prediction mode; or alternatively, the process may be performed,

if the width is less than a third threshold and the height is less than a fourth threshold, determining that the prediction mode parameter indicates that the current block uses a different prediction mode to determine a prediction value for the current block.
The method of claim 27, wherein the method further comprises:

determining a total number of pixels of the current block;

and judging whether the prediction mode parameter indicates the current block to use different prediction modes to determine the prediction value of the current block according to the total number of pixels and a sixth threshold.
The method of claim 39, wherein,

determining an aspect ratio of the current block according to the width and the length;

if the aspect ratio is greater than a fifth threshold, determining that the prediction mode parameter indicates the current block to determine a prediction value of the current block using a different prediction mode; or alternatively, the process may be performed,

if the inverse aspect ratio is greater than the fifth threshold, determining that the prediction mode parameter indicates that the current block uses a different prediction mode to determine a prediction value for the current block.
The method of claim 30, wherein the method further comprises:

determining the weight matrix deriving mode according to a first mode index range; the first mode index range is used for limiting the index sequence number of the weight matrix derived mode.
The method of claim 30, wherein the method further comprises:

If the size parameter of the current block is smaller than a first size threshold, determining the weight matrix deriving mode according to a first mode index range; the first mode index range is used for limiting the index sequence number of the weight matrix derived mode.
The method of claim 42 or 43, wherein the first pattern index range includes a first lower threshold and a first upper threshold, the method further comprising:

if the index sequence number of the weight matrix deriving mode is smaller than the first lower threshold, setting the index sequence number of the weight matrix deriving mode as the first lower threshold;

and if the index sequence number of the weight matrix deriving mode is larger than the first upper threshold, setting the index sequence number of the weight matrix deriving mode as the first upper threshold.
The method of claim 28, wherein the method further comprises:

determining the intra-prediction mode according to a second mode index range; the second mode index range is used for limiting the index sequence number of the intra-frame prediction mode.
The method of claim 28, wherein the method further comprises:

If the size parameter of the current block is smaller than a second size threshold, determining the one intra-frame prediction mode according to a second mode index range; the second mode index range is used for limiting the index sequence number of the intra-frame prediction mode.
The method of claim 28, wherein the method further comprises:

and determining the inter prediction mode according to the candidate inter prediction mode.
The method of claim 28, wherein the method further comprises:

and if the size parameter of the current block is larger than a third size threshold, determining the inter prediction mode according to the candidate inter prediction mode.
The method of claim 27, wherein the method further comprises:

and when the current block is used for predicting other blocks to be predicted, setting the first prediction mode and/or the second prediction mode as the prediction mode of the current block.
The method of claim 27, wherein the method further comprises:

and when the prediction mode parameter indicates the current block to determine the prediction value of the current block by using different prediction modes, not writing the prediction mode parameter of a preset mutual exclusion mode into the code stream.
The method of claim 27, wherein the method further comprises:

and writing the first mode index and the second mode index into the code stream.
The method of claim 31, wherein the method further comprises:

determining a first weight mark and a second weight mark, and writing the first weight mark and the second weight mark into the code stream;

determining a first weight corresponding to the first predicted value in the weight matrix according to the first weight mark; and determining a first weight corresponding to the first predicted value in the weight matrix according to the first weight mark.
The method of claim 27, wherein the method further comprises:

determining a prediction mode parameter of the current block;

when the prediction mode parameter indicates that the current block uses different prediction modes to determine a prediction value of the current block, determining a first prediction mode of the current block, and determining a first prediction value of the current block according to the first prediction mode;

determining a second prediction mode of the current block, and determining a second prediction value of the current block according to the second prediction mode;

A predictor of the current block is determined based on the first predictor and the second predictor.
An encoder, the encoder comprising: the first determining section may determine, based on the first determining section,

the first determining part is configured to determine a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.
An encoder comprising a first processor, a first memory storing instructions executable by the first processor, which when executed, implement the method of any of claims 1-26.
A decoder, the decoder comprising: a decoding section, a second determining section

The decoding part is configured to parse the code stream;

the second determining part is configured to determine a prediction mode parameter of the current block; determining a first prediction mode and a second prediction mode of the current block when the prediction mode parameter indicates that the current block determines a prediction value of the current block using different prediction modes; wherein the first prediction mode and the second prediction mode are any two modes among the following prediction modes respectively: intra prediction mode, inter prediction mode, intra block copy mode, palette mode, string prediction mode; a prediction value of the current block is determined based on the first prediction mode and the second prediction mode.
A decoder comprising a second processor, a second memory storing instructions executable by the second processor, which when executed, implement the method of any of claims 27-53.
A computer storage medium storing a computer program which when executed by a first processor performs the method of any one of claims 1-26 or when executed by a second processor performs the method of any one of claims 27-53.