WO2021127923A1 - Procédé de prédiction d'image, codeur, décodeur et support de stockage - Google Patents

Procédé de prédiction d'image, codeur, décodeur et support de stockage Download PDF

Info

Publication number
WO2021127923A1
WO2021127923A1 PCT/CN2019/127635 CN2019127635W WO2021127923A1 WO 2021127923 A1 WO2021127923 A1 WO 2021127923A1 CN 2019127635 W CN2019127635 W CN 2019127635W WO 2021127923 A1 WO2021127923 A1 WO 2021127923A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
sampling
prediction block
mip
current block
Prior art date
Application number
PCT/CN2019/127635
Other languages
English (en)
Chinese (zh)
Inventor
霍俊彦
马彦卓
万帅
杨付正
李新伟
王海鑫
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to CN201980087536.2A priority Critical patent/CN113302934B/zh
Priority to PCT/CN2019/127635 priority patent/WO2021127923A1/fr
Publication of WO2021127923A1 publication Critical patent/WO2021127923A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding

Definitions

  • the embodiments of the present application relate to the field of image processing technologies, and in particular, to an image prediction method, an encoder, a decoder, and a storage medium.
  • VVC High Efficiency Video Coding
  • VTM VVC Test Model
  • MIP matrix-based intra prediction
  • the embodiments of the present application provide an image prediction method, an encoder, a decoder, and a storage medium, which can make full use of reference pixel information and can improve the accuracy of MIP prediction.
  • an embodiment of the present application provides an image prediction method, which is applied to an encoder or a decoder, and the method includes:
  • the first filtering process includes a first up-sampling process
  • the first up-sampling process represents Up-sampling the MIP prediction block in the horizontal direction and then up-sampling in the vertical direction to obtain a first up-sampling block, up-sampling the MIP prediction block in the vertical direction and then up-sampling in the horizontal direction to obtain a second up-sampling block, Taking a weighted average value of the first up-sampling block and the second up-sampling block as the first filtered prediction block;
  • the first filtered prediction block is set as an intra prediction block of the current block; wherein the intra prediction block includes prediction sample values of all pixel positions in the current block.
  • an encoder which includes a first determination unit, a first calculation unit, a first processing unit, and a first prediction unit;
  • the first determining unit is configured to determine the MIP input sampling matrix of the current block
  • the first calculation unit is configured to calculate the MIP prediction block of the current block according to the MIP input sampling matrix, wherein the MIP prediction block includes predicted sampling values of some pixel positions in the current block;
  • the first processing unit is configured to perform a first filtering process on the MIP prediction block according to the parameters of the current block to obtain a first filtered prediction block, wherein the first filtering process includes a first upsampling process, and
  • the first up-sampling processing means that the MIP prediction block is up-sampled in the horizontal direction and then up-sampled in the vertical direction to obtain a first up-sampling block, and the MIP prediction block is up-sampled in the vertical direction and then up-sampling in the horizontal direction.
  • the first prediction unit is configured to set the first filtered prediction block as an intra prediction block of the current block; wherein the intra prediction block includes prediction sample values of all pixel positions in the current block.
  • an encoder which includes a first memory and a first processor; wherein,
  • the first memory is used to store a computer program that can run on the first processor
  • the first processor is configured to execute the method described in the first aspect when the computer program is running.
  • an embodiment of the present application provides a decoder, which includes a second determination unit, a second calculation unit, a second processing unit, and a second prediction unit; wherein,
  • the second determining unit is configured to determine the MIP input sampling matrix of the current block
  • the second calculation unit is configured to calculate the MIP prediction block of the current block according to the MIP input sampling matrix, wherein the MIP prediction block includes predicted sampling values of some pixel positions in the current block;
  • the second processing unit is configured to perform a first filtering process on the MIP prediction block according to the parameters of the current block to obtain a first filtered prediction block, wherein the first filtering process includes a first upsampling process, and
  • the first up-sampling processing means that the MIP prediction block is up-sampled in the horizontal direction and then up-sampled in the vertical direction to obtain a first up-sampling block, and the MIP prediction block is up-sampled in the vertical direction and then up-sampling in the horizontal direction.
  • the second prediction unit is configured to set the first filtered prediction block as an intra prediction block of the current block; wherein the intra prediction block includes prediction sample values of all pixel positions in the current block.
  • an embodiment of the present application provides a decoder, which includes a second memory and a second processor; wherein,
  • the second memory is used to store a computer program that can run on the second processor
  • the second processor is configured to execute the method described in the first aspect when the computer program is running.
  • an embodiment of the present application provides a computer storage medium that stores an image prediction program, and when the image prediction program is executed by a first processor or a second processor, the implementation is as described in the first aspect. Methods.
  • the embodiment of the present application provides an image prediction method, encoder, decoder, and storage medium.
  • the MIP input sampling matrix of the current block is determined; the MIP prediction block of the current block is calculated according to the MIP input sampling matrix, where the MIP prediction block Containing predicted sampling values of some pixel positions in the current block; performing a first filtering process on the MIP prediction block according to the parameters of the current block to obtain a first filtering prediction block, wherein the first filtering process includes a first upsampling process;
  • the first filtered prediction block is set as the intra prediction block of the current block, and the intra prediction block contains the predicted sample values of all pixel positions in the current block.
  • the first upsampling process means that the MIP prediction block is upsampled in the horizontal direction and then the vertical direction is upsampled to obtain the first upsampling block, and the MIP prediction block is upsampled in the vertical direction and then the horizontal direction is upsampled to obtain the first upsampling block.
  • the weighted average of the first up-sampling block and the second up-sampling block is used as the first filtered prediction block; in this way, the reference of the current block can be fully utilized through the first up-sampling block and the second up-sampling block Pixel information can also fully consider the prediction characteristics of the prediction sampling in different directions, so that the final up-sampling result of the intra-frame prediction block is more accurate, which not only improves the accuracy of MIP prediction, but also improves the video image quality.
  • Figure 1 is a flow chart of a MIP prediction process provided by related technical solutions
  • FIG. 2A is a block diagram of a video encoding system provided by an embodiment of this application.
  • 2B is a block diagram of a video decoding system provided by an embodiment of this application.
  • FIG. 3 is a schematic flowchart of an image prediction method provided by an embodiment of the application.
  • FIG. 4 is a schematic flowchart of another image prediction method provided by an embodiment of the application.
  • FIG. 5A is a schematic structural diagram of a reference pixel position provided by an embodiment of the application.
  • 5B is a schematic structural diagram of a reference pixel down-sampling process provided by an embodiment of the application.
  • 6A is a schematic structural diagram of a buffer filling provided by related technical solutions
  • 6B is a schematic structural diagram of another buffer filling provided by related technical solutions.
  • FIG. 7 is a schematic flowchart of another image prediction method provided by an embodiment of the application.
  • FIG. 8 is a schematic diagram of a structure for generating a predicted value according to an embodiment of the application.
  • FIG. 9 is a schematic structural diagram of filling prediction sampling pixels provided by an embodiment of the application.
  • FIG. 10 is a schematic structural diagram of up-sampling and horizontal interpolation provided by an embodiment of this application.
  • FIG. 11 is a schematic diagram of the distribution of linear interpolation weights according to an embodiment of the application.
  • FIG. 12 is a schematic structural diagram of an up-sampling and vertical interpolation provided by an embodiment of this application.
  • FIG. 13 is a schematic structural diagram of another upsampling first vertical interpolation provided by an embodiment of this application.
  • FIG. 14 is a schematic structural diagram of another up-sampling and re-horizontal interpolation provided by an embodiment of this application.
  • 15 is a schematic diagram of another structure for filling prediction sampling pixels provided by an embodiment of the application.
  • FIG. 16 is a schematic diagram of a structure for predicting the row interpolation of sampling pixels provided by an embodiment of the application.
  • FIG. 17 is a schematic diagram of a structure for predicting the row and column interpolation of sampling pixels provided by an embodiment of the application.
  • FIG. 18 is a schematic structural diagram of upsampling and interpolation within a sub-block provided by an embodiment of this application.
  • FIG. 19 is a schematic structural diagram of yet another upsampling interpolation provided by an embodiment of this application.
  • FIG. 20 is a schematic diagram of the composition structure of an encoder provided by an embodiment of the application.
  • FIG. 21 is a schematic diagram of a specific hardware structure of an encoder provided by an embodiment of the application.
  • FIG. 22 is a schematic diagram of the composition structure of a decoder provided by an embodiment of the application.
  • FIG. 23 is a schematic diagram of a specific hardware structure of a decoder provided by an embodiment of the application.
  • the first image component, the second image component, and the third image component are generally used to represent the coding block (CB); among them, the three image components are a luminance component and a blue chrominance component. And a red chrominance component, specifically, the luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V; in this way, the video image can be in YCbCr format It can also be expressed in YUV format.
  • CB coding block
  • the first image component may be a luminance component
  • the second image component may be a blue chrominance component
  • the third image component may be a red chrominance component
  • the input data of MIP prediction mainly includes: the position of the current block (xTbCmp, yTbCmp), the MIP prediction mode applied to the current block (which can be represented by modeId), the height of the current block (represented by nTbH), and the width of the current block ( It is indicated by nTbW) and the transposition processing indication flag (which can be indicated by isTransposed) whether to be transposed or not, etc.;
  • the output data of MIP prediction mainly includes the prediction block of the current block.
  • the MIP prediction process can be divided into four steps: configuring core parameters 11, obtaining reference pixels 12, constructing input samples 13, and generating predicted values 14.
  • the core parameter 11 can be divided into three categories, and mipSizeId is used to record the type of the current block; and for different types of current blocks, refer to the number of sampling points and matrix multiplication The number of output sampling points is different.
  • the upper and left blocks of the current block are both already coded blocks.
  • the reference pixels of the MIP technology are the reconstructed values of the upper row of pixels and the left column of the current block.
  • the process of obtaining the reference pixel adjacent to the upper side of the current block (indicated by refT) and the reference pixel adjacent to the left side (indicated by refL) is the process of obtaining the reference pixels.
  • this step is used for the input of matrix multiplication, and mainly includes: obtaining reference samples 131, constructing reference sample buffer 132 and deriving matrix multiplication input samples 133; among them, the process of obtaining reference samples is downsampling
  • the process of constructing the reference sampling buffer 132 may include a filling method 1321 of the buffer when transposition is not required and a filling method 1322 of the buffer when transposition is required.
  • this step is used to obtain the MIP predicted value of the current block, which may mainly include: constructing a matrix multiplication output sampling block 141, matrix multiplication output sampling clamp 142, matrix multiplication output sampling transposition 143 and generating MIP The final predicted value 144; wherein, the construction of the matrix multiplication output sampling block 141 may include obtaining the weight matrix 1411, obtaining the shift factor and offset factor 1412, and the matrix multiplication operation 1413. Generating the MIP final predicted value 144 may also include generating unneeded upper The sampled predicted value 1441 and the generated predicted value 1442 that need to be up-sampled. In this way, after the four steps, the predicted value of at least one pixel in the current block can be obtained.
  • the current solution is a fixed up-sampling method, that is, “first upsampling in the horizontal direction, then Upsampling in the vertical direction”.
  • this fixed up-sampling method some reference pixels will not participate in the up-sampling process.
  • some of the reference pixels on the left side are not used as references for up-sampling, that is, in the up-sampling process.
  • the reference level of the above characteristics is also different, which makes the upsampling results obtained by the two upsampling methods different, that is, the MIP prediction value is different; so, for any current block, if you use "first horizontal upsampling, then vertical direction"
  • the fixed upsampling method of "upsampling” will make the upsampling results inaccurate and reduce the accuracy of MIP prediction; thus, more deviations between the current predicted value and the original value will be introduced in the upsampling process of MIP prediction , Resulting in a decrease in video image quality.
  • the embodiment of the application provides an image prediction method. Firstly, determine the MIP input sampling matrix of the current block; then calculate the MIP prediction block of the current block according to the MIP input sampling matrix, where the MIP prediction block contains some pixel positions in the current block. Predict the sample value; and then perform the first filtering process on the MIP prediction block according to the parameters of the current block to obtain the first filtered prediction block, where the first filtering process includes the first upsampling process; finally, the first filtered prediction block is set to The intra-frame prediction block of the current block, which contains the predicted sample values of all pixel positions in the current block.
  • the first upsampling process means that the MIP prediction block is upsampled in the horizontal direction and then the vertical direction is upsampled to obtain the first upsampling block, and the MIP prediction block is upsampled in the vertical direction and then the horizontal direction is upsampled to obtain the first upsampling block.
  • the weighted average of the first up-sampling block and the second up-sampling block is used as the first filtered prediction block; in this way, the reference of the current block can be fully utilized through the first up-sampling block and the second up-sampling block Pixel information can also fully consider the prediction characteristics of the prediction sampling in different directions, so that the final up-sampling result of the intra-frame prediction block is more accurate, which not only improves the accuracy of MIP prediction, but also improves the video image quality.
  • the video encoding system 10 includes a transform and quantization unit 101, an intra-frame estimation unit 102, and an intra-frame
  • the encoding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (Context-based Adaptive Binary Arithmatic Coding, CABAC).
  • CABAC Context-based Adaptive Binary Arithmatic Coding
  • a video coding block can be obtained by dividing the coding tree unit (CTU), and then the residual pixel information obtained after intra-frame or inter-frame prediction is paired by the transformation and quantization unit 101
  • the video coding block is transformed, including transforming the residual information from the pixel domain to the transform domain, and quantizing the resulting transform coefficients to further reduce the bit rate;
  • the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used for Perform intra-frame prediction on the video encoding block; specifically, the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to determine the intra-frame prediction mode to be used to encode the video encoding block;
  • the motion compensation unit 104 and the motion estimation unit 105 is used to perform inter-frame predictive coding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information;
  • the motion estimation performed by the motion estimation unit 105 is a method for generating a motion vector In the process, the motion vector can estimate the motion of
  • the context content can be based on adjacent coding blocks, can be used to encode information indicating the determined intra prediction mode, and output the code stream of the video signal; and the decoded image buffer unit 110 is used to store reconstructed video coding blocks for Forecast reference. As the video image encoding progresses, new reconstructed video encoding blocks will be continuously generated, and these reconstructed video encoding blocks will be stored in the decoded image buffer unit 110.
  • the video decoding system 20 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, and an intra-frame
  • the prediction unit 203, the motion compensation unit 204, the filtering unit 205, and the decoded image buffer unit 206, etc. wherein the decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement deblocking filtering and SAO filtering.
  • the code stream of the video signal is output; the code stream is input into the video decoding system 20, and first passes through the decoding unit 201 to obtain the decoded transform coefficient; the transform coefficient is passed through
  • the inverse transform and inverse quantization unit 202 performs processing to generate a residual block in the pixel domain; the intra prediction unit 203 can be used to generate data based on the determined intra prediction mode and the data from the previous decoded block of the current frame or picture The prediction data of the current video decoding block; the motion compensation unit 204 determines the prediction information for the video decoding block by analyzing the motion vector and other associated syntax elements, and uses the prediction information to generate the predictability of the video decoding block being decoded Block; by summing the residual block from the inverse transform and inverse quantization unit 202 and the corresponding predictive block generated by the intra prediction unit 203 or the motion compensation unit 204 to form a decoded video block; the decoded video signal Through the filtering unit 205 in order to remove the block effect artifacts, the video quality can
  • the image prediction method in the embodiment of the present application is mainly applied to the part of the intra prediction unit 103 shown in FIG. 2A and the part of the intra prediction unit 203 shown in FIG. 2B. That is to say, the image prediction method in the embodiment of this application can be applied to a video encoding system, a video decoding system, or even a video encoding system and a video decoding system at the same time. However, the embodiment of this application does not apply to it. Specific restrictions.
  • the "current block” specifically refers to the current coding block in the intra prediction; when the image prediction method is applied to the part of the intra prediction unit 203 At this time, the “current block” specifically refers to the current decoded block in intra-frame prediction.
  • FIG. 3 shows a schematic flowchart of an image prediction method provided by an embodiment of the present application.
  • the method may include:
  • each image block currently to be encoded may be referred to as a coding block (CB).
  • each encoding block may include a first image component, a second image component, and a third image component; and the current block is the encoding of the first image component, the second image component, or the third image component currently to be predicted in the video image.
  • the current block is the encoding of the first image component, the second image component, or the third image component currently to be predicted in the video image.
  • the current block performs the first image component prediction, and the first image component is the brightness component, that is, the image component to be predicted is the brightness component, then the current block can also be called the brightness block; or, it is assumed that the current block performs the second image component Prediction, and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, then the current block can also be called a chrominance block.
  • the MIP core parameters need to be configured first.
  • the MIP technology divides the current block into three categories based on the width and height of the current block, and mipSizeId represents the type of the current block.
  • mipSizeId represents the type of the current block.
  • the number of reference samples each side requires boundySize reference sample points
  • the number of matrix multiplication input samples inSize each side requires boundySize reference sample points
  • the number of matrix multiplication output samples arranged as predSize ⁇ predSize
  • an implementation manner for determining the MIP prediction block size index sequence number mipSizeId is:
  • the value of mipSizeId can be set to 0;
  • the value of mipSizeId can be set to 1;
  • the value of mipSizeId can be set to 2.
  • Another implementation manner for determining the MIP prediction block size index sequence number mipSizeId according to the size of the current block is:
  • the value of mipSizeId can be set to 0;
  • the value of mipSizeId can be set to 1;
  • the value of mipSizeId can be set to 2.
  • the number of adjacent boundary reference samples (variable boundarySize) and the size of the MIP prediction block (variable Is predSize, the size of the MIP prediction block is predSize ⁇ predSize), and the number of input samples used in the MIP matrix multiplication process (variable is inSize) is calculated.
  • the calculation formula is as follows:
  • the value of boundarySize can be 2, and the value of predSize can be 4; that is, at this time, the reference pixel selects two pixels for each side, and the matrix
  • the output of the multiplication is a 4 ⁇ 4 MIP prediction block; or, when the value of mipSizeId is 1, the value of boundarySize can be 4, and the value of predSize can be 4; that is, the reference pixel at this time is each side Select four pixels, and the matrix multiplication output is a 4 ⁇ 4 MIP prediction block; or, when the value of mipSizeId is 2, the value of boundarySize can be 4, and the value of predSize can be 8; that is, this At this time, the reference pixel selects four pixels for each side, and the matrix multiplication output is an 8 ⁇ 8 MIP prediction block.
  • the LUT shown in Table 2 can also be used according to the value of mipSizeId, and the values of boundarySize, inSize, and predSize can be determined at the same time.
  • the value of boundarySize can be 2
  • the value of inSize can be 4, and the value of predSize can be 4; that is, the reference pixel at this time is each Two pixels are selected, the number of input sampling points of the matrix multiplication is four, and the output of the matrix multiplication is a 4 ⁇ 4 MIP prediction block; or, when the value of mipSizeId is 1, the value of boundarySize can be 4, and the value of inSize The value can be 8 and the value of predSize can be 4; that is, at this time, the reference pixel selects four pixels for each side, the number of input sampling points for matrix multiplication is eight, and the output of matrix multiplication is 4 ⁇ 4 MIP prediction block; or, when the value of mipSizeId is 2, the value of boundarySize can be 4, the value of inSize can be 7, and the value of predSize can be 8; that is, the reference pixel is each Four pixels are selected, the number of input
  • the determining the MIP input sampling matrix of the current block may include:
  • S301-1 Determine an adjacent reference sample set of the current block; wherein the adjacent reference sample set includes at least one reference sample value;
  • the adjacent reference sample set can be from the current It is filtered from the reference pixels in the adjacent area on the left side and the adjacent area on the upper side of the block, or it can be obtained by filtering from the reference pixels in the adjacent area on the left side and the adjacent area on the lower left side of the current block. It may be obtained by filtering from the reference pixels in the upper adjacent area and the upper right adjacent area of the current block, which is not specifically limited in the embodiment of the present application.
  • the determining the adjacent reference sample set of the current block may include:
  • At least one side of the current block includes at least one of the following: an upper side, an upper right side, a left side, and a lower left side;
  • the adjacent reference sample set of the current block is determined.
  • At least one side of the current block can be the upper side (also called the upper side), the upper right side (also called the right upper side), or the left side (also called the left column) Or the lower left side (also referred to as the lower left side); it can even be a combination of two sides, such as the upper side and the left side; however, the embodiment of the present application does not limit it.
  • the determining the adjacent reference sample set of the current block may include :
  • Acquiring reference pixels adjacent to at least one side of the current block wherein at least one side of the current block includes an upper side and/or a left side;
  • the adjacent reference sample set of the current block is determined.
  • At least one side of the current block may include the left side of the current block and/or the upper side of the current block; that is, at least one side of the current block may refer to the upper side of the current block, or it may refer to the upper side of the current block.
  • the left side of the current block may even refer to the upper side and the left side of the current block, which is not specifically limited in the embodiment of the present application.
  • the adjacent reference sample set at this time can be from the reference pixels adjacent to the left side of the current block and the upper side of the current block Filtered from adjacent reference pixels.
  • the adjacent area on the left is the valid area and the adjacent area on the upper side is the invalid area
  • the adjacent reference sample set can be from the left side adjacent to the current block.
  • the adjacent area on the left is an invalid area and the adjacent area on the upper side is an effective area
  • the adjacent reference sample set can be from the reference adjacent to the upper side of the current block. Filtered in the pixel.
  • the filtering here may be down-sampling filtering processing or low-pass filtering processing, which is not specifically limited in the embodiment of the present application.
  • the reference pixels of the MIP technology can be the reconstructed value of the adjacent reference pixels on the current block and the left column of the current block The reconstructed value of the adjacent reference pixel.
  • the reference pixel of the current block may be obtained by obtaining the reference pixel corresponding to the upper side and the reference pixel corresponding to the left side of the current block.
  • FIG. 5A it shows a schematic diagram of the position of a reference pixel provided by an embodiment of the present application.
  • the adjacent area on the left and the adjacent area on the upper side are all valid areas; among them, the reference pixels corresponding to the upper side of the current block are pixels filled with gray, which can be represented by refT; the left side of the current block
  • the reference pixels corresponding to the edges are pixels filled with diagonal lines, which can be represented by refL, then the reference pixels of the current block can include refT and refL, and the adjacent reference sample set is obtained by filtering refT and refL.
  • the same method as the acquisition of reference pixels in the traditional intra prediction technology can be used for filling.
  • the obtained reference pixels (including refT and refL) need to be down-sampled at this time. Get the adjacent reference sample set.
  • the determining the adjacent reference sample set of the current block according to the obtained reference pixels may include:
  • the adjacent reference sample set of the current block is formed.
  • the method may also include:
  • the second filter processing includes down-sampling filter processing or low-pass filter processing.
  • the determining the adjacent reference sample set of the current block according to the obtained reference pixels may include:
  • the reference sampling value corresponding to the sampling position is selected from the obtained reference pixels, and the parameter sampling value obtained by the selection is formed into the adjacent reference sampling set.
  • the determining the sampling position of the reference pixel based on at least one edge of the current block may include:
  • boundarySize reference samples need to be sampled separately on each reference edge of the current block (including the upper side and the left side), and the value of boundarySize is related to the type of mipSizeId of the current block, as shown in Table 1 or Table 2 shows.
  • the number of reference pixels (that is, the side length of the current block) is nTbS; at this time, the number of reference samples that need to be down-sampled is boundarySize.
  • the sampling rate of each reference edge can be expressed by bDwn.
  • an average operation is performed on each bDwn reference pixel, and the average value obtained is used as a sampling point in the reference sample redS, and the redS can be calculated according to formula (2).
  • S can be replaced by W and H respectively, where W represents the upper side and H represents the left side.
  • two parts of the reference sample can be obtained: the upper-side reference sample redT obtained by down-sampling the upper-side reference pixel refT of the current block, and the left-side reference sample redT of the current block
  • the left side reference sample redL obtained by down-sampling the reference pixel refL that is, the down-sampling value (boundarySize total) obtained after the down-sampling of the upper side reference pixel refT is stored in the one-dimensional array redT
  • the down-sampling value (a total of boundarySize) obtained after down-sampling of the reference pixel refL on the left side is stored in a one-dimensional array redL.
  • the redL obtained by down-sampling on the left side includes two reference pixels, namely reference pixel 1 and reference pixel 2, and down-sampling is performed on the upper side.
  • the processed redT includes two reference pixels, that is, reference pixel 3 and reference pixel 4; in this way, the adjacent reference sample set of the current block includes four reference pixels.
  • an input reference sample set can be constructed first to facilitate the subsequent construction of the MIP input sampling matrix.
  • the buffering the adjacent reference sample set to construct the input reference sample set may include:
  • the method of optimizing the utilization rate distortion determines the value of the transposition processing indicator
  • the reference sample value corresponding to the upper side of the adjacent reference sample set is stored before the reference sample value corresponding to the left side in the buffer, then The buffer is determined to be the input reference sample set; or,
  • the reference sample value corresponding to the upper side of the adjacent reference sample set is stored after the reference sample value corresponding to the left side, and all The buffer is transposed, and the transposed buffer is determined as the input reference sample set.
  • the utilization rate distortion optimization (Rate Distortion Optimization, RDO) method can determine the value of the transposition processing indicator; for example, the value of the first generation of transposition processing and the value of the transposition processing are calculated separately.
  • Second generation value if the value of the first generation is less than the value of the second generation, the value of the transposition processing indicator can be determined to be 1, at this time, the reference sample value corresponding to the upper side of the adjacent reference sample set can be stored in After the reference sample value corresponding to the left side, or the reference sample value corresponding to the left side of the adjacent reference sample set is stored before the reference sample value corresponding to the upper side, it needs to be transposed; if the value of the first generation is not Less than the value of the second generation, at this time, it can be determined that the value of the transposition processing indicator is 0.
  • the reference sample value corresponding to the upper side of the adjacent reference sample set can be stored before the reference sample value corresponding to the left side. Or the reference sample value corresponding to the left side of the adjacent reference sample set is stored after the reference sample value corresponding to the upper side, that is, no transposition processing is required.
  • the buffering the adjacent reference sample set to construct the input reference sample set may include:
  • the reference sample value corresponding to the upper side of the adjacent reference sample set is stored before the reference sample value corresponding to the left side in the buffer, then The buffer is determined to be the input reference sample set; or,
  • the reference sample value corresponding to the upper side of the adjacent reference sample set is stored after the reference sample value corresponding to the left side, and all The buffer is transposed, and the transposed buffer is determined as the input reference sample set.
  • the value of the transposition processing indicator can be directly obtained by parsing the code stream; then, according to the value of the transposition processing indicator, it is determined whether the buffer needs to be transposed.
  • RedT includes reference pixel 3 and reference pixel 4; in this way, the buffer sequence in pTemp is reference pixel 3, reference pixel 4, reference pixel 1, and reference pixel 2; because the reference pixels corresponding to the upper side are all stored in the left side corresponding Before the reference pixels of, there is no need to transpose at this time, and the resulting buffer is the input reference sample set.
  • redL includes reference pixel 1 and reference pixel 2
  • redT includes reference pixel 3 and reference pixel 4
  • the buffer order in pTemp is reference pixel 1, reference pixel 2, Reference pixel 3 and reference pixel 4; since the reference pixels corresponding to the upper side are all stored after the reference pixels corresponding to the left side, transpose is required at this time, and then the transposed buffer is determined as the input reference sample set .
  • S301-3 Determine the MIP input sampling matrix according to the input reference sample value set.
  • the matrix multiplication input samples are inputs for the matrix multiplication process; by multiplying with the corresponding matrix, the matrix multiplication output samples (denoted by predMip) can be obtained.
  • the matrix multiplication input sample P is determined by the buffer (represented by pTemp), the type of the current block (represented by mipSizeId), and the bit depth value corresponding to the image component to be predicted (represented by BitDepth), and the matrix multiplication input sample P
  • the included input reference sample number inSize is only related to the type mipSizeId of the current block, and finally the x-th input sample value (indicated by P[x]) in the input sample matrix can be obtained.
  • the sample value corresponding to each position in the other positions in the input sampling matrix is obtained by subtracting the sample value corresponding to the zeroth position in pTemp from the sample value at the next position in the corresponding position in pTemp;
  • S302 Calculate the MIP prediction block of the current block according to the MIP input sampling matrix, where the MIP prediction block includes prediction sampling values of some pixel positions in the current block;
  • the weight matrix (which can be represented by mWeight), the shift factor (which can be represented by sW), and the offset factor (which can be represented by fO) can be combined to perform matrix multiplication operations, so as to be able to Get the MIP prediction block.
  • the weight matrix mWeight, the shift factor sW, and the offset factor f0 can be obtained according to the type of the current block mipSizeId and the MIP prediction mode modeId; then the input sampling matrix P[x], weight matrix mWeight, The shift factor sW and the offset factor f0 are input to the process of matrix multiplication to obtain the MIP prediction block output by the matrix multiplication (which can be represented by predMip).
  • the method may include:
  • S401 Obtain the MIP prediction block of the current block according to the MIP input sampling matrix, where the MIP prediction block includes prediction sampling values of some pixel positions in the current block;
  • the weight matrix (indicated by mWeight), shift factor (indicated by sW) and offset factor (indicated by fO) corresponding to the current block can also be obtained, and then use the first preset
  • the calculation model is designed to calculate the temporary prediction value of at least one pixel in the MIP prediction block (represented by predMip) to obtain the MIP prediction block.
  • the obtaining the MIP prediction block of the current block according to the MIP input sampling matrix may include:
  • a first preset calculation model is used to perform matrix multiplication processing on the input sampling matrix, the weight matrix, the shift factor, and the offset factor to obtain the MIP prediction block by calculation.
  • a weight matrix table is established in advance, and the weight matrix table is stored in the encoder or decoder.
  • the weight matrix mWeight[x][y] that needs to be used for the current block can be determined by looking up the table.
  • the size of the weight matrix mWeight[x][y] is only related to the type mipSizeId of the current block, as shown in Table 3.
  • the number of columns is the number of matrix multiplication input samples inSize, and the number of rows is matrix multiplication.
  • the number of output samples is predSized ⁇ predSized.
  • a shift factor table is also pre-established, and the shift factor table is also stored in the encoder or decoder.
  • the determination of the shift factor sW may include the following three methods:
  • the shift factor sW that needs to be used in the matrix multiplication can also be determined by looking up the table.
  • the shift factor sW that needs to be used in the matrix multiplication can also be determined by looking up the table based only on the type mipSizeId of the current block.
  • sW can also be set to a fixed constant value. For example, sW can be set to 5 for different types of current blocks and different MIP prediction modes; or, for different types of current blocks and different MIP prediction modes, sW can be set to 6; or, for different types of current blocks and different MIP prediction modes, sW can be set to 6; For the current block and different MIP prediction modes, sW can be set to 7.
  • an offset factor table is also pre-established, and the offset factor table is also stored in the encoder or decoder.
  • the determination of the offset factor f0 may also include the following three methods:
  • the shift factor f0 that needs to be used in the matrix multiplication can also be determined by looking up the table.
  • the shift factor f0 that needs to be used in the matrix multiplication can also be determined by looking up the table based only on the type mipSizeId of the current block.
  • fO can also be set to a fixed constant value, usually in the range of 0-100.
  • fO can be set to 32; or, for different types of current blocks and different MIP prediction modes, fO can be set to 46; or, for different types of current blocks and different MIP prediction modes.
  • fO can be set to 56; or, for different types of current blocks and different MIP prediction modes, fO can be set to 66.
  • the weight matrix mWeight[x][y], the shift factor sW and the offset factor fO can be determined by looking up the table, and the MIP prediction can be calculated Block predMip[x][y], and arrange the sampling points in predMip[x][y] into a matrix form according to predSize ⁇ predSize.
  • the first preset calculation model is as follows,
  • [x][y] represents the position coordinates of the pixel, x represents the horizontal direction, and y represents the vertical direction; inSize represents the number of input samples, and predSize represents the side length of the MIP prediction block predMip.
  • the temporary prediction value of at least one pixel in the MIP prediction block predMip can be calculated to obtain the MIP prediction block.
  • mWeight[x][y] is a MIP matrix
  • the corresponding MIP matrix can be selected from a set of preset MIP matrices according to the value of mipSizeId and the value of modeId, that is, the weight matrix.
  • modeId represents the MIP prediction mode of the current block; on the encoder side, the rate-distortion optimization method can still be used to determine the MIP prediction mode of the current block, that is, the value of modeId, and then the value of modeId is written into the code stream ; On the decoder side, the value of modeId can be obtained by parsing the code stream.
  • S402 Clamping processing is performed on the prediction sample value in the MIP prediction block to obtain the MIP prediction block of the current block;
  • the temporary prediction value of at least one pixel in the MIP prediction block may be clamped. Specifically, if the temporary prediction value of the sampling point in the MIP prediction block is less than 0, then it can be set to 0; if the temporary prediction value of the sampling point in the MIP prediction block is greater than (1 ⁇ BitDepth)-1, then it can be set Set to (1 ⁇ BitDepth)-1; thus, the predicted values of all sampling points in the MIP prediction block can be clamped between 0 and (1 ⁇ BitDepth)-1.
  • the prediction value of at least one pixel in the MIP prediction block can be obtained, and the range of the prediction value is between 0 and (1 ⁇ BitDepth)-1; and then according to the transposition
  • the sign bit isTransposed determines whether transposition processing is needed to determine the final MIP prediction block.
  • S403 Determine whether to perform transposition processing on the MIP prediction block
  • the determining whether to perform transposition processing on the MIP prediction block may include:
  • the utilization rate distortion optimization method separately calculates the first-generation value of transposing the MIP prediction block and the second-generation value of not transposing the MIP prediction block;
  • the first-generation value is less than the second-generation value, it is determined to transpose the MIP prediction block; or,
  • the first generation value is not less than the second generation value, it is determined not to perform transposition processing on the MIP prediction block.
  • the determining whether to perform transposition processing on the MIP prediction block may include:
  • the transposition processing indication flag it is determined whether to perform transposition processing on the MIP prediction block.
  • the transposed processing indicator is represented by isTransposed, and it can be judged whether the MIP prediction block needs transposed processing according to the value of isTransposed. Specifically, on the encoder side, if the value of the first generation is less than the value of the second generation, the value of isTransposed is set to 1, then it can be determined that the MIP prediction block needs to be transposed; or, if the value of the first generation is not If the value is less than the second generation value, the value of isTransposed is set to 0 at this time, and it can be determined that the MIP prediction block does not need to be transposed.
  • the value of the transposition processing indicator can be obtained by parsing the code stream; if the value of isTransposed is analyzed to be 1, then it can be determined that the MIP prediction block needs to be transposed; or, if isTransposed is resolved The value of is 0, then it can be determined that the MIP prediction block does not need to be transposed.
  • the transposed MIP prediction block after transposing the MIP prediction block, the transposed MIP prediction block can be obtained, and the transposed MIP prediction block is set as the MIP prediction block.
  • the subsequent steps can be performed according to the MIP prediction block and the parameters of the current block, such as performing the first filtering process on the MIP prediction block, so as to finally determine the intra prediction block of the current block.
  • S303 Perform first filtering processing on the MIP prediction block according to the parameters of the current block to obtain a first filtered prediction block;
  • S304 Set the first filtered prediction block as an intra prediction block of the current block; wherein the intra prediction block includes prediction sample values of all pixel positions in the current block.
  • the current block can be the current brightness block, and finally the brightness prediction block of the current brightness block can be obtained, and the brightness prediction block provides the brightness prediction value of at least one pixel; or, Assuming that the image component to be predicted is a chroma component, the current block can be the current chroma block, and finally the chroma prediction block of the current chroma block can be obtained.
  • the chroma prediction block provides the chroma prediction value of at least one pixel.
  • the first filtering process includes a first up-sampling process
  • the first up-sampling process indicates that the MIP prediction block is up-sampled in the horizontal direction and then up-sampled in the vertical direction to obtain the first up-sampling block, Perform vertical upsampling on the MIP prediction block and then perform horizontal upsampling to obtain a second upsampling block, and use the weighted average of the first upsampling block and the second upsampling block as the first filter Prediction block.
  • the size of the MIP prediction block only includes two types: 4 ⁇ 4 MIP prediction block and 8 ⁇ 8 MIP prediction block; in this way, the size of the current block is the same as the size of the MIP prediction block. It may be the same or different; that is, the sampling value corresponding to the MIP prediction block may not be able to fill the current block, so that the generation of the final prediction value may require the first filtering process of the MIP prediction block, that is, by judging the MIP prediction Whether the size of the block is the same as the size of the current block to determine whether to perform the first filtering process on the MIP prediction block.
  • the method may further include:
  • S406 Determine whether the size of the MIP prediction block is the same as the size of the current block
  • the size of the MIP prediction block only includes two types: 4 ⁇ 4 MIP prediction block and 8 ⁇ 8 MIP prediction block; in this way, the size of the current block is the same as that of the MIP prediction block.
  • the size may be the same or different; that is, the sampling value corresponding to the MIP prediction block may not be able to fill the current block, so that the generation of the final prediction value may require an up-sampling operation on the MIP prediction block, that is, by judging the MIP prediction Whether the size of the block is the same as the size of the current block to determine whether to perform upsampling processing on the MIP prediction block.
  • the MIP prediction block when the size of the MIP prediction block is the same as the size of the current block, that is, the width and height of the MIP prediction block are the same as the current block, indicating that the MIP prediction block does not need to be up-sampling.
  • the MIP prediction can be The block is directly filled into the current block, that is, there are no vacant pixels in the filled current block.
  • the predicted value of each pixel in the current block can be directly set to the predicted value of each pixel in the MIP prediction block, as shown below ,
  • predSamples[x][y] predMip[x][y] (7)
  • [x][y] represents the position coordinates of the pixel, x represents the horizontal direction, and y represents the vertical direction; predSamples[x][y] represents the pixel corresponding to the location coordinate [x][y] in the current block
  • the predicted value, predMip[x][y] represents the predicted value corresponding to the pixel at the position coordinate [x][y] in the MIP prediction block.
  • the MIP prediction block predMip[x][y] can be directly used as the intra prediction block predSamples[x][y] of the current block.
  • the MIP prediction block cannot fill the current block, that is, There are vacant pixels in the current block after filling, which indicates that the MIP prediction block needs to be subjected to the first filtering process at this time.
  • the MIP prediction block can be up-sampled in the horizontal direction and then up-sampled in the vertical direction to obtain the first up-sampling.
  • the block can be represented by predSamples[x][y]; then the MIP prediction block is upsampled vertically and then upsampled horizontally to obtain the second upsampled block, which can be represented by predSamplesTemp[x][y]; Finally, predSamples[x][y] and predSamplesTemp[x][y] are calculated on the weighted average, and finally the intra prediction block of the current block is obtained.
  • the side length nTbS of the current block (here, S can be replaced by W and H respectively) is equal to the side length predSize of predMip (here, predSize is only related to the type mipSizeId of the current block)
  • predSize is only related to the type mipSizeId of the current block
  • the generation process of the intra prediction block predSamples can be seen in Figure 8, still taking the 4 ⁇ 4 current block as an example. In Figure 8, since the size of the current block and the MIP prediction block are the same, it is not necessary at this time.
  • the MIP prediction block can be directly set as the intra prediction block of the current block.
  • the performing the first filtering process on the MIP prediction block according to the parameters of the current block to obtain the first filtered prediction block may include:
  • the first filtering process is performed on the MIP prediction block to obtain the first filtered prediction block.
  • width and height of the current block are equal.
  • the width and height of the current block can also determine whether the width and height of the current block are equal to determine the selected filtering processing method or upsampling processing method, and then determine the intra prediction of the current block Piece. Among them, if the width and height of the current block are equal, it indicates that the current block is a square block.
  • the first filtering process ie, the first up-sampling process
  • the MIP prediction block can be used for the MIP prediction block to calculate the first filtered prediction block. Then use it as the intra prediction block of the current block.
  • the width and height of the MIP prediction block are both predSize, the width of the current block is nTbW, and the height of the current block is nTbH; then according to the side length predSize of the MIP prediction block and the width nTbW of the current block, Calculate the horizontal upsampling factor (indicated by upHor); also according to the side length predSize of the MIP prediction block and the high nTbH of the current block, the vertical upsampling factor can be calculated (indicated by upVer); the specific calculation formula is as follows,
  • the current block can be filled according to the horizontal upsampling factor upHor and the vertical upsampling factor upVer, that is, the up-sampling operation is performed.
  • the method of filling the corresponding position is as follows:
  • the size of the MIP prediction block is the same as the size of the current block, it can determine whether to perform filtering, that is, whether to perform filtering is determined according to whether the width and height of the MIP prediction block are the same as the width and height of the current block; and the horizontal upsampling factor upHor And the vertical upsampling factor upVer can reflect whether the width and height of the MIP prediction block are the same as the width and height of the current block, that is, the horizontal upsampling factor upHor and the vertical upsampling factor upVer can be used to determine whether the first filtering is required Processing, such as the first up-sampling filter processing.
  • the size of the MIP prediction block at this time is the same as the size of the current block.
  • the size of the MIP prediction block is different from the size of the current block at this time, and the first filtering process is required to obtain the intra prediction block of the current block. .
  • the MIP prediction block predMip can be filled into the corresponding position ((x+1) ⁇ upHor-1, (y+1) ⁇ upVer-1) in the current block predSamples.
  • the current block is equally divided into predSize ⁇ predSize sub-blocks, and filled to the lower right corner of each sub-block.
  • the pixels filled by the cross line are The upper side reference pixel refT
  • the diagonally filled pixels are the left side reference pixels refL
  • the black filled pixels are the positions filled by the predicted sampling pixels in the MIP prediction block.
  • the horizontal upsampling factor upHor is greater than 1, then perform upsampling in the horizontal direction first, and specifically determine the first predicted value of the pixel position to be filled in the horizontal direction in the current block according to the second preset calculation model; the second preset calculation model As follows,
  • the horizontal up-sampling process is shown in Figure 10, still taking the 32 ⁇ 32 current block as an example, the left side reference pixel and the MIP prediction block filled to the horizontal position during horizontal up-sampling will all be regarded as the horizontal position.
  • the reference point for up-sampling is also filled with black here; in Figure 10, all black-filled pixels are reference points for horizontal up-sampling; among them, every two black-filled pixels in the horizontal direction are linearly interpolated Way to get the predicted value of the gray filled pixels.
  • the up-sampling method is the up-sampling method using linear interpolation.
  • the value of each interpolation point (pixels filled with gray) between every two up-sampling reference points (pixels filled with black) is It is obtained by the weighted average of two up-sampling reference points.
  • the weight of the up-sampling reference point on the left is (upHor-dX)/upHor
  • the weight of the right reference point is dX/upHor
  • dX represents the distance between the current interpolation point and the left reference point
  • the first predicted value of the pixel position to be filled in the vertical direction in the current block is determined according to the third preset calculation model; the third preset calculation model is as follows:
  • the vertical upsampling process is shown in Figure 12, still taking the 32 ⁇ 32 current block as an example.
  • the upper side reference pixels and all filled in the current block Pixels will be used as reference points for vertical up-sampling, and they are still filled with black here; in Figure 12, all black-filled pixels are reference points for vertical up-sampling; among them, every two black-filled in the vertical direction
  • the predicted value of gray-filled pixels is obtained by linear interpolation between pixels.
  • the up-sampling method is the up-sampling method using linear interpolation.
  • the value of each interpolation point (pixels filled with gray) between every two up-sampling reference points (pixels filled with black) is It is obtained by the weighted average of two up-sampling reference points.
  • the weight of the upper sampling reference point on the upper side is (upVer-dY)/upVer
  • the weight of the lower reference point is dY/upVer
  • dY represents the distance between the current interpolation point and the upper reference point
  • dY 1, 2,...,upVer-1. That is to say, in the vertical upsampling process, the weights are all related to the vertical upsampling factor upVer.
  • predSamplesTemp After constructing the temporary prediction block predSamplesTemp, first fill the upper side reference pixel refT to the corresponding position of the previous line of the current block predSamplesTemp[x][-1], and then fill the left side reference pixel refL to the left of the current block
  • the upsampling process in the vertical direction is first performed at this time, and the second predicted value of the pixel position to be filled in the vertical direction in the current block is determined according to the fourth preset calculation model;
  • the four preset calculation models are as follows,
  • the upsampling process in the vertical direction is shown in Figure 13, still taking the 32 ⁇ 32 current block as an example.
  • the upper side reference pixels and the MIP prediction block filled to the vertical position will be filled with the prediction sampling pixels corresponding to the vertical position.
  • the reference point for up-sampling is also filled with black here; in Figure 13, all black-filled pixels are reference points for vertical up-sampling; among them, every two black-filled pixels in the vertical direction are linearly interpolated Way to get the predicted value of the gray filled pixels.
  • the up-sampling method is the up-sampling method using linear interpolation.
  • the value of each interpolation point (pixels filled with gray) between every two up-sampling reference points (pixels filled with black) is It is obtained by the weighted average of two up-sampling reference points.
  • the weight of the upper sampling reference point on the upper side is (upVer-dY)/upVer
  • the weight of the lower reference point is dY/upVer
  • dY represents the distance between the current interpolation point and the upper reference point
  • dY 1, 2,...,upVer-1. That is to say, in the vertical upsampling process, the weights are all related to the vertical upsampling factor upVer.
  • the up-sampling processing in the horizontal direction is performed again, which is similar to the up-sampling processing in the vertical direction.
  • the second predicted value of the pixel position to be filled in the horizontal direction in the current block is determined according to the fifth preset calculation model; the fifth preset calculation model is as follows:
  • the horizontal up-sampling process is shown in Figure 14. Still taking the 32 ⁇ 32 current block as an example, after the vertical up-sampling shown in Figure 13 is completed, the left side reference pixel and all filled in the current block Pixels will be used as reference points for horizontal up-sampling, and they are still filled with black here; in Figure 14, all black-filled pixels are reference points for horizontal up-sampling; among them, every two black-filled pixels in the horizontal direction
  • the predicted value of gray-filled pixels is obtained by linear interpolation between pixels.
  • the up-sampling method is the up-sampling method using linear interpolation.
  • the value of each interpolation point (pixels filled with gray) between every two up-sampling reference points (pixels filled with black) is It is obtained by the weighted average of two up-sampling reference points.
  • the weight of the up-sampling reference point on the left is (upHor-dX)/upHor
  • the weight of the right reference point is dX/upHor
  • dX represents the distance between the current interpolation point and the left reference point
  • the average value can be calculated to obtain the intra prediction block of the current block.
  • it may include: using a sixth preset calculation model to calculate the average value of the predicted value of the first pixel position in the first up-sampling block and the predicted value of the first pixel position in the second up-sampling block, and calculate The intra-frame prediction value of the first pixel position in the current block is obtained to obtain the intra-frame prediction block; wherein, the intra-frame prediction block includes at least the first pixel.
  • the sixth preset calculation model can be used to find Take the average value to obtain the intra prediction block of the current block.
  • the sixth preset calculation model is shown in the above equation (15), as shown below,
  • predSamples[x][y] (predSamples[x][y]+predSamplesTemp[x][y]+1)>>1 (15)
  • the upsampling process can be optimized, that is, using the "upsampling in the horizontal direction first, then upsampling in the vertical direction” and “upsampling in the vertical direction first, then upsampling in the horizontal direction”
  • the predicted values obtained by the two up-sampling methods are averaged to obtain the final target predicted value.
  • all pixels in the reference pixels refT and refL will be used as references for upsampling, and the prediction characteristics in different directions are also fully considered, so that the MIP prediction is more accurate.
  • the image prediction method is applied to VTM7.0 and tested under All Intra conditions at 24 frame intervals.
  • BD-rate changes Based on the average bit rate change under the same peak signal to noise ratio (Peak Signal to Noise Ratio, PSNR), it can be tested in three BD-rate changes of -0.02%, -0.02%, and -0.01% are obtained on each image component (such as Y, Cb, and Cr); especially on large-resolution sequences, there will be better performance, Specifically, a BD-rate change of -0.04% for Y can be achieved on Class A1, which not only improves the accuracy of MIP prediction, but also improves the video image quality.
  • PSNR Peak Signal to Noise Ratio
  • the MIP prediction block to fill the prediction pixels in the horizontal row and vertical column where the prediction sampling pixels in the current block are located, when the first filtering process is In the first up-sampling processing mode, the predicted values of the two up-sampling operations of "upsampling in the horizontal direction, then upsampling in the vertical direction” and “upsampling in the vertical direction first, then upsampling in the horizontal direction” are the same. At this time, there is redundant operation. In order to avoid redundant operations, other upsampling processing methods will be described below.
  • the performing the first filtering process on the MIP prediction block according to the parameters of the current block to obtain the first filtered prediction block may include:
  • the first filter prediction block is obtained; wherein, the first filter processing further includes one or more of the following: a second up-sampling process, a third up-sampling process, and a fourth up-sampling process. Sampling processing and fifth up-sampling processing;
  • the second up-sampling processing means performing horizontal up-sampling processing on the MIP prediction block to obtain the first filtered prediction block;
  • the third up-sampling processing means performing vertical up-sampling processing on the MIP prediction block to obtain the first filtered prediction block;
  • the fourth up-sampling processing means performing up-sampling in the horizontal direction on the MIP prediction block and then up-sampling in the vertical direction to obtain the first filtered prediction block;
  • the fifth up-sampling processing means that the MIP prediction block is vertically up-sampled and then horizontally up-sampled to obtain the first filtered prediction block.
  • the first filtering process may also be at least one of the second up-sampling process, the third up-sampling process, the fourth up-sampling process, and the fifth up-sampling process. Item or multiple items; wherein, the selection of the first filtering process may be determined according to the parameters of the current block (such as the width and height of the current block).
  • the selected up-sampling processing method can be determined by judging whether the width and height of the current block are equal, and then the intra prediction block of the current block can be determined. Therefore, in some embodiments, specifically, the setting the first filtering process according to the size of the current block and the size of the MIP prediction block may include:
  • the first filtering process is set as the first up-sampling process.
  • width and height of the current block are equal.
  • the selected upsampling processing method can be determined by judging whether the width and height of the current block are equal, and then the intra prediction block of the current block can be determined. Among them, if the width and height of the current block are equal, it indicates that the current block is a square block. At this time, the first up-sampling process can be used for the MIP prediction block to calculate the first filtered prediction block, and then use it as the current block Intra prediction block.
  • the setting of the first filtering process according to the size of the current block and the size of the MIP prediction block may include:
  • the first filtering process is set as the second upsampling process.
  • the second upsampling process can be used to perform the first filtering process on the MIP prediction block, that is, the second preset calculation model is used to perform the first filtering process on the MIP prediction block.
  • the prediction block is subjected to horizontal up-sampling processing to obtain a first filtered prediction block, and the obtained first filtered prediction block is determined as the intra prediction block of the current block.
  • the setting of the first filtering process according to the size of the current block and the size of the MIP prediction block may include:
  • the first filtering process is set to the third upsampling process.
  • the third upsampling process can be used to perform the first filtering process on the MIP prediction block, that is, the fourth preset calculation model is used to perform the first filtering process on the MIP prediction block.
  • the prediction block is subjected to vertical upsampling processing to obtain the first filtered prediction block, and the obtained first filtered prediction block is determined as the intra prediction block of the current block.
  • the setting of the first filtering process according to the size of the current block and the size of the MIP prediction block may include:
  • the first filtering process is set to the fourth upper Sampling processing.
  • the second preset calculation model can be used to upsample the MIP prediction block in the horizontal direction
  • the third preset calculation model is used to perform vertical upsampling processing on the MIP prediction block to obtain the first filtered prediction block, and the obtained first filtered prediction block is determined as the intra prediction block of the current block.
  • the setting of the first filtering process according to the size of the current block and the size of the MIP prediction block may include:
  • the first filtering process is set to the fifth upper Sampling processing.
  • the fourth preset calculation model can be used to upsample the MIP prediction block vertically
  • the fifth preset calculation model is used to perform horizontal upsampling processing on the MIP prediction block to obtain the first filtered prediction block, and the obtained first filtered prediction block is determined as the intra prediction block of the current block.
  • the predSamples at this time is the intra-frame prediction value of the current block; otherwise, it needs to be based on The horizontal upsampling factor upHor and the vertical upsampling factor upVer determine the upsampling processing method, as shown below:
  • upHor is greater than 1 and upVer is equal to 1, then the predicted value of the pixel position to be filled in the horizontal line where the predicted sample pixel in the intra prediction block predSamples is located can be obtained by interpolation through the second upsampling process, and there is no need at this time Perform vertical upsampling;
  • upHor is greater than 1 and upVer is equal to 1
  • the predicted value of the pixel position to be filled in the vertical column where the predicted sample pixel in the intra prediction block predSamples is located can be obtained by interpolation through the third upsampling process. There is no need to perform horizontal upsampling.
  • the current block can be divided into multiple sub-blocks, and the uninterpolated pixels in each sub-block can be filtered using a PLANAR prediction method, which can be Determine the predicted value of the pixel position to be filled in each sub-block to obtain an intra-frame prediction block.
  • a PLANAR prediction method which can be Determine the predicted value of the pixel position to be filled in each sub-block to obtain an intra-frame prediction block.
  • the method may further include:
  • a second preset calculation model is used to determine the first predicted value of the pixel position to be filled in the horizontal line where the predicted sampling pixel in the current block is located , Using a fourth preset calculation model to determine the second predicted value of the position of the pixel to be filled in the vertical column where the predicted sampling pixel in the current block is located;
  • the seventh predicted value is used.
  • the design calculation model determines the predicted value of the pixel position to be filled in at least one sub-block to obtain an intra-frame prediction block.
  • the predicted value of the pixel position to be filled in the horizontal line where the predicted sampling pixel is located is obtained by horizontal upsampling interpolation, for example, it can be calculated according to the second preset calculation model shown in equation (10);
  • all black-filled pixels are reference points for upsampling; among them, the predicted value of gray-filled pixels is obtained by linear interpolation between every two black-filled pixels in the horizontal direction.
  • the predicted value of the pixel position to be filled in the vertical column where the predicted sampling pixel is located is obtained by vertical upsampling interpolation, for example, it can be calculated according to the fourth preset calculation model shown in equation (13) ;
  • all black-filled pixels are reference points for upsampling; among them, the predicted value of gray-filled pixels is obtained by linear interpolation between every two black-filled pixels in the vertical direction; and the prediction in Figure 17 The predicted values of all to-be-filled pixel positions in the horizontal row and vertical column where the sampled pixel is located have been interpolated.
  • the using the seventh preset calculation model to determine the predicted value of the pixel position to be filled in the at least one sub-block may include:
  • Figure 18 is an example of a sub-block in Figure 17; the four corners filled with black are the predicted sampling pixels, and the pixels filled with gray are the predicted points obtained by interpolation; at this time Interpolate for white-filled pixels.
  • the interpolation method can be bilinear interpolation, that is, the average of linear interpolation in two directions; among them, the position of the pixel to be filled marked by a circle is taken as an example, and the four reference pixels are up, down, left, and right. Also marked with a circle.
  • the bilinear interpolation method is as follows,
  • the first step is to perform horizontal interpolation; among them, the weight of the pixel on the left is (upHor-dX)/upHor, and the weight of the pixel on the right is dX/upHor, and dX represents the difference between the current interpolation point and the left pixel. distance;
  • the second step is to perform vertical interpolation; where the weight of the upper pixel is (upVer-dY)/upVer, the weight of the lower pixel is dY/upVer, and dY represents the difference between the current interpolation point and the upper pixel distance;
  • the interpolation value obtained in the first step and the interpolation value obtained in the second step are averaged to obtain the predicted value of the pixel position to be filled.
  • the calculation result of the embodiment of the present application is the same as the calculation result of the average value obtained according to the first up-sampling block and the second up-sampling block in the foregoing embodiment.
  • it can also be realized by calculating the weighted average of four points. Therefore, in some embodiments, using the seventh preset calculation model to determine the predicted value of the pixel position to be filled in at least one sub-block may include:
  • a seventh preset calculation model is used to perform a weighted average calculation on the pixel position to be filled in the at least one sub-block to obtain a predicted value of the pixel position to be filled in the at least one sub-block.
  • the weight of each point is as follows: the weight of the upper point is (upVer-dY)/(2 ⁇ upVer), and the weight of the lower point is dY/(2 ⁇ upVer), the weight of the left point is (upHor-dX)/(2 ⁇ upHor), and the weight of the right point is dX/(2 ⁇ upHor); where dY is the distance between the current interpolation point and the upper point , DX is the distance between the current interpolation point and the left point.
  • dY is the distance between the current interpolation point and the upper point
  • DX is the distance between the current interpolation point and the left point.
  • the weight of each point can also be as follows: the weight of the upper point is (upVer-dY)/(upHor+upVer), and the weight of the lower point is dY/ (upHor+upVer), the weight of the left point is (upHor-dX)/(upHor+upVer), and the weight of the right point is dX/(upHor+upVer); where dY is the difference between the current interpolation point and the upper point DX is the distance between the current interpolation point and the left point.
  • upHor and upVer are equal, the two weighted average calculation results are the same; when upHor and upVer are not equal, the two weighted average calculation results are also different.
  • some current block size, current block type, or MIP prediction mode can be added at the same time under the MIP prediction mode of all current block types.
  • mipSizeId is equal to 2
  • the non-fixed upsampling method in the embodiment of this application is used at this time, that is, the first upsampling block can be obtained by using “upsampling in the horizontal direction and then upsampling in the vertical direction” and using “vertical upsampling first" Up-sampling in the direction and then up-sampling in the horizontal direction” to obtain the second up-sampling block and get the average value of it; otherwise, the fixed up-sampling method in the relevant technical solution is applied at this time, that is, "the first horizontal up-sampling, then the vertical direction Upsampling".
  • the current block of large size can be restricted to use the non-fixed upsampling method in the embodiment of the present application, and the current block of small size can be fixed in the related technical solution.
  • the restriction condition may also be mipSizeId, or information such as the width and height of the current prediction block, which are not specifically limited in the embodiments of the present application.
  • the intra prediction block of the current block can also be fully considered, and then different upsampling methods can be designed.
  • the following will take the size of the current block (that is, the width and height of the current block) as an example for detailed description.
  • the method may further include:
  • the first filtering process is set as the first up-sampling process.
  • the method may further include:
  • the first filtering process is set as the fourth up-sampling process
  • the first filtering process is set as the fifth up-sampling process.
  • the method may further include:
  • the first filtering process is set as the fourth upsampling process
  • the first filtering process is set as the fifth upsampling process.
  • the selected upsampling processing method can be determined by judging whether the width and height of the current block are equal, and then the intra prediction block of the current block can be determined.
  • the first determination method is to directly use the first upsampling process to calculate the first filtered prediction block as the intra prediction block. That is, first perform "first horizontal upsampling, then vertical upsampling" to calculate the first upsampling block of the current block (which can be represented by predSamples0[x][y]), and then perform "first vertical upsampling, then Horizontal upsampling” to calculate the second upsampling block of the current block (which can be represented by predSample1[x][y]), and then calculate the average value between predSamples0[x][y] and predSamples1[x][y] , Determine the average value as the intra prediction block of the current block.
  • the second determination method is that for a square block (that is, the current block with equal width and height), you can directly use the first determination method to calculate predSamples; for a rectangular block (that is, the current block with unequal width and height), The following methods can be used:
  • the intra prediction block is determined according to the width and height of the current block; specifically, if the width of the current block is greater than the height, the fourth upsampling process can be directly used to calculate the first filtered prediction block as the frame Intra-prediction block, that is, perform "first horizontal upsampling, then vertical upsampling" to calculate the first filtered prediction block; conversely, if the height of the current block is greater than the width, you can directly use the fifth upsampling process to calculate the first
  • a filtered prediction block is used as an intra-frame prediction block, that is, "upsampling in the vertical direction and then upsampling in the horizontal direction" is performed to calculate the first filtered prediction block.
  • the current prediction block is still determined according to the width and height of the current block; specifically, if the width of the current block is greater than a certain multiple of the height (for example, the width is greater than or equal to 4 times the height), then you can directly Use the fourth up-sampling process to calculate the first filtered prediction block as an intra-frame prediction block; on the contrary, if the height of the current block is greater than a certain multiple of the width (for example, the height is greater than or equal to 4 times the width), then the fifth one can be used directly
  • the sampling process calculates the first filtered prediction block as the intra prediction block; on the contrary, for other cases, the first determination method can be used to calculate the intra prediction block directly, or the fourth upsampling process (ie " Upsampling in the horizontal direction and then upsampling in the vertical direction") to calculate the intra prediction block, or the fifth upsampling process (ie, "upsampling in the vertical direction first and then upsampling in the horizontal direction”) can be used to calculate the intra prediction
  • the horizontal upsampling factor in the horizontal direction or the vertical upsampling factor in the vertical direction for example, judging whether the horizontal upsampling factor is greater than 1 and/or the vertical Whether the up-sampling factor is greater than 1, determines the up-sampling mode, and then determines the intra-frame prediction block.
  • the up-sampling operation in the horizontal or vertical direction may be used. Upsampling can also be performed in a sub-block manner.
  • the black-filled pixels represent the corresponding positions in the MIP prediction block predMip that are filled to the current block
  • the white-filled pixels represent the positions of the pixels to be filled in the current block to be up-sampled
  • the filled pixels are reference pixels (including the left side reference pixel refL and the upper side reference pixel refT).
  • one implementation is to calculate the predicted value of each sub-block in the order of raster scanning starting from sub-block 0; or another implementation is to first calculate the predicted value of sub-block 0 , And then calculate the predicted value of the two sub-block 1 at the same time, and then calculate the predicted value of the sub-block 2 at the same time.
  • the image prediction method when the image prediction method is applied to the encoder side, the image prediction method can be used to calculate the predicted value of at least one pixel in the current block, and then the actual value of at least one pixel in the current block can be compared with the actual value of the pixel in the current block.
  • the difference between the predicted values calculates the residual corresponding to at least one pixel, and writes the calculated residual into the code stream; in addition, after obtaining the transposed processing indicator (isTransposed), it is also necessary to take the value of isTransposed
  • the value is also written into the code stream; then the code stream is transmitted from the encoder side to the decoder side; correspondingly, when the image prediction method is applied to the decoder side, the value of isTransposed can be determined by parsing the code stream, and then Determine whether transposition processing is required; in addition, the image prediction method can also be used to calculate the predicted value of at least one pixel in the current block, and then the residual error corresponding to at least one pixel can be directly obtained by parsing the code stream, and then based on the current block The predicted value and residual of at least one pixel in, the true value of at least one pixel in the current block can be obtained.
  • This embodiment provides an image prediction method to determine the MIP input sampling matrix of the current block; calculate the MIP prediction block of the current block according to the MIP input sampling matrix, where the MIP prediction block contains the predicted sampling values of some pixel positions in the current block ; According to the parameters of the current block, perform the first filtering process on the MIP prediction block to obtain the first filtered prediction block, wherein the first filtering process includes the first up-sampling process; the first filtered prediction block is set as the intra-frame of the current block The prediction block, where the intra prediction block contains the predicted sample values of all pixel positions in the current block.
  • the first upsampling process means that the MIP prediction block is upsampled in the horizontal direction and then the vertical direction is upsampled to obtain the first upsampling block, and the MIP prediction block is upsampled in the vertical direction and then the horizontal direction is upsampled to obtain the first upsampling block.
  • the weighted average of the first up-sampling block and the second up-sampling block is used as the first filtered prediction block; in this way, the reference of the current block can be fully utilized through the first up-sampling block and the second up-sampling block Pixel information can also fully consider the prediction characteristics of the prediction sampling in different directions, so that the final up-sampling result of the intra-frame prediction block is more accurate, which not only improves the accuracy of MIP prediction, but also improves the video image quality.
  • FIG. 20 shows a schematic diagram of the composition structure of an encoder 200 provided by an embodiment of the present application.
  • the encoder 200 may include: a first determining unit 2001, a first calculating unit 2002, a first processing unit 2003, and a first predicting unit 2004; wherein,
  • the first determining unit 2001 is configured to determine the MIP input sampling matrix of the current block
  • the first calculation unit 2002 is configured to calculate the MIP prediction block of the current block according to the MIP input sampling matrix, where the MIP prediction block includes predicted sampling values of some pixel positions in the current block;
  • the first processing unit 2003 is configured to perform a first filtering process on the MIP prediction block according to the parameters of the current block to obtain a first filtered prediction block, wherein the first filtering process includes a first upsampling process,
  • the first up-sampling process means that the MIP prediction block is up-sampled in the horizontal direction and then up-sampled in the vertical direction to obtain a first up-sampling block, and the MIP prediction block is up-sampled in the vertical direction before performing the up-sampling in the horizontal direction. Sampling to obtain a second up-sampling block, and using a weighted average value of the first up-sampling block and the second up-sampling block as the first filtered prediction block;
  • the first prediction unit 2004 is configured to set the first filtered prediction block as an intra prediction block of the current block; wherein the intra prediction block includes prediction sample values of all pixel positions in the current block.
  • the first processing unit 2003 is specifically configured to perform the first filtering process on the MIP prediction block when the width and height of the current block are both greater than the width and height of the MIP prediction block, Obtain the first filtered prediction block.
  • the width and height of the current block are equal.
  • the encoder 200 may further include a first setting unit 2005 configured to set the first filtering process according to the size of the current block and the size of the MIP prediction block;
  • the first processing unit 2003 is configured to obtain the first filtered prediction block according to the set first filter processing; wherein, the first filter processing further includes one or more of the following: second upsampling processing, first Three up-sampling processing, fourth up-sampling processing and fifth up-sampling processing;
  • the second up-sampling processing means performing horizontal up-sampling processing on the MIP prediction block to obtain the first filtered prediction block;
  • the third up-sampling processing means performing vertical up-sampling processing on the MIP prediction block to obtain the first filtered prediction block;
  • the fourth up-sampling processing means performing up-sampling in the horizontal direction on the MIP prediction block and then up-sampling in the vertical direction to obtain the first filtered prediction block;
  • the fifth up-sampling processing means that the MIP prediction block is vertically up-sampled and then horizontally up-sampled to obtain the first filtered prediction block.
  • the first setting unit 2005 is further configured to set the first filtering process as the first upper filter when the height and width of the current block are both greater than the height and width of the MIP prediction block. Sampling processing.
  • the width and height of the current block are equal.
  • the first setting unit 2005 is specifically configured to: when the height of the current block is equal to the height of the MIP prediction block, and the width of the current block is greater than the width of the MIP prediction block, the The first filtering process is set as the second up-sampling process.
  • the first setting unit 2005 is specifically configured to: when the width of the current block is equal to the width of the MIP prediction block, and the height of the current block is greater than the height of the MIP prediction block, the The first filtering process sets the third up-sampling process.
  • the first setting unit 2005 is specifically configured to: when the height and width of the current block are both greater than the height and width of the MIP prediction block, and the width of the current block is greater than the height of the current block,
  • the first filtering process is set as the fourth up-sampling process.
  • the first setting unit 2005 is specifically configured to, when the height and width of the current block are both greater than the height and width of the MIP prediction block, and the height of the current block is greater than the width of the current block,
  • the first filtering process is set as the fifth up-sampling process.
  • a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, may also be a module, or may be non-modular.
  • the various components in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be realized in the form of hardware or software function module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this embodiment is essentially or It is said that the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can It is a personal computer, a server, or a network device, etc.) or a processor (processor) that executes all or part of the steps of the method described in this embodiment.
  • the aforementioned storage media include: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.
  • an embodiment of the present application provides a computer storage medium, which is applied to the encoder 200, and the computer storage medium stores an image prediction program.
  • the image prediction program is executed by a first processor, any one of the foregoing embodiments is implemented. The method described.
  • FIG. 21 shows an example of a specific hardware structure of the encoder 200 provided by an embodiment of the present application, which may include: a first communication interface 2101, a first memory 2102, and a first communication interface 2101; A processor 2103; various components are coupled together through a first bus system 2104.
  • the first bus system 2104 is used to implement connection and communication between these components.
  • the first bus system 2104 also includes a power bus, a control bus, and a status signal bus.
  • various buses are marked as the first bus system 2104 in FIG. 21. among them,
  • the first communication interface 2101 is used for receiving and sending signals in the process of sending and receiving information with other external network elements;
  • the first memory 2102 is configured to store a computer program that can run on the first processor 2103;
  • the first processor 2103 is configured to execute the following when running the computer program:
  • the first filtering process includes a first up-sampling process
  • the first up-sampling process represents Up-sampling the MIP prediction block in the horizontal direction and then up-sampling in the vertical direction to obtain a first up-sampling block, up-sampling the MIP prediction block in the vertical direction and then up-sampling in the horizontal direction to obtain a second up-sampling block, Taking a weighted average value of the first up-sampling block and the second up-sampling block as the first filtered prediction block;
  • the first filtered prediction block is set as an intra prediction block of the current block; wherein the intra prediction block includes prediction sample values of all pixel positions in the current block.
  • the first memory 2102 in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory.
  • the non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), and electrically available Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory.
  • the volatile memory may be a random access memory (Random Access Memory, RAM), which is used as an external cache.
  • RAM static random access memory
  • DRAM dynamic random access memory
  • DRAM synchronous dynamic random access memory
  • DDRSDRAM Double Data Rate Synchronous Dynamic Random Access Memory
  • Enhanced SDRAM, ESDRAM Synchronous Link Dynamic Random Access Memory
  • Synchlink DRAM Synchronous Link Dynamic Random Access Memory
  • DRRAM Direct Rambus RAM
  • the first processor 2103 may be an integrated circuit chip with signal processing capability. In the implementation process, the steps of the foregoing method can be completed by an integrated logic circuit of hardware in the first processor 2103 or instructions in the form of software.
  • the above-mentioned first processor 2103 may be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (ASIC), a ready-made programmable gate array (Field Programmable Gate Array, FPGA) Or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • DSP Digital Signal Processor
  • ASIC application specific integrated circuit
  • FPGA ready-made programmable gate array
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the first memory 2102, and the first processor 2103 reads the information in the first memory 2102, and completes the steps of the foregoing method in combination with its hardware.
  • the embodiments described in this application can be implemented by hardware, software, firmware, middleware, microcode, or a combination thereof.
  • the processing unit can be implemented in one or more application specific integrated circuits (ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable Logic device (Programmable Logic Device, PLD), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, and others for performing the functions described in this application Electronic unit or its combination.
  • ASIC application specific integrated circuits
  • DSP Digital Signal Processing
  • DSP Device digital signal processing equipment
  • PLD programmable Logic Device
  • PLD Field-Programmable Gate Array
  • FPGA Field-Programmable Gate Array
  • the technology described in this application can be implemented through modules (for example, procedures, functions, etc.) that perform the functions described in this application.
  • the software codes can be stored in the memory and executed by
  • the first processor 2103 is further configured to execute the method described in any one of the foregoing embodiments when running the computer program.
  • This embodiment provides an encoder, which may include a first determination unit, a first calculation unit, a first processing unit, and a first prediction unit; wherein the first determination unit is configured to determine the MIP input of the current block A sampling matrix; a first calculation unit, configured to calculate the MIP prediction block of the current block according to the MIP input sampling matrix; a first processing unit, configured to perform the MIP prediction block according to the parameters of the current block A first filtering process to obtain a first filtered prediction block, wherein the first filtering process includes a first upsampling process; a first prediction unit configured to set the first filtered prediction block as a frame of the current block Intra-prediction block; where the intra-frame prediction block includes the predicted sample values of all pixel positions in the current block.
  • the first determination unit is configured to determine the MIP input of the current block A sampling matrix
  • a first calculation unit configured to calculate the MIP prediction block of the current block according to the MIP input sampling matrix
  • a first processing unit configured to perform the MIP prediction block according to
  • the first upsampling process means that the MIP prediction block is upsampled in the horizontal direction and then the vertical direction is upsampled to obtain the first upsampling block, and the MIP prediction block is upsampled in the vertical direction and then the horizontal direction is upsampled to obtain the first upsampling block.
  • the weighted average of the first up-sampling block and the second up-sampling block is used as the first filtered prediction block; in this way, the reference of the current block can be fully utilized through the first up-sampling block and the second up-sampling block Pixel information can also fully consider the prediction characteristics of the prediction sampling in different directions, so that the final up-sampling result of the intra-frame prediction block is more accurate, which not only improves the accuracy of MIP prediction, but also improves the video image quality.
  • FIG. 22 shows a schematic diagram of the composition structure of a decoder 220 provided by an embodiment of the present application.
  • the decoder 220 may include: a second determination unit 2201, a second calculation unit 2202, a second processing unit 2203, and a second prediction unit 2204; wherein,
  • the second determining unit 2201 is configured to determine the MIP input sampling matrix of the current block
  • the second calculation unit 2202 is configured to calculate the MIP prediction block of the current block according to the MIP input sampling matrix, where the MIP prediction block includes predicted sampling values of some pixel positions in the current block;
  • the second processing unit 2203 is configured to perform a first filtering process on the MIP prediction block according to the parameters of the current block to obtain a first filtered prediction block, wherein the first filtering process includes a first upsampling process,
  • the first up-sampling process means that the MIP prediction block is up-sampled in the horizontal direction and then up-sampled in the vertical direction to obtain a first up-sampling block, and the MIP prediction block is up-sampled in the vertical direction before performing the up-sampling in the horizontal direction. Sampling to obtain a second up-sampling block, and using a weighted average value of the first up-sampling block and the second up-sampling block as the first filtered prediction block;
  • the second prediction unit 2204 is configured to set the first filtered prediction block as an intra prediction block of the current block; wherein the intra prediction block includes prediction sample values of all pixel positions in the current block.
  • the second processing unit 2203 is specifically configured to perform the first filtering process on the MIP prediction block when the width and height of the current block are both greater than the width and height of the MIP prediction block, Obtain the first filtered prediction block.
  • the width and height of the current block are equal.
  • the decoder 220 may further include a second setting unit 2205 configured to set the first filtering process according to the size of the current block and the size of the MIP prediction block;
  • the second processing unit 2203 is configured to obtain the first filter prediction block according to the set first filter processing; wherein, the first filter processing further includes one or more of the following: second upsampling processing, Three up-sampling processing, fourth up-sampling processing and fifth up-sampling processing;
  • the second up-sampling processing means performing horizontal up-sampling processing on the MIP prediction block to obtain the first filtered prediction block;
  • the third up-sampling processing means performing vertical up-sampling processing on the MIP prediction block to obtain the first filtered prediction block;
  • the fourth up-sampling processing means performing up-sampling in the horizontal direction on the MIP prediction block and then up-sampling in the vertical direction to obtain the first filtered prediction block;
  • the fifth up-sampling processing means that the MIP prediction block is vertically up-sampled and then horizontally up-sampled to obtain the first filtered prediction block.
  • the second setting unit 2205 is further configured to set the first filtering process as the first upper filter when the height and width of the current block are both greater than the height and width of the MIP prediction block. Sampling processing.
  • the width and height of the current block are equal.
  • the second setting unit 2205 is specifically configured to: when the height of the current block is equal to the height of the MIP prediction block, and the width of the current block is greater than the width of the MIP prediction block, set the The first filtering process is set as the second up-sampling process.
  • the second setting unit 2205 is specifically configured to: when the width of the current block is equal to the width of the MIP prediction block, and the height of the current block is greater than the height of the MIP prediction block, the The first filtering process sets the third up-sampling process.
  • the second setting unit 2205 is specifically configured to: when the height and width of the current block are both greater than the height and width of the MIP prediction block, and the width of the current block is greater than the height of the current block,
  • the first filtering process is set as the fourth up-sampling process.
  • the second setting unit 2205 is specifically configured to: when the height and width of the current block are both greater than the height and width of the MIP prediction block, and the height of the current block is greater than the width of the current block, The first filtering process is set as the fifth up-sampling process.
  • a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, it may also be a module, or it may also be non-modular.
  • the various components in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be realized in the form of hardware or software function module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium.
  • this embodiment provides a computer storage medium applied to the decoder 220.
  • the computer storage medium stores an image prediction program, and when the image prediction program is executed by the second processor, any of the foregoing embodiments is implemented. The method described in one item.
  • FIG. 23 shows an example of a specific hardware structure of the decoder 220 provided by an embodiment of the present application, which may include: a second communication interface 2301, a second memory 2302, and a second communication interface 2301; Two processors 2303; various components are coupled together through a second bus system 2304.
  • the second bus system 2304 is used to implement connection and communication between these components.
  • the second bus system 2304 also includes a power bus, a control bus, and a status signal bus.
  • various buses are marked as the second bus system 2304 in FIG. 23. among them,
  • the second communication interface 2301 is used for receiving and sending signals in the process of sending and receiving information with other external network elements;
  • the second memory 2302 is configured to store a computer program that can run on the second processor 2303;
  • the second processor 2303 is configured to execute when the computer program is running:
  • the first filtering process includes a first up-sampling process
  • the first up-sampling process represents Up-sampling the MIP prediction block in the horizontal direction and then up-sampling in the vertical direction to obtain a first up-sampling block, up-sampling the MIP prediction block in the vertical direction and then up-sampling in the horizontal direction to obtain a second up-sampling block, Taking a weighted average value of the first up-sampling block and the second up-sampling block as the first filtered prediction block;
  • the first filtered prediction block is set as an intra prediction block of the current block; wherein the intra prediction block includes prediction sample values of all pixel positions in the current block.
  • the second processor 2303 is further configured to execute the method described in any one of the foregoing embodiments when the computer program is running.
  • the hardware function of the second memory 2302 is similar to that of the first memory 2102, and the hardware function of the second processor 2303 is similar to that of the first processor 2103; details are not repeated here.
  • This embodiment provides a decoder, which may include a second determination unit, a second calculation unit, a second processing unit, and a second prediction unit; wherein the second determination unit is configured to determine the MIP input of the current block A sampling matrix; a second calculation unit, configured to calculate the MIP prediction block of the current block according to the MIP input sampling matrix; a second processing unit, configured to perform the MIP prediction block according to the parameters of the current block A first filtering process to obtain a first filtered prediction block, wherein the first filtering process includes a first upsampling process; a second prediction unit is configured to set the first filtered prediction block as a frame of the current block Intra-prediction block; where the intra-frame prediction block includes the predicted sample values of all pixel positions in the current block.
  • the second determination unit is configured to determine the MIP input of the current block A sampling matrix
  • a second calculation unit configured to calculate the MIP prediction block of the current block according to the MIP input sampling matrix
  • a second processing unit configured to perform the MIP prediction
  • the first upsampling process means that the MIP prediction block is upsampled in the horizontal direction and then the vertical direction is upsampled to obtain the first upsampling block, and the MIP prediction block is upsampled in the vertical direction and then the horizontal direction is upsampled to obtain the first upsampling block.
  • the weighted average of the first up-sampling block and the second up-sampling block is used as the first filtered prediction block; in this way, the reference of the current block can be fully utilized through the first up-sampling block and the second up-sampling block Pixel information can also fully consider the prediction characteristics of the prediction sampling in different directions, so that the final up-sampling result of the intra-frame prediction block is more accurate, which not only improves the accuracy of MIP prediction, but also improves the video image quality.
  • the MIP input sampling matrix of the current block is determined first; then the MIP prediction block of the current block is calculated according to the MIP input sampling matrix, where the MIP prediction block contains the predicted sampling values of some pixel positions in the current block; For the parameters of the current block, perform the first filtering process on the MIP prediction block to obtain the first filtered prediction block, where the first filtering process includes the first upsampling process; finally, the first filtered prediction block is set as the intra prediction of the current block Block, the intra-frame prediction block contains the predicted sample values of all pixel positions in the current block.
  • the first upsampling process means that the MIP prediction block is upsampled in the horizontal direction and then the vertical direction is upsampled to obtain the first upsampling block, and the MIP prediction block is upsampled in the vertical direction and then the horizontal direction is upsampled to obtain the first upsampling block.
  • the weighted average of the first up-sampling block and the second up-sampling block is used as the first filtered prediction block; in this way, the reference of the current block can be fully utilized through the first up-sampling block and the second up-sampling block Pixel information can also fully consider the prediction characteristics of the prediction sampling in different directions, so that the final up-sampling result of the intra-frame prediction block is more accurate, which not only improves the accuracy of MIP prediction, but also improves the video image quality.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Selon des modes de réalisation, la présente invention divulgue un procédé de prédiction d'image, un codeur, un décodeur et un support de stockage. Le procédé consiste à : déterminer une matrice d'échantillonnage d'entrée en mode de prédiction intra basé sur une matrice (MIP) d'un bloc courant ; selon la matrice d'échantillonnage d'entrée MIP, calculer un bloc de prédiction MIP du bloc courant ; en fonction de paramètres du bloc courant, effectuer un premier traitement de filtrage sur le bloc de prédiction MIP pour obtenir un premier bloc de prédiction filtré, ledit premier traitement de filtration comprenant un premier traitement de sur-échantillonnage, ledit premier traitement de sur-échantillonnage signifiant le sur-échantillonnage du bloc de prédiction MIP dans la direction horizontale et ensuite le sur-échantillonnage dans la direction verticale pour obtenir un premier bloc de sur-échantillonnage et le sur-échantillonnage du bloc de prédiction MIP dans la direction verticale et ensuite le sur-échantillonnage dans la direction horizontale pour obtenir un second bloc de sur-échantillonnage et la valeur moyenne pondérée du premier bloc de sur-échantillonnage et du second bloc de sur-échantillonnage est utilisée en tant que premier bloc de prédiction filtré ; et configurer le premier bloc de prédiction filtré en tant que bloc de prédiction intra du bloc courant, le bloc de prédiction intra contenant des valeurs d'échantillonnage de prédiction de toutes les positions de pixel dans le bloc courant.
PCT/CN2019/127635 2019-12-23 2019-12-23 Procédé de prédiction d'image, codeur, décodeur et support de stockage WO2021127923A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980087536.2A CN113302934B (zh) 2019-12-23 2019-12-23 图像预测方法、编码器、解码器以及存储介质
PCT/CN2019/127635 WO2021127923A1 (fr) 2019-12-23 2019-12-23 Procédé de prédiction d'image, codeur, décodeur et support de stockage

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/127635 WO2021127923A1 (fr) 2019-12-23 2019-12-23 Procédé de prédiction d'image, codeur, décodeur et support de stockage

Publications (1)

Publication Number Publication Date
WO2021127923A1 true WO2021127923A1 (fr) 2021-07-01

Family

ID=76573453

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/127635 WO2021127923A1 (fr) 2019-12-23 2019-12-23 Procédé de prédiction d'image, codeur, décodeur et support de stockage

Country Status (2)

Country Link
CN (1) CN113302934B (fr)
WO (1) WO2021127923A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012172779A1 (fr) * 2011-06-13 2012-12-20 Panasonic Corporation Procédé et appareil de codage décodage vidéo au moyen d'une matrice de quantification adaptative dépendant d'un mode de prédiction intra
CN103098473A (zh) * 2010-09-08 2013-05-08 三星电子株式会社 对帧内预测使用自适应dct/dst的低复杂度变换编码
CN103493491A (zh) * 2010-07-15 2014-01-01 株式会社东芝 图像编码方法以及图像解码方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100370076B1 (ko) * 2000-07-27 2003-01-30 엘지전자 주식회사 다운 컨버젼 기능을 갖는 비디오 디코더 및 비디오 신호를디코딩 하는 방법
CN101715135B (zh) * 2009-09-30 2013-01-09 武汉大学 基于匹配模板的自适应帧内预测滤波编码方法
WO2012047047A2 (fr) * 2010-10-06 2012-04-12 에스케이텔레콤 주식회사 Procédé et appareil destinés à coder/décoder une vidéo en utilisant un filtre à haute précision
GB2552323B (en) * 2016-07-18 2020-04-29 Imagination Tech Ltd Mip map compression
CN108810552B (zh) * 2017-04-28 2021-11-09 华为技术有限公司 图像预测方法和相关产品

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103493491A (zh) * 2010-07-15 2014-01-01 株式会社东芝 图像编码方法以及图像解码方法
CN103098473A (zh) * 2010-09-08 2013-05-08 三星电子株式会社 对帧内预测使用自适应dct/dst的低复杂度变换编码
WO2012172779A1 (fr) * 2011-06-13 2012-12-20 Panasonic Corporation Procédé et appareil de codage décodage vidéo au moyen d'une matrice de quantification adaptative dépendant d'un mode de prédiction intra

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
H. LIU (BYTEDANCE), L. ZHANG (BYTEDANCE), K. ZHANG (BYTEDANCE), Z. DENG (BYTEDANCE), J. XU (BYTEDANCE), Y. WANG (BYTEDANCE): "CE3-3: Up-sampling with a fixed order in MIP", 16. JVET MEETING; 20191001 - 20191011; GENEVA; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ), no. JVET-P0054, 17 September 2019 (2019-09-17), XP030216077 *
K. KONDO (SONY), J.-Y. HUO (XIDIAN UNIV.): "CE3-2: On rounding shift of MIP", 16. JVET MEETING; 20191001 - 20191011; GENEVA; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ), no. JVET-P0056 ; m49976, 18 September 2019 (2019-09-18), XP030216081 *

Also Published As

Publication number Publication date
CN113302934B (zh) 2022-10-18
CN113302934A (zh) 2021-08-24

Similar Documents

Publication Publication Date Title
WO2021120122A1 (fr) Procédé de prédiction de composante d'image, codeur, décodeur et support de mémoire
WO2021004155A1 (fr) Procédé de prédiction de composante d'image, encodeur, décodeur et support d'informations
WO2021196235A1 (fr) Procédé de prédiction inter-trame, codeur, décodeur et support d'enregistrement
WO2021139572A1 (fr) Procédé de codage, procédé de décodage, codeur, décodeur, et support de stockage
JP2010514300A (ja) ビデオ画像のブロックを復号化する方法
WO2022087901A1 (fr) Procédé de prédiction d'image, codeur, décodeur et support de stockage informatique
WO2021134635A1 (fr) Procédé de transformation, codeur, décodeur et support de stockage
CN112616057A (zh) 图像预测方法、编码器、解码器以及存储介质
AU2019357929A1 (en) Video image component prediction method and apparatus, and computer storage medium
US20220329809A1 (en) Transform method, encoder, decoder, and storage medium
WO2021127923A1 (fr) Procédé de prédiction d'image, codeur, décodeur et support de stockage
CN112532997B (zh) 图像预测方法、编码器、解码器以及存储介质
WO2022077490A1 (fr) Procédé de prédiction intra, codeur, décodeur et support de stockage
WO2021134327A1 (fr) Procédé de transformation, codeur, décodeur, et support de stockage
WO2020258052A1 (fr) Procédé et dispositif de prédiction de composante d'image et support de stockage informatique
CN113395520A (zh) 解码预测方法、装置及计算机存储介质
WO2024007120A1 (fr) Procédé de codage et de décodage, codeur, décodeur et support de stockage
WO2023197192A1 (fr) Procédé et appareil de codage, procédé et appareil de décodage, dispositif de codage, dispositif de décodage et support d'enregistrement
CN113411588B (zh) 预测方向的确定方法、解码器以及计算机存储介质
EP3952308B1 (fr) Procédé et dispositif de prédiction de composante d'image et support de stockage informatique
WO2023197195A1 (fr) Procédé de codage et de décodage vidéo, codeur, décodeur et support de stockage
WO2021134637A1 (fr) Procédé de transformée, encodeur, décodeur, et support de stockage
EP4124036A1 (fr) Procédé, appareil et dispositif de codage/décodage vidéo
WO2023197189A1 (fr) Procédé et appareil de codage, procédé et appareil de décodage, et dispositif de codage, dispositif de décodage et support d'enregistrement
WO2023197193A1 (fr) Procédé et appareil de codage, procédé et appareil de décodage, et dispositif de codage, dispositif de décodage et support d'enregistrement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19957404

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19957404

Country of ref document: EP

Kind code of ref document: A1