WO2021139572A1 - 编码方法、解码方法、编码器、解码器以及存储介质 - Google Patents

编码方法、解码方法、编码器、解码器以及存储介质 Download PDF

Info

Publication number
WO2021139572A1
WO2021139572A1 PCT/CN2020/141080 CN2020141080W WO2021139572A1 WO 2021139572 A1 WO2021139572 A1 WO 2021139572A1 CN 2020141080 W CN2020141080 W CN 2020141080W WO 2021139572 A1 WO2021139572 A1 WO 2021139572A1
Authority
WO
WIPO (PCT)
Prior art keywords
value
current block
mip
block
parameter
Prior art date
Application number
PCT/CN2020/141080
Other languages
English (en)
French (fr)
Inventor
霍俊彦
万帅
马彦卓
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to CN202210689703.4A priority Critical patent/CN115314710A/zh
Priority to EP20912114.4A priority patent/EP4090014A4/en
Priority to JP2022541305A priority patent/JP2023509951A/ja
Priority to CN202080068055.XA priority patent/CN114450945A/zh
Priority to KR1020227023785A priority patent/KR20220122666A/ko
Publication of WO2021139572A1 publication Critical patent/WO2021139572A1/zh
Priority to US17/858,161 priority patent/US11843781B2/en
Priority to US18/494,947 priority patent/US20240056582A1/en
Priority to US18/494,880 priority patent/US20240056581A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop

Definitions

  • the embodiments of the present application relate to the field of video coding and decoding technologies, and in particular, to an encoding method, a decoding method, an encoder, a decoder, and a storage medium.
  • H.265/High Efficiency Video Coding (HEVC) has been unable to meet the needs of the rapid development of video applications.
  • JVET Joint Video Exploration Team
  • VVC Very Video Coding
  • Matrix-based Intra Prediction is an intra prediction mode to obtain the intra prediction block of the current block; then it uses low-frequency inseparable secondary transformation ( Low-Frequency Non-Separable Transform (LFNST) technology to determine the prediction residual of the current block.
  • LFNST Low-Frequency Non-Separable Transform
  • the embodiments of the present application provide an encoding method, a decoding method, an encoder, a decoder, and a storage medium, which can improve encoding and decoding efficiency.
  • an embodiment of the present application provides an encoding method, which is applied to an encoder, and the method includes:
  • the prediction mode parameter indicates that the matrix-based intra prediction MIP mode is used for the chrominance component of the current block to determine the intra prediction value
  • the adjacent sample value of the current block is obtained, and the value of the current block is determined according to the Adjacent sample values determine the MIP input sample value of the current block
  • the LFNST parameters are encoded and written into the code stream.
  • an embodiment of the present application provides a decoding method, which is applied to a decoder, and the method includes:
  • Parse the code stream to obtain the prediction parameters and LFNST parameters of the current block, where the prediction parameters include prediction mode parameters;
  • the prediction mode parameter indicates that the matrix-based intra prediction MIP mode is used for the chrominance component of the current block to determine the intra prediction value
  • the adjacent sample value of the current block is obtained, and the value of the current block is determined according to the Adjacent sample values determine the MIP input sample value of the current block
  • the LFNST parameter indicates to perform LFNST on the current block, determine the reconstructed transform coefficient block of the current block, and perform LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second transform coefficient Piece;
  • an encoder which includes a first determination unit, a first prediction unit, a first transformation unit, and a coding unit; wherein,
  • the first determining unit is configured to determine a prediction parameter of a current block, where the prediction parameter includes a prediction mode parameter;
  • the first determining unit is further configured to, when the prediction mode parameter indicates that a matrix-based intra prediction MIP mode is used to determine an intra prediction value for the chrominance component of the current block, obtain the neighbors of the current block Sample value, and determine the MIP input sample value of the current block according to the adjacent sample value of the current block;
  • the first prediction unit is configured to determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weight matrix, and the shift parameter; wherein the MIP prediction value is the current The predicted value of some sampling points in the chrominance component of the block; and filtering the MIP predicted value to determine the intra-frame predicted value of the chrominance component of the current block;
  • the first transformation unit is configured to determine the prediction residual value of the chrominance component of the current block according to the intra prediction value of the chrominance component of the current block; Separate the secondary transformation LFNST and determine the LFNST parameters;
  • the encoding unit is configured to encode the LFNST parameter and write it into a code stream.
  • an encoder which includes a first memory and a first processor; wherein,
  • the first memory is configured to store a computer program that can run on the first processor
  • the first processor is configured to execute the method described in the first aspect when the computer program is running.
  • an embodiment of the present application provides a decoder, which includes a parsing unit, a second determination unit, a second prediction unit, and a second transformation unit; wherein,
  • the parsing unit is configured to parse the code stream to obtain prediction parameters and LFNST parameters of the current block, where the prediction parameters include prediction mode parameters;
  • the second determining unit is configured to obtain adjacent samples of the current block when the prediction mode parameter indicates that a matrix-based intra prediction MIP mode is used to determine an intra prediction value for the chrominance component of the current block Value, and determine the MIP input sample value of the current block according to the adjacent sample value of the current block;
  • the second prediction unit is configured to determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weight matrix, and the shift parameter; wherein the MIP prediction value is the current The predicted value of some sampling points in the chrominance component of the block; and in order to filter the MIP predicted value, determine the intra-frame predicted value of the chrominance component of the current block;
  • the second transform unit is configured to determine a reconstructed transform coefficient block of the current block when the LFNST parameter indicates to perform LFNST on the current block, and reconstruct and transform at least part of the reconstructed transform coefficient block Performing LFNST on the coefficients to obtain a second transform coefficient block; and performing a first transform on the second transform coefficient block to obtain a reconstructed residual block of the chrominance component of the current block;
  • the second determining unit is further configured to determine a reconstruction block of the chrominance component of the current block according to the intra prediction value of the chrominance component of the current block and the reconstruction residual block.
  • an embodiment of the present application provides a decoder, which includes a second memory and a second processor; wherein,
  • the second memory is configured to store a computer program that can run on the second processor
  • the second processor is configured to execute the method described in the second aspect when the computer program is running.
  • an embodiment of the present application provides a computer storage medium, the computer storage medium stores a computer program, and when the computer program is executed by a first processor, the method described in the first aspect is implemented, or when the computer program is executed by the second The processor implements the method described in the second aspect when executed.
  • the embodiments of the present application provide an encoding method, a decoding method, an encoder, a decoder, and a storage medium.
  • the prediction mode parameter indicates that the chrominance component of the current block is used based on
  • the intra prediction MIP mode of the matrix determines the intra prediction value
  • the adjacent sampling value of the current block is obtained, and the MIP input sampling value of the current block is determined according to the adjacent sampling value of the current block; according to the MIP input sampling value, MIP weighting matrix And shift parameters to determine the MIP prediction value of the chroma component of the current block; filter the MIP prediction value to determine the intra prediction value of the chroma component of the current block; according to the intraframe prediction value of the chroma component of the current block
  • the prediction value determines the prediction residual value of the chrominance component of the current block; performs the low-frequency inseparable secondary transformation LFNST on the prediction residual value to determine the LFNST parameter;
  • the prediction mode parameter indicates that the MIP mode is used to determine the intra prediction value for the chrominance component of the current block, the adjacent sample value of the current block is obtained , And determine the MIP input sample value of the current block according to the adjacent sample value of the current block; determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix and the shift parameter; perform the MIP prediction value Filter processing to determine the intra-frame prediction value of the chrominance component of the current block; when the LFNST parameter indicates to perform LFNST on the current block, determine the reconstructed transform coefficient block of the current block, and reconstruct at least part of the reconstructed transform coefficient block Perform LFNST on the transform coefficients to obtain the second transform coefficient block; perform the first transform on the second transform coefficient block to obtain the reconstructed residual block of the chrominance component of the current block; according to the
  • the complexity can be reduced on the basis of ensuring the coding and decoding performance, while the storage space required in the coding and decoding process is reduced, and the coding and decoding efficiency is effectively improved; in addition, the LFNST technology is being applied to the prediction of the MIP mode. At this time, due to the introduction of MIP parameters, the LFNST transform is more flexible, and the coding and decoding efficiency is further improved.
  • FIG. 1 is a schematic diagram of the composition structure of an encoder provided by an embodiment of the application.
  • FIG. 2 is a schematic diagram of the composition structure of a decoder provided by an embodiment of the application
  • FIG. 3 is a schematic flowchart of an encoding method provided by an embodiment of the application.
  • FIG. 4 is a schematic flowchart of another encoding method provided by an embodiment of the application.
  • FIG. 5 is a schematic flowchart of an MPI prediction process provided by an embodiment of the application.
  • FIG. 6 is a schematic diagram of a process for acquiring an intra prediction block using MIP mode in an embodiment of the application
  • FIG. 7 is a schematic diagram of a process framework for executing LFNST according to an embodiment of the application.
  • FIG. 8 is a schematic diagram of a process for encoding LFNST parameters according to an embodiment of the application.
  • FIG. 9 is a schematic flowchart of a decoding method provided by an embodiment of this application.
  • FIG. 10 is a schematic diagram of a process for parsing LFNST parameters according to an embodiment of the application.
  • FIG. 11 is a schematic diagram of another process framework for executing LFNST according to an embodiment of the application.
  • FIG. 12 is a schematic diagram of the composition structure of another encoder provided by an embodiment of the application.
  • FIG. 13 is a schematic diagram of a specific hardware structure of an encoder provided by an embodiment of the application.
  • FIG. 14 is a schematic diagram of the composition structure of another decoder provided by an embodiment of the application.
  • 15 is a schematic diagram of a specific hardware structure of a decoder provided by an embodiment of the application.
  • FIG. 16 is a schematic diagram of the composition structure of a sending device according to an embodiment of the application.
  • FIG. 17 is a schematic diagram of the composition structure of a target device provided by an embodiment of this application.
  • FIG. 18 is a schematic diagram of the composition structure of a communication system provided by an embodiment of this application.
  • VVC International Telecommunication Union-Telecommunication
  • ISO International Organization for Standardization
  • IEC International Electrotechnical Commission
  • JVET Joint Video Exploration Team
  • the matrix-based intra prediction (MIP) mode is a kind of intra prediction mode.
  • the encoder or decoder can call the MIP mode to obtain the intra prediction block of the current block.
  • a low frequency non-separable transform (LFNST) as a secondary transform (or called a second transform) can be applied to the prediction residual of the current block.
  • LNNST low frequency non-separable transform
  • LFNST can be applied to MIP mode prediction with a fixed configuration of LFNST transform core candidate sets and LFNST transposition indicator parameters, it has a negative impact on the coding and decoding efficiency of VVC.
  • an embodiment of the present application provides an encoding method.
  • the basic idea of the encoding method is to determine the prediction parameters of the current block, where the prediction parameters include prediction mode parameters;
  • the chrominance component of the block uses the matrix-based intra prediction MIP mode to determine the intra prediction value, the adjacent sample value of the current block is obtained, and the MIP input sample value of the current block is determined according to the adjacent sample value of the current block;
  • the MIP input sample value, the MIP weighting matrix, and the shift parameter are used to determine the MIP predicted value of the chroma component of the current block; filter the MIP predicted value to determine the intra-frame predicted value of the chroma component of the current block;
  • the intra prediction value of the chrominance component of the current block is determined to determine the prediction residual value of the chrominance component of the current block;
  • the low-frequency inseparable secondary transformation LFNST is performed on the prediction residual value to determine the LFNST parameter;
  • the LFNST parameter is determined Encode and
  • Another embodiment of the present application provides a decoding method.
  • the basic idea of the decoding method is to parse the code stream to obtain the prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters; when the prediction The mode parameter indicates that when the matrix-based intra prediction MIP mode is used to determine the intra prediction value of the chrominance component of the current block, the adjacent sample value of the current block is obtained, and the adjacent sample value of the current block is determined.
  • the MIP input sample value determines the MIP predicted value of the chrominance component of the current block according to the MIP input sample value, MIP weighting matrix, and shift parameter; filter the MIP predicted value to determine the chroma component of the current block
  • the LFNST parameter indicates to perform LFNST on the current block, determine the reconstructed transform coefficient block of the current block, and perform LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second Transform coefficient block; perform the first transform on the second transform coefficient block to obtain the reconstruction residual block of the chrominance component of the current block; according to the intra prediction value of the chrominance component of the current block and the reconstruction residual
  • the difference block determines the reconstruction block of the chrominance component of the current block.
  • the complexity can be reduced on the basis of ensuring the coding and decoding performance, while the storage space required in the coding and decoding process is reduced, and the coding and decoding efficiency is effectively improved; in addition, the LFNST technology is being applied to the prediction of the MIP mode. At this time, due to the introduction of MIP parameters, the LFNST transform is more flexible, and the coding and decoding efficiency is further improved.
  • the encoder 100 may include: a division unit 101, a prediction unit 102, a first adder 107, a transform unit 108, a quantization unit 109, an inverse quantization unit 110, an inverse transform unit 111, a second adder 112, The filtering unit 113, a decoded picture buffer (DPB) unit 114, and an entropy encoding unit 115.
  • the input of the encoder 100 may be a video composed of a series of pictures or a static picture
  • the output of the encoder 100 may be a bit stream (also referred to as a "code stream") used to represent a compressed version of the input video.
  • the dividing unit 101 divides the picture in the input video into one or more coding tree units (CTUs).
  • the dividing unit 101 divides the picture into multiple tiles, and may further divide a tile into one or more bricks.
  • a tile or a brick may include one or more complete and/or Part of the CTUs.
  • the dividing unit 101 may form one or more slices, where one slice may include one or more tiles arranged in a grid order in the picture, or one or more tiles covering a rectangular area in the picture.
  • the dividing unit 101 may also form one or more sub-pictures, where one sub-picture may include one or more slices, tiles, or bricks.
  • the segmentation unit 101 transmits the CTU to the prediction unit 102.
  • the prediction unit 102 may be composed of a block division unit 103, a motion estimation (ME) unit 104, a motion compensation (MC) unit 105, and an intra prediction unit 106.
  • the block segmentation unit 103 iteratively uses quadtree segmentation, binary tree segmentation, and triple tree segmentation to further divide the input CTU into smaller coding units (CUs).
  • the prediction unit 102 may use the ME unit 104 and the MC unit 105 to obtain the inter prediction block of the CU.
  • the intra prediction unit 106 may obtain the intra prediction block of the CU using various intra prediction modes including the MIP mode.
  • the rate-distortion optimized motion estimation method can be called by the ME unit 104 and the MC unit 105 to obtain the inter prediction block, and the rate-distortion optimized mode determination method can be called by the intra prediction unit 106 to obtain the intra prediction block .
  • the prediction unit 102 outputs the prediction block of the CU, and the first adder 107 calculates the difference between the CU and the prediction block of the CU in the output of the segmentation unit 101, that is, the residual CU.
  • the transform unit 108 reads the residual CU and performs one or more transform operations on the residual CU to obtain coefficients.
  • the quantization unit 109 quantizes the coefficients and outputs quantized coefficients (that is, levels).
  • the inverse quantization unit 110 performs a scaling operation on the quantized coefficient to output a reconstructed coefficient.
  • the inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108 and outputs a reconstruction residual.
  • the second adder 112 calculates the reconstructed CU by adding the reconstructed residual and the prediction block of the CU from the prediction unit 102.
  • the second adder 112 also sends its output to the prediction unit 102 to be used as an intra prediction reference.
  • the filtering unit 113 performs loop filtering on the reconstructed picture or sub-picture.
  • the filtering unit 113 includes one or more filters, such as a deblocking filter, a sample adaptive offset (Sample Adaptive Offset, SAO) filter, an adaptive loop filter (Adaptive Loop Filter, ALF), and a brightness map.
  • filters such as a deblocking filter, a sample adaptive offset (Sample Adaptive Offset, SAO) filter, an adaptive loop filter (Adaptive Loop Filter, ALF), and a brightness map.
  • chroma scaling (Luma Mapping with Chroma Scaling, LMCS) filters and neural network-based filters, etc.
  • the filtering unit 113 determines that the CU is not used as a reference when encoding other CUs, the filtering unit 113 performs loop filtering on one or more target pixels in the CU.
  • the output of the filtering unit 113 is a decoded picture or sub-picture, and these decoded pictures or sub-pictures are buffered in the DPB unit 114.
  • the DPB unit 114 outputs decoded pictures or sub-pictures according to timing and control information.
  • the picture stored in the DPB unit 114 can also be used as a reference for the prediction unit 102 to perform inter prediction or intra prediction.
  • the entropy encoding unit 115 converts the parameters (such as control parameters and supplementary information, etc.) necessary for decoding the picture from the encoder 100 into a binary form, and writes such a binary form into the code stream according to the syntax structure of each data unit , That is, the encoder 100 finally outputs the code stream.
  • the encoder 100 may be a first memory having a first processor and recording a computer program. When the first processor reads and runs the computer program, the encoder 100 reads the input video and generates a corresponding code stream.
  • the encoder 100 may also be a computing device having one or more chips. These units implemented as integrated circuits on a chip have similar connection and data exchange functions to the corresponding units in FIG. 1.
  • the decoder 200 may include: a parsing unit 201, a prediction unit 202, a scaling unit 205, a transformation unit 206, an adder 207, a filtering unit 208, and a decoded picture buffering unit 209.
  • the input of the decoder 200 is a bitstream used to represent a compressed version of a video or a still picture
  • the output of the decoder 200 may be a decoded video composed of a series of pictures or a decoded still picture.
  • the input code stream of the decoder 200 may be the code stream generated by the encoder 100.
  • the parsing unit 201 parses the input code stream and obtains the value of the syntax element from the input code stream.
  • the parsing unit 201 converts the binary representation of the syntax element into a digital value and sends the digital value to a unit in the decoder 200 to obtain one or more decoded pictures.
  • the parsing unit 201 may also parse one or more syntax elements from the input code stream to display the decoded picture.
  • the parsing unit 201 sends the value of the syntax element and one or more variables set or determined according to the value of the syntax element and used to obtain one or more decoded pictures to the decoder 200. unit.
  • the prediction unit 202 determines the prediction block of the currently decoded block (for example, CU).
  • the prediction unit 202 may include a motion compensation unit 203 and an intra prediction unit 204.
  • the prediction unit 202 passes the relevant parameters from the parsing unit 201 to the motion compensation unit 203 to obtain the inter-frame prediction block; when the intra-frame prediction mode (When the MIP mode (including the MIP mode indicated based on the MIP mode index value) is used to decode the current decoded block, the prediction unit 202 transmits the relevant parameters from the parsing unit 201 to the intra prediction unit 204 to obtain the intra prediction block.
  • the scaling unit 205 has the same function as the dequantization unit 110 in the encoder 100.
  • the scaling unit 205 performs a scaling operation on the quantized coefficients (ie, levels) from the analysis unit 201 to obtain reconstruction coefficients.
  • the transform unit 206 has the same function as the inverse transform unit 111 in the encoder 100.
  • the transform unit 206 performs one or more transform operations (that is, the inverse operation of one or more transform operations performed by the inverse transform unit 111 in the encoder 100) to obtain the reconstruction residual.
  • the adder 207 performs an addition operation on its inputs (the prediction block from the prediction unit 202 and the reconstruction residual from the transformation unit 206) to obtain a reconstructed block of the current decoded block.
  • the reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks encoded in the intra prediction mode.
  • the filtering unit 208 After all CUs in the picture or sub-picture are reconstructed, the filtering unit 208 performs loop filtering on the reconstructed picture or sub-picture.
  • the filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive compensation filter, an adaptive loop filter, a luminance mapping and chrominance scaling filter, and a neural network-based filter.
  • filters such as a deblocking filter, a sample adaptive compensation filter, an adaptive loop filter, a luminance mapping and chrominance scaling filter, and a neural network-based filter.
  • the filtering unit 208 determines that the reconstructed block is not used as a reference when decoding other blocks, the filtering unit 208 performs loop filtering on one or more target pixels in the reconstructed block.
  • the output of the filtering unit 208 is a decoded picture or sub-picture
  • the decoded picture or sub-picture is buffered in the DPB unit 209.
  • the DPB unit 209 outputs the decoded picture or sub-picture according to the timing and control information.
  • the picture stored in the DPB unit 209 can also be used as a reference for performing inter prediction or intra prediction by the prediction unit 202.
  • the decoder 200 may be a second memory having a second processor and recording computer programs. When the first processor reads and runs the computer program, the decoder 200 reads the input code stream and generates a corresponding decoded video.
  • the decoder 200 may also be a computing device having one or more chips. These units implemented as integrated circuits on the chip have similar connection and data exchange functions to the corresponding units in FIG. 2.
  • the encoding method provided by the embodiment of the present application mainly acts on the intra prediction unit 106 and the transform unit 108 of the encoder 100
  • the decoding method provided by the embodiment of the present application mainly acts on the frame of the decoder 200.
  • the intra prediction unit 204 and the transformation unit 206 that is to say, the embodiments of the present application can be applied to both the encoder and the decoder, or even the encoder and the decoder at the same time, but there is no limitation here.
  • the "current block” specifically refers to the image block currently to be encoded in the intra prediction (also referred to as the "encoding block”); when this application When the embodiment is applied to the decoder 200, the “current block” specifically refers to the image block currently to be decoded in the intra prediction (may also be referred to as the "decoding block”).
  • the encoder 100 can obtain a better prediction effect through the encoding method provided by the embodiment of the present application to improve the encoding performance; then, correspondingly, the decoder 200 can also be improved through the decoding method provided by the embodiment of the present application.
  • Video decoding restores quality, thereby improving decoding performance.
  • FIG. 3 shows a schematic flowchart of an encoding method provided in an embodiment of the present application.
  • the method may include:
  • S301 Determine a prediction parameter of the current block, where the prediction parameter includes a prediction mode parameter.
  • each image block to be encoded may be referred to as an encoding block.
  • each coding block may include a first image component, a second image component, and a third image component.
  • the current block is a coding block in the video image for which prediction of the first image component, the second image component, or the third image component is currently to be performed.
  • the current block can also be called a brightness block to obtain the predicted value of the brightness component of the current block .
  • the second image component is predicted for the current block
  • the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component
  • the current block can also be called a chrominance block to obtain the luminance component of the current block Predicted value.
  • the prediction parameters may include prediction mode parameters, where the prediction mode parameters are used to indicate the prediction mode adopted by the current block, and different prediction modes correspond to different prediction mode parameters.
  • simple decision-making strategies can be used, such as determination based on the magnitude of the distortion value; complex decision-making strategies, such as determination based on the result of Rate Distortion Optimization (RDO), can also be used.
  • RDO Rate Distortion Optimization
  • the embodiments of this application do not make any limitation.
  • the RDO method can be used to determine the prediction mode parameters of the current block, that is, to determine the prediction parameters of the current block.
  • the determining the prediction parameter of the current block may include:
  • Precoding the current block by using multiple prediction modes to obtain rate-distortion cost values corresponding to the multiple prediction modes;
  • the optimal rate-distortion cost value is selected from the obtained multiple rate-distortion cost values, and the prediction parameter in the prediction mode corresponding to the optimal rate-distortion cost value is determined as the prediction parameter of the current block.
  • multiple prediction modes may be used for the current block to perform pre-encoding processing on the current block respectively.
  • multiple prediction modes usually include inter prediction mode, traditional intra prediction mode, and non-traditional intra prediction mode.
  • the traditional intra prediction mode may include a direct current (DC) mode, a planar (PLANAR) mode, an angular intra prediction mode, and so on.
  • Non-traditional intra prediction modes may include MIP mode, Cross-component Linear Model Prediction (CCLM) mode, Intra Block Copy (IBC) mode, and PLT (Palette) mode.
  • the inter-frame prediction mode may include the traditional inter-frame prediction mode and the geometrical partitioning for inter-blocks (Geometrical partitioning for inter-blocks, GEO) mode.
  • the rate-distortion cost value corresponding to the multiple prediction modes can be obtained. Then select the optimal rate-distortion cost value from the obtained multiple rate-distortion cost values (usually, the minimum rate-distortion cost value is the optimal rate-distortion cost value), and the optimal rate-distortion cost value corresponds to The prediction parameter in the prediction mode is determined as the prediction parameter of the current block.
  • multiple prediction modes may be used to respectively pre-encode the current block to obtain distortion values corresponding to the multiple prediction modes. Then, the optimal distortion value is selected from the obtained multiple distortion values, and the prediction parameter in the prediction mode corresponding to the optimal distortion value is determined as the prediction parameter of the current block. In this way, the encoder finally uses the prediction mode indicated by the prediction mode parameter in the determined prediction parameters to perform predictive coding on the current block, which can make the prediction residual smaller and improve the coding efficiency.
  • the embodiment of the present application uses the MIP mode to perform intra-frame prediction on the chrominance component of the current block. In this process, it is first necessary to obtain the adjacent sampling value of the current block; then according to the adjacent sampling value of the current block, it is used to determine the MIP input sampling value of the current block.
  • the MIP core parameters need to be configured first.
  • the MIP core parameters can include the type of the current block (represented by mipSizeId), the number of reference samples for each edge (represented by boundySize), the number of MIP input samples (represented by inSize), and the MIP prediction block output by matrix multiplication Size (arranged as predSize ⁇ predSize).
  • the MIP mode can divide the current block into three types according to the width and height of the current block. mipSizeId can be equal to 0, 1, or 2.
  • the block size index value of the block can be equal to 0, 1, or 2.
  • the number of reference samples (boundySize reference sample points are required on each side), the number of MIP input samples (inSize), and the MIP prediction block size (arranged as predSize ⁇ predSize) output by matrix multiplication are also different.
  • the prediction parameter may also include: the size parameter of the current block.
  • the size parameter of the current block may include: the width (represented by nTbW) and the height (represented by nTbH) of the current block.
  • the block size index value (ie mipSizeId) of the current block can be determined.
  • the determining the block size index value of the current block according to the size parameter of the current block may include:
  • the block size index value of the current block can be set to 0;
  • the block size index value of the current block can be set to 1;
  • the block size index value of the current block can be set to 2.
  • the determining the block size index value of the current block according to the size parameter of the current block may include:
  • the block size index value of the current block can be set to 0;
  • the block size index value of the current block can be set to 1;
  • the block size index value of the current block can be set to 2.
  • the look-up table (LUT) shown in Table 1 can be used to determine the number of adjacent boundary reference samples (variable boundarySize) and the MIP prediction block
  • the size (variable is predSize, the size of the MIP prediction block is predSize ⁇ predSize), and the number of MIP input sample values used in the MIP matrix multiplication operation process (represented by inSize) is calculated.
  • the calculation formula is as follows:
  • the reference pixel selects two pixels for each side, and the matrix multiplication output is a 4 ⁇ 4 MIP prediction block.
  • the reference pixel selects four pixels for each side, and the matrix multiplication output is a 4 ⁇ 4 MIP prediction block.
  • the value of boundarySize can be 4, and the value of predSize can be 8; that is to say, four pixels are selected for each side of the reference pixel at this time, and the matrix multiplication output is 8 ⁇ 8 The MIP prediction block.
  • boundarySize, inSize, and predSize can also be determined according to the block size index value of the current block and the lookup table shown in Table 2.
  • the value of boundarySize can be 2
  • the value of inSize can be 4
  • the value of predSize can be 4; that is, the reference pixel at this time is each side Two pixels are selected, the number of input sampling points of the matrix multiplication is four, and the output of the matrix multiplication is a 4 ⁇ 4 MIP prediction block.
  • the value of boundarySize can be 4
  • the value of inSize can be 8
  • the value of predSize can be 4; that is, four pixels are selected for each side as the reference pixel at this time.
  • the number of input sampling points of the matrix multiplication is eight, and the output of the matrix multiplication is a 4 ⁇ 4 MIP prediction block.
  • the value of boundarySize can be 4
  • the value of inSize can be 7
  • the value of predSize can be 8; that is, at this time, the reference pixel is four pixels for each side.
  • the number of input sampling points of matrix multiplication is seven, and the output of matrix multiplication is an 8 ⁇ 8 MIP prediction block.
  • the reference pixel is the adjacent sample value of the current block, which may include: the left adjacent sample value of the current block and the upper adjacent sample value of the current block.
  • the MIP input sample value of the current block can be determined according to the left adjacent sample value and the upper adjacent sample value of the current block.
  • the determining the MIP input sample value of the current block according to the adjacent sample value of the current block may include:
  • the second constant value is determined according to the bit depth of the adjacent sample value of the current block; the index number in the MIP input sample value is 0 The corresponding value is set equal to the difference between the second constant value and the value corresponding to the index number of 0 in the first temporary reference value; the value corresponding to the index number of i in the MIP input sample value is set Is equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, where i is an integer greater than 0;
  • the value corresponding to the index number equal to j in the MIP input sample value is set to be equal to the index number equal to j+1 in the first temporary reference value
  • the method may further include:
  • the first temporary reference value can be obtained after performing down-sampling filtering processing on adjacent sample values of the current block.
  • the adjacent sample value after the filter processing may be buffered in a buffer (indicated by pTemp).
  • the value corresponding to the index number of 0 in the first temporary reference value refers to pTemp[0]
  • the value corresponding to the index number of i in the first temporary reference value refers to pTemp[i].
  • the MIP input sample value is determined by the buffer (indicated by pTemp), the block size index value of the current block (indicated by MipSizeId), and the bit depth of the adjacent sample value of the current block (indicated by BitDepth), and MIP
  • the number of input samples included in the input sample value is only related to the block size index value of the current block, and finally the value corresponding to the index number x in the MIP input sample value (represented by p[x]) can be obtained.
  • the determining the second constant value according to the bit depth of adjacent sample values of the current block may include:
  • the second constant value is set to an integer exponential power equal to 2, wherein the exponent of the power is equal to the bit depth of the adjacent sample value of the current block minus one.
  • the determining the second constant value according to the bit depth of adjacent sample values of the current block may include:
  • the second constant value may be expressed as 1 ⁇ (BitDepth-1) or 2 ⁇ (BitDepth-1). In this way, for the situation that the size parameter value of the current block is within the preset range, the second constant value can be combined to determine the MIP input sample value of the current block.
  • the MIP input samples are matrix vectors used for matrix multiplication operations.
  • the current related technical solution consists of the buffer (represented by pTemp), the type of the current block (that is, the block size index value of the current block, represented by mipSizeId), the bit depth of the adjacent sample value of the current block (represented by BitDepth), and The number of MIP input samples is determined, and finally the value corresponding to the index number of x in the MIP input sample value is obtained (denoted by p[x]).
  • the construction process may include:
  • i is a positive integer greater than 0 and less than N
  • N represents the number of elements contained in the input sampling matrix
  • the subtract in the calculation of the difference is set to be equal to the second constant value, and the subtract in the calculation of the difference is set to be equal to the value corresponding to the index number of 0 in the first temporary reference value.
  • the construction process may include:
  • i is a positive integer greater than 0 and less than N
  • N represents the number of elements contained in the input sampling matrix
  • the subtracted number in the calculation of the difference is set to be equal to the value corresponding to the index number of 0 in the first temporary reference value, and the subtracted in the calculation of the difference is set to be equal to the second constant value.
  • the value corresponding to the index number of 0 in the MIP input sample value (represented by p[0]) can be determined by the index number in the first temporary reference value as The value corresponding to 0 (that is, pTemp[0]) is obtained by subtracting the second constant value (that is, 1 ⁇ (BitDepth-1)).
  • the index number in the MIP input sample value is the value corresponding to x (represented by p[x]), and the index number in the first temporary reference value is the value corresponding to x (ie pTemp[x] ) Is obtained by subtracting the value corresponding to the index number of 0 in the first temporary reference value (ie, pTemp[0]).
  • the construction process may include:
  • the four MIP input sampling values they can also be formed into a 1 ⁇ 4 MIP input sampling matrix.
  • the embodiment of the present application may also use a unified calculation method to obtain the value of p[x] without judging the value of mipSizeId.
  • the determining the MIP input sample value of the current block according to the adjacent sample value of the current block may include:
  • a second constant value ie (1 ⁇ (BitDepth-1)
  • the MIP prediction value of the current block can be further determined.
  • S303 Determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weight matrix, and the shift parameter.
  • the MIP predicted value is the predicted value of some sampling points in the chrominance component of the current block.
  • the shift parameter may include a shift offset parameter and a shift quantity parameter.
  • the following steps may be included:
  • S303-1 Determine the product of the sum of the MIP input sample values and the shift offset parameter according to the value of the shift offset parameter.
  • the value of the shift offset parameter is a fixed constant.
  • the shift offset parameter can also be referred to as a shift compensation parameter or an offset factor, which can be represented by f0.
  • the value of the shift offset parameter can be set to a fixed constant, such as 32, or 46, or 56, or 66. It is also possible to set the value of the shift offset parameter to be related to the shift offset parameter table, and determine the value of the shift offset parameter by looking up the table, which is not limited here.
  • the shift quantity parameter may also be referred to as shift factor, shift bit number, weight shift value, etc., and may be represented by sw, shift, or weight shift.
  • the shift amount parameter is represented by sW.
  • the first offset can be represented by oW, and the first offset is related to both the shift quantity parameter and the shift offset parameter.
  • the value of the shift amount parameter may be set to a fixed constant, such as 5, or 6, or 7, etc. It is also possible to set the value of the shift amount parameter to be related to the shift amount parameter table, and determine the value of the shift amount parameter by looking up the table, which is not limited here.
  • S303-4 Determine the MIP weight matrix of the current block according to the prediction parameter.
  • the prediction parameter may include a prediction mode parameter, and may also include a size parameter of the current block.
  • a weight matrix table can be established in advance at this time, and the weight matrix table is stored in a memory or a storage unit, where The memory or storage unit can be integrated in the encoder, or can be set separately.
  • the MIP weight matrix or MIP weight matrix, or MIP weight matrix
  • the block size index value (mipSizeId) of the current block is determined by the size parameter of the current block, and the size of the MIP weighting matrix mWeight[x][y] is only related to the block size index value of the current block, as shown in Table 3. Show.
  • the number of columns is the number of input samples of the matrix multiplication inSize, and the number of rows is the number of output samples of the matrix multiplication predSized ⁇ predSized, so that the MIP weighting matrix of the current block can be determined.
  • a shift quantity parameter table may also be established in advance, and the shift quantity parameter table is also stored in the memory or storage unit, where the memory or storage unit may be integrated in the encoder, or It can also be set individually.
  • the determination of the shift amount parameter (sW) may include the following methods:
  • the shift amount parameter may be different.
  • the method may further include:
  • the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determining the MIP mode index value of the current block;
  • the MIP mode index value query the value corresponding to the MIP mode index value from a first preset lookup table, where the first preset lookup table is used to record the MIP mode index value and the shift amount Correspondence between the values of the parameters;
  • the queried value is determined as the value of the shift amount parameter.
  • the value of the shift quantity parameter can be queried according to the block size index value (indicated by mipSizeId) and the MIP mode index value (modeId) of the current block.
  • the shift quantity parameter that needs to be used in the matrix multiplication can be determined through the lookup table.
  • Table 4 needs to be stored in the memory or storage unit in the form of a lookup table; however, storage is costly, and the search process also requires cost; because the value of the shift quantity parameter in Table 4 is different from the current
  • the block size of the block is related to the index value of the MIP mode, which increases the memory usage and also increases the computational complexity.
  • the embodiment of the present application may simplify the method for determining the shift amount parameter.
  • the value of the shift amount parameter can be set to a fixed constant, and it is a fixed constant independent of the block size index value and the MIP mode index value.
  • the value of the shift amount parameter can be set to 5; or, for different block size index values and different MIP mode index values, the shift amount The value of the parameter can be set to 6; or, for different block size index values and different MIP mode index values, the value of the shift amount parameter can be set to 7.
  • the value of the shift amount parameter is set to be equal to 6, but there is no limitation on this.
  • the method may further include:
  • the block size index value of the current block can be determined.
  • the determining the block size index value of the current block according to the size parameter of the current block may include:
  • the block size index value of the current block is set to 0;
  • the block size index value of the current block is set to 1;
  • the block size index value of the current block is set to 2.
  • the value of the shift amount parameter can be further determined according to the block size index value of the current block.
  • the determining the value of the shift amount parameter according to the block size index value of the current block may include:
  • the determining the value of the shift amount parameter according to the block size index value of the current block may include:
  • the value of the shift amount parameter is set to be equal to the ratio between the width or height of the current block and the first preset value corresponding to the block size index value of the current block.
  • the first preset value represents the number of MIP input sample values obtained from the boundary of the current block.
  • the method can also include:
  • the first preset value represents the number of MIP input sample values obtained from the boundary of the current block
  • the corresponding first preset value is equal to 2
  • the block size index value of the current block is equal to 1
  • the corresponding first preset value is equal to 4
  • the block size index value of the current block is equal to 2
  • the corresponding first preset value is equal to 4
  • the block size index value of the current block is equal to 2
  • the corresponding first preset value is equal to 4
  • the determining the value of the shift amount parameter according to the block size index value of the current block may include:
  • the value of the shift amount parameter is set to be equal to the ratio between the width or height of the current block and the second preset value corresponding to the block size index value of the current block.
  • the second preset value represents the size of the MIP prediction block of the current block obtained by calculation directly using the MIP weight matrix.
  • the method can also include:
  • the second preset value indicates the size of the MIP prediction block of the current block obtained by directly using the MIP weight matrix calculation
  • the corresponding second preset value Equal to 4
  • the block size index value of the current block is equal to 1
  • the corresponding second preset value is equal to 4
  • the block size index value of the current block is equal to 2
  • the corresponding second preset value is equal to 8 which can be based on The ratio between the width or height of the current block and the corresponding second preset value determines the value of the shift quantity parameter.
  • the shift quantity parameter table can be minimized, and a look-up table is still used to determine the value of the shift quantity parameter.
  • the determining the value of the shift amount parameter according to the block size index value of the current block may include:
  • the block size index value query a value corresponding to the block size index value from a second preset lookup table, where the second preset lookup table is used to record the block size index value and the shift amount Correspondence between the values of the parameters;
  • the queried value is determined as the value of the shift amount parameter.
  • the value of the shift amount parameter can be queried only according to the block size index value (indicated by mipSizeId) of the current block.
  • the block size index value indicated by mipSizeId
  • it can correspond to a fixed value, that is, the size of each block or the size set of each block can have the following Table 5 shows the value of the fixed number of shift parameters.
  • the storage of the look-up table can be minimized, so that the storage of the look-up table can be minimized without increasing
  • reduce the memory occupied by the shift quantity parameter table storage in the MIP mode reduce the memory occupied by the shift quantity parameter table storage in the MIP mode.
  • a shift offset parameter table may also be established in advance, and the shift offset parameter table is also stored in a memory or a storage unit, where the memory or storage unit may be integrated in the encoder, or Can be set individually.
  • the determination of the shift offset parameter (fO) may include the following methods:
  • the shift offset parameters may also be different.
  • the method may further include:
  • the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determining the MIP mode index value of the current block;
  • the value corresponding to the MIP mode index value is queried from a third preset lookup table, where the third preset lookup table is used to record the MIP mode index value and the shift offset.
  • the third preset lookup table is used to record the MIP mode index value and the shift offset.
  • the queried value is determined as the value of the shift offset parameter.
  • the value of the shift offset parameter can be queried according to the block size index value (represented by MipSizeId) and the MIP mode index value (modeId) of the current block.
  • the shift offset parameter that needs to be used in the matrix multiplication can be determined through the lookup table.
  • Table 6 also needs to be stored in the memory or storage unit in the form of a lookup table; however, storage is costly, and the search process also requires cost; because the value of the shift offset parameter in Table 6 is The block size of the current block is related to the MIP mode index value, which increases the memory usage and also increases the computational complexity.
  • the embodiment of the present application may also simplify the method for determining the shift offset parameter.
  • the value of the shift offset parameter can be set to a fixed constant, and it is a fixed constant independent of the block size index value and the MIP mode index value.
  • the value range of the fixed constant is 0-100.
  • the value of the shift offset parameter can be set to 32; or, for different block size index values and different MIP mode index values, shift The value of the offset parameter can be set to 46; or, for different block size index values and different MIP mode index values, the value of the shift offset parameter can be set to 56; or, for different block sizes
  • the index value, the index value of different MIP modes, and the value of the shift offset parameter can be set to 66.
  • the value of the shift offset parameter is set to be equal to 32, but there is no limitation on this.
  • the method may further include:
  • the block size index value of the current block can be determined according to the size parameter of the current block; and then the value of the shift offset parameter can be further determined according to the block size index value of the current block.
  • the determining the value of the shift offset parameter according to the block size index value of the current block may include:
  • the shift offset parameter table can also be minimized, and a look-up table is still used to determine the offset factor.
  • the determining the value of the shift offset parameter according to the block size index value of the current block may include:
  • the value corresponding to the block size index value is queried from a fourth preset lookup table, where the fourth preset lookup table is used to record the block size index value and the shift offset.
  • the queried value is determined as the value of the shift offset parameter.
  • the value of the shift offset parameter can be queried only according to the block size index value (indicated by mipSizeId) of the current block.
  • the block size index value indicated by mipSizeId
  • the fourth preset lookup table shown in Table 7 for each block size index value, it can correspond to a fixed value, that is, the size of each block or the size set of each block can have the following Table 7 shows the values of the fixed shift offset parameters.
  • the storage of the look-up table can be minimized, thereby enabling Without increasing the computational complexity, the memory occupied by the shift offset parameter table storage in the MIP mode is reduced.
  • the selection of the shift quantity parameter Value, to determine the first constant value which can include:
  • the first constant value is set to an integer exponential power equal to 2, wherein the exponent of the power is equal to the value of the shift amount parameter minus one.
  • the determining the first constant value according to the value of the shift amount parameter may include:
  • the first constant value can be expressed as 1 ⁇ (sW-1) or 2 ⁇ (sW-1). At this time, when the value of the shift amount parameter is set to 6, the first constant value equal to 32 can be obtained.
  • the product of the sum of the MIP input sample values and the shift offset parameter can be calculated according to the shift offset parameter (fO), which is Then the value of oW can be set as
  • the MIP weight matrix, the MIP input sample value, the shift quantity parameter, and the first offset can be obtained, so as to subsequently determine the MIP prediction value of the current block.
  • S303-5 Determine the MIP prediction value of the chrominance component of the current block according to the MIP weighting matrix, the MIP input sample value, the shift quantity parameter, and the first offset.
  • the MIP predicted value represents the predicted value of some sampling points in the chrominance component of the current block.
  • the determining the MIP prediction value of the current block according to the MIP weighting matrix, the MIP input sample value, the shift quantity parameter, and the first offset may include:
  • the MIP prediction value of the current block is set equal to the sum of the first right shift value and the value corresponding to the index number of 0 in the first temporary reference value; wherein, the first temporary reference value is based on It is obtained by performing down-sampling filtering processing on adjacent sample values of the current block.
  • the MIP weighting matrix (indicated by mWeight) and the shift amount parameter (indicated by mWeight) can be determined according to the block size index value (indicated by mipSizeId) and the MIP mode index value (indicated by modeId) of the current block in the MIP mode.
  • predMip[x][y] can be calculated to obtain the MIP prediction block.
  • the MIP prediction block can be directly determined as the MIP prediction block of the current block to obtain the MIP prediction value of the current block, and the MIP prediction value is the prediction value of some sampling points in the current block.
  • S304 Perform filtering processing on the MIP predicted value, and determine the intra-frame predicted value of the chrominance component of the current block.
  • performing filtering processing on the MIP prediction value to determine the intra prediction value of the current block may include:
  • the intra prediction block of the current block is set equal to the MIP prediction block; wherein, the MIP prediction block includes the Predicted sampling values of all pixel positions in the current block;
  • the filter processing may include up-sampling filter processing or low-pass filter processing.
  • the MIP prediction block is composed of MIP prediction values.
  • the size parameters of the MIP prediction block only include two types: 4 ⁇ 4 MIP prediction block and 8 ⁇ 8 MIP prediction block; in this way, the size parameter of the current block and the size parameter of the MIP prediction block It may be the same, or it may not be the same. That is to say, the sampling value corresponding to the MIP prediction block may not be able to fill the current block, so that the generation of the final prediction value may require an up-sampling operation on the MIP prediction block, that is, by judging the size parameter of the MIP prediction block and the size of the current block Whether the parameters are the same to determine whether to perform up-sampling processing on the MIP prediction block.
  • the size parameter of the MIP prediction block is the same as the size parameter of the current block, that is, the width and height of the MIP prediction block are the same as the current block, which indicates that there is no need to perform upsampling processing on the MIP prediction block.
  • the MIP prediction block can be directly filled into the current block, that is, there are no vacant pixels in the filled current block.
  • the intra prediction value of each pixel in the current block can be directly set to each pixel in the MIP prediction block
  • the predicted value of is as follows,
  • predSamples[x][y] predMip[x][y] (7)
  • [x][y] represents the position coordinates of the pixel, x represents the horizontal direction, and y represents the vertical direction; predSamples[x][y] represents the pixel corresponding to the location coordinate [x][y] in the current block Intra-frame prediction value, predMip[x][y] represents the prediction value corresponding to the pixel at the position coordinate [x][y] in the MIP prediction block.
  • the MIP prediction block predMip[x][y] can be directly used as the intra prediction block predSamples[x][y] of the current block, that is, the intra prediction value of at least one pixel in the current block .
  • the MIP prediction block when the size parameter of the MIP prediction block is different from the size parameter of the current block, that is, at least one of the width and height of the MIP prediction block is different from the current block. At this time, the MIP prediction block cannot be filled. The current block, that is, there are free pixels in the filled current block, which indicates that the MIP prediction block needs to be filtered at this time. In other words, if upsampling is required in both the horizontal and vertical directions, the MIP prediction block can be up-sampled in the horizontal direction and then up-sampled in the vertical direction to obtain the first up-sampling block.
  • PredSamples can be used [x][y] means; then the MIP prediction block is up-sampled vertically and then up-sampled horizontally to obtain the second up-sampling block, which can be represented by predSamplesTemp[x][y]; finally, predSamples[x ][y] and predSamplesTemp[x][y] perform weighted average calculation, and finally obtain the intra prediction value of the chrominance component of the current block, that is, obtain the intra prediction block of the current block.
  • nTbS of the current block (here, S can be replaced by W and H respectively) is equal to the side length predSize of predMip (here, predSize is only related to the blocksizeIdx of the current block)
  • predSize is only related to the blocksizeIdx of the current block
  • the MIP prediction block can be directly set as the frame of the current block Intra prediction block, so that the intra prediction value of at least one pixel in the current block can be obtained.
  • the input data of the MIP mode may include: the position of the current block (xTbCmp, yTbCmp), the MIP mode index value applied to the current block (which can be represented by modeId), and the current block The height (indicated by nTbH), the width of the current block (indicated by nTbW), and the transposition indication flag (that is, the MIP transposition indicator parameter, which can be expressed by isTransposed) whether transposition processing is required.
  • the output data of the MIP mode may include: the intra prediction block of the current block.
  • the MIP prediction process specifically includes the following steps:
  • MIP core parameters can include the type of the current block (represented by mipSizeId), the number of reference samples for each edge (represented by boundySize), the number of MIP input samples (represented by inSize), and the output of matrix multiplication MIP prediction block size (arranged as predSize ⁇ predSize).
  • the current block can be divided into three types, and mipSizeId is used to record the type of the current block.
  • mipSizeId can be equal to 0, 1, or 2.
  • the number of reference sampling points and the size of the MIP prediction block output by the matrix multiplication are different.
  • the upper neighboring block and the left neighboring block of the current block are all already coded blocks.
  • the reference pixels of the MIP mode are the previous row of pixels and the left column of pixels adjacent to the current block.
  • the reconstruction value In other words, the process of obtaining the reference pixel adjacent to the side (indicated by refT) and the reference pixel adjacent to the left side (indicated by refL) on the current block is the process of obtaining the reference pixel.
  • this step is used for the input of matrix multiplication, which may specifically include:
  • the process of obtaining reference samples is a process of down-sampling the reference pixels.
  • the process of obtaining reference samples is a process of down-sampling the reference pixels.
  • the S5032 in the construction of the reference sampling buffer, there are two filling methods, which can specifically include:
  • this step is used to obtain the MIP prediction value of the current block, which may specifically include:
  • S5042 Clamping processing is performed on the MIP prediction block sampled by the matrix multiplication output
  • the process of constructing the MIP prediction block may specifically include:
  • generating the final MIP prediction value includes two situations, which can specifically include:
  • S5044-1 Generate a predicted value that does not require up-sampling filtering processing
  • S5044-2 Generate predicted values that require up-sampling filtering processing.
  • the embodiment of the present application can obtain the intra-frame prediction value of at least one pixel in the chrominance component of the current block.
  • S305 Determine the prediction residual value of the chroma component of the current block according to the intra-frame prediction value of the chroma component of the current block.
  • the difference between the true value of the chrominance component of the current block and the intra-frame prediction value of the chrominance component of the current block can be calculated to obtain The prediction residual value of the chrominance component of the current block, that is, the residual block of the current block is obtained.
  • S306 Perform low-frequency inseparable secondary transformation LFNST on the prediction residual value, and determine LFNST parameters.
  • the transform unit 108 may perform the first transformation on the residual block of the current block, such as an initial design based on discrete cosine transform (DCT) Integer transformation; it is also possible to perform a second transformation on the residual block of the current block, such as Low-Frequency Non-Separable Transform (LFNST).
  • the performing the low-frequency inseparable secondary transformation LFNST on the prediction residual value to determine the LFNST parameter may include:
  • the encoder can perform the first transformation (also called “Core Transform” or “primary transformation” or “main transformation”) on the prediction residual value in the residual block to obtain the first transformation.
  • the first transform coefficient block after primary transformation then LFNST transform (also called “Secondary Transform” or “secondary transform”) is performed on part or all of the transform coefficients in the first transform coefficient block to determine the LFNST parameter. It should be noted that the first transformation here is different from LFNST.
  • any current block can execute LFNST.
  • some conditions such as the minimum value in the size parameter of the current block, the block size index value of the current block, etc.
  • Transform coefficients perform LFNST.
  • the determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block may include:
  • the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determining the smallest value among the size parameters of the current block;
  • the determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block according to the minimum value may include:
  • the minimum value is greater than or equal to the first preset threshold, it is determined to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.
  • the first preset threshold may be used to indicate a preset threshold for measuring whether to perform LFNST.
  • the first preset threshold may be set to 8, but it is not specifically limited.
  • the minimum value of the size parameter of the current block can be determined at this time; for example, when the minimum value of the width and height is greater than or equal to 8, this It may be determined to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.
  • the determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block may include:
  • the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determining the block size index value of the current block according to the size parameter of the current block;
  • the block size index value it is determined whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.
  • the determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block according to the block size index value may include:
  • the second preset threshold may also be used to indicate a preset threshold for measuring whether to perform LFNST.
  • the second preset threshold may be set to a preset value (for example, 2), or one of multiple preset values (for example, 1, 2), but is not specifically limited.
  • the block size index value (mipSizeId) of the current block can be determined according to the size parameter of the current block; for example, when the value of mipSizeId is equal to 2 At this time, it may be determined to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.
  • the determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block may include:
  • the first cost result and the second cost result it is determined whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.
  • the determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block according to the first cost result and the second cost result may include:
  • the first cost result is greater than the second cost result, it is determined to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.
  • the embodiment of the present application can adopt the traditional rate-distortion optimization method.
  • the first cost result is calculated without performing LFNST on at least part of the transform coefficients in the first transform coefficient block;
  • calculate the second cost result compare the first cost result with the second cost result, and when the first cost result is greater than the second cost result, it can be determined at this time At least part of the transform coefficients in the first transform coefficient block perform LFNST.
  • LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block, and the LFNST parameters are determined.
  • the LFNST parameter includes the LFNST index number, which can be represented by lfnst_index.
  • the determining the LFNST parameter may include:
  • the value of the LFNST index number is set to be equal to zero.
  • the encoding the LFNST parameters and writing the code stream may include:
  • the LFNST index sequence number is coded and written into the code stream.
  • the Context-based Adaptive Binary Arithmetic Coding (CABAC) with the descriptor "ae(v)" can be used to encode the lfnst_index.
  • CABAC Context-based Adaptive Binary Arithmetic Coding
  • the judgment condition can also be set as: both the width and the height are greater than or equal to the first value (for example, 8), and the sum of the width and the height is greater than or equal to the second value (for example, 16).
  • the judgment condition can also be set as: mipSizeId is equal to one of a plurality of preset values (for example, 1, 2).
  • the performing LFNST on the prediction residual value to determine the LFNST parameters may include: determining the LFNST transformation core used by the current block; using the LFNST transformation core to perform LFNST on the prediction residual value to determine LFNST parameters.
  • the four transform core candidate sets may include set0, set1, set2, and set3.
  • the embodiment of the present application may use the MIP parameter to determine the LFNST transform core candidate set, and then select the LFNST transform core used by the current block from the LFNST transform core candidate set. Therefore, in some embodiments, the method may further include:
  • the MIP parameter is determined from the prediction parameter of the current block
  • the transform core indicated by the LFNST index number is set as the LFNST transform core used by the current block;
  • the LFNST transform core candidate set includes two or more preset LFNST transform cores.
  • the MIP parameter may include at least one of the following: a MIP mode index value (represented by modeId) and a MIP transposition indication parameter (represented by isTransposed).
  • modeId a MIP mode index value
  • MIP transposition indication parameter represented by isTransposed
  • the determining the LFNST transformation core candidate set according to the MIP parameters may include:
  • the LFNST transform core candidate set is selected from a plurality of candidate LFNST transform core candidate sets.
  • the LFNST intra prediction mode index number can be represented by predModeIntra
  • the index value of the LFNST transform core candidate set can be represented by lfnstTrSetIdx.
  • the fifth preset lookup table is shown in Table 10.
  • the value of predModeIntra can be determined; then according to the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in conjunction with Table 10, that is, the LFNST transform selected for the current block is determined Nuclear candidate set.
  • the value of lfnstTrSetIdx indicates the candidate set of transform cores used in LFNST; since the value of modeId can include 0, 1, 2, 3, 4, and 5, the value of predModeIntra is also 0, 1, 2 , 3, 4, 5; its corresponding relationship with lfnstTrSetIdx is as follows,
  • the determining the value of the LFNST intra prediction mode index sequence number according to the MIP mode index value may include: setting the value of the LFNST intra prediction mode index sequence number to be equal to the value of the index sequence number of the LFNST intra prediction mode. MIP mode index value.
  • the value of predModeIntra can be set equal to the value of modeId, and then the value of lfnstTrSetIdx can be directly determined according to the value of predModeIntra in conjunction with Table 10, that is, the LFNST transform selected for the current block is determined Nuclear candidate set.
  • the embodiment of the present application can also directly map the value of modeId to the value of the PLANAR mode, and then use the value of predModeIntra corresponding to the PLANAR mode to determine The value of lfnstTrSetIdx determines the candidate set of LFNST transform cores selected by the current block.
  • the LFNST parameters may also include LFNST coefficients.
  • the determining the LFNST parameter may include:
  • the encoding the LFNST parameters and writing the code stream includes:
  • the quantized coefficient is coded and written into the code stream.
  • lfnst_index is not equal to 0, that is, when it is determined to perform LFNST on the current block, at this time, by performing LFNST on at least part of the transform coefficients in the first transform coefficient block, the LFNST coefficients can be obtained; and then the LFNST coefficients are quantized And encode, and write the bitstream.
  • the horizontal scanning sequence may be directly used to perform LFNST on at least part of the transform coefficients in the first transform coefficient block to obtain LFNST coefficients. details as follows,
  • log2LfnstSize is as follows:
  • the scanning sequence may include a horizontal scanning sequence and a vertical scanning sequence.
  • the determination of the scan order is related to the value of the MIP transposition indication parameter (indicated by isTransposed).
  • the performing LFNST on at least part of the transform coefficients in the first transform coefficient block to obtain the LFNST coefficients may include:
  • the LFNST is performed on at least part of the transform coefficients in the first transform coefficient block according to the vertical scanning order to obtain the LFNST coefficient
  • the LFNST is performed on at least part of the transform coefficients in the first transform coefficient block according to the horizontal scanning order to obtain the LFNST coefficient.
  • the performing LFNST on at least part of the transform coefficients in the first transform coefficient block to obtain the LFNST coefficients may include:
  • the LFNST is performed on at least part of the transform coefficients in the first transform coefficient block according to the horizontal scanning order to obtain the LFNST coefficient
  • the LFNST is performed on at least part of the transform coefficients in the first transform coefficient block according to the vertical scanning order to obtain the LFNST coefficient.
  • the LFNST in the embodiments of the present application can be applied to traditional intra prediction modes (for example, angular intra prediction mode, DC mode, PLANAR mode, etc.) in addition to the MIP mode. Its specific implementation is similar to that of MIP mode.
  • the determining the LFNST transform core candidate set according to the MIP parameters may include:
  • the LFNST transform core candidate set is selected from a plurality of candidate LFNST transform core candidate sets.
  • the sixth preset lookup table is shown in Table 11.
  • the value of predModeIntra can be determined; then according to the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in conjunction with Table 11, that is, the LFNST transform core candidate set selected for the current block is determined .
  • the index value of the traditional intra prediction mode can be assigned to predModeIntra to determine the value of lfnstTrSetIdx.
  • the current block in the embodiment of the present application may be a CU or a partition of a CU (for example, a transform block).
  • the intra prediction unit 106 determines that the MIP mode is used to encode the current block, or when the intra prediction unit 106 uses the rate-distortion optimization method to evaluate the MIP mode used to encode the current block, the intra prediction unit 106 106 will obtain the intra prediction block of the current block; wherein, the intra prediction block of the current block includes the intra prediction value of at least one pixel (sample) in the current block.
  • the intra prediction unit 106 uses the MIP mode to obtain the intra prediction block, and the steps are as follows:
  • the intra prediction unit 106 obtains one or more reference pixels from neighboring pixels of the current block. For example, it can be obtained by down-sampling adjacent pixels, or directly extracted from adjacent pixels.
  • the intra prediction unit 106 uses the acquired reference pixels, MIP matrix, and shift parameters to determine one or more partial prediction pixels corresponding to the pixel positions in the current block.
  • the pixel position may be a preset pixel position in the current block.
  • the pixel position has uniform horizontal and vertical coordinate values.
  • the shift parameter includes a shift amount parameter and a shift offset parameter, and the shift parameter can be used for an offset operation in the process of obtaining intra-frame prediction values of some pixels.
  • the intra prediction unit 106 also needs to obtain the predicted pixels corresponding to the remaining pixels in the current block except for some pixels.
  • the intra prediction unit 106 may use an interpolation filter to obtain predicted pixels corresponding to the remaining pixels, where the input of the interpolation filter may be some pixels and adjacent pixels.
  • FIG. 6 shows an example of a block diagram of a process for acquiring an intra prediction block using MIP mode in an embodiment of the present application.
  • the process can be implemented on the encoder 100, and the "current block” here refers to the "encoded block”.
  • the intra prediction unit 106 obtains adjacent pixels of the current block, for example, the adjacent pixels are marked as gray filled squares adjacent to the current block in step 601 shown in FIG. 6.
  • the intra prediction unit 106 obtains one or more reference pixels from adjacent pixels.
  • the intra prediction unit 106 may calculate an average value of two adjacent pixels and use the average value as a reference pixel.
  • the intra prediction unit 106 selects every other adjacent pixel as the reference pixel. For example, in the example of step 601 shown in FIG. 6, the intra prediction unit 106 selects 4 reference pixels from the 8 adjacent pixels on the upper side of the current block, and selects 4 reference pixels from the 8 adjacent pixels on the left side of the current block. The other 4 reference pixels are selected from the pixels of.
  • step 601 The specific process of step 601 is as follows:
  • the intra prediction unit 106 obtains the width and height of the current block from the block dividing unit 103, and the width and height are represented by variables cbWidth and cbHeight, respectively.
  • the intra prediction unit 206 invokes a rate-distortion optimized mode determination method to determine the intra prediction mode, where the current block is divided into one or more transform blocks. Let the variables nTbW and nTbH be the width and height of the transform block, respectively.
  • the intra prediction unit 106 determines the block size index value of the MIP, that is, a variable expressed as mipSizeId.
  • the intra prediction unit 106 determines the value of mipSizeId as follows:
  • mipSizeId is set equal to 0;
  • mipSizeId is set equal to 1;
  • mipSizeId is set equal to 2.
  • mipSizeId is set equal to 2.
  • the intra prediction unit 106 determines the value of mipSizeId as follows:
  • mipSizeId is set equal to 0;
  • mipSizeId is set equal to 1;
  • mipSizeId is set equal to 2.
  • mipSizeId is set equal to 1.
  • the intra-frame prediction unit 106 obtains the values of boundarySize and predSize according to mipSizeId, which are specifically as follows:
  • boundarySize is set to be equal to 2
  • predSize is set to be equal to 4
  • mipSizeId is equal to 1
  • boundarySize is set to be equal to 4
  • predSize is set to be equal to 4
  • boundarySize is set to be equal to 4
  • predSize is set to be equal to 8
  • boundarySize represents the number of reference pixels obtained from each side of the reference pixels adjacent to the upper side of the current block and the reference pixels adjacent to the left side of the current block.
  • the intra prediction unit 106 can also obtain the variable isTransposed to indicate the order of the reference pixels stored in the buffer pTemp.
  • isTransposed equal to 0 means: the intra prediction unit 106 first presents the reference pixels obtained from the adjacent reference pixels on the upper side of the current block, and then presents the reference pixels obtained from the adjacent reference pixels on the left side; otherwise, isTransposed is equal to 1 means: the intra prediction unit 106 first presents the reference pixels obtained from the reference pixels adjacent to the left side of the current block, and then presents the reference pixels obtained from the reference pixels adjacent to the upper side.
  • the value of isTransposed will be sent to the entropy encoding unit 115 as one of the parameters of the MIP mode that needs to be encoded and written into the code stream.
  • the intra prediction unit 106 obtains the value of inSize to indicate the number of reference pixels used when the MIP mode is used, as shown in the above formula (1).
  • the intra prediction unit 106 calls the following process to obtain a group of reference pixels (stored as an array p[x], where x is 0,..., inSize-1) using reference pixels adjacent to the current block.
  • the intra prediction unit 106 obtains the nTbW reference pixel from the reference pixel adjacent to the upper side of the current block (for example, stored in the array refT), and obtains the nTbW reference pixel from the reference pixel adjacent to the left side of the current block (for example, stored in the array refL). ) Obtain nTbH reference pixels.
  • the intra prediction unit 106 calls down-sampling processing on refT to obtain boundarySize reference pixels, and stores the boundarySize reference pixels in refT.
  • the intra prediction unit 106 calls down-sampling processing on refL to obtain boundarySize reference pixels, and stores the boundarySize reference pixels in refL.
  • the intra prediction unit 106 may also obtain the variable isTransposed to indicate the order of the reference pixels stored in the buffer pTemp.
  • isTransposed equal to 0 means: the intra prediction unit 106 first presents the reference pixels obtained from the adjacent reference pixels on the upper side of the current block, and then presents the reference pixels obtained from the adjacent reference pixels on the left side; Otherwise, isTransposed equal to 1 (or TRUE) means that the intra prediction unit 106 first presents the reference pixels obtained from the reference pixels adjacent to the left side of the current block, and then presents the reference pixels obtained from the reference pixels adjacent to the upper side.
  • the intra prediction unit 106 may determine the value of isTransposed by using a rate-distortion optimization method or based on a correlation comparison between adjacent reference pixels and the current block. The value of isTransposed will be sent to the entropy encoding unit 115 as one of the parameters of the MIP mode that needs to be encoded and written into the code stream.
  • the intra prediction unit 106 puts the elements in refT and refL into the buffer pTemp according to the order indicated by isTransposed.
  • BitDepth is the bit depth of the color component of the pixel in the current block.
  • the color component may be one of the RGB components, or one of the YUV components, or one of the YCbCr components, for example, the Y component.
  • the intra prediction unit 106 may use a unified calculation method to obtain the value of p[x] without determining the value of mipSizeId. For example, adding (1 ⁇ (BitDepth-1)) as an additional element to the buffer pTemp, at this time, the intra prediction unit 106 calculates p[x] as pTemp[x+1]-pTemp[0].
  • Step 602 the intra prediction unit 106 uses a set of reference pixels and MIP weighting matrix to obtain the MIP prediction value of the current block.
  • the MIP weighting matrix is selected from a set of predefined MIP weighting matrices according to the MIP mode index value (represented as ModeId) corresponding to the MIP mode and the block size index value of the MIP (represented as mipSizeId).
  • the intra-frame prediction unit 106 obtains the MIP prediction value (denoted as predMip[x][y]) of a part of the predicted pixel points corresponding to one or more pixel point positions in the current block.
  • part of the predicted pixels are pixels marked as gray filled squares in the current block.
  • the input of the prediction module 601 is the reference pixel p[x] obtained in step 601, and the prediction module 601 uses the MIP weighting matrix and the shift parameter to calculate part of the predicted pixel; here, the shift parameter includes the shift quantity parameter and the shift offset parameter.
  • the prediction module 601 can make its coordinates equal to (x, y) and obtain the predicted pixel points as represented by predMip[x][y], then the calculation formula of predMip[x][y] is as follows: (5) or formula (6).
  • mWeight[i][j] is the MIP weighting matrix; here, the matrix element can be a predetermined constant value; or, it can also be updated adaptively using, for example, a training method.
  • the input of is one or more coded pictures or blocks, or pictures in other code streams provided by an external device to the encoder 100; f0 is a shift offset parameter used to determine oW; sW is a shift amount parameter; p [i] is the MIP input sample value calculated using reference pixels; pTemp[0] represents the value corresponding to the index number of 0 in the first temporary reference value; ">>" is the binary right shift operation as defined in VVC symbol.
  • the intra prediction unit 106 may also send mWeight[i][j] to the entropy encoding unit 115.
  • the entropy encoding unit 115 may write mWeight[i][j] into one or more data units in the code stream.
  • the prediction module 601 may determine the values of sW and f0 according to the size of the current block and the MIP mode used for the current block. In one example, the prediction module 601 uses a look-up table to obtain the values of sW and fO.
  • the prediction module 601 can use the aforementioned Table 4 to determine sW according to the size parameter of the current block and the MIP mode.
  • the prediction module 601 can also use the aforementioned Table 5 to determine sW according to the size parameter of the current block.
  • the prediction module 601 can also directly set sW as a constant value. For example, for blocks of various size parameters and different MIP modes, the prediction module 601 may set sW to 5; or, for blocks of various size parameters and different MIP modes, the prediction module 601 may set sW to 6; or For blocks of various size parameters and different MIP modes, the prediction module 601 sets sW to 7.
  • the prediction module 601 may use the aforementioned Table 6 to determine f0 according to the size parameter of the current block and the MIP mode.
  • the prediction module 601 can also use the aforementioned Table 7 to determine f0 according to the size parameter of the current block.
  • the prediction module 601 can also directly set f0 as a constant value (such as 0-100). For example, for blocks of various size parameters and different MIP modes, the prediction module 601 sets fO to 32; or, the prediction module 601 sets fO to 46; or, the prediction module 601 sets fO to 56; or, the prediction module 601 sets fO to 66.
  • f0 a constant value (such as 0-100). For example, for blocks of various size parameters and different MIP modes, the prediction module 601 sets fO to 32; or, the prediction module 601 sets fO to 46; or, the prediction module 601 sets fO to 56; or, the prediction module 601 sets fO to 66.
  • the intra prediction unit 106 may perform a clamp processing operation on the MIP predicted value in predMip.
  • isTransposed is equal to 1 (or TRUE)
  • the intra prediction unit 106 determines that mipSizeId is equal to 2 and obtains an 8 ⁇ 8 predMip.
  • the intra prediction unit 106 determines that mipSizeId is equal to 1 and obtains a 4 ⁇ 4 predMip.
  • the intra prediction unit 106 calls an upsampling process to obtain predSamples using predMip.
  • the intra prediction unit 106 obtains predicted pixel points corresponding to the remaining pixels in the current block except for some pixels.
  • the intra prediction unit 106 may use the filtering module 602 to obtain predicted pixels corresponding to the remaining pixels except for some pixels in the current block.
  • the input of the filtering module 602 is the pixels marked as gray filled squares in step 602.
  • the filtering module 602 may use one or more interpolation filters, and use the input to obtain predicted pixels corresponding to the remaining pixels in the current block except for some pixels.
  • the input may include a reference pixel and a partial prediction of one or more pixel positions in the current block; or, the input may include a partial prediction of neighboring pixels and one or more pixel positions in the current block Pixels; or, the input may include reference pixels, neighboring pixels, and partial predicted pixels at one or more pixel positions in the current block.
  • the intra prediction unit 106 determines that mipSizeId is equal to 1, and the intra prediction unit 106 applies the upsampling process to 4 ⁇ 4 predMip, to obtain an 8 ⁇ 8 intra prediction block of the current block.
  • the intra prediction unit 106 determines that mipSizeId is equal to 2, and the intra prediction unit 106 obtains the predSamples of the current block, whose size is equal to 8 ⁇ 8 (that is, both cbWidth and cbHeight are equal to 8).
  • the intra prediction unit 106 can obtain the intra prediction block of the current block (ie, CU), that is, determine the intra prediction value of at least one pixel in the current block.
  • the prediction unit 102 outputs the intra prediction block of the current block.
  • the first adder 107 calculates the difference between the current block and the intra prediction block of the current block in the output of the segmentation unit 101, that is, the residual block (that is, the residual CU).
  • the transform unit 108 reads the residual block and performs one or more transform operations on the residual block to obtain coefficients.
  • the quantization unit 109 quantizes the coefficients and outputs quantized coefficients (that is, levels).
  • the transform unit 108 performs a first transform on the residual block, for example, an integer transform originally designed based on DCT.
  • the transformation unit 108 determines whether to allow secondary transformation to be applied to the block. When it is determined that the application is allowed, the transformation unit 108 further determines whether to apply the secondary transformation to the coefficient obtained after the first transformation is performed.
  • LFNST is an example of secondary transformation.
  • FIG. 7 shows a schematic diagram of a process framework for executing LFNST according to an embodiment of the present application.
  • a block 701 represents a residual block.
  • the transform unit 108 performs the first transform (ie core transform) on the block 701, for example, the integer transform originally designed based on DCT, and obtains the transform coefficients corresponding to the first transform (referred to as "first transform coefficients"). ) Of block 702.
  • step 702 the transform unit 108 determines whether to allow secondary transform, such as LFNST, to be performed on all or part of the first transform coefficients in the block 702.
  • secondary transform such as LFNST
  • the transform unit 108 is allowed to perform LFNST on the transform block.
  • the transform unit is allowed 108 performs LFNST on the transform block.
  • the transform unit 108 determines the LFNST parameter, such as lfnst_index.
  • Lfnst_index is equal to 0, it means that the transform unit 108 does not perform LFNST on the transform block.
  • lfnst_index is greater than 0, it means that the transformation core in the LFNST transformation core candidate set is used to perform LFNST on the transformation block.
  • the transformation unit 108 may use a traditional rate-distortion optimization method to determine the value of lfnst_index. First, the transform unit 108 obtains the first cost result without performing LFNST on the transform block.
  • the transformation unit 108 selects the LFNST transformation core candidate set according to the MIP parameters.
  • the MIP parameter may include at least one of: a MIP mode index value (ie modeId) and a MIP transposition indication parameter (ie isTransposed).
  • the conversion unit 108 sets the value of the variable predModeIntra to be equal to the value of modeId.
  • the transform unit 108 selects a part of the first transform coefficient in the block 702, for example, the coefficients in the sub-blocks 7001, 7002, and 7003, and determines a transform core used to perform LFNST on the coefficients on the sub-blocks.
  • the size of the sub-block may be a preset value, such as 8 ⁇ 8.
  • the transformation unit 108 determines the index value of the LFNST transformation core candidate set (that is, lfnstTrSetIdx) according to the value of predModeIntra.
  • the transform unit 108 may use a look-up table method such as Table 10 to obtain the value of lfnstTrSetIdx.
  • the transformation unit 108 may use a traditional rate-distortion optimization method to determine the value of lfnst_index (that is, if the value is greater than 0, it means: lfnstTrSetIdx indicates the transformation core used in the LFNST transformation core candidate set).
  • the transform unit 108 sets the value of lfnst_index to the index number of the transform core in the LFNST transform core candidate set, so that the cost function has a minimum value and the second cost result is also obtained.
  • the transform unit 108 determines to perform LFNST on the transform block, and sends lfnst_index indicating the index number of the transform core to the entropy encoding unit 115. Otherwise, if the transform unit 108 determines that LFNST is not to be performed on the transform block, lfnst_value is set equal to 0, and lfnst_index indicating that LFNST is not to be performed is sent to the entropy encoding unit 115.
  • the transform unit 108 When lfnst_index is not equal to 0, the transform unit 108 performs LFNST on the transform block using the coefficients from the sub-blocks 7001, 7002, and 7003, obtains the LFNST coefficients, and puts the LFNST coefficients into the sub-blocks 7011, 7012, and 7013 of the block 703.
  • the transform unit 108 uses the following instructions to obtain the block 703 of transform coefficients, and transmits the block 703 of the LFNST coefficients in d[x][y] to the quantization unit 109, specifically as shown in the above equation (8).
  • log2LfnstSize is as shown in the above formula (9).
  • the transform unit 108 uses the following instruction to obtain the block 703 of LFNST coefficients, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (10 ) Shows;
  • the transform unit 108 uses the following instructions to obtain the block 703 of LFNST coefficients, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above equation (11) .
  • the transform unit 208 uses the following instruction to obtain the block 703 of LFNST coefficients, and transmits the block 703 of the LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above equation (10 ) Shows;
  • the transform unit 208 obtains the block 703 of LFNST coefficients using the following instructions, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (11) .
  • the transform unit 108 can also determine the value of lfnst_index for the transform block encoded in the traditional intra prediction mode (ie, angular intra prediction mode, DC mode, PLANAR mode, etc.). Similar to the rate-distortion optimization method for determining the value of lfnst_index of the transform block in the MIP mode, the first cost result is determined without performing LFNST on the transform block at this time.
  • the traditional intra prediction mode ie, angular intra prediction mode, DC mode, PLANAR mode, etc.
  • the transform unit 108 uses a look-up table method to determine the value of lfnstTrSetIdx according to the mode index value of the traditional intra prediction mode (assigned to predModeIntra), that is, determines the LFNST transform core candidate set selected for the current block, as shown in Table 11 above. .
  • the transformation unit 108 may determine the value of lfnst_index by adopting a traditional rate-distortion optimization method (that is, if the value is greater than 0, it means: lfnstTrSetIdx indicates the transformation core used in the LFNST transformation core candidate set).
  • the transform unit 108 sets the value of lfnst_index to the index number of the transform core in the LFNST transform core candidate set, so that the cost function has a minimum value and the second cost result is also obtained.
  • the transform unit 108 determines to perform LFNST on the transform block, and sends lfnst_index indicating the index number of the transform core to the entropy encoding unit 115. Otherwise, if the transform unit 108 determines that LFNST is not to be performed on the transform block, lfnst_value is set equal to 0, and lfnst_index indicating that LFNST is not to be performed is sent to the entropy encoding unit 115.
  • the transform unit 108 When lfnst_index is not equal to 0, the transform unit 108 performs LFNST on the transform block using the coefficients from the sub-blocks 7001, 7002, and 7003, obtains the LFNST coefficients, and puts the LFNST coefficients into the sub-blocks 7011, 7012, and 7013 of the block 703.
  • the transform unit 108 uses the following instructions to obtain the block 703 of LFNST coefficients, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (10) Shown;
  • the transform unit 108 uses the following instructions to obtain the block 703 of LFNST coefficients, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above equation (11) .
  • the transform unit 108 uses the following instructions to obtain the block 703 of LFNST coefficients, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above equation (10 ) Shows;
  • the transform unit 108 uses the following instructions to obtain the block 703 of LFNST coefficients, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (11) Shown.
  • the inverse quantization unit 110 performs a scaling operation on the quantized coefficient to output the reconstructed coefficient.
  • the inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108 and outputs a reconstruction residual.
  • the second adder 112 calculates the reconstructed CU by adding the reconstruction residual and the intra prediction block of the current block from the prediction unit 102.
  • the second adder 112 also sends its output to the prediction unit 102 to be used as an intra prediction reference.
  • the output of the filtering unit 113 is a decoded picture or sub-picture, and these decoded pictures or sub-pictures are sent to the DPB unit 114.
  • the DPB unit 114 outputs the decoded picture according to the timing and control information.
  • the pictures stored in the DPB unit 114 can also be used as references for the prediction unit 102 to perform inter prediction or intra prediction.
  • the entropy encoding unit 115 converts the parameters (necessary to obtain decoded pictures), control parameters, and supplementary information from the units in the encoder 100 into binary representations, and writes such binary representations into the generation according to the syntax structure of each data unit.
  • the code stream (or called "video bit stream”) is specifically shown in Table 9A and Table 9B.
  • the entropy encoding unit 115 uses the MIP mode to perform The lfnst_index is encoded. Another optional condition may be set as: the width and height are greater than or equal to the first value (for example, 8), and the sum of the width and height is greater than or equal to the second value (for example, 16).
  • the entropy encoding unit 115 uses the MIP mode to encode the lfnst_index of the coding unit.
  • Another optional condition may be set as: the mipSizeId of the coding unit is equal to one of multiple values (for example, 1 and 2).
  • FIG. 8 shows a schematic flowchart of encoding LFNST parameters according to an embodiment of the present application. As shown in Figure 8, the process may include:
  • the entropy encoding unit 115 checks the above conditions to determine whether to encode the LFNST parameter.
  • the entropy encoding unit 115 uses context-based adaptive binary arithmetic coding to encode the LFNST parameter.
  • step S801 the entropy encoding unit 115 checks to determine the LFNST parameter, such as the value of lfnst_index; then in step S802, the descriptor shown in Table 9A and Table 9B can be used as "ae(v) "CABAC to encode the value of lfnst_index.
  • the encoder 100 may be a computing device having a processor and a storage medium recording an encoding program. When the processor reads and runs the encoding program, the encoder 100 reads the input video and generates a corresponding code stream.
  • the encoder 100 may also be a computing device with one or more chips. A unit implemented as an integrated circuit on a chip has similar functions through similar connections and data exchanges to the corresponding unit in FIG. 1.
  • This embodiment provides an encoding method by determining the prediction parameters of the current block, where the prediction parameters include prediction mode parameters; when the prediction mode parameters indicate the use of matrix-based frames for the chrominance components of the current block
  • the intra prediction MIP mode determines the intra prediction value
  • the adjacent sample value of the current block is obtained, and the MIP input sample value of the current block is determined according to the adjacent sample value of the current block; according to the MIP input sample Value, MIP weight matrix and shift parameter, determine the MIP prediction value of the chroma component of the current block; perform filtering processing on the MIP prediction value to determine the intra prediction value of the chroma component of the current block;
  • the intra-frame prediction value of the chrominance component of the current block is determined to determine the prediction residual value of the chrominance component of the current block;
  • the low-frequency inseparable secondary transformation LFNST is performed on the prediction residual value to determine the LFNST parameter;
  • the LFNST parameters are encoded and written into the code stream.
  • the complexity can be reduced on the basis of ensuring the coding performance, while the storage space required in the coding process is reduced, and the coding efficiency is effectively improved.
  • the LFNST technology is applied to MIP mode prediction, due to The introduction of MIP parameters makes the LFNST transformation more flexible and further improves the coding efficiency.
  • FIG. 9 shows a schematic flowchart of a decoding method provided in an embodiment of the present application.
  • the method may include:
  • S901 Parse the code stream to obtain prediction parameters and LFNST parameters of the current block, where the prediction parameters include prediction mode parameters.
  • each image block to be decoded may be referred to as a decoding block.
  • each decoding block may include a first image component, a second image component, and a third image component; and the current block is the decoding of the first image component, the second image component, or the third image component currently to be predicted in the video image. Piece.
  • the prediction parameters may include prediction mode parameters, where the prediction mode parameters are used to indicate the prediction mode adopted by the current block, and different prediction modes correspond to different prediction mode parameters.
  • the prediction modes usually include inter prediction mode, traditional intra prediction mode, and non-traditional intra prediction mode, etc.
  • traditional intra prediction modes can include DC mode, PLANAR mode and angular intra prediction mode, etc.
  • non-traditional The intra prediction modes may include MIP mode, CCLM mode, IBC mode, PLT mode, and so on.
  • the encoder will select the optimal prediction mode to encode the current block.
  • the prediction mode of the current block can be determined, so as to obtain the prediction mode parameters correspondingly, and then the prediction parameters including the prediction mode parameters Write the code stream and transmit it from the encoder to the decoder.
  • the prediction parameters of the current block can be obtained by parsing the code stream, and the prediction mode parameters included in the prediction parameters obtained according to the analysis can be used to determine whether the current block uses the MIP mode.
  • the embodiment of the present application uses the MIP mode to perform intra-frame prediction on the chrominance component of the current block. In this process, it is first necessary to obtain the adjacent sampling value of the current block; then according to the adjacent sampling value of the current block, it is used to determine the MIP input sampling value of the current block.
  • the adjacent sample value of the current block may include the left adjacent sample value of the current block and the upper adjacent sample value of the current block.
  • the MIP input sample value of the current block can be determined according to the left adjacent sample value and the upper adjacent sample value of the current block.
  • the determining the MIP input sample value of the current block according to the adjacent sample value of the current block may include:
  • the second constant value is determined according to the bit depth of the adjacent sample value of the current block; the index number in the MIP input sample value is 0 The corresponding value is set equal to the difference between the second constant value and the value corresponding to the index number of 0 in the first temporary reference value; the value corresponding to the index number of i in the MIP input sample value is set Is equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, where i is an integer greater than 0;
  • the value corresponding to the index number equal to j in the MIP input sample value is set to be equal to the index number equal to j+1 in the first temporary reference value
  • the method may further include:
  • the first temporary reference value can be obtained after performing down-sampling filtering processing on adjacent sample values of the current block.
  • the adjacent sample value after filtering processing may be buffered in a buffer (represented by pTemp).
  • the value corresponding to the index number of 0 in the first temporary reference value refers to pTemp[0]
  • the value corresponding to the index number of i in the first temporary reference value refers to pTemp[i].
  • the prediction parameter may also include: the size parameter of the current block.
  • the size parameter of the current block may include: the width (represented by nTbW) and the height (represented by nTbH) of the current block.
  • the block size index value (ie mipSizeId) of the current block can be determined.
  • the MIP input sample value is determined by the buffer (indicated by pTemp), the block size index value of the current block (indicated by MipSizeId), and the bit depth of the adjacent sample value of the current block (indicated by BitDepth), and MIP
  • the number of input samples included in the input sample value is only related to the block size index value of the current block, and finally the value corresponding to the index number x in the MIP input sample value (represented by p[x]) can be obtained.
  • the determining the second constant value according to the bit depth of adjacent sample values of the current block may include:
  • the second constant value is set to an integer exponential power equal to 2, wherein the exponent of the power is equal to the bit depth of the adjacent sample value of the current block minus one.
  • the determining the second constant value according to the bit depth of adjacent sample values of the current block may include:
  • the second constant value may be expressed as 1 ⁇ (BitDepth-1) or 2 ⁇ (BitDepth-1). In this way, for the situation that the size parameter value of the current block is within the preset range, the second constant value can be combined to determine the MIP input sample value of the current block.
  • the MIP input samples are matrix vectors used for matrix multiplication operations.
  • the current related technical solution consists of the buffer (represented by pTemp), the type of the current block (that is, the block size index value of the current block, represented by mipSizeId), the bit depth of the adjacent sample value of the current block (represented by BitDepth), and The number of MIP input samples is determined, and finally the value corresponding to the index number of x in the MIP input sample value is obtained (denoted by p[x]).
  • the embodiment of the present application may also use a unified calculation method to obtain the value of p[x] without judging the value of mipSizeId.
  • the MIP prediction value of the current block can be further determined.
  • S903 Determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weight matrix, and the shift parameter.
  • the MIP predicted value is the predicted value of some sampling points in the chrominance component of the current block.
  • the shift parameter may include a shift offset parameter and a shift quantity parameter.
  • determining the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix, and the shift parameter may include:
  • the shift offset parameter can also be referred to as a shift compensation parameter or an offset factor, which can be represented by f0.
  • the value of the shift offset parameter can be set to a fixed constant, such as 32, or 46, or 56, or 66, or 32, etc.; the value of the shift offset parameter can also be set
  • the value of the shift offset parameter is determined by looking up the table, and there is no limitation here.
  • the shift quantity parameter may also be referred to as shift factor, shift bit number, weight shift value, etc., and may be represented by sw, shift, or weight shift.
  • the shift amount parameter is usually represented by sW.
  • the value of the shift amount parameter may be set to a fixed constant, such as 5, or 6, or 7, etc. It is also possible to set the value of the shift amount parameter to be related to the shift amount parameter table, and determine the value of the shift amount parameter by looking up the table, which is not limited here.
  • the first offset can be represented by oW, and the first offset is related to both the shift quantity parameter and the shift offset parameter.
  • the prediction parameters may include prediction mode parameters and size parameters of the current block.
  • a weight matrix table can be established in advance at this time, and the weight matrix table is stored in a memory or a storage unit, where the memory or storage unit It can be integrated in the decoder, or it can be set separately.
  • the MIP weight matrix or MIP weight matrix, or MIP weight matrix
  • MIP matrix represented by mWeight[x][y].
  • a shift quantity parameter table may also be established in advance, and the shift quantity parameter table is also stored in a memory or a storage unit, where the memory or storage unit may be integrated in the decoder In, or can also be set separately.
  • the determination of the shift amount parameter (sW) may include the following methods:
  • the shift amount parameter may be different.
  • the method may further include:
  • the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, analyzing the code stream to obtain the MIP mode index value of the current block;
  • the MIP mode index value query the value corresponding to the MIP mode index value from a first preset lookup table, where the first preset lookup table is used to record the MIP mode index value and the shift amount Correspondence between the values of the parameters;
  • the queried value is determined as the value of the shift amount parameter.
  • the value of the shift quantity parameter can be queried according to the block size index value (indicated by mipSizeId) and the MIP mode index value (modeId) of the current block.
  • the shift quantity parameter that needs to be used in the matrix multiplication can be determined through the look-up table.
  • Table 4 needs to be stored in the memory or storage unit in the form of a lookup table; however, storage is costly, and the search process also requires cost; due to the shift factor in Table 4 and the block size of the current block The size and the index value of the MIP mode are all related, which increases the memory footprint and also increases the computational complexity.
  • the embodiment of the present application simplifies the method for determining the shift factor.
  • the value of the shift amount parameter can be set to a fixed constant, and it is a fixed constant independent of the block size index value and the MIP mode index value.
  • the value of the shift amount parameter can be set to 5; or, for different block size index values and different MIP mode index values, the shift amount The value of the parameter can be set to 6; or, for different block size index values and different MIP mode index values, the value of the shift amount parameter can be set to 7.
  • the value of the shift amount parameter is equal to 6, but there is no limitation on this.
  • the method may further include:
  • the block size index value of the current block can be determined.
  • the determining the block size index value of the current block according to the size parameter of the current block may include:
  • the block size index value of the current block is set to 0;
  • the block size index value of the current block is set to 1;
  • the block size index value of the current block is set to 2.
  • the value of the shift amount parameter can be further determined according to the block size index value of the current block.
  • the determining the value of the shift amount parameter according to the block size index value of the current block may include:
  • the determining the value of the shift amount parameter according to the block size index value of the current block may include:
  • the value of the shift amount parameter is set to be equal to the ratio between the width or height of the current block and the first preset value corresponding to the block size index value of the current block.
  • the first preset value represents the number of MIP input sample values obtained from the boundary of the current block.
  • the method can also include:
  • the first preset value represents the number of MIP input sample values obtained from the boundary of the current block
  • the corresponding first preset value is equal to 2
  • the block size index value of the current block is equal to 1
  • the corresponding first preset value is equal to 4
  • the block size index value of the current block is equal to 2
  • the corresponding first preset value is equal to 4
  • the block size index value of the current block is equal to 2
  • the corresponding first preset value is equal to 4
  • the determining the value of the shift amount parameter according to the block size index value of the current block may include:
  • the value of the shift amount parameter is set to be equal to the ratio between the width or height of the current block and the second preset value corresponding to the block size index value of the current block.
  • the second preset value represents the size of the MIP prediction block of the current block obtained by calculation directly using the MIP weight matrix.
  • the method can also include:
  • the second preset value indicates the size of the MIP prediction block of the current block obtained by directly using the MIP weight matrix calculation
  • the corresponding second preset value Equal to 4
  • the block size index value of the current block is equal to 1
  • the corresponding second preset value is equal to 4
  • the block size index value of the current block is equal to 2
  • the corresponding second preset value is equal to 8 which can be based on The ratio between the width or height of the current block and the corresponding second preset value determines the value of the shift quantity parameter.
  • the shift quantity parameter table can be minimized, and a look-up table is still used to determine the value of the shift quantity parameter.
  • the determining the value of the shift amount parameter according to the block size index value of the current block may include:
  • the block size index value query a value corresponding to the block size index value from a second preset lookup table, where the second preset lookup table is used to record the block size index value and the shift amount Correspondence between the values of the parameters;
  • the queried value is determined as the value of the shift amount parameter.
  • the value of the shift amount parameter can be queried only according to the block size index value (indicated by mipSizeId) of the current block.
  • the block size index value for each block size index value, it can correspond to a fixed value, that is, the size of each block or the size set of each block can have As shown in Table 5, the value of the fixed shift quantity parameter.
  • a shift offset parameter table may also be established in advance, and the shift offset parameter table is also stored in a memory or a storage unit, where the memory or storage unit may be integrated in the decoder , Or it can be set separately.
  • the determination of the shift offset parameter (fO) may include the following methods:
  • the shift offset parameters may also be different.
  • the method may further include:
  • the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, analyzing the code stream to determine the MIP mode index value of the current block;
  • the value corresponding to the MIP mode index value is queried from a third preset lookup table, where the third preset lookup table is used to record the MIP mode index value and the shift offset.
  • the third preset lookup table is used to record the MIP mode index value and the shift offset.
  • the queried value is determined as the value of the shift offset parameter.
  • the value of the shift offset parameter can be queried according to the block size index value (represented by mipSizeId) and the MIP mode index value (modeId) of the current block.
  • the shift offset parameter that needs to be used in the matrix multiplication can be determined through the look-up table.
  • Table 6 also needs to be stored in the memory or storage unit in the form of a lookup table; however, storage is costly, and the search process also requires cost; because the value of the shift offset parameter in Table 6 is The block size of the current block is related to the MIP mode index value, which increases the memory footprint and also increases the computational complexity.
  • the embodiment of the present application also simplifies the method for determining the shift offset parameter.
  • the value of the shift offset parameter can be set to a fixed constant, and it is a fixed constant independent of the block size index value and the MIP mode index value.
  • the value range of the fixed constant is 0-100.
  • the value of the shift offset parameter can be set to 32; or, for different block size index values and different MIP mode index values, shift The value of the offset parameter can be set to 46; or, for different block size index values and different MIP mode index values, the value of the shift offset parameter can be set to 56; or, for different block sizes
  • the index value, the index value of different MIP modes, and the value of the shift offset parameter can be set to 66.
  • the value of the shift offset parameter is equal to 32, but there is no limitation on this.
  • the method may further include:
  • the block size index value of the current block can be determined according to the size parameter of the current block; and then the value of the shift offset parameter can be further determined according to the block size index value of the current block.
  • the determining the value of the shift offset parameter according to the block size index value of the current block may include:
  • the shift offset parameter table can also be minimized, and a look-up table is still used to determine the offset factor.
  • the determining the value of the shift offset parameter according to the block size index value of the current block may include:
  • the value corresponding to the block size index value is queried from a fourth preset lookup table, where the fourth preset lookup table is used to record the block size index value and the shift offset.
  • the queried value is determined as the value of the shift offset parameter.
  • the value of the shift offset parameter can be queried only according to the block size index value (indicated by mipSizeId) of the current block.
  • the block size index value indicated by mipSizeId
  • the fourth preset lookup table shown in Table 7 above for each block size index value, it can correspond to a fixed value, that is, the size of each block or the size set of each block can have The value of the fixed shift offset parameter is shown in Table 7.
  • the shift amount parameter can include:
  • the first constant value is set to an integer exponential power equal to 2, wherein the exponent of the power is equal to the value of the shift amount parameter minus one.
  • the determining the first constant value according to the value of the shift amount parameter may include:
  • the first constant value can be expressed as 1 ⁇ (sW-1) or 2 ⁇ (sW-1). At this time, when the value of the shift amount parameter is set to 6, the first constant value equal to 32 can be obtained.
  • the product of the sum of the MIP input sample values and the shift offset parameter can be calculated according to the shift offset parameter (fO), which is Then the value of oW can be set as
  • the MIP weight matrix, the MIP input sample value, the shift quantity parameter, and the first offset can be obtained, so as to subsequently determine the MIP prediction value of the current block.
  • the MIP prediction value of the current block is determined according to the MIP weighting matrix, the MIP input sample value, the shift amount parameter, and the first offset, Can include:
  • the MIP prediction value of the current block is set equal to the sum of the first right shift value and the value corresponding to the index number of 0 in the first temporary reference value; wherein, the first temporary reference value is based on It is obtained by performing down-sampling filtering processing on adjacent sample values of the current block.
  • the MIP weight matrix (indicated by mWeight) and the shift amount parameter (indicated by mWeight) can be determined according to the block size index value (indicated by mipSizeId) and the MIP mode index value (indicated by modeId) of the current block in the MIP mode.
  • S904 Perform filtering processing on the MIP predicted value, and determine the intra-frame predicted value of the chrominance component of the current block.
  • the MIP prediction block is composed of MIP prediction values. After the obtained MIP prediction block, it can be determined whether the size parameter of the MIP prediction block is the same as the size parameter of the current block, so as to further determine the intra prediction value of the current block. Specifically, according to the judgment result, when the size parameter of the MIP prediction block is the same as the size parameter of the current block, the intra prediction block of the current block may be set equal to the MIP prediction block; The MIP prediction block contains the predicted values of all pixel positions in the current block; when the size parameter of the MIP prediction block is different from the size parameter of the current block, the MIP prediction block is filtered to obtain Filter the prediction block, and set the filtered prediction block as the intra prediction block of the current block.
  • the filter processing may include up-sampling filter processing or low-pass filter processing.
  • the parsing of the code stream to obtain the LFNST parameters may include:
  • the code stream is parsed to obtain the LFNST parameter.
  • the determining whether the LFNST parameter exists in the code stream may include:
  • the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determining the smallest value among the size parameters of the current block;
  • the determining whether the LFNST parameter exists in the code stream according to the minimum value may include:
  • the first preset threshold may be used to indicate a preset threshold for measuring whether to perform LFNST.
  • the first preset threshold may be set to 8, but it is not specifically limited.
  • the minimum value of the size parameter of the current block can be determined at this time; for example, when the minimum value of the width and height is greater than or equal to 8, this
  • the LFNST parameter can be obtained by parsing the code stream, such as the value of lfnst_index.
  • the determining whether the LFNST parameter exists in the code stream may include:
  • the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determining the block size index value of the current block according to the size parameter of the current block;
  • the block size index value it is determined whether the LFNST parameter exists in the code stream.
  • the determining whether the LFNST parameter exists in the code stream according to the block size index value may include:
  • the second preset threshold may also be used to indicate a preset threshold for measuring whether to perform LFNST.
  • the second preset threshold may be set to a preset value (for example, 2), or one of multiple preset values (for example, 1, 2), but is not specifically limited.
  • the block size index value (mipSizeId) of the current block can be determined according to the size parameter of the current block; for example, when the value of mipSizeId is equal to 2
  • the LFNST parameters such as the value of lfnst_index, can be obtained by parsing the code stream.
  • the parsing the code stream and obtaining the LFNST parameters may include:
  • the code stream is parsed to obtain the value of the LFNST index number.
  • the LFNST index number can be represented by lfnst_index, and the value of the LFNST index number is used to determine whether to perform LFNST on the current block.
  • the method may further include:
  • the determining the reconstructed transform coefficient block of the current block may include:
  • the inverse quantization process is the scaling process.
  • the inverse quantization of the quantized coefficients to obtain the reconstructed transform coefficient block of the current block may include: performing a scaling operation on the quantized coefficients to obtain the reconstructed transform coefficient block of the current block. Construct a block of transform coefficients.
  • LFNST when the LFNST parameter indicates to perform LFNST on the current block, at this time, LFNST can be performed on all or part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second transform coefficient block.
  • the performing LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second transform coefficient block may include: verifying at least part of the reconstructed transform coefficient block by using the LFNST transform Perform LFNST to reconstruct the transform coefficients to obtain the second transform coefficient block.
  • the four transform core candidate sets may include set0, set1, set2, and set3.
  • the embodiment of the present application may use the MIP parameter to determine the LFNST transform core candidate set, and then select the LFNST transform core used by the current block from the LFNST transform core candidate set. Therefore, in some embodiments, the method may further include:
  • the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determining the MIP parameter from the prediction mode parameter of the current block;
  • the transform core indicated by the value of the LFNST index number is determined as the LFNST transform core used by the current block; wherein the LFNST transform core candidate set includes two preset ORs Multiple LFNST transform cores.
  • the MIP parameter may include at least one of the following: a MIP mode index value (represented by modeId) and a MIP transposition indication parameter (represented by isTransposed).
  • modeId a MIP mode index value
  • MIP transposition indication parameter represented by isTransposed
  • the determining the LFNST transformation core candidate set according to the MIP parameters may include:
  • the LFNST transform core candidate set is selected from a plurality of candidate LFNST transform core candidate sets.
  • the LFNST intra prediction mode index number can be represented by predModeIntra
  • the index value of the LFNST transform core candidate set can be represented by lfnstTrSetIdx.
  • the fifth preset lookup table is shown in Table 10 above.
  • the value of predModeIntra can be determined; then according to the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in conjunction with Table 10, that is, the LFNST transform core candidate set selected for the current block is determined .
  • the value of lfnstTrSetIdx indicates the candidate set of transform cores used in LFNST; since the value of modeId can include 0, 1, 2, 3, 4, and 5, the value of predModeIntra is also 0, 1, 2 , 3, 4, 5; its corresponding relationship with lfnstTrSetIdx is shown in Table 10.
  • the determining the value of the LFNST intra prediction mode index sequence number according to the MIP mode index value may include: setting the value of the LFNST intra prediction mode index sequence number to be equal to the value of the index sequence number of the LFNST intra prediction mode. MIP mode index value.
  • the value of predModeIntra can be set equal to the value of modeId, and then the value of lfnstTrSetIdx can be directly determined according to the value of predModeIntra in conjunction with Table 10, that is, the LFNST transformation selected for the current block is determined Nuclear candidate set.
  • the MIP mode index value can be directly mapped to the value of the PLANAR mode in the embodiment of the present application, and then the value of predModeIntra corresponding to the PLANAR mode is used to determine The value of lfnstTrSetIdx determines the candidate set of LFNST transform cores selected by the current block.
  • the scanning sequence may include a horizontal scanning sequence and a vertical scanning sequence.
  • the determination of the scan order is related to the value of the MIP transposition indication parameter (indicated by isTransposed).
  • the performing LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second transform coefficient block may include:
  • the LFNST is performed on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block according to the vertical scanning order to obtain The second transform coefficient block;
  • the LFNST is performed on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block according to the horizontal scanning order to obtain The second transform coefficient block.
  • the sampling point input vector used in the MIP mode is transposed at this time, and then LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block according to the horizontal scanning order. , Specifically as shown in the above formula (10).
  • the performing LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second transform coefficient block may include:
  • the LFNST is performed on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block according to the horizontal scanning order to obtain The second transform coefficient block;
  • the LFNST is performed on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block according to the vertical scanning order to obtain The second transform coefficient block.
  • S906 Perform a first transformation on the second transform coefficient block to obtain a reconstructed residual block of the chrominance component of the current block.
  • S907 Determine a reconstruction block of the chrominance component of the current block according to the intra prediction value of the chrominance component of the current block and the reconstruction residual block.
  • the quantized value of the LFNST coefficient can be obtained by parsing the code stream, and the quantized value is inversely quantized by the inverse quantization unit (which can be called "Scaling") to obtain the reconstructed transform coefficient Block, the inverse LFNST transform is performed on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second transform coefficient block; and then through the inverse first transform, the current block can be recovered The reconstructed residual block of the chrominance component.
  • the reverse first transformation is the inverse transformation corresponding to the "first transformation (Core Transform)" in the encoder.
  • the standard only defines the "inverse transform” operation in the decoder, so the "inverse LFNST transform” can also be called “LFNST transform” in the standard.
  • the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual block can be added to determine Get the reconstructed block of the chrominance component of the current block.
  • the input code stream of the decoder 200 may be a code stream generated by the encoder 100.
  • the parsing unit 201 parses the input code stream and obtains the value of the syntax element from the input code stream.
  • the parsing unit 201 converts the binary representation of the syntax element into a digital value and sends the digital value to a unit in the decoder 200 to obtain one or more decoded pictures.
  • the parsing unit 201 can also parse one or more syntax elements from the input code stream to render the decoded picture.
  • FIG. 10 shows a schematic flow chart of parsing LFNST parameters provided by an embodiment of the present application. As shown in Figure 10, the process may include:
  • the parsing unit 201 checks to determine whether the LFNST parameter exists in the input code stream.
  • the LFNST parameter may refer to the value of lfnst_index.
  • the example of the syntax structure used to express the LFNST parameter may be the same as the example of the syntax structure shown in Table 8 above, or may be the same as the example of the syntax structure shown in Table 9A and Table 9B described above.
  • the parsing unit 201 obtains the coded bits of lfnst_index from the input code stream in the following manner.
  • the parsing unit 201 uses the MIP mode Read the lfnst_index bit.
  • Another optional condition may be set as: the width and height are greater than or equal to the first value (for example, 8), and the sum of the width and height is greater than or equal to the second value (for example, 16).
  • the parsing unit 201 uses the MIP mode to read the lfnst_index bit.
  • Another optional condition may be set as: the mipSizeId of the current block is equal to one of multiple values (for example, 1 and 2).
  • the parsing unit 201 performs adaptive binary arithmetic decoding on the LFNST parameter, and obtains the LFNST parameter.
  • the entropy encoding unit 115 uses the CABAC with the descriptor "ae(v)" as shown in Table 9A and Table 9B to encode the LFNST parameter (for example, the value of lfnst_index), then the decoding On the side of the device 200, the parsing unit 201 performs adaptive binary arithmetic decoding, and the value of lfnst_index can be obtained from the lfnst_index bit.
  • the parsing unit 201 sends the value of the syntax element and one or more variables set or determined according to the value of the syntax element and used to obtain one or more decoded pictures to a unit in the decoder 200.
  • the prediction unit 202 determines the intra prediction block of the current block (for example, CU). When the inter-frame decoding mode is indicated to be used for decoding the current decoded block, the prediction unit 202 transmits the relative parameters from the parsing unit 201 to the MC unit 203 to obtain the inter-frame prediction block. When the intra prediction mode (including the MIP mode indicated based on the MIP mode index value) is indicated for decoding the current block, the prediction unit 202 transmits the relative parameters from the parsing unit 201 to the intra prediction unit 204 to obtain intra prediction Piece.
  • the intra prediction mode including the MIP mode indicated based on the MIP mode index value
  • the current block (or referred to as a "decoding block") in the embodiment of the present application may be a CU or a partition of the CU (for example, a transform block).
  • the intra prediction unit 204 will obtain the intra prediction block of the current block, where the intra prediction block of the current block includes at least one sample of the current block. Intra prediction value.
  • the intra prediction unit 204 uses the MIP mode to obtain the intra prediction block, and the steps are as follows:
  • the intra prediction unit 204 obtains one or more reference pixels from adjacent pixels of the current block, for example, by down-sampling the adjacent pixels, or directly extracts from the adjacent pixels.
  • the intra prediction unit 204 uses the acquired reference pixels, MIP matrix, and shift parameters to determine one or more partially predicted pixels corresponding to the pixel positions in the current block.
  • the pixel position may be a preset pixel position in the current block.
  • the pixel position has uniform horizontal and vertical coordinate values.
  • the shift parameter includes a shift amount parameter and a shift offset parameter, and the shift parameter can be used for an offset operation in the process of obtaining intra-frame prediction values of some pixels.
  • the intra prediction unit 204 also needs to obtain the predicted pixels corresponding to the remaining pixels except for some pixels in the current block.
  • the intra prediction unit 204 may use an interpolation filter to obtain predicted pixels corresponding to the remaining pixels, where the input of the interpolation filter may be some pixels and adjacent pixels.
  • the block diagram example of the process of acquiring the intra prediction block using the MIP mode can also be implemented on the decoder 200, where the "current block” here refers to the "decoding block”.
  • the intra prediction unit 204 obtains adjacent pixels of the current block, for example, the adjacent pixels are marked as gray filled squares adjacent to the current block in step 601 shown in FIG. 6.
  • the intra prediction unit 204 obtains one or more reference pixels from adjacent pixels.
  • the intra prediction unit 204 may calculate the average value of two adjacent pixels and use the average value as a reference pixel.
  • the intra prediction unit 204 selects every other adjacent pixel as the reference pixel. For example, in the example in step 601 shown in FIG. 6, the intra prediction unit 204 selects 4 reference pixels from 8 adjacent pixels on the upper side of the current block, and selects 4 reference pixels from the 8 left sides of the current block. Four other reference pixels are selected from adjacent pixels.
  • step 601 The specific process of step 601 is as follows:
  • the intra prediction unit 204 obtains the width and height of the current block from the parsing unit 201, and the width and height are represented by variables cbWidth and cbHeight, respectively. According to the parameters from the parsing unit 201, the current block is divided into one or more transform blocks. Let the variables nTbW and nTbH be the width and height of the transform block, respectively.
  • the intra prediction unit 204 determines the block size index value of the MIP, that is, a variable expressed as mipSizeId.
  • the intra prediction unit 204 determines the value of mipSizeId as follows:
  • mipSizeId is set equal to 0;
  • mipSizeId is set equal to 1;
  • mipSizeId is set equal to 2.
  • mipSizeId is set equal to 2.
  • the intra prediction unit 204 determines the value of mipSizeId as follows:
  • mipSizeId is set equal to 0;
  • mipSizeId is set equal to 1;
  • mipSizeId is set equal to 2.
  • mipSizeId is set equal to 1.
  • the intra prediction unit 204 obtains the values of boundarySize and predSize according to mipSizeId, which are specifically as follows:
  • boundarySize is set to be equal to 2
  • predSize is set to be equal to 4
  • mipSizeId is equal to 1
  • boundarySize is set to be equal to 4
  • predSize is set to be equal to 4
  • boundarySize is set to be equal to 4
  • predSize is set to be equal to 8
  • boundarySize represents the number of reference pixels obtained from each side of the reference pixels adjacent to the upper side of the current block and the reference pixels adjacent to the left side of the current block.
  • the intra-prediction unit 204 can also use the parameter indicating the order of the reference pixels stored in the buffer pTemp from the parsing unit 701 to obtain the reference pixels used to calculate the MIP prediction value.
  • the intra prediction unit 204 sets the variable isTransposed to be equal to the parameter from the parsing unit 701 indicating the order of the reference pixels stored in the buffer pTemp.
  • isTransposed equal to 0 means that: the intra prediction unit 204 first presents the reference pixels obtained from the adjacent reference pixels on the upper side of the current block, and then presents the reference pixels obtained from the adjacent reference pixels on the left side; Otherwise, isTransposed equal to 1 (or TRUE) means that the intra prediction unit 204 first presents the reference pixels obtained from the reference pixels adjacent to the left side of the current block, and then presents the reference pixels obtained from the reference pixels adjacent to the upper side.
  • the intra prediction unit 204 obtains the value of the variable inSize to indicate the number of reference pixels used when the MIP mode is used, as shown in the above formula (1).
  • the intra prediction unit 204 calls the following process to obtain a group of reference pixels (stored as an array p[x], where x is 0,..., inSize-1) using reference pixels adjacent to the current block.
  • the intra prediction unit 204 obtains nTbW reference pixels from the reference pixels adjacent to the upper side of the current block (for example, stored in the array refT), and obtains nTbW reference pixels from the reference pixels adjacent to the left side of the current block (for example, stored in the array refL). ) Obtain nTbH reference pixels.
  • the intra prediction unit 204 calls down-sampling processing on refT to obtain boundarySize reference pixels, and stores the boundarySize reference pixels in refT.
  • the intra prediction unit 204 calls down-sampling processing on refL to obtain boundarySize reference pixels, and stores the boundarySize reference pixels in refL.
  • the intra prediction unit 204 may also obtain the order indicated by isTransposed to put the elements in refT and refL into the buffer pTemp.
  • BitDepth is the bit depth of the color component of the pixel in the current block.
  • the color component may be one of the RGB components, or one of the YUV components, or one of the YCbCr components, for example, the Y component.
  • the intra prediction unit 204 may use a unified calculation method to obtain the value of p[x] without determining the value of mipSizeId. For example, adding (1 ⁇ (BitDepth-1)) as an additional element to the buffer pTemp, at this time, the intra prediction unit 204 calculates p[x] as pTemp[x+1]-pTemp[0].
  • Step 602 The intra prediction unit 204 uses a set of reference pixels and MIP weighting matrix to obtain the MIP prediction value of the current block.
  • the MIP weighting matrix is selected from a set of predefined MIP weighting matrices according to the MIP mode index value (denoted as ModeId) corresponding to the MIP mode and the block size index value of the MIP (denoted as mipSizeId).
  • the intra prediction unit 204 obtains the value of ModeId from the parsing unit 201.
  • the intra-frame prediction unit 204 obtains MIP prediction values (denoted as predMip[x][y]) of a part of predicted pixels corresponding to one or more pixel positions in the current block.
  • part of the predicted pixels are pixels marked as gray filled squares in the current block.
  • the input of the prediction module 601 is the reference pixel p[x] obtained in step 601.
  • the prediction module 601 uses the MIP weighting matrix and the shift parameter to calculate part of the predicted pixels; here, the shift parameter includes a shift quantity parameter and a shift offset parameter.
  • the prediction module 601 can use coordinates (x, y) to represent the pixel, and the predicted pixel is represented by predMip[x][y], then the calculation formula of predMip[x][y] is as follows: (5) or formula (6).
  • mWeight[i][j] is the MIP weighting matrix; here, the matrix element can be a predetermined constant value; or, it can also be updated adaptively using a training method such as
  • the input is one or more coded pictures or blocks, or pictures from other code streams provided by an external device to the decoder 100, or from the parsing unit 201 by parsing special data units in the input code stream containing the MIP weighting matrix
  • the obtained picture; mWeight[i][j] can be determined according to the MIP mode indicated by the corresponding one or more parameters from the parsing unit 201; fO is the shift offset parameter used to determine oW; sW is the shift Quantity parameter; p[i] is calculated using reference pixels, that is, the MIP input sampling value in the embodiment of this application; pTemp[0] represents the first pixel in the reference pixel, that is, the first temporary reference in the embodiment of this application The index number in the value is the value corresponding to 0; "
  • the prediction module 601 may determine the values of sW and f0 according to the size of the current block and the MIP mode used for the current block. In one example, the prediction module 601 uses a look-up table to obtain the values of sW and fO.
  • the prediction module 601 can use the aforementioned Table 4 to determine sW according to the size parameter of the current block and the MIP mode.
  • the prediction module 601 can also use the aforementioned Table 5 to determine sW according to the size parameter of the current block.
  • the prediction module 601 can also directly set sW as a constant value. For example, for blocks of various size parameters and different MIP modes, the prediction module 601 may set sW to 5; or, for blocks of various size parameters and different MIP modes, the prediction module 601 may set sW to 6; or For blocks of various size parameters and different MIP modes, the prediction module 601 sets sW to 7.
  • the prediction module 601 may use the aforementioned Table 6 to determine f0 according to the size parameter of the current block and the MIP mode.
  • the prediction module 601 can also use the aforementioned Table 7 to determine f0 according to the size parameter of the current block.
  • the prediction module 601 can also directly set f0 as a constant value (such as 0-100). For example, for blocks of various size parameters and different MIP modes, the prediction module 601 sets fO to 32; or, the prediction module 601 sets fO to 46; or, the prediction module 601 sets fO to 56; or, the prediction module 601 sets fO to 66.
  • f0 a constant value (such as 0-100). For example, for blocks of various size parameters and different MIP modes, the prediction module 601 sets fO to 32; or, the prediction module 601 sets fO to 46; or, the prediction module 601 sets fO to 56; or, the prediction module 601 sets fO to 66.
  • the intra prediction unit 204 may perform a clamp processing operation on the MIP predicted value in predMip.
  • isTransposed is equal to 1 (or TRUE)
  • the intra prediction unit 204 determines that mipSizeId is equal to 2 and obtains an 8 ⁇ 8 predMip.
  • the intra prediction unit 204 determines that mipSizeId is equal to 1 and obtains a 4 ⁇ 4 predMip.
  • the intra prediction unit 204 invokes an upsampling process to obtain predSamples using predMip.
  • the intra prediction unit 204 obtains predicted pixel points corresponding to the remaining pixels in the current block except for some pixels.
  • the intra prediction unit 204 may use the filtering module 602 to obtain predicted pixels corresponding to the remaining pixels except for some pixels in the current block.
  • the input of the filtering module 602 is the pixels marked as gray filled squares in step 602.
  • the filtering module 602 may use one or more interpolation filters, and use the input to obtain predicted pixels corresponding to the remaining pixels in the current block except for some pixels.
  • the input may include a reference pixel and a partial prediction of one or more pixel positions in the current block; or, the input may include a partial prediction of neighboring pixels and one or more pixel positions in the current block Pixels; or, the input may include reference pixels, neighboring pixels, and partial predicted pixels at one or more pixel positions in the current block.
  • the intra prediction unit 204 determines that mipSizeId is equal to 1, and the intra prediction unit 204 applies the upsampling process to 4 ⁇ 4predMip , To obtain the 8 ⁇ 8 intra prediction block of the current block.
  • the intra prediction unit 204 can obtain the intra prediction block of the current block (ie, CU), that is, determine the intra prediction value of at least one pixel in the current block.
  • the scaling unit 205 has the same function as that of the inverse quantization unit 110 in the encoder 100.
  • the scaling unit 205 performs a scaling operation on the quantized coefficients (ie, levels) from the analysis unit 201 to obtain reconstruction coefficients.
  • the transform unit 206 has the same function as that of the inverse transform unit 111 in the encoder 100.
  • the transform unit 206 performs one or more transform operations (that is, the inverse operation of the one or more transform operations performed by the inverse transform unit 111 in the encoder 100) to obtain a reconstruction residual.
  • the transformation unit 206 When lfnst_index is not equal to 0, the transformation unit 206 performs secondary transformation on the reconstruction coefficient to obtain a secondary transformation coefficient block, and then performs a first transformation on the secondary transformation coefficient block, for example, an integer transformation originally designed based on DCT to Obtain the reconstruction residual block.
  • LFNST is an example of secondary transformation.
  • FIG. 11 shows a schematic diagram of another process framework for executing LFNST provided by an embodiment of the present application.
  • a block 1101 is a reconstructed transform coefficient block including reconstructed transform coefficients.
  • step 1101 the transform unit 206 performs a secondary transform LFNST on all or part of the reconstructed transform coefficients in the block 1101.
  • the transform unit 206 selects the LFNST transform core candidate set according to the MIP parameters.
  • the MIP parameter may include at least one of the following: a MIP mode index value (that is, modeId) and a MIP transposition indication parameter (that is, isTransposed).
  • the transformation unit 206 sets the value of the variable predModeIntra to be equal to the value of modeId.
  • the transform unit 206 determines a transform kernel used to perform LFNST on the coefficients on the sub-blocks 11001, 11002, and 11003.
  • the size of the sub-block may be a preset value, such as 8 ⁇ 8.
  • the transformation unit 206 determines the index value of the LFNST transformation core candidate set (that is, lfnstTrSetIdx) according to the value of predModeIntra. For block decoding in the MIP mode, the transform unit 206 uses a look-up table method as shown in Table 10 to obtain the value of lfnstTrSetIdx.
  • the transformation unit 206 determines the transformation core for performing LFNST as the transformation core indicated by lfnst_index in the LFNST transformation core candidate set indicated by lfnstTrSetIdx.
  • the transform unit 206 When lfnst_index is not equal to 0, the transform unit 206 performs LFNST on the transform block using the coefficients from the sub-blocks 11001, 11002, and 11003, obtains the LFNST coefficients, and puts the LFNST coefficients into the sub-blocks 11011, 11012, and 11013 of the block 1102.
  • the transform unit 206 uses the following instructions to obtain the block 1102 of transform coefficients, specifically as shown in the above equation (8).
  • log2LfnstSize is as shown in the above formula (9).
  • the transform unit 206 uses the following instructions to obtain the block 1102 of transform coefficients, which is specifically shown in the above formula (10);
  • the transform unit 206 uses the following instructions to obtain the block 1102 of transform coefficients, which is specifically shown in the above-mentioned equation (11).
  • the transform unit 206 uses the following instruction to obtain the block 1102 of transform coefficients, which is specifically shown in the above formula (10);
  • the transform unit 706 uses the following instructions to obtain the block 902 of transform coefficients, specifically as shown in the above equation (11).
  • the transform unit 206 uses the look-up table shown in Table 11, according to the traditional frame
  • the mode index value of the intra prediction mode (assigned to predModeIntra) determines the value of lfnstTrSetIdx, that is, determines the LFNST transform core candidate set selected for the current block.
  • the transformation unit 206 determines the transformation core for performing LFNST as the transformation core indicated by lfnst_index in the LFNST transformation core candidate set indicated by lfnstTrSetIdx.
  • the transform unit 206 When lfnst_index is not equal to 0, the transform unit 206 performs LFNST on the transform block using the coefficients from the sub-blocks 11001, 11002, and 11003, obtains the LFNST coefficients, and puts the LFNST coefficients into the sub-blocks 11011, 11012, and 11013 of the block 1102.
  • the transform unit 206 uses the following instructions to obtain the block 1102 of transform coefficients, as specifically shown in the above formula (10);
  • the transform unit 206 uses the following instructions to obtain the block 1102 of transform coefficients, which is specifically shown in the above-mentioned equation (11).
  • the transform unit 206 uses the following instructions to obtain the block 1102 of transform coefficients, specifically as shown in the above formula (10);
  • the transform unit 706 uses the following instructions to obtain the block 902 of transform coefficients, specifically as shown in the above formula (11).
  • the transform unit 206 performs a first transform (ie, core transform) on the transform coefficients in the block 1102, for example, an integer transform originally designed based on DCT to obtain a reconstructed residual block 1103 of the current block.
  • a first transform ie, core transform
  • the adder 207 performs an addition operation on its inputs (the intra prediction block from the prediction unit 202 and the reconstructed residual block from the transform unit 206) to obtain a reconstructed block of the current block.
  • the reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks encoded in the intra prediction mode.
  • the filtering unit 208 After all CUs in the picture or sub-picture are reconstructed, the filtering unit 208 performs loop filtering on the reconstructed picture or sub-picture.
  • the filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive offset filter, an adaptive loop filter, a luminance mapping and chrominance scaling filter, and a neural network-based filter.
  • filters such as a deblocking filter, a sample adaptive offset filter, an adaptive loop filter, a luminance mapping and chrominance scaling filter, and a neural network-based filter.
  • the filtering unit 208 determines that the reconstructed block is not used as a reference for decoding other blocks, the filtering unit 208 performs loop filtering on one or more target pixels in the reconstructed block.
  • the output of the filtering unit 208 is a decoded picture or sub-picture, and the decoded picture or sub-picture is sent to the DPB unit 209.
  • the DPB unit 209 outputs decoded pictures according to timing and control information.
  • the picture stored in the DPB unit 209 can also be used as a reference for performing inter prediction or intra prediction by the prediction unit 202.
  • the decoder 200 may be a computing device having a processor and a storage medium recording a decoding program. When the processor reads and runs the decoding program, the decoder 200 reads the input code stream and generates a corresponding decoded video.
  • the decoder 200 may be a computing device having one or more chips. A unit implemented as an integrated circuit on a chip has similar functions through similar connections and data exchanges to the corresponding unit in FIG. 2.
  • This embodiment provides an encoding method that obtains the prediction parameters and LFNST parameters of the current block by parsing the code stream, where the prediction parameters include prediction mode parameters; when the prediction mode parameters indicate the color of the current block When the degree component uses the matrix-based intra prediction MIP mode to determine the intra prediction value, the adjacent sample value of the current block is obtained, and the MIP input sample value of the current block is determined according to the adjacent sample value of the current block ; Determine the MIP predicted value of the chrominance component of the current block according to the MIP input sample value, the MIP weight matrix and the shift parameter; filter the MIP predicted value to determine the chrominance component of the current block When the LFNST parameter indicates to perform LFNST on the current block, determine the reconstructed transform coefficient block of the current block, and perform LFNST on at least part of the reconstructed transform coefficient in the reconstructed transform coefficient block , Obtain the second transform coefficient block; perform the first transform on the second transform coefficient block to obtain the reconstructed residual block of the chrominance component of the
  • the complexity can be reduced on the basis of ensuring the decoding performance, while the storage space required in the decoding process is reduced, and the decoding efficiency is effectively improved.
  • the LFNST technology is applied to MIP mode prediction, due to The introduction of MIP parameters makes the LFNST transform more flexible and further improves the decoding efficiency.
  • FIG. 12 shows a schematic diagram of the composition structure of an encoder 120 provided in an embodiment of the present application.
  • the encoder 120 may include: a first determination unit 1201, a first prediction unit 1202, a first transformation unit 1203, and an encoding unit 1204; wherein,
  • the first determining unit 1201 is configured to determine a prediction parameter of the current block, where the prediction parameter includes a prediction mode parameter;
  • the first determining unit 1201 is further configured to obtain adjacent samples of the current block when the prediction mode parameter indicates that the matrix-based intra prediction MIP mode is used to determine the intra prediction value for the chrominance component of the current block Value, and determine the MIP input sample value of the current block according to the adjacent sample value of the current block;
  • the first prediction unit 1202 is configured to determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weight matrix, and the shift parameter; wherein the MIP prediction value is the current block The predicted value of some sampling points in the chrominance component; and filtering the MIP predicted value to determine the intra-frame predicted value of the chrominance component of the current block;
  • the first transform unit 1203 is configured to determine the prediction residual value of the chrominance component of the current block according to the intra prediction value of the chrominance component of the current block; and perform low-frequency inseparability on the prediction residual value Transform LFNST twice to determine LFNST parameters;
  • the encoding unit 1204 is configured to encode the LFNST parameters and write them into the code stream.
  • the adjacent sample value of the current block includes the left adjacent sample value and the upper adjacent sample value of the current block.
  • the shift parameter includes a shift offset parameter and a shift amount parameter
  • the first determining unit 1201 is specifically configured to determine the product of the sum of the MIP input sample values and the shift offset parameter according to the value of the shift offset parameter, wherein the value of the shift offset parameter The value is a fixed constant; and the first constant value is determined according to the value of the shift amount parameter, wherein the value of the shift amount parameter is a fixed constant; and the value of the first offset is set to be all The difference between the first constant value and the product; and determining the MIP weight matrix of the current block according to the prediction parameter; and according to the MIP weight matrix, the MIP input sample value, and the shift amount parameter And the first offset to determine the MIP prediction value of the chrominance component of the current block.
  • the prediction parameter further includes: a size parameter of the current block.
  • the first determining unit 1201 is further configured to determine the block size index value of the current block according to the size parameter of the current block; and perform down-sampling filtering on adjacent sample values of the current block Processing to obtain a first temporary reference value; and when the block size index value of the current block is within a preset range, a second constant value is determined according to the bit depth of adjacent sample values of the current block; The value corresponding to the index number of 0 in the MIP input sample value is set equal to the difference between the second constant value and the value corresponding to the index number of 0 in the first temporary reference value; and the MIP input sample The value corresponding to the index number i in the value is set to be equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, where , I is an integer greater than 0; when the block size index value of the current block is outside the preset range, the value corresponding to the index number equal to
  • the first determining unit 1201 is further configured to, when the block size index value of the current block is within a preset range, determine the second block according to the bit depth of adjacent sample values of the current block. Constant value; set the value corresponding to index number 0 in the MIP input sample value to be equal to the difference between the value corresponding to index number 0 in the first temporary reference value and the second constant value; The value corresponding to the index number i in the MIP input sample value is set to be equal to the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value The difference between, where i is an integer greater than 0.
  • the first determining unit 1201 is further configured to set the block size index value of the current block to 0 when the width and height of the current block are both equal to 4.
  • the block size index value of the current block is set to 1; when the width and height of the current block do not meet the foregoing conditions, The block size index value of the current block is set to 2.
  • the first determining unit 1201 is further configured to perform down-sampling and filtering processing on adjacent sample values of the current block to obtain a first temporary reference value; and a calculation based on the adjacent sample values of the current block Bit depth, determining a second constant value, buffering the second constant value in a data unit after the first temporary reference value to obtain a second temporary reference value; and inputting the MIP to the index number in the sample value
  • the value corresponding to j is set to be equal to the difference between the value corresponding to the index number equal to j+1 in the second temporary reference value and the value corresponding to the index number equal to 0 in the second temporary reference value, where j Is an integer greater than or equal to 0.
  • the first determining unit 1201 is further configured to set the second constant value to an integer exponential power equal to 2, wherein the exponent of the power is equal to the bit of the adjacent sample value of the current block. Decrease the depth by 1.
  • the first determining unit 1201 is further configured to perform a binary bit left shift of "1" to obtain the second constant value, wherein the number of bits shifted to the left is equal to the phase of the current block.
  • the bit depth of the adjacent sample value is reduced by one.
  • the value of the shift amount parameter is a fixed constant independent of the block size index value and the MIP mode index value.
  • the value of the shift amount parameter is set to be equal to 6.
  • the value of the shift offset parameter is a fixed constant independent of the block size index value and the MIP mode index value.
  • the value of the shift offset parameter is set to be equal to 32.
  • the first determining unit 1201 is further configured to set the first constant value to an integer exponent power equal to 2, wherein the exponent of the power is equal to the value of the shift amount parameter minus 1. .
  • the first determining unit 1201 is further configured to perform a binary bit shift to the left of "1" to obtain the first constant value, wherein the number of bits shifted to the left is equal to the shift quantity parameter The value of minus 1.
  • the encoder 120 may further include a first calculation unit 1205 configured to calculate a first weighted sum of the MIP weight matrix and the MIP input sample value; and calculate the first weight And the first sum value of the sum and the first offset; and performing a binary bit right shift on the first sum value to obtain a first right shift value, wherein the number of bits of the right shift is equal to the shift The value of the quantity parameter;
  • the first determining unit 1201 is further configured to set the MIP prediction value of the chrominance component of the current block to be equal to the sum of the first right shift value and the value corresponding to the index number of 0 in the first temporary reference value; Wherein, the first temporary reference value is obtained based on performing down-sampling filtering processing on adjacent sample values of the current block.
  • the first transform unit 1203 is configured to determine a residual block of the current block according to the prediction residual value, and perform a first transform on the residual block to obtain a first transform coefficient block;
  • the first determining unit 1201 is further configured to determine whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block;
  • the first transform unit 1203 is further configured to perform LFNST on at least part of the transform coefficients in the first transform coefficient block to determine the LFNST parameter when the determination result is yes.
  • the first determining unit 1201 is further configured to determine the smallest value among the size parameters of the current block when the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value; and if so If the minimum value is greater than or equal to the first preset threshold, it is determined to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.
  • the LFNST parameter includes an LFNST index number.
  • the first determining unit 1201 is further configured to set the LFNST index number if it is determined to perform LFNST on at least part of the transform coefficients in the first transform coefficient block. The value is greater than zero; if it is determined not to perform LFNST on at least part of the transform coefficients in the first transform coefficient block, set the value of the LFNST index number to be equal to zero;
  • the encoding unit 1204 is further configured to encode the LFNST index sequence number and write it into the code stream.
  • the LFNST parameters further include LFNST coefficients.
  • the encoder 120 may further include a quantization unit 1206; wherein,
  • the first transform unit 1203 is configured to perform LFNST on at least some transform coefficients in the first transform coefficient block to obtain the LFNST coefficients;
  • the quantization unit 1206 is configured to quantize the LFNST coefficients to obtain quantized coefficients
  • the encoding unit 1204 is further configured to encode the quantized coefficients and write them into the code stream.
  • a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, may also be a module, or may be non-modular.
  • the various components in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be realized in the form of hardware or software function module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this embodiment is essentially or It is said that the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can It is a personal computer, a server, or a network device, etc.) or a processor (processor) that executes all or part of the steps of the method described in this embodiment.
  • the aforementioned storage media include: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.
  • an embodiment of the present application provides a computer storage medium applied to the encoder 120, and the computer storage medium stores a computer program that, when executed by a first processor, implements any one of the foregoing embodiments. Methods.
  • FIG. 13 shows a schematic diagram of a specific hardware structure of the encoder 120 provided in an embodiment of the present application.
  • the first bus system 1304 may include: a first communication interface 1301, a first memory 1302, and a first processor 1303; various components are coupled together through a first bus system 1304.
  • the first bus system 1304 is used to implement connection and communication between these components.
  • the first bus system 1304 also includes a power bus, a control bus, and a status signal bus.
  • various buses are marked as the first bus system 1304 in FIG. 13. among them,
  • the first communication interface 1301 is used for receiving and sending signals in the process of sending and receiving information with other external network elements;
  • the first memory 1302 is configured to store a computer program that can run on the first processor 1303;
  • the first processor 1303 is configured to execute: when the computer program is running:
  • the prediction mode parameter indicates that the matrix-based intra prediction MIP mode is used for the chrominance component of the current block to determine the intra prediction value
  • the adjacent sample value of the current block is obtained, and the value of the current block is determined according to the Adjacent sample values determine the MIP input sample value of the current block
  • the LFNST parameters are encoded and written into the code stream.
  • the first memory 1302 in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory.
  • the non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), and electrically available Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory.
  • the volatile memory may be a random access memory (Random Access Memory, RAM), which is used as an external cache.
  • RAM static random access memory
  • DRAM dynamic random access memory
  • DRAM synchronous dynamic random access memory
  • DDRSDRAM Double Data Rate Synchronous Dynamic Random Access Memory
  • Enhanced SDRAM, ESDRAM Synchronous Link Dynamic Random Access Memory
  • Synchlink DRAM Synchronous Link Dynamic Random Access Memory
  • DRRAM Direct Rambus RAM
  • the first processor 1303 may be an integrated circuit chip with signal processing capability. In the implementation process, the steps of the foregoing method can be completed by an integrated logic circuit of hardware in the first processor 1303 or instructions in the form of software.
  • the aforementioned first processor 1303 may be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (ASIC), a ready-made programmable gate array (Field Programmable Gate Array, FPGA) Or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • DSP Digital Signal Processor
  • ASIC application specific integrated circuit
  • FPGA ready-made programmable gate array
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the first memory 1302, and the first processor 1303 reads the information in the first memory 1302, and completes the steps of the foregoing method in combination with its hardware.
  • the embodiments described in this application can be implemented by hardware, software, firmware, middleware, microcode, or a combination thereof.
  • the processing unit can be implemented in one or more application specific integrated circuits (ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable Logic device (Programmable Logic Device, PLD), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, and others for performing the functions described in this application Electronic unit or its combination.
  • ASIC application specific integrated circuits
  • DSP Digital Signal Processing
  • DSP Device digital signal processing equipment
  • PLD programmable Logic Device
  • PLD Field-Programmable Gate Array
  • FPGA Field-Programmable Gate Array
  • the technology described in this application can be implemented through modules (for example, procedures, functions, etc.) that perform the functions described in this application.
  • the software codes can be stored in the memory and executed by
  • the first processor 1303 is further configured to execute the method described in any one of the foregoing embodiments when the computer program is running.
  • This embodiment provides an encoder, which may include a first determination unit, a first prediction unit, a first transformation unit, and a coding unit.
  • the complexity can be reduced on the basis of ensuring the coding performance, while the storage space required in the coding process is reduced, and the coding efficiency is effectively improved.
  • the LFNST technology is applied to MIP mode prediction, due to The introduction of MIP parameters makes the LFNST transformation more flexible and further improves the coding efficiency.
  • FIG. 14 shows a schematic diagram of the composition structure of a decoder 140 provided by an embodiment of the present application.
  • the decoder 140 may include: an analysis unit 1401, a second determination unit 1402, a second prediction unit 1403, and a second transformation unit 1404; wherein,
  • the parsing unit 1401 is configured to parse the code stream to obtain prediction parameters and LFNST parameters of the current block, where the prediction parameters include prediction mode parameters;
  • the second determining unit 1402 is configured to obtain adjacent sample values of the current block when the prediction mode parameter indicates that the matrix-based intra prediction MIP mode is used to determine the intra prediction value for the chrominance component of the current block , And determine the MIP input sample value of the current block according to the adjacent sample value of the current block;
  • the second prediction unit 1403 is configured to determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weight matrix, and the shift parameter; wherein the MIP prediction value is the current block The predicted value of some sampling points in the chrominance component; and in order to filter the MIP predicted value, determine the intra-frame predicted value of the chrominance component of the current block;
  • the second transform unit 1404 is configured to determine a reconstructed transform coefficient block of the current block when the LFNST parameter indicates to perform LFNST on the current block, and reconstruct transform coefficients for at least part of the reconstructed transform coefficient block Performing LFNST to obtain a second transform coefficient block; and performing a first transform on the second transform coefficient block to obtain a reconstructed residual block of the chrominance component of the current block;
  • the second determining unit 1402 is further configured to determine the reconstruction block of the chroma component of the current block according to the intra prediction value of the chroma component of the current block and the reconstruction residual block.
  • the adjacent sample value of the current block includes the left adjacent sample value and the upper adjacent sample value of the current block.
  • the shift parameter includes a shift offset parameter and a shift amount parameter
  • the second determining unit 1402 is specifically configured to determine the product of the sum of the MIP input sample values and the shift offset parameter according to the value of the shift offset parameter, where the value of the shift offset parameter The value is a fixed constant; and the first constant value is determined according to the value of the shift amount parameter, wherein the value of the shift amount parameter is a fixed constant; and the value of the first offset is set to be all The difference between the first constant value and the product; and determining the MIP weight matrix of the current block according to the prediction parameter; and according to the MIP weight matrix, the MIP input sample value, and the shift amount parameter And the first offset to determine the MIP prediction value of the chrominance component of the current block.
  • the prediction parameter further includes: a size parameter of the current block.
  • the second determining unit 1402 is further configured to determine the block size index value of the current block according to the size parameter of the current block; and perform down-sampling filtering on adjacent sample values of the current block Processing to obtain a first temporary reference value; and when the block size index value of the current block is within a preset range, a second constant value is determined according to the bit depth of adjacent sample values of the current block; The value corresponding to the index number of 0 in the MIP input sample value is set equal to the difference between the second constant value and the value corresponding to the index number of 0 in the first temporary reference value; and the MIP input sample The value corresponding to the index number i in the value is set to be equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, where , I is an integer greater than 0; when the block size index value of the current block is outside the preset range, the value corresponding to the index number equal to
  • the second determining unit 1402 is further configured to, when the block size index value of the current block is within a preset range, determine the second Constant value; set the value corresponding to index number 0 in the MIP input sample value to be equal to the difference between the value corresponding to index number 0 in the first temporary reference value and the second constant value; The value corresponding to the index number i in the MIP input sample value is set to be equal to the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value The difference between, where i is an integer greater than 0.
  • the second determining unit 1402 is further configured to set the block size index value of the current block to 0 when the width and height of the current block are both equal to 4.
  • the block size index value of the current block is set to 1; when the width and height of the current block do not meet the foregoing conditions, The block size index value of the current block is set to 2.
  • the second determining unit 1402 is further configured to perform down-sampling filtering processing on the adjacent sample values of the current block to obtain the first temporary reference value; and the calculation according to the adjacent sample values of the current block Bit depth, determining a second constant value, buffering the second constant value in a data unit after the first temporary reference value to obtain a second temporary reference value; and inputting the MIP to the index number in the sample value
  • the value corresponding to j is set to be equal to the difference between the value corresponding to the index number equal to j+1 in the second temporary reference value and the value corresponding to the index number equal to 0 in the second temporary reference value, where j Is an integer greater than or equal to 0.
  • the second determining unit 1402 is further configured to set the second constant value to an integer exponential power equal to 2, wherein the exponent of the power is equal to the bit of the adjacent sample value of the current block. Decrease the depth by 1.
  • the second determining unit 1402 is further configured to perform a binary bit left shift of "1" to obtain the second constant value, wherein the number of bits shifted to the left of the bit is equal to the phase of the current block.
  • the bit depth of the adjacent sample value is reduced by one.
  • the value of the shift amount parameter is a fixed constant independent of the block size index value and the MIP mode index value.
  • the value of the shift amount parameter is set to be equal to 6.
  • the value of the shift offset parameter is a fixed constant independent of the block size index value and the MIP mode index value.
  • the value of the shift offset parameter is set to be equal to 32.
  • the second determining unit 1402 is further configured to set the first constant value to an integer exponential power equal to 2, wherein the exponent of the power is equal to the value of the shift amount parameter minus 1. .
  • the second determining unit 1402 is further configured to perform a binary bit shift to the left of "1" to obtain the first constant value, wherein the number of bits shifted to the left is equal to the shift quantity parameter The value of minus 1.
  • the decoder 140 may further include a second calculation unit 1405 configured to calculate a first weighted sum of the MIP weight matrix and the MIP input sample value; and calculate the first weight And the first sum value of the sum and the first offset; and performing a binary bit right shift on the first sum value to obtain a first right shift value, wherein the number of bits of the right shift is equal to the shift The value of the quantity parameter;
  • the second determining unit 1402 is further configured to set the MIP prediction value of the chrominance component of the current block to be equal to the sum of the first right shift value and the value corresponding to the index number of 0 in the first temporary reference value; Wherein, the first temporary reference value is obtained based on performing down-sampling filtering processing on adjacent sample values of the current block.
  • the second determining unit 1402 is further configured to determine whether there are LFNST parameters in the code stream;
  • the parsing unit 1401 is further configured to parse the code stream to obtain the LFNST parameter when it is determined that the LFNST parameter exists in the code stream.
  • the second determining unit 1402 is further configured to determine the smallest value among the size parameters of the current block when the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value; and if so If the minimum value is greater than or equal to the first preset threshold, it is determined that the LFNST parameter exists in the code stream.
  • the decoder 140 may further include an inverse quantization unit 1406;
  • the parsing unit 1401 is further configured to parse the code stream to obtain the quantization coefficient of the current block when the LFNST parameter indicates to perform LFNST on the current block;
  • the inverse quantization unit 1406 is configured to perform inverse quantization on the quantized coefficient to obtain a reconstructed transform coefficient block of the current block.
  • the inverse quantization unit 1406 is specifically configured to perform a scaling operation on the quantized coefficient to obtain the reconstructed transform coefficient block of the current block.
  • the parsing unit 1401 is specifically configured to parse the code stream to obtain the value of the LFNST index number; and if the value of the LFNST index number is greater than zero, determine to perform LFNST on the current block ; If the value of the LFNST index sequence number is equal to zero, it is determined not to perform LFNST on the current block.
  • a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, it may also be a module, or it may be non-modular.
  • the various components in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be realized in the form of hardware or software function module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium.
  • this embodiment provides a computer storage medium applied to the decoder 140.
  • the computer storage medium stores a computer program that, when executed by the second processor, implements any of the foregoing embodiments. The method described.
  • FIG. 15 shows a schematic diagram of a specific hardware structure of the decoder 140 provided in an embodiment of the present application.
  • the decoder 140 may include: a second communication interface 1501, a second memory 1502, and a second processor 1503; various components are coupled together through a second bus system 1504.
  • the second bus system 1504 is used to implement connection and communication between these components.
  • the second bus system 1504 also includes a power bus, a control bus, and a status signal bus.
  • various buses are marked as the second bus system 1504 in FIG. 15. among them,
  • the second communication interface 1501 is used for receiving and sending signals in the process of sending and receiving information with other external network elements;
  • the second memory 1502 is configured to store a computer program that can run on the second processor 1503;
  • the second processor 1503 is configured to execute: when the computer program is running:
  • Parse the code stream to obtain the prediction parameters and LFNST parameters of the current block, where the prediction parameters include prediction mode parameters;
  • the prediction mode parameter indicates that the matrix-based intra prediction MIP mode is used for the chrominance component of the current block to determine the intra prediction value
  • the adjacent sample value of the current block is obtained, and the value of the current block is determined according to the Adjacent sample values determine the MIP input sample value of the current block
  • the LFNST parameter indicates to perform LFNST on the current block, determine the reconstructed transform coefficient block of the current block, and perform LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second transform coefficient Piece;
  • a reconstruction block of the chrominance component of the current block is determined.
  • the second processor 1503 is further configured to execute the method described in any one of the foregoing embodiments when the computer program is running.
  • the hardware function of the second memory 1502 is similar to that of the first memory 1302, and the hardware function of the second processor 1503 is similar to that of the first processor 1303; it will not be described in detail here.
  • This embodiment provides a decoder, which may include an analysis unit, a second determination unit, a second prediction unit, and a second transformation unit.
  • a decoder which may include an analysis unit, a second determination unit, a second prediction unit, and a second transformation unit.
  • FIG. 16 shows a schematic diagram of the composition structure of a sending device provided in an embodiment of the present application.
  • the acquisition unit 1601 acquires the video signal and sends the video signal to the encoder 1602.
  • the acquisition unit 1601 may be a device including one or more cameras (including a depth camera).
  • the acquisition unit 1601 may be a device that partially or completely decodes the code stream to obtain a video.
  • the acquisition unit 1601 may also include one or more elements to capture audio signals.
  • a specific implementation of the encoder 1602 is the encoder 100 or the encoder 120 described in the foregoing embodiment, which encodes the video signal as its input video from the acquisition unit 1601 and generates a video code stream.
  • the encoder 1602 may also include one or more audio encoders to encode audio signals to generate audio code streams.
  • the storage/transmission unit 1603 receives the video code stream from the encoder 1602.
  • the storage/transmission unit 1603 may also receive an audio code stream from the encoder 1602 and compress the video code stream and the audio code stream together to form a media file (for example, an ISO-based media file format) or a transport stream.
  • the storage/transmission unit 1603 writes media files or transport streams into a storage unit, such as a hard disk, a DVD disk, a cloud, or a portable storage device.
  • a storage unit such as a hard disk, a DVD disk, a cloud, or a portable storage device.
  • the storage/transmission unit 1603 transmits the bit stream to a transmission network, such as the Internet, a wired network, a cellular network, a wireless local area network, and so on.
  • a transmission network such as the Internet, a wired network, a cellular network, a wireless local area network, and so on.
  • FIG. 17 shows a schematic diagram of the composition structure of a target device provided in an embodiment of the present application.
  • the receiving unit 1701 receives a media file or transport stream from the network, or reads a media file or transport stream from a storage device.
  • the receiving unit 1701 separates the video code stream and the audio code stream from the media file or transport stream.
  • the receiving unit 1701 may also generate a new video code stream by extracting the video code stream.
  • the receiving unit 1701 can also generate a new audio code stream by extracting the audio code stream.
  • the decoder 1702 includes one or more video decoders, for example, a specific implementation is the decoder 200 or the decoder 140 described in the foregoing embodiment.
  • the decoder 1702 may also include one or more audio decoders.
  • the decoder 1702 decodes the video code stream and the audio code stream from the receiving unit 1701 to obtain a decoded video and one or more decoded audios corresponding to one or more channels.
  • the rendering unit 1703 performs operations on the reconstructed video to make it suitable for display. Such operations may include one or more of the following operations to improve the perceived quality: noise reduction, synthesis, color space conversion, up-sampling, down-sampling, and so on.
  • the rendering unit 1703 may also perform operations on the decoded audio to improve the perceived quality of the displayed audio signal.
  • FIG. 18 shows a schematic diagram of the composition structure of a communication system provided by an embodiment of the present application.
  • the source device 1801 may be the sending device 1600 shown in FIG. 16.
  • the output of the storage/transmission unit 1603 is processed by the storage medium/transmission network 1802 to store or transmit the code stream.
  • the target device 1803 may be the target device 1700 shown in FIG. 17.
  • the receiving unit 1701 may obtain the code stream from the storage medium/transmission network 1802.
  • the receiving unit 1701 can extract a new video code stream from the media file or transport stream.
  • the receiving unit 1701 can also extract a new audio code stream from the media file or transport stream.
  • the encoder side after determining the prediction parameter of the current block, obtains the current block when the prediction mode parameter indicates that the matrix-based intra prediction MIP mode is used to determine the intra prediction value for the chrominance component of the current block And determine the MIP input sample value of the current block according to the adjacent sample value of the current block; determine the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix, and the shift parameter MIP prediction value; filter the MIP prediction value to determine the intra prediction value of the chroma component of the current block; determine the prediction residual of the chroma component of the current block according to the intra prediction value of the chroma component of the current block Difference; perform low-frequency inseparable secondary transformation LFNST on the prediction residual value to determine the LFNST parameter; encode the LFNST parameter and write it into the code stream.
  • the prediction mode parameter indicates to use the matrix-based intra prediction MIP mode for the chrominance component of the current block to determine the intra prediction value
  • obtain the current The adjacent sample value of the block, and the MIP input sample value of the current block is determined according to the adjacent sample value of the current block; the chrominance component of the current block is determined according to the MIP input sample value, the MIP weight matrix and the shift parameter MIP prediction value; filter the MIP prediction value to determine the intra-frame prediction value of the chrominance component of the current block; when the LFNST parameter indicates to perform LFNST on the current block, determine the reconstructed transform coefficient block of the current block, Perform LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block; perform the first transform on the second transform coefficient block to obtain the reconstruction residual of the chrominance component of the current block Block; according to the
  • the complexity can be reduced on the basis of ensuring the coding and decoding performance, while the storage space required in the coding and decoding process is reduced, and the coding and decoding efficiency is effectively improved; in addition, the LFNST technology is being applied to the prediction of the MIP mode. At this time, due to the introduction of MIP parameters, the LFNST transform is more flexible, and the coding and decoding efficiency is further improved.

Abstract

本申请实施例公开了一种编码方法、解码方法、编码器、解码器以及存储介质，该方法包括：确定当前块的预测参数，所述预测参数包括预测模式参数；当预测模式参数指示对当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取当前块的相邻采样值，并根据当前块的相邻采样值确定当前块的MIP输入采样值；根据MIP输入采样值、MIP加权矩阵和移位参数，确定当前块的色度分量的MIP预测值；对所述MIP预测值进行滤波处理，确定当前块的色度分量的帧内预测值；根据当前块的色度分量的帧内预测值，确定当前块的色度分量的预测残差值；对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；对所述LFNST参数进行编码，并写入码流。

Description

编码方法、解码方法、编码器、解码器以及存储介质
相关申请的交叉引用
本申请要求以Junyan Huo、Shuai Wan、Yanzhuo Ma的名义于2020年01月08日提交的、申请号为62/958,582的题为“VIDEO ENCODING AND DECODING METHOD,APPARATUS AND COMMUNICATION SYSTEM”在先美国临时专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及视频编解码技术领域，尤其涉及一种编码方法、解码方法、编码器、解码器以及存储介质。
背景技术
随着人们对视频显示质量要求的提高，高清和超高清视频等新视频应用形式应运而生。H.265/高效率视频编码(High Efficiency Video Coding，HEVC)已经无法满足视频应用迅速发展的需求，联合视频专家组(Joint Video Exploration Team，JVET)提出了新一代视频编码标准H.266/多功能视频编码(Versatile Video Coding，VVC)。
在H.266/VVC中，基于矩阵的帧内预测(Matrix-based Intra Prediction，MIP)是一种帧内预测模式，用以获得当前块的帧内预测块；然后使用低频不可分二次变换(Low-Frequency Non-Separable Transform，LFNST)技术来确定当前块的预测残差。但是LFNST技术在应用于MIP模式预测时，对VVC的编解码效率具有负面影响。
发明内容
本申请实施例提供一种编码方法、解码方法、编码器、解码器以及存储介质，可以提高编解码效率。
本申请实施例的技术方案可以如下实现：
第一方面，本申请实施例提供了一种编码方法，应用于编码器，该方法包括：
确定当前块的预测参数，其中，所述预测参数包括预测模式参数；
当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；
对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
根据所述当前块的色度分量的帧内预测值，确定所述当前块的色度分量的预测残差值；
对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；
对所述LFNST参数进行编码，并写入码流。
第二方面，本申请实施例提供了一种解码方法，应用于解码器，该方法包括：
解析码流，获取当前块的预测参数和LFNST参数，其中，所述预测参数包括预测模式参数；
当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；
对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
当所述LFNST参数指示对所述当前块执行LFNST时，确定所述当前块的重构变换系数块，对所 述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块；
对所述第二变换系数块进行第一次变换，得到所述当前块的色度分量的重构残差块；
根据所述当前块的色度分量的帧内预测值以及所述重构残差块，确定所述当前块的色度分量的重构块。
第三方面，本申请实施例提供了一种编码器，该编码器包括第一确定单元、第一预测单元、第一变换单元和编码单元；其中，
所述第一确定单元，配置为确定当前块的预测参数，其中，所述预测参数包括预测模式参数；
所述第一确定单元，还配置为当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
所述第一预测单元，配置为根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；以及对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
所述第一变换单元，配置为根据所述当前块的色度分量的帧内预测值，确定所述当前块的色度分量的预测残差值；以及对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；
所述编码单元，配置为对所述LFNST参数进行编码，并写入码流。
第四方面，本申请实施例提供了一种编码器，该编码器包括第一存储器和第一处理器；其中，
所述第一存储器，用于存储能够在所述第一处理器上运行的计算机程序；
所述第一处理器，用于在运行所述计算机程序时，执行如第一方面所述的方法。
第五方面，本申请实施例提供了一种解码器，该解码器包括解析单元、第二确定单元、第二预测单元和第二变换单元；其中，
所述解析单元，配置为解析码流，获取当前块的预测参数和LFNST参数，其中，所述预测参数包括预测模式参数；
所述第二确定单元，配置为当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
所述第二预测单元，配置为根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；以及为对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
所述第二变换单元，配置为当所述LFNST参数指示对所述当前块执行LFNST时，确定所述当前块的重构变换系数块，对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块；以及对所述第二变换系数块进行第一次变换，得到所述当前块的色度分量的重构残差块；
所述第二确定单元，还配置为根据所述当前块的色度分量的帧内预测值以及所述重构残差块，确定所述当前块的色度分量的重构块。
第六方面，本申请实施例提供了一种解码器，该解码器包括第二存储器和第二处理器；其中，
所述第二存储器，用于存储能够在所述第二处理器上运行的计算机程序；
所述第二处理器，用于在运行所述计算机程序时，执行如第二方面所述的方法。
第七方面，本申请实施例提供了一种计算机存储介质，该计算机存储介质存储有计算机程序，所述计算机程序被第一处理器执行时实现如第一方面所述的方法、或者被第二处理器执行时实现如第二方面所述的方法。
本申请实施例提供了一种编码方法、解码方法、编码器、解码器以及存储介质，编码器侧，在确定当前块的预测参数后，当预测模式参数指示对当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取当前块的相邻采样值，并根据当前块的相邻采样值确定当前块的MIP输入采样值；根据MIP输入采样值、MIP加权矩阵和移位参数，确定当前块的色度分量的MIP预测值；对所述MIP预测值进行滤波处理，确定当前块的色度分量的帧内预测值；根据当前块的色度分量的帧内预测值，确定当前块的色度分量的预测残差值；对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；对所述LFNST参数进行编码，并写入码流。解码器侧，在解析码流，获取当前块的预测参数和LFNST参数后，当预测模式参数指示对当前块的色度分量使用MIP模式确定帧内预测值时，获取当前块的相邻采样值，并根据当前块的相邻采样值确定当前块的MIP输入采样值；根据MIP输入采样值、MIP加权矩阵和移位参数，确定当前块的色度分量的MIP预测值；对MIP预测值进行滤波处理，确定当前块的色度分量的帧内预测值；当LFNST参数指示对当前块执行LFNST时，确定当前块的重构变换系数块，对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块； 对所述第二变换系数块进行第一次变换，得到当前块的色度分量的重构残差块；根据当前块的色度分量的帧内预测值以及所述重构残差块，确定当前块的色度分量的重构块。这样，针对MIP模式，能够在保证编解码性能的基础上，降低复杂度，同时减小编解码过程中所需要的存储空间，有效地提高编解码效率；另外，LFNST技术在应用于MIP模式预测时，由于引入了MIP参数，使得LFNST变换更加灵活，进一步提高了编解码效率。
附图说明
图1为本申请实施例提供的一种编码器的组成结构示意图；
图2为本申请实施例提供的一种解码器的组成结构示意图；
图3为本申请实施例提供的一种编码方法的流程示意图；
图4为本申请实施例提供的另一种编码方法的流程示意图；
图5为本申请实施例提供的一种MPI预测过程的流程示意图；
图6为本申请实施例中一种使用MIP模式获取帧内预测块的流程示意图；
图7为本申请实施例提供的一种执行LFNST的流程框架示意图；
图8为本申请实施例提供的一种对LFNST参数进行编码的流程示意图；
图9为本申请实施例提供的一种解码方法的流程示意图；
图10为本申请实施例提供的一种对LFNST参数进行解析的流程示意图；
图11为本申请实施例提供的另一种执行LFNST的流程框架示意图；
图12为本申请实施例提供的另一种编码器的组成结构示意图；
图13为本申请实施例提供的一种编码器的具体硬件结构示意图；
图14为本申请实施例提供的另一种解码器的组成结构示意图；
图15为本申请实施例提供的一种解码器的具体硬件结构示意图；
图16为本申请实施例提供的一种发送设备的组成结构示意图；
图17为本申请实施例提供的一种目标设备的组成结构示意图；
图18为本申请实施例提供的一种通信系统的组成结构示意图。
具体实施方式
为了能够更加详尽地了解本申请实施例的特点与技术内容，下面结合附图对本申请实施例的实现进行详细阐述，所附附图仅供参考说明之用，并非用来限定本申请实施例。
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中所使用的术语只是为了描述本申请实施例的目的，不是旨在限制本申请。
在以下的描述中，涉及到“一些实施例”，其描述了所有可能实施例的子集，但是可以理解，“一些实施例”可以是所有可能实施例的相同子集或不同子集，并且可以在不冲突的情况下相互结合。还需要指出，本申请实施例所涉及的术语“第一\第二\第三”仅仅是是区别类似的对象，不代表针对对象的特定排序，可以理解地，“第一\第二\第三”在允许的情况下可以互换特定的顺序或先后次序，以使这里描述的本申请实施例能够以除了在这里图示或描述的以外的顺序实施。
随着视频压缩技术的发展，国际电信联盟-电信标准化部(International Telecommunication Union-Telecommunication，ITU-T)和国际标准化组织(International Organization for Standardization，ISO)/国际电工委道员会(International Electrotechnical Commission，IEC)启动了一项名为VVC的标准化项目以开发新一代视频编码标准，其目的是在对具有高分辨率、高帧率、高位深、高动态范围、宽色域和全向视角中的一个或多个特征的高质量视频进行编码时，与H.265/HEVC标准相比，可以使得VVC性能提高约50％。由ITU-T SG 16 WP 3和ISO/IEC JTC 1/SC 29/WG 11的联合视频专家组(Joint Video Exploration Team，JVET)负责该标准化项目。其中，各种帧内预测模式和帧间预测模式已被验证在对高质量视频进行编码时能够实现高压缩效率，因此在VVC的工作草案(Working Draft，WD)中被采纳。
这里，基于矩阵的帧内预测(Matrix-based Intra Prediction，MIP)模式是一种帧内预测模式。在VVC中，编码器或解码器可以调用MIP模式以获得当前块的帧内预测块。然后，作为次要变换(或称为第二次变换)的低频不可分离变换(Low Frequency Non-Separable Transform，LFNST)能够应用于当前块的预测残差。
在相关技术中，LFNST虽然能够应用于具有LFNST变换核候选集和LFNST转置指示参数的固定配置的MIP模式预测，但是这对VVC的编解码效率具有负面影响。
基于此，本申请一实施例提供了一种编码方法，该编码方法的基本思想是：确定当前块的预测参数，其中，所述预测参数包括预测模式参数；当所述预测模式参数指示对当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取当前块的相邻采样值，并根据当前块的相邻采样值确定当前块的MIP输入采样值；根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定当前块的色度分量的MIP预测值；对所述MIP预测值进行滤波处理，确定当前块的色度分量的帧内预测值；根据当前块的色度分量的帧内预测值，确定当前块的色度分量的预测残差值；对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；对所述LFNST参数进行编码，并写入码流。本申请另一实施例提供了一种解码方法，该解码方法的基本思想是：解析码流，获取当前块的预测参数和LFNST参数，其中，所述预测参数包括预测模式参数；当所述预测模式参数指示对当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取当前块的相邻采样值，并根据当前块的相邻采样值确定所述当前块的MIP输入采样值；根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定当前块的色度分量的MIP预测值；对所述MIP预测值进行滤波处理，确定当前块的色度分量的帧内预测值；当所述LFNST参数指示对当前块执行LFNST时，确定当前块的重构变换系数块，对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块；对所述第二变换系数块进行第一次变换，得到当前块的色度分量的重构残差块；根据当前块的色度分量的帧内预测值以及所述重构残差块，确定当前块的色度分量的重构块。这样，针对MIP模式，能够在保证编解码性能的基础上，降低复杂度，同时减小编解码过程中所需要的存储空间，有效地提高编解码效率；另外，LFNST技术在应用于MIP模式预测时，由于引入了MIP参数，使得LFNST变换更加灵活，进一步提高了编解码效率。
下面将结合附图对本申请各实施例进行详细阐述。
参见图1，其示出了本申请实施例提供的一种编码器的系统组成框图示例。如图1所示，编码器100可以包括：分割单元101、预测单元102、第一加法器107、变换单元108、量化单元109、反量化单元110、逆变换单元111、第二加法器112、滤波单元113、解码图片缓存(Decoded Picture Buffer，DPB)单元114和熵编码单元115。这里，编码器100的输入可以是由一系列图片或者一张静态图片组成的视频，编码器100的输出可以是用于表示输入视频的压缩版本的比特流(也可以称为“码流”)。
其中，分割单元101将输入视频中的图片分成一个或多个编码树单元(coding tree units，CTUs)。分割单元101将图片分成多个图块(tiles)，还可以进一步将一个tile分成一个或多个砖块(bricks)，这里,一个tile或者一个brick中可以包括一个或多个完整的和/或部分的CTUs。另外，分割单元101可以形成一个或多个切片(slices)，其中一个slice可以包括图片中按照栅格顺序排列的一个或多个tiles，或者覆盖图片中矩形区域的一个或多个tiles。分割单元101还可形成一个或多个子图片，其中，一个子图片可以包括一个或多个slices、tiles或bricks。
在编码器100的编码过程中，分割单元101将CTU传送到预测单元102。通常，预测单元102可以由块分割单元103、运动估计(Motion Estimation，ME)单元104、运动补偿(Motion Compensation，MC)单元105和帧内预测单元106组成。具体地，块分割单元103迭代地使用四叉树分割、二叉树分割和三叉树分割而进一步将输入CTU划分成更小的编码单元(Coding Units，CUs)。预测单元102可使用ME单元104和MC单元105获取CU的帧间预测块。帧内预测单元106可使用包括MIP模式的各种帧内预测模式获取CU的帧内预测块。在示例中，率失真优化的运动估计方式可被ME单元104和MC单元105调用以获取帧间预测块，以及率失真优化的模式确定方式可被帧内预测单元106调用以获取帧内预测块。
预测单元102输出CU的预测块，第一加法器107计算分割单元101的输出中的CU和CU的预测块之间的差值，即残差CU。变换单元108读取残差CU并对残差CU执行一个或多个变换操作以获取系数。量化单元109对系数进行量化并输出量化系数(即levels)。反量化单元110对量化系数执行缩放操作以输出重构系数。逆变换单元111执行对应于变换单元108中的变换的一个或多个逆变换并输出重构残差。第二加法器112通过使重构残差和来自预测单元102的CU的预测块相加而计算出重构CU。第二加法器112还将其输出发送到预测单元102以用作帧内预测参考。在图片或子图片中的所有CU被重构之后，滤波单元113对重构图片或子图片执行环路滤波。这里，滤波单元113包含一个或多个滤波器，例如去方块滤波器、采样自适应偏移(Sample Adaptive Offset，SAO)滤波器、自适应环路滤波器(Adaptive Loop Filter，ALF)、亮度映射和色度缩放(Luma Mapping with Chroma Scaling，LMCS)滤波器以及基于神经网络的滤波器等。或者，当滤波单元113确定CU不用作其它CU编码时的参考时，滤波单元113对CU中的一个或多个目标像素执行环路滤波。
滤波单元113的输出是解码图片或子图片，这些解码图片或子图片缓存至DPB单元114。DPB单 元114根据时序和控制信息输出解码图片或子图片。这里，存储在DPB单元114中的图片还可用作预测单元102执行帧间预测或帧内预测的参考。最后熵编码单元115将来自编码器100中解码图片所必需的参数(比如控制参数和补充信息等)转换成二进制形式，并根据每个数据单元的语法结构将这样的二进制形式写入码流中，即编码器100最终输出码流。
进一步地，编码器100可以是具有第一处理器和记录计算机程序的第一存储器。当第一处理器读取并运行计算机程序时，编码器100读取输入视频并生成对应的码流。另外，编码器100还可以是具有一个或多个芯片的计算设备。在芯片上实现为集成电路的这些单元具有与图1中相应单元类似的连接和数据交换功能。
参见图2，其示出了本申请实施例提供的一种解码器的系统组成框图示例。如图2所示，该解码器200可以包括：解析单元201、预测单元202、缩放单元205、变换单元206、加法器207、滤波单元208和解码图片缓存单元209。这里，解码器200的输入是用于表示视频或者一张静态图片的压缩版本的比特流，解码器200的输出可以是由一系列图片组成的解码视频或者一张解码的静态图片。
其中，解码器200的输入码流可以是编码器100所生成的码流。解析单元201对输入码流进行解析并从输入码流获取语法元素的值。解析单元201将语法元素的二进制表示转换成数字值并将数字值发送到解码器200中的单元以获取一个或多个解码图片。解析单元201还可从输入码流解析一个或多个语法元素以显示解码图片。
在解码器200的解码过程中，解析单元201将语法元素的值以及根据语法元素的值设置或确定的、用于获取一个或多个解码图片的一个或多个变量发送到解码器200中的单元。
预测单元202确定当前解码块(例如CU)的预测块。这里，预测单元202可以包括运动补偿单元203和帧内预测单元204。具体地，当指示帧间解码模式用于对当前解码块进行解码时，预测单元202将来自解析单元201的相关参数传递到运动补偿单元203以获取帧间预测块；当指示帧内预测模式(包括基于MIP模式索引值指示的MIP模式)用于对当前解码块进行解码时，预测单元202将来自解析单元201的相关参数传送到帧内预测单元204以获取帧内预测块。
缩放单元205具有与编码器100中的反量化单元110相同的功能。缩放单元205对来自解析单元201的量化系数(即levels)执行缩放操作以获取重构系数。
变换单元206具有与编码器100中的逆变换单元111相同的功能。变换单元206执行一个或多个变换操作(即通过编码器100中的逆变换单元111执行的一个或多个变换操作的反操作)以获取重构残差。
加法器207对其输入(来自预测单元202的预测块和来自变换单元206的重构残差)执行相加操作以获取当前解码块的重构块。重构块还发送到预测单元202以用作在帧内预测模式下编码的其它块的参考。
在图片或子图片中的所有CU被重构之后，滤波单元208对重构图片或子图片执行环路滤波。滤波单元208包含一个或多个滤波器，例如去方块滤波器、采样自适应补偿滤波器、自适应环路滤波器、亮度映射和色度缩放滤波器以及基于神经网络的滤波器等。或者，当滤波单元208确定重构块不用作对其它块解码时的参考时，滤波单元208对重构块中的一个或多个目标像素执行环路滤波。这里，滤波单元208的输出是解码图片或子图片，解码图片或子图片缓存至DPB单元209。DPB单元209根据时序和控制信息输出解码图片或子图片。存储在DPB单元209中的图片还可用作通过预测单元202执行帧间预测或帧内预测的参考。
进一步地，解码器200可以是具有第二处理器和记录计算机程序的第二存储器。当第一处理器读取并运行计算机程序时，解码器200读取输入码流并生成对应的解码视频。另外，解码器200还可以是具有一个或多个芯片的计算设备。在芯片上实现为集成电路的这些单元具有与图2中相应单元类似的连接和数据交换功能。
需要说明的是，本申请实施例提供的一种编码方法主要作用于编码器100的帧内预测单元106和变换单元108，本申请实施例提供的一种解码方法主要作用于解码器200的帧内预测单元204和变换单元206。也就是说，本申请实施例既可以应用于编码器，也可以应用于解码器，甚至还可以同时应用于编码器和解码器，但是这里不作任何限定。
还需要说明的是，当本申请实施例应用于编码器100时，“当前块”具体是指帧内预测中的当前待编码的图像块(也可以称为“编码块”)；当本申请实施例应用于解码器200时，“当前块”具体是指帧内预测中的当前待解码的图像块(也可以称为“解码块”)。
另外，如果编码器100能够通过本申请实施例提供的编码方法得到一个较好的预测效果，用以提高编码性能；那么，对应地，解码器200通过本申请实施例提供的解码方法也能够改善视频解码恢复质量，从而提高解码性能。
本申请一实施例中，参见图3，其示出了本申请实施例提供的一种编码方法的流程示意图。如图3所示，该方法可以包括：
S301：确定当前块的预测参数，其中，所述预测参数包括预测模式参数。
需要说明的是，待编码图像可以划分为多个图像块，每个待编码的图像块可以称为编码块。这里，每个编码块可以包括第一图像分量、第二图像分量和第三图像分量。而当前块为视频图像中当前待进行第一图像分量、第二图像分量或者第三图像分量预测的编码块。
其中，假定对当前块进行第一图像分量预测，第一图像分量为亮度分量，即待预测图像分量为亮度分量，那么当前块也可以称为亮度块，以得到当前块的亮度分量的预测值。或者，假定对当前块进行第二图像分量预测，第二图像分量为色度分量，即待预测图像分量为色度分量，那么当前块也可以称为色度块，以得到当前块的亮度分量的预测值。
还需要说明的是，预测参数中可以包括有预测模式参数，其中，预测模式参数用于指示当前块采用的预测模式，而且不同的预测模式对应不同的预测模式参数。这里，针对预测模式参数的确定，可以采用简单的决策策略，比如根据失真值的大小进行确定；也可以采用复杂的决策策略，比如根据率失真优化(Rate Distortion Optimization，RDO)的结果进行确定，本申请实施例不作任何限定。通常而言，可以采用RDO方式来确定当前块的预测模式参数，也即确定出当前块的预测参数。
具体地，在一些实施例中，所述确定当前块的预测参数，可以包括：
利用多种预测模式对所述当前块进行预编码处理，获取所述多种预测模式对应的率失真代价值；
从所获取的多个率失真代价值中选择最优率失真代价值，将所述最优率失真代价值对应的预测模式下的预测参数确定为所述当前块的预测参数。
也就是说，在编码器侧，针对当前块可以采用多种预测模式分别对当前块进行预编码处理。这里，多种预测模式通常包括有帧间预测模式、传统帧内预测模式和非传统帧内预测模式。其中，传统帧内预测模式可以包括有直流(Direct Current，DC)模式、平面(PLANAR)模式和角度帧内预测模式等。非传统帧内预测模式可以包括有MIP模式、跨分量线性模型预测(Cross-component Linear Model Prediction，CCLM)模式、帧内块复制(Intra Block Copy，IBC)模式和PLT(Palette)模式等。而帧间预测模式可以包括有传统帧间预测模式和帧间几何划分预测(Geometrical partitioning for inter blocks，GEO)模式等。
这样，在利用多种预测模式分别对当前块进行预编码之后，可以得到所述多种预测模式对应的率失真代价值。然后从所得到的多个率失真代价值中选取最优率失真代价值(通常情况下，最小率失真代价值即为最优率失真代价值)，并将该最优率失真代价值对应的预测模式下的预测参数确定为当前块的预测参数。
除此之外，还可以利用多种预测模式分别对当前块进行预编码，得到所述多种预测模式对应的失真值。然后从所得到的多个失真值中选取最优失真值，并将该最优失真值对应的预测模式下的预测参数确定为当前块的预测参数。如此，编码器最终使用所确定的预测参数中预测模式参数所指示的预测模式对当前块进行预测编码，可以使得预测残差较小，能够提高编码效率。
S302：当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值。
需要说明的是，对于当前块而言，本申请实施例是利用MIP模式对当前块的色度分量进行帧内预测。在这过程中，首先需要获取当前块的相邻采样值；然后根据当前块的相邻采样值，用以确定出当前块的MIP输入采样值。
应理解，对于MIP模式，首先需要配置MIP核心参数。这里，MIP核心参数可包括当前块的种类(用mipSizeId表示)、每个边的参考采样个数(用boundySize表示)、MIP输入采样的个数(用inSize表示)以及矩阵乘法输出的MIP预测块大小(排列为predSize×predSize)。其中，MIP模式可根据当前块的宽度和高度将当前块划分为三类，mipSizeId可以等于0、1或2，这里的mipSizeId表示当前块的种类，也即本申请实施例中所示的“当前块的块尺寸索引值”。针对不同的mipSizeId，参考采样的个数(每边需要boundySize个参考采样点)、MIP输入采样的个数(inSize)和矩阵乘法输出的MIP预测块大小(排列成predSize×predSize)也是不同的。
还需要说明的是，预测参数除了预测模式参数之外，还可以包括：当前块的尺寸参数。其中，当前块的尺寸参数可包括：当前块的宽度(用nTbW表示)和高度(用nTbH表示)。而且根据当前块的尺寸参数，可以确定出当前块的块尺寸索引值(即mipSizeId)。
在一种可能的实施方式中，所述根据当前块的尺寸参数，确定当前块的块尺寸索引值，可以包括：
如果当前块的宽度和高度均等于4，那么可以将当前块的块尺寸索引值设置为0；
反之，如果当前块的宽度和高度均等于8，或者当前块的宽度和高度之一等于4，那么可以将当前块的块尺寸索引值设置为1；
反之，如果当前块为其他大小的块，那么可以将当前块的块尺寸索引值设置为2。
在另一种可能的实施方式中，所述根据当前块的尺寸参数，确定当前块的块尺寸索引值，可以包括：
如果当前块的宽度和高度均等于4，那么可以将当前块的块尺寸索引值设置为0；
反之，如果当前块的宽度和高度之一等于4，那么可以将当前块的块尺寸索引值设置为1；
反之，如果当前块为其他大小的块，那么可以将当前块的块尺寸索引值设置为2。
如此，根据当前块的块尺寸索引值，可以依据表1所示的查找表(Look-Up Table，LUT)，能够确定出相邻边界参考采样的个数(变量为boundarySize)和MIP预测块的大小(变量为predSize，MIP预测块的大小为predSize×predSize)，并计算出用于MIP矩阵乘法运算过程的MIP输入采样值的个数(用inSize表示)，该计算公式如下，
inSize＝(2×boundarySize)-(mipSizeId＝＝2)？1:0   (1)
其中，式(1)中的运算符的运算规则与ITU-TH.265标准中定义的运算符相同，例如“＝＝”为逻辑“相等(equal to)”运算符。
表1
mipSizeId boundarySize predSize
0 2 4
1 4 4
2 4 8
这样，根据表1，当mipSizeId的取值为0时，boundarySize的取值可以为2，predSize的取值可以为4。也就是说，这时候参考像素为每个边选取两个像素点，矩阵乘法输出为4×4的MIP预测块。
当mipSizeId的取值为1时，boundarySize的取值可以为4，predSize的取值可以为4。也就是说，这时候参考像素为每个边选取四个像素点，矩阵乘法输出为4×4的MIP预测块。
当mipSizeId的取值为2时，boundarySize的取值可以为4，predSize的取值可以为8；也就是说，这时候参考像素为每个边选取四个像素点，矩阵乘法输出为8×8的MIP预测块。
另外，还可以根据当前块的块尺寸索引值，依据表2所示的查找表，同时能够确定出boundarySize、inSize和predSize的取值。
表2
mipSizeId boundarySize inSize predSize
0 2 4 4
1 4 8 4
2 4 7 8
这样，根据表2，当mipSizeId的取值为0时，boundarySize的取值可以为2，inSize的取值可以为4，predSize的取值可以为4；也即，这时候参考像素为每个边选取两个像素点，矩阵乘法输入采样点数量为四个，矩阵乘法输出为4×4的MIP预测块。
当mipSizeId的取值为1时，boundarySize的取值可以为4，inSize的取值可以为8，predSize的取值可以为4；也即，这时候参考像素为每个边选取四个像素点，矩阵乘法输入采样点数量为八个，矩阵乘法输出为4×4的MIP预测块。
当mipSizeId的取值为2时，boundarySize的取值可以为4，inSize的取值可以为7，predSize的取值可以为8；也即，这时候参考像素为每个边选取四个像素点，矩阵乘法输入采样点数量为七个，矩阵乘法输出为8×8的MIP预测块。
进一步地，在配置MIP核心参数之后，还需要获取参考像素；然后根据参考像素和MIP核心参数，用以构造出MIP输入采样值。这里的参考像素即是当前块的相邻采样值，可以包括：当前块的左相邻采样值和当前块的上相邻采样值。也就是说，可以根据当前块的左相邻采样值和上相邻采样值，用以确定出当前块的MIP输入采样值。
在一些实施例中，所述根据当前块的相邻采样值确定当前块的MIP输入采样值，可以包括：
根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；
对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；
当所述当前块的块尺寸索引值处于预设范围之内时，根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第二常数值与所述第一临时参考值中索引序号为0对应的值之间的差值；将所述MIP输入采样值中索引序号为i对应的值 设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数；
当所述当前块的块尺寸索引值处于预设范围之外时，将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第一临时参考值中索引序号等于j+1对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
进一步地，当所述当前块的块尺寸索引值处于预设范围之内时，该方法还可以包括：
根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第一临时参考值中索引序号为0对应的值与所述第二常数值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数。
需要说明的是，将当前块的相邻采样值进行下采样滤波处理之后，可以得到第一临时参考值。这里，对于第一临时参考值，具体来讲，在将当前块的相邻采样值进行下采样滤波处理之后，可以将滤波处理后的相邻采样值缓存到缓冲区(用pTemp表示)中。其中，第一临时参考值中索引序号为0对应的值即指pTemp[0]，第一临时参考值中索引序号为i对应的值即指pTemp[i]。
还需要说明的是，根据当前块的块尺寸索引值(用mipSizeId表示)是否处于预设范围之内，可以确定出当前块的尺寸参数取值是否在预设范围之内。具体来讲，当mipSizeId＝0或1时，表明了当前块的块尺寸索引值处于预设范围之内，即当前块的尺寸参数取值在预设范围之内；当mipSizeId＝2时，表明了当前块的块尺寸索引值处于预设范围之外，即当前块的尺寸参数取值不在预设范围之内。
也就是说，MIP输入采样值是由缓冲区(用pTemp表示)、当前块的块尺寸索引值(用MipSizeId表示)、当前块的相邻采样值的比特深度(用BitDepth表示)确定，而且MIP输入采样值中所包含的输入采样个数则仅与当前块的块尺寸索引值有关，最终可以获取MIP输入采样值中索引序号为x对应的值(用p[x]表示)。
进一步地，在一些实施例中，所述根据所述当前块的相邻采样值的比特深度，确定第二常数值，可以包括：
将所述第二常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述当前块的相邻采样值的比特深度减1。
或者，在一些实施例中，所述根据所述当前块的相邻采样值的比特深度，确定第二常数值，可以包括：
对“1”进行二进制比特左移，得到所述第二常数值，其中，所述比特左移的位数等于所述当前块的相邻采样值的比特深度减1。
也就是说，在获取到当前块的相邻采样值的比特深度(用BitDepth表示)之后，第二常数值可以表示为1<<(BitDepth-1)或者2^(BitDepth-1)。这样，针对当前块的尺寸参数取值在预设范围之内的情况，可以结合第二常数值以确定出当前块的MIP输入采样值。
还需要说明的是，MIP输入采样是用于矩阵乘法运算的矩阵向量。目前的相关技术方案是由缓冲区(用pTemp表示)、当前块的种类(即当前块的块尺寸索引值，用mipSizeId表示)、当前块的相邻采样值的比特深度(用BitDepth表示)和MIP输入采样的个数确定，最终获取MIP输入采样值中索引序号为x对应的值(用p[x]表示)。
具体来讲，第x输入采样值p[x]的构造过程如下：
在一种可能的实施方式中，当所述当前块的块尺寸索引值处于预设范围之内时，该构造过程可以包括：
计算第二常数值与所述第一临时参考值中索引序号为0对应的值之间的差值，得到MIP输入采样值中索引序号为0对应的值；
对所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号为0对应的值进行减法运算，得到MIP输入采样值中索引序号为i对应的值；其中，i为大于0且小于N的正整数，N表示所述输入采样矩阵中包含的元素数量。
在本申请实施例中，将差值的计算中被减数设置为等于第二常数值，将差值的计算中减数设置为等于第一临时参考值中索引序号为0对应的值。
也就是说，当mipSizeId＝0或1时，还可以由1<<(BitDepth-1)减去pTemp[0]作为p[0]；然后当x不等于0时，则由pTemp[x]减去pTemp[0]作为p[x]。具体如下所示，
Figure PCTCN2020141080-appb-000001
在另一种可能的实施方式中，当所述当前块的块尺寸索引值处于预设范围之内时，该构造过程可以包括：
计算所述第一临时参考值中索引序号为0对应的值与第二常数值之间的差值，得到MIP输入采样值中索引序号为0对应的值；
对所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号为0对应的值进行减法运算，得到MIP输入采样值中索引序号为i对应的值；其中，i为大于0且小于N的正整数，N表示所述输入采样矩阵中包含的元素数量。
在本申请实施例中，将差值的计算中被减数设置为等于所述第一临时参考值中索引序号为0对应的值，将差值的计算中减数设置为等于第二常数值。
也就是说，当mipSizeId＝0或1时，在x等于0时，MIP输入采样值中索引序号为0对应的值(用p[0]表示)，可以由第一临时参考值中索引序号为0对应的值(即pTemp[0])减去第二常数值(即1<<(BitDepth-1))得到。在x不等于0时，MIP输入采样值中索引序号为x对应的值(用p[x]表示)，则需要由第一临时参考值中索引序号为x对应的值(即pTemp[x])减去第一临时参考值中索引序号为0对应的值(即pTemp[0])得到。
具体如下所示，
Figure PCTCN2020141080-appb-000002
在又一种可能的实施方式中，当所述当前块的块尺寸索引值处于预设范围之外时，该构造过程可以包括：
对所述第一临时参考值中索引序号为i+1对应的值与所述第一临时参考值中索引序号为0对应的值进行减法运算，得到MIP输入采样值中索引序号为i对应的值；其中，i为大于或等于0且小于N的正整数，N表示所述输入采样矩阵中包含的元素数量。
也就是说，当mipSizeId＝2时，可以忽略第一临时参考值中索引序号为0对应的值，即pTemp[0]，然后MIP输入采样值中索引序号为x对应的值(用p[x]表示)，可以由第一临时参考值中索引序号为x+1对应的值(即pTemp[x+1])减去第一临时参考值中索引序号为0对应的值(即pTemp[0])得到。这里，x为大于或等于0的正整数。具体如下所示，
p[x]＝pTemp[x+1]-pTemp[0]for x＝0,...,inSize-1   (4)
这样，仍以4×4的当前块为例，缓冲区pTemp中存储有四个值，即第一临时参考值包括有：索引序号为0对应的值(即pTemp[0])、索引序号为1对应的值(即pTemp[1])、索引序号为2对应的值(即pTemp[2])和索引序号为3对应的值(即pTemp[3])；这时候根据式(2)或式(3)或式(4)可以确定出四个MIP输入采样值，用p[x]表示，x＝0,1,2,3。这里，针对这四个MIP输入采样值，还可以将其组成1×4的MIP输入采样矩阵。
除此之外，本申请实施例还可以利用统一的计算方式来获取p[x]的值，而无需判断mipSizeId的取值。在一些实施例中，所述根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值，可以包括：
对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；
根据所述当前块的相邻采样值的比特深度，确定第二常数值，将所述第二常数值缓存在所述第一临时参考值之后的一个数据单元中，得到第二临时参考值；
将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第二临时参考值中索引序号等于j+1对应的值与所述第二临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
也就是说，本申请实施例可以通过在缓冲区pTemp的末尾追加第二常数值(即(1<<(BitDepth-1)))作为附加元素，这时候不再考虑当前块的尺寸参数(即无需考虑mipSizeId的取值)，可以直接将p[x]设置为等于pTemp[x+1]-pTemp[0]，x＝0,…,inSize-1。
这样，在根据当前块的相邻采样值确定出当前块的MIP输入采样值后，可以进一步确定当前块的MIP预测值。
S303：根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值。
其中，MIP预测值为当前块的色度分量中部分采样点的预测值。
需要说明的是，移位参数可以包括移位偏移参数和移位数量参数。这时候，如图4所示，对于S303来说，可以包括如下步骤：
S303-1：根据移位偏移参数的取值，确定所述MIP输入采样值的和值与所述移位偏移参数的乘积。其中，所述移位偏移参数的取值为固定常数。
需要说明的是，移位偏移参数还可称为移位补偿参数或者偏移因子，可以用fO表示。在本申请实施例中，可以将移位偏移参数的取值设置为固定常数，比如32，或者46，或者56，或者66等。还可以将移位偏移参数的取值设置为与移位偏移参数表有关，通过查表的方式来确定移位偏移参数的取值，这里并不作任何限定。
S303-2：根据移位数量参数的取值，确定第一常数值。
S303-3：将第一偏移量的取值设置为所述第一常数值与所述乘积之差。
需要说明的是，移位数量参数还可称为移位因子、移位比特数或者权重移位值等，可以用sW、shift或者weight shift表示。在本申请实施例中，移位数量参数用sW表示。另外，第一偏移量可以用oW表示，第一偏移量与移位数量参数和移位偏移参数均有关。
在本申请实施例中，可以将移位数量参数的取值设置为固定常数，比如5，或者6，或者7等。还可以将移位数量参数的取值设置为与移位数量参数表有关，通过查表的方式来确定移位数量参数的取值，这里并不作任何限定。
S303-4：根据所述预测参数，确定所述当前块的MIP加权矩阵。
需要说明的是，预测参数可以包括预测模式参数，还可以包括当前块的尺寸参数。在预测参数中，当预测模式参数指示当前块的色度分量使用MIP模式确定帧内预测值时，这时候可以预先建立有权重矩阵表，并且该权重矩阵表存储在存储器或者存储单元中，这里的存储器或者存储单元可以是集成在编码器中，或者也可以单独设置。这样，根据当前块的块尺寸索引值(mipSizeId)和MIP模式索引值(modeId)，可以通过查表的方式确定出当前块所需要使用的MIP加权矩阵(或者称为MIP权重矩阵，也可简称为MIP矩阵)，用mWeight[x][y]表示。其中，当前块的块尺寸索引值(mipSizeId)是由当前块的尺寸参数确定的，而MIP加权矩阵mWeight[x][y]的大小仅与当前块的块尺寸索引值有关，如表3所示。
在表3所示的该MIP加权矩阵中，列数为矩阵乘法输入采样的个数inSize，行数为矩阵乘法输出采样的个数predSized×predSized，从而能够确定出当前块的MIP加权矩阵。
表3
mipSizeId 列数 行数
0 4 16
1 8 16
2 7 64
可以理解，在编码器中，还可以预先建立有移位数量参数表，并且该移位数量参数表也存储在存储器或者存储单元中，这里的存储器或者存储单元可以是集成在编码器中，或者也可以单独设置。本申请实施例中，针对移位数量参数(sW)的确定可以包括如下几种方式：
一种可能的实施方式中，对于不同的块尺寸索引值和不同的MIP模式索引值，移位数量参数可以是不同的。在一些实施例中，该方法还可以包括：
在所述当前块的色度分量使用MIP模式确定帧内预测值时，确定当前块的MIP模式索引值；
根据所述MIP模式索引值，从第一预设查找表中查询与所述MIP模式索引值对应的取值，其中，所述第一预设查找表用于记录MIP模式索引值与移位数量参数的取值之间的对应关系；
将查询到的取值确定为所述移位数量参数的取值。
需要说明的是，可以根据当前块的块尺寸索引值(用mipSizeId表示)和MIP模式索引值(modeId)来查询移位数量参数的取值。如表4所示的第一预设查找表，针对不同的mipSizeId和modeId，可以通过查找表方式确定出矩阵乘法中所需要使用的移位数量参数。
表4
Figure PCTCN2020141080-appb-000003
但是在编码器侧，表4是需要以查找表形式存储入存储器或者存储单元中；然而存储是需要代价的，而且查找过程也是需要代价的；由于表4中移位数量参数的取值与当前块的块尺寸大小和MIP模式索引值均相关，从而增加了内存占用，而且还增加了计算复杂度。
为了减少内存占用和降低计算复杂度，本申请实施例可针对移位数量参数的确定方式进行简化。
在另一种可能的实施方式中，移位数量参数的取值可以设置为一个固定常数，而且是与块尺寸索引值和MIP模式索引值无关的固定常数。例如，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位数量参数的取值均可以设置为5；或者，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位数量参数的取值均可以设置为6；或者，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位数量参数的取值均可以设置为7。在本申请实施例中，优选地，移位数量参数的取值设置为等于6，但对此不作任何限定。
在又一种可能的实施方式中，对于移位数量参数的取值来说，该方法还可以包括：
根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；
根据所述当前块的块尺寸索引值，确定所述移位数量参数的取值。
需要说明的是，根据所述当前块的尺寸参数，可以确定出当前块的块尺寸索引值。在一些实施例中，所述根据所述当前块的大小参数，确定所述当前块的块尺寸索引值，可以包括：
当所述当前块的宽度和高度均等于4时，将所述当前块的块尺寸索引值设置为0；
当所述当前块的宽度和高度均等于8、或者所述当前块的宽度和高度之一等于4时，将所述当前块的块尺寸索引值设置为1；
当所述当前块的宽度和高度不满足前述条件时，将所述当前块的块尺寸索引值设置为2。
这样，在确定出当前块的块尺寸索引值之后，根据当前块的块尺寸索引值，可以进一步确定出移位数量参数的取值。
可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位数量参数的取值，可以包括：
当所述块尺寸索引值分别等于0、1和2时，确定所述当前块的块尺寸索引值对应的所述移位数量参数的取值分别等于5、6和5。
可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位数量参数的取值，可以包括：
将所述移位数量参数的取值设置为等于所述当前块的宽度或高度与对应于所述当前块的块尺寸索引值的第一预设值之间的比值。
这里，第一预设值表示从所述当前块的边界获取的MIP输入采样值的数量。在这种情况下，该方法还可以包括：
当所述当前块的块尺寸索引值分别等于0、1和2时，确定所述当前块的块尺寸索引值对应的所述第一预设值分别等于2、4和4。
也就是说，当第一预设值表示从当前块的边界获取的MIP输入采样值的数量时，这时候如果当前块的块尺寸索引值等于0，那么对应的第一预设值等于2；如果当前块的块尺寸索引值等于1，那么对应的第一预设值等于4；如果当前块的块尺寸索引值等于2，那么对应的第一预设值等于4，从而可以根据当前块的宽度或高度与对应的第一预设值之间的比值来确定出移位数量参数的取值。
可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位数量参数的取值，可以包括：
将所述移位数量参数的取值设置为等于所述当前块的宽度或高度与对应于所述当前块的块尺寸索引值的第二预设值之间的比值。
这里，第二预设值表示直接使用所述MIP加权矩阵计算获取的所述当前块的MIP预测块的大小。在这种情况下，该方法还可以包括：
当所述当前块的块尺寸索引值分别等于0、1和2时，确定所述当前块的块尺寸索引值对应的所述第二预设值分别等于4、4和8。
也就是说，当第二预设值表示直接使用MIP加权矩阵计算获取的当前块的MIP预测块的大小时，这时候如果当前块的块尺寸索引值等于0，那么对应的第二预设值等于4；如果当前块的块尺寸索引值等于1，那么对应的第二预设值等于4；如果当前块的块尺寸索引值等于2，那么对应的第二预设值等于8，从而可以根据当前块的宽度或高度与对应的第二预设值之间的比值来确定出移位数量参数的取值。
在又一种可能的实施方式中，可以最小化移位数量参数表，并且仍然使用查找表的方式来确定移位数量参数的取值。可选地，在一些实施例中，所述根据当前块的块尺寸索引值，确定所述移位数量参数的取值，可以包括：
根据所述块尺寸索引值，从第二预设查找表中查询与所述块尺寸索引值对应的取值，其中，所述第二预设查找表用于记录块尺寸索引值与移位数量参数的取值之间的对应关系；
将查询到的取值确定为所述移位数量参数的取值。
需要说明的是，可以仅根据当前块的块尺寸索引值(用mipSizeId表示)来查询移位数量参数的取值。如表5所示的第二预设查找表，针对每一种的块尺寸索引值，可以对应固定的取值，也就是说，每个块的尺寸大小或者每个块的尺寸集合可以具有如表5所示固定的移位数量参数的取值。
表5
mipSizeId sW
0 5
1 6
2 5
根据表5，当所述当前块的块尺寸索引值分别等于0、1和2时，可以确定出所述块尺寸索引值对应的所述移位数量参数的取值分别等于5、6和5。
在上述的实施方式中，通过简化移位数量参数的确定方式，尤其是最小化移位数量参数表或者固定移位数量参数的取值，可以实现最小化查找表的存储，从而能够在不增加计算复杂度的情况下，减小MIP模式下移位数量参数表存储所占用的内存。
在编码器中，还可以预先建立有移位偏移参数表，并且该移位偏移参数表也存储在存储器或者存储单元中，这里的存储器或者存储单元可以是集成在编码器中，或者也可以单独设置。本申请实施例中，针对移位偏移参数(fO)的确定可以包括如下几种方式：
在一种可能的实施方式中，对于不同的块尺寸和不同的MIP模式索引值，移位偏移参数也可以是不同的。在一些实施例中，该方法还可以包括：
在所述当前块的色度分量使用MIP模式确定帧内预测值时，确定当前块的MIP模式索引值；
根据所述MIP模式索引值，从第三预设查找表中查询与所述MIP模式索引值对应的取值，其中，所述第三预设查找表用于记录MIP模式索引值与移位偏移参数的取值之间的对应关系；
将查询到的取值确定为所述移位偏移参数的取值。
需要说明的是，可以根据当前块的块尺寸索引值(用MipSizeId表示)和MIP模式索引值(modeId)来查询移位偏移参数的取值。如表6所示的第三预设查找表，针对不同的mipSizeId和modeId，可以通过查找表方式确定出矩阵乘法中所需要使用的移位偏移参数。
表6
Figure PCTCN2020141080-appb-000004
但是在编码器侧，表6也是需要以查找表形式存储入存储器或者存储单元中；然而存储是需要代价的，而且查找过程也是需要代价的；由于表6中移位偏移参数的取值与当前块的块尺寸大小和MIP模式索引值均相关，从而增加了内存占用，而且还增加了计算复杂度。
为了减少内存占用和降低计算复杂度，本申请实施例也可以针对移位偏移参数的确定方式也进行简化。
在另一种可能的实施方式中，移位偏移参数的取值可以设置为一个固定常数，而且是与块尺寸索引值和MIP模式索引值无关的固定常数。通常情况下，该固定常数的取值范围为0～100。例如，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位偏移参数的取值均可以设置为32；或者，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位偏移参数的取值均可以设置为46；或者，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位偏移参数的取值均可以设置为56；或者，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位偏移参数的取值均可以设置为66。在本申请实施例中，优选地，移位偏移参数的取值设置为等于32，但对此不作任何限定。
在又一种可能的实施方式中，对于移位偏移参数的取值来说，该方法还可以包括：
根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；
根据所述当前块的块尺寸索引值，确定所述移位偏移参数的取值。
需要说明的是，根据所述当前块的尺寸参数，可以确定出当前块的块尺寸索引值；然后根据当前块的块尺寸索引值，可以进一步确定出移位偏移参数的取值。
可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位偏移参数的取值，可以包括：
当所述块尺寸索引值分别等于0、1和2时，确定所述当前块的块尺寸索引值对应的所述移位偏移 参数的取值分别等于34、23和46。
可选地，在一些实施例中，还可以最小化移位偏移参数表，并且仍然使用查找表的方式来确定偏移因子。可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位偏移参数的取值，可以包括：
根据所述块尺寸索引值，从第四预设查找表中查询与所述块尺寸索引值对应的取值，其中，所述第四预设查找表用于记录块尺寸索引值与移位偏移参数的取值之间的对应关系；
将查询到的取值确定为所述移位偏移参数的取值。
需要说明的是，可以仅根据当前块的块尺寸索引值(用mipSizeId表示)来查询移位偏移参数的取值。如表7所示的第四预设查找表，针对每一种的块尺寸索引值，可以对应固定的取值，也就是说，每个块的尺寸大小或者每个块的尺寸集合可以具有如表7所示固定的移位偏移参数的取值。
表7
mipSizeId fO
0 34
1 23
2 46
根据表7，当所述当前块的块尺寸索引值分别等于0、1和2时，可以确定出所述块尺寸索引值对应的所述移位偏移参数的取值分别等于34、23和46。
在上述的实施方式中，通过简化移位偏移参数的确定方式，尤其是最小化移位偏移参数表或者固定移位偏移参数的取值，可以实现最小化查找表的存储，从而能够在不增加计算复杂度的情况下，减小MIP模式下移位偏移参数表存储所占用的内存。
如此，在确定出移位偏移参数(fO)和移位数量参数(sW)之后，针对第一常数值而言，可选地，在一些实施例中，所述根据移位数量参数的取值，确定第一常数值，可以包括：
将第一常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述移位数量参数的取值减1。
可选地，在一些实施例中，所述根据移位数量参数的取值，确定第一常数值，可以包括：
对“1”进行二进制比特左移，得到所述第一常数值，其中，所述比特左移的位数等于所述移位数量参数的取值减1。
也就是说，在得到移位数量参数(sW)之后，第一常数值可以表示为1<<(sW-1)，也可以表示为2^(sW-1)。这时候当移位数量参数的取值设置为6时，可以得到第一常数值等于32。
这里，如果第一偏移量用oW表示，根据移位偏移参数(fO)可以计算得到MIP输入采样值的和值与移位偏移参数的乘积，即为
Figure PCTCN2020141080-appb-000005
那么oW的取值可以设置为
Figure PCTCN2020141080-appb-000006
这样，在MIP模式下，可以得到MIP加权矩阵、MIP输入采样值、移位数量参数和第一偏移量，以便后续确定当前块的MIP预测值。
S303-5：根据所述MIP加权矩阵、所述MIP输入采样值、所述移位数量参数和所述第一偏移量，确定所述当前块的色度分量的MIP预测值。
这里，MIP预测值表示所述当前块的色度分量中部分采样点的预测值。
需要说明的是，所述根据所述MIP加权矩阵、所述MIP输入采样值、所述移位数量参数和所述第一偏移量，确定所述当前块的MIP预测值，可以包括：
计算所述MIP加权矩阵与所述MIP输入采样值的第一加权和；
计算所述第一加权和与所述第一偏移量的第一和值；
对所述第一和值进行二进制比特右移，得到第一右移值，其中，所述右移的位数等于所述移位数量参数的取值；
将所述当前块的MIP预测值设置为等于所述第一右移值与所述第一临时参考值中索引序号为0对应的值的和值；其中，所述第一临时参考值是基于对所述当前块的相邻采样值进行下采样滤波处理得到的。
具体来讲，在MIP模式下，可根据当前块的块尺寸索引值(用mipSizeId表示)和MIP模式索引值(用modeId表示)来确定出MIP加权矩阵(用mWeight表示)、移位数量参数(用sW表示)以及移位偏移参数(用fO表示)；然后将MIP输入采样值(用p[x]表示)、mWeight、sW以及fO输入矩阵乘法的过程，以获取矩阵乘法输出的MIP预测值(可以用predMip[x][y]表示)，并且将predMip[x][y]中的采样点按照predSize×predSize排列成矩阵/阵列形式。其中，其计算公 式如下，
Figure PCTCN2020141080-appb-000007
其中，[x][y]表示像素点的位置坐标，x表示水平方向，y表示垂直方向；inSize表示输入采样个数，p[i]表示MIP输入采样值中索引序号为i对应的值；pTemp[0]表示第一临时参考值中索引序号为0对应的值；“>>”表示二进制右移运算符，“<<”表示二进制左移运算符；x＝0,…,predSize-1,y＝0,…,predSize-1。如此，根据上述式(5)，可以计算出predMip[x][y]，以得到MIP预测块。
在一些实施例中，如果sW固定值为6，fO固定值为32，那么这时候的计算公式如下，
Figure PCTCN2020141080-appb-000008
进一步地，还需要对MIP预测块中采样点的预测值进行嵌位处理，以获取当前块的MIP预测块；然后判断是否对该MIP预测块进行转置处理；如果判断结果为是，那么还需要对该MIP预测块中的预测采样值进行转置处理，将转置后的MIP预测块确定为当前块的MIP预测块；如果判断结果为否，那么不需要对该MIP预测块中的预测采样值进行转置处理，可以直接将该MIP预测块确定为当前块的MIP预测块，以得到当前块的MIP预测值，而且该MIP预测值是当前块中部分采样点的预测值。
S304：对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值。
需要说明的是，所述对所述MIP预测值进行滤波处理，确定所述当前块的帧内预测值，可以包括：
判断MIP预测块的尺寸参数与当前块的尺寸参数是否相同；
当所述MIP预测块的尺寸参数与所述当前块的尺寸参数相同时，将所述当前块的帧内预测块设置为等于所述MIP预测块；其中，所述MIP预测块中包含所述当前块中全部像素位置的预测采样值；
当所述MIP预测块的尺寸参数与所述当前块的尺寸参数不相同时，对所述MIP预测块进行滤波处理，得到滤波预测块，将所述滤波预测块设置为所述当前块的帧内预测块。
这里，滤波处理可以包括上采样滤波处理或低通滤波处理。
需要说明的是，MIP预测块是由MIP预测值组成的。在得到MIP预测块之后，由于MIP预测块的尺寸参数只包括有两种：4×4的MIP预测块和8×8的MIP预测块；这样，当前块的尺寸参数与MIP预测块的尺寸参数可能相同，也可能不相同。也就是说，MIP预测块对应的采样值不一定能够填满当前块，使得最终预测值的生成可能需要对MIP预测块进行上采样操作，即通过判断MIP预测块的尺寸参数与当前块的尺寸参数是否相同，以确定是否对MIP预测块进行上采样处理。
还需要说明的是，当MIP预测块的尺寸参数与当前块的尺寸参数相同时，即MIP预测块的宽度和高度均与当前块相同，表明了不需要对MIP预测块进行上采样处理，这时候可以将MIP预测块直接填充进当前块中，即填充后的当前块内没有空余的像素点，此时当前块中每一像素的帧内预测值可以直接置为MIP预测块中每一像素的预测值，如下所示，
predSamples[x][y]＝predMip[x][y]   (7)
其中，[x][y]表示像素点的位置坐标，x表示水平方向，y表示垂直方向；predSamples[x][y]表示当前块中位置坐标[x][y]的像素点所对应的帧内预测值，predMip[x][y]表示MIP预测块中位置坐标[x][y]的像素点所对应的预测值。如此，根据式(7)，可以将MIP预测块predMip[x][y]直接作为当前块的帧内预测块predSamples[x][y]，即当前块内至少一个像素点的帧内预测值。
还需要说明的是，当MIP预测块的尺寸参数与当前块的尺寸参数不相同时，即MIP预测块的宽度和高度中至少之一与当前块不相同，这时候MIP预测块不能够填满当前块，即填充后的当前块内存在有空余的像素点，表明了这时候需要对MIP预测块进行滤波处理。也就是说，如果在水平方向和垂直方向上都需要进行上采样处理，那么可以先对MIP预测块进行水平方向上采样后再进行垂直方向上采样，以得到第一上采样块，可以用predSamples[x][y]表示；然后对MIP预测块进行垂直方向上采样后再进行水平方向上采样，以得到第二上采样块，可以用predSamplesTemp[x][y]表示；最后将predSamples[x][y]和predSamplesTemp[x][y]进行加权均值计算，最终得到当前块的色度分量的帧内预测值，也即得到当前块的帧内预测块。
示例性地，如果当前块的边长nTbS(这里，S可分别被W和H替代)均等于predMip的边长predSize (这里，predSize仅与当前块的blocksizeIdx有关)时，这时候可以直接将MIP预测块置为当前块的帧内预测块；否则，将需要对MIP预测块进行滤波处理，以得到当前块的帧内预测块。这里，以4×4的当前块为例，如果当前块与MIP预测块的尺寸参数相同，这时候是不需要对MIP预测块进行滤波处理，那么可以将MIP预测块直接置为当前块的帧内预测块，从而能够得到当前块内至少一个像素的帧内预测值。
简言之，对于MIP模式的预测过程，其中，MIP模式的输入数据，可以包括：当前块的位置(xTbCmp，yTbCmp)、当前块所应用的MIP模式索引值(可以用modeId表示)、当前块的高度(用nTbH表示)、当前块的宽度(用nTbW表示)以及是否需要转置处理的转置指示标识(即MIP转置指示参数，可以用isTransposed表示)等。MIP模式的输出数据，可以包括：当前块的帧内预测块。该帧内预测块中像素坐标[x][y]所对应的预测值为predSamples[x][y]；其中，x＝0,1,…,nTbW-1；y＝0,1,…,nTbH-1。
如图5所示，MIP预测过程具体包括如下步骤：
S501：配置核心参数。
对于S501来说，MIP核心参数可包括当前块的种类(用mipSizeId表示)、每个边的参考采样个数(用boundySize表示)、MIP输入采样的个数(用inSize表示)以及矩阵乘法输出的MIP预测块大小(排列为predSize×predSize)。其中，根据当前块的大小，可以将当前块划分为三类，用mipSizeId记录当前块的种类。这里，mipSizeId可以等于0、1或2。而且不同种类的当前块，参考采样点数量和矩阵乘法输出的MIP预测块大小是不同的。
S502：获取参考像素。
对于S502来说，预测当前块时，这时候当前块的上相邻块和左相邻块都是已编码的块，MIP模式的参考像素为与当前块相邻的上一行像素和左一列像素的重建值。换句话说，获取当前块上侧边相邻的参考像素(用refT表示)和左侧边相邻的参考像素(用refL表示)的过程即为参考像素的获取过程。
S503：构造输入采样。
对于S503来说，该步骤用于矩阵乘法的输入，具体可包括：
S5031：获取参考采样；
S5032：构造参考采样缓冲区；
S5033：推导矩阵乘法输入采样。
其中，对于S5031来说，获取参考采样的过程为对参考像素进行下采样处理的过程。对于S5032来说，在参考采样缓冲区的构造中，存在有两种填充方式，具体可包括：
S5032-1：不需要转置时缓冲区的填充方式；
S5032-2：需要转置时缓冲区的填充方式。
S504：生成帧内预测值。
对于S504来说，该步骤用于获取当前块的MIP预测值，具体可包括：
S5041：构造矩阵乘法输出采样的MIP预测块；
S5042：对矩阵乘法输出采样的MIP预测块进行嵌位处理；
S5043：对矩阵乘法输出采样的MIP预测块进行转置处理；
S5044：生成MIP预测值。
其中，对于S5041来说，在构造MIP预测块的过程中，具体可包括：
S5041-1：获取权重矩阵；
S5041-2：获取移位参数；
S5041-3：矩阵乘法运算。
也就是说，在构造MIP预测块的过程中，不仅需要获取权重矩阵，又需要获取移位参数(包括：移位偏移参数和移位数量参数)；然后进行矩阵乘法运算，用以得到矩阵乘法输出采样的MIP预测块。而对于S5044来说，生成最终的MIP预测值又包括两种情况，具体可包括：
S5044-1：生成不需要上采样滤波处理的预测值；
S5044-2：生成需要上采样滤波处理的预测值。
这样，在经过上述的S501～S504这四个步骤之后，本申请实施例即可得到当前块的色度分量中至少一个像素的帧内预测值。
S305：根据所述当前块的色度分量的帧内预测值，确定所述当前块的色度分量的预测残差值。
需要说明的是，在得到当前块的色度分量的帧内预测值后，可以对当前块的色度分量的真实值与当前块的色度分量的帧内预测值进行差值计算，进而得到当前块的色度分量的预测残差值，也即得到当前块的残差块。
S306：对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数。
需要说明的是，对于图1中所示的变换单元108来说，变换单元108可以对当前块的残差块进行第一次变换，如基于离散余弦变换(Discrete Cosine Transform，DCT)初始设计的整数变换；还可以对当前块的残差块进行第二次变换，如低频不可分离二次变换(Low-Frequency Non-Separable Transform，LFNST)。具体地，在一些实施例中，所述对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数，可以包括：
根据所述预测残差值确定当前块的残差块，对所述残差块进行第一次变换，得到第一变换系数块；
确定是否对所述第一变换系数块中至少部分变换系数执行LFNST；
在确定结果为是的情况下，对所述第一变换系数块中至少部分变换系数执行LFNST，确定所述LFNST参数。
在本申请实施例中，编码器可以针对残差块中的预测残差值进行第一次变换(也可称之为“Core Transform”或者“一次变换”或者“主变换”)，以得到第一次变换后的第一变换系数块；然后针对第一变换系数块中的部分或者全部变换系数进行LFNST变换(也可称之为“Secondary Transform”或者“二次变换”)，以确定出LFNST参数。需要注意的是，这里的第一次变换为不同于LFNST的变换。
还需要说明的是，并不是任意的当前块都可以执行LFNST。在经过第一次变换后，这时候需要检测(check)一些条件(比如当前块的尺寸参数中的最小值、当前块的块尺寸索引值等)以确定是否对第一变换系数块中至少部分变换系数执行LFNST。可选地，在一些实施例中，所述确定是否对所述第一变换系数块中至少部分变换系数执行LFNST，可以包括：
在当前块的色度分量使用MIP模式确定帧内预测值时，确定所述当前块的尺寸参数中的最小值；
根据所述最小值，确定是否对所述第一变换系数块中至少部分变换系数执行LFNST。
进一步地，所述根据所述最小值，确定是否对所述第一变换系数块中至少部分变换系数执行LFNST，可以包括：
若所述最小值大于或等于第一预设阈值，则确定对所述第一变换系数块中至少部分变换系数执行LFNST。
这里，第一预设阈值可以用于表示预先设定的衡量是否执行LFNST的门限值。在本申请实施例中，第一预设阈值可以设置为8，但是不作具体限定。
也就是说，当对当前块的色度分量使用MIP模式进行帧内预测时，这时候可以确定当前块的尺寸参数中的最小值；比如当宽度和高度的最小值大于或等于8时，此时可以确定对第一变换系数块中至少部分变换系数执行LFNST。
可选地，在一些实施例中，所述确定是否对所述第一变换系数块中至少部分变换系数执行LFNST，可以包括：
在所述当前块的色度分量使用MIP模式确定帧内预测值时，根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；
根据所述块尺寸索引值，确定是否对所述第一变换系数块中至少部分变换系数执行LFNST。
进一步地，所述根据所述块尺寸索引值，确定是否对所述第一变换系数块中至少部分变换系数执行LFNST，可以包括：
若所述块尺寸索引值等于第二预设阈值，则确定对所述第一变换系数块中至少部分变换系数执行LFNST。
这里，第二预设阈值也可用于表示预先设定的衡量是否执行LFNST的门限值。在本申请实施例中，第二预设阈值可以设置为预设值(例如2)，或者多个预设值(例如1、2)之一，但是不作具体限定。
也就是说，当对当前块的色度分量使用MIP模式进行帧内预测时，这时候可以根据当前块的尺寸参数确定当前块的块尺寸索引值(mipSizeId)；比如当mipSizeId的取值等于2时，此时可以确定对第一变换系数块中至少部分变换系数执行LFNST。
可选地，在一些实施例中，所述确定是否对所述第一变换系数块中至少部分变换系数执行LFNST，可以包括：
在不对所述第一变换系数块中至少部分变换系数执行LFNST的情况下，计算第一代价结果；
在对所述第一变换系数块中至少部分变换系数执行LFNST的情况下，计算第二代价结果；
根据所述第一代价结果和所述第二代价结果，确定是否对所述第一变换系数块中至少部分变换系数执行LFNST。
进一步地，所述根据所述第一代价结果和所述第二代价结果，确定是否对所述第一变换系数块中至少部分变换系数执行LFNST，可以包括：
若所述第一代价结果大于所述第二代价结果，则确定对所述第一变换系数块中至少部分变换系数执行LFNST。
需要说明的是，本申请实施例可以采用传统的率失真优化方式，首先在不对第一变换系数块中至少部分变换系数执行LFNST的情况下，计算出第一代价结果；然后在对第一变换系数块中至少部分变换系数执行LFNST的情况下，计算出第二代价结果；将第一代价结果与第二代价结果进行比较，当第一代价结果大于第二代价结果时，此时可以确定对第一变换系数块中至少部分变换系数执行LFNST。
这样，当确定结果为是，即确定需要对第一变换系数块中至少部分变换系数执行LFNST时，这时候可以对第一变换系数块中至少部分变换系数执行LFNST，并确定出LFNST参数。
S307：对所述LFNST参数进行编码，并写入码流。
需要说明的是，LFNST参数包括LFNST索引序号，可以用lfnst_index表示。在一些实施例中，所述确定LFNST参数，可以包括：
若确定对所述第一变换系数块中至少部分变换系数执行LFNST，则设置所述LFNST索引序号的取值为大于零；
若确定不对所述第一变换系数块中至少部分变换系数执行LFNST，则设置所述LFNST索引序号的取值为等于零。
进一步地，对于S307来说，所述对所述LFNST参数进行编码，并写入码流，可以包括：
对所述LFNST索引序号进行编码，并写入码流。
也就是说，在确定LFNST参数，即lfnst_index之后，如果lfnst_index等于0，那么表示不执行LFNST；如果lfnst_index大于0，那么表示需要执行LFNST。在编码过程中，可以使用描述符为“ae(v)”的基于上下文的自适应二进制算术编码(Context-based Adaptive Binary Arithmetic Coding，CABAC)来对lfnst_index进行编码。
示例性地，参见表8，其示出了相关技术提供的一种LFNST参数的语法结构描述示例。这时候确定是否对第一变换系数块中至少部分变换系数块执行LFNST的判断条件，如可以包括下述至少之一：treeType＝＝DUAL_TREE_CHROMA、！IntraMipFlag[x0][y0]、Min(lfnstWidth,lfnstHeight)>＝16。
表8
Figure PCTCN2020141080-appb-000009
参见表9A和表9B，其分别示出了本申请实施例提供的一种LFNST参数的语法结构描述示例。这时候确定是否对第一变换系数块中至少部分变换系数块执行LFNST的判断条件，如可以包括下述至少之一：intra_mip_flag[x0][y0]＝＝1&&Min(lfnstWidth,lfnstHeight)>＝8、intra_mip_flag[x0][y0]＝＝1&&mipSizeId＝＝2。
具体来说，如表9A所示的语法结构，在当前块的尺寸参数中的最小值大于或等于8(即，宽度和高度均大于或等于8)时，这时候可以使用MIP模式(即，intra_mip_flag[x0][y0]＝＝1时)对当前块的lfnst_index进行编码。或者，该判断条件也可以设置为：宽度和高度均大于或等于第一值(例如8)，且宽度和高度之和大于或等于第二值(例如16)。如表9B所示的语法结构，在当前块的块尺寸索引值(mipSizeId)等于2时，这时候可以使用MIP模式(即，intra_mip_flag[x0][y0]＝＝1时)对当前块的lfnst_index进行编码。或者，该判断条件也可以设置为：mipSizeId等于多个预设值(例如1、2)之一。
表9A
coding_unit(x0,y0,cbWidth,cbHeight,cqtDepth,treeType，modeType){ Descriptor
……  
if(…&&(intra_mip_flag[x0][y0]＝＝1&&Min(lfnstWidth,lfnstHeight)>＝8)&&…){  
……  
lfnst_idx ae(v)
}  
……  
}  
表9B
coding_unit(x0,y0,cbWidth,cbHeight,cqtDepth,treeType，modeType){ Descriptor
……  
if(…&&(intra_mip_flag[x0][y0]＝＝1&&mipSizeId＝＝2)&&…){  
……  
lfnst_idx ae(v)
}  
……  
}  
还需要说明的是，在确定当前块可以执行LFNST时，这时候还需要确定当前块使用的LFNST变换核(用kernel表示)。在一些实施例中，所述对所述预测残差值执行LFNST，确定LFNST参数，可以包括：确定当前块使用的LFNST变换核；利用所述LFNST变换核对所述预测残差值执行LFNST，确定LFNST参数。
在本申请实施例中，LFNST中共有4个变换核候选集，这四个变换核候选集可以包括有set0、set1、set2和set3。这里，本申请实施例可以利用MIP参数来确定LFNST变换核候选集，然后从LFNST变换核候选集中选择当前块使用的LFNST变换核。因此，在一些实施例中，该方法还可以包括：
在当前块的色度分量使用MIP模式确定帧内预测值时，从当前块的预测参数中确定MIP参数；
根据所述MIP参数，确定LFNST变换核候选集；
在所述LFNST变换核候选集中，将所述LFNST索引序号指示的变换核设置为所述当前块使用的LFNST变换核；
将所述LFNST索引序号的取值设置为指示对所述当前块执行LFNST、且所述LFNST变换核在所述LFNST变换核候选集中的索引序号；
其中，所述LFNST变换核候选集包含预设的两个或多个LFNST变换核。
需要说明的是，MIP参数可以包括下述至少之一：MIP模式索引值(用modeId表示)和MIP转置指示参数(用isTransposed表示)。其中，针对当前块使用的LFNST变换核，当所述MIP转置指示参数的取值指示对MIP模式使用的采样点输入向量进行转置处理时，这时候还需要对所述选择的变换核进行矩阵转置处理，以得到所述当前块使用的LFNST变换核。
进一步地，在一些实施例中，所述根据所述MIP参数，确定LFNST变换核候选集，可以包括：
根据所述MIP模式索引值，确定LFNST帧内预测模式索引序号的取值；
根据所述LFNST帧内预测模式索引序号的取值，通过第五预设查找表确定LFNST变换核候选集的索引值；
根据所述LFNST变换核候选集的索引值，从多个候选LFNST变换核候选集中选择所述LFNST变换核候选集。
这里，LFNST帧内预测模式索引序号可以用predModeIntra表示，LFNST变换核候选集的索引值可以用lfnstTrSetIdx表示。
需要说明的是，第五预设查找表如表10所示。根据MIP模式索引值(即modeId的取值)，可以确定出predModeIntra的取值；然后根据predModeIntra的取值，可以结合表10直接确定出lfnstTrSetIdx的取值，即确定出当前块所选择的LFNST变换核候选集。这里，lfnstTrSetIdx的取值指示了LFNST时所使用的变换核候选集；由于modeId的取值可以包括有0、1、2、3、4、5，那么predModeIntra的取值也为0、1、2、3、4、5；其与lfnstTrSetIdx的对应关系如下，
表10
predModeIntra lfnstTrSetIdx
0<＝predModeIntra<＝2 0
predModeIntra＝＝3 2
predModeIntra＝＝4 1
predModeIntra＝＝5 3
在一种具体的实施例中，所述根据所述MIP模式索引值，确定LFNST帧内预测模式索引序号的取值，可以包括：将LFNST帧内预测模式索引序号的取值设置为等于所述MIP模式索引值。
也就是说，本申请实施例可以将predModeIntra的取值设置为等于modeId的取值，然后根据predModeIntra的取值，结合表10直接确定出lfnstTrSetIdx的取值，即确定出当前块所选择的LFNST变换核候选集。还需要注意的是，在另一种具体的实施例中，本申请实施例还可以将modeId的取值直接映射为PLANAR模式的取值，然后使用PLANAR模式所对应的predModeIntra的取值来确定出 lfnstTrSetIdx的取值，即确定出当前块所选择的LFNST变换核候选集。
在一些实施例中，所述LFNST参数还可以包括LFNST系数。所述确定LFNST参数，可以包括：
对所述第一变换系数块中至少部分变换系数执行LFNST，得到所述LFNST系数；
相应地，所述对所述LFNST参数进行编码，并写入码流，包括：
对所述LFNST系数进行量化，得到量化系数；
对所述量化系数进行编码，并写入码流。
需要说明的是，当lfnst_index不等于0，即确定对当前块执行LFNST时，这时候通过对第一变换系数块中至少部分变换系数执行LFNST，可以得到所述LFNST系数；然后对LFNST系数进行量化和编码，并写入码流(bitstream)。
需要说明的是，假定d[x][y]是第一变换系数块，其中，x＝0,...,nTbW-1，y＝0,…,nTbH-1，其中nTbW和nTbH分别表示当前块的宽度和高度。假定v[x]是表示包括LFNST系数的块，其中，x＝0,…,nLfnstOutSize-1，这里的nLfnstOutSize＝(nTbW>＝8&&nTbH>＝8)？48:16。
在一种可能的实施方式中，可以直接利用水平扫描顺序对第一变换系数块中至少部分变换系数执行LFNST，以得到LFNST系数。具体如下，
Figure PCTCN2020141080-appb-000010
其中，log2LfnstSize的确定如下：
Log2LfnstSize＝(nTbW>＝8&&nTbH>＝8)？3:2   (9)
在另一种可能的实施方式中，扫描顺序可以包括水平扫描顺序和垂直扫描顺序。这里，扫描顺序的确定与MIP转置指示参数(用isTransposed表示)的取值有关。
可选地，在一些实施例中，所述对所述第一变换系数块中至少部分变换系数执行LFNST，得到所述LFNST系数，可以包括：
当所述MIP转置指示参数的取值指示对MIP模式使用的采样点输入向量进行转置处理时，根据垂直扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，得到所述LFNST系数；
当所述MIP转置指示参数的取值指示不对MIP模式使用的采样点输入向量进行转置处理时，根据水平扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，得到所述LFNST系数。
具体来讲，当isTransposed的取值等于0时，这时候不对MIP模式使用的采样点输入向量进行转置处理，那么可以根据水平扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，具体如下，
Figure PCTCN2020141080-appb-000011
而当isTransposed的取值等于1时，这时候对MIP模式使用的采样点输入向量进行转置处理，那么可以根据垂直扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，具体如下，
Figure PCTCN2020141080-appb-000012
其中，log2LfnstSize的确定如上述式(9)所示。
可选地，在一些实施例中，所述对所述第一变换系数块中至少部分变换系数执行LFNST，得到所述LFNST系数，可以包括：
当所述MIP转置指示参数的取值指示对MIP模式使用的采样点输入向量进行转置处理时，根据水平扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，得到所述LFNST系数；
当所述MIP转置指示参数的取值指示不对MIP模式使用的采样点输入向量进行转置处理时，根据垂直扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，得到所述LFNST系数。
具体来讲，当isTransposed的取值等于1时，这时候对MIP模式使用的采样点输入向量进行转置处理，那么可以根据水平扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，具体如上述式(10)所示。
而当isTransposed的取值等于0时，这时候不对MIP模式使用的采样点输入向量进行转置处理，那么可以根据垂直扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，具体如上述式(11)所示。
其中，log2LfnstSize的确定如上述式(9)所示。
除此之外，本申请实施例中的LFNST，除了可以应用于MIP模式之外，还可以应用于传统帧内预测模式(例如，角度帧内预测模式、DC模式和PLANAR模式等)。其具体的实现方式与MIP模式的实 现方式类似。
在一些实施例中，所述根据所述MIP参数，确定LFNST变换核候选集，可以包括：
根据传统帧内预测模式的索引值，确定LFNST帧内预测模式索引序号的取值；
根据所述LFNST帧内预测模式索引序号的取值，通过第六预设查找表确定LFNST变换核候选集的索引值；
根据所述LFNST变换核候选集的索引值，从多个候选LFNST变换核候选集中选择所述LFNST变换核候选集。
需要说明的是，第六预设查找表如表11所示。根据传统帧内预测模式的索引值，可以确定出predModeIntra的取值；然后根据predModeIntra的取值，可以结合表11直接确定出lfnstTrSetIdx的取值，即确定出当前块所选择的LFNST变换核候选集。这里，可以将传统帧内预测模式的索引值分配给predModeIntra，用以确定出lfnstTrSetIdx的取值。
表11
predModeIntra lfnstTrSetIdx
predModeIntra<0 1
0<＝predModeIntra<＝1 0
2<＝predModeIntra<＝12 1
13<＝predModeIntra<＝23 2
24<＝predModeIntra<＝44 3
45<＝predModeIntra<＝55 2
56<＝predModeIntra<＝80 1
下面将结合图1所示的编码器100对本申请实施例的编码方法进行具体说明。
应理解，本申请实施例中的当前块(或称为“编码块”)可以是CU或者CU的分区(例如，变换块)。具体地，当帧内预测单元106确定MIP模式用于对当前块进行编码时，或者当帧内预测单元106使用率失真优化方式评价用于对当前块进行编码的MIP模式时，帧内预测单元106将获取当前块的帧内预测块；其中，当前块的帧内预测块包括当前块内至少一个像素(samples)的帧内预测值。
具体地，对于帧内预测单元106而言，其使用MIP模式获取帧内预测块，步骤如下：
首先，帧内预测单元106从当前块的相邻像素点获取一个或多个参考像素点。例如通过对相邻像素点进行下采样获取，或者直接从相邻像素点中提取。
然后，帧内预测单元106使用所获取的参考像素点、MIP矩阵和移位参数以确定当前块内的像素点位置对应的一个或多个部分预测像素点。这里，像素点位置可以是当前块中的预设像素点位置。例如，像素点位置具有均匀的水平和垂直坐标值。移位参数包括移位数量参数和移位偏移参数，而且移位参数可以在获取部分像素点的帧内预测值的过程中用于偏移操作。
最后，如果获取到当前块的部分像素点对应的预测像素点，那么帧内预测单元106还需要获取当前块中除了部分像素点之外的剩余像素点对应的预测像素点。例如，帧内预测单元106可以使用插值滤波器获取剩余像素点对应的预测像素点，这里插值滤波器的输入可以是部分像素点和相邻像素点。
如图6所示，其示出了本申请实施例中一种使用MIP模式获取帧内预测块的流程框图示例。其中，该流程可在编码器100上实现，这里的“当前块”指的是“编码块”。
步骤601，帧内预测单元106获取当前块的相邻像素点，例如相邻像素点被标记为与图6所示步骤601的当前块相邻的灰色填充正方形。帧内预测单元106从相邻像素点获取一个或多个参考像素点。在图6所示步骤601的示例中，可选地，帧内预测单元106可计算两个相邻像素点的平均值并将该平均值用作参考像素点。可选地，帧内预测单元106每隔一个相邻像素点就选择一个相邻像素点作为参考像素点。例如，在图6所示步骤601的示例中，帧内预测单元106从当前块的8个上侧相邻的像素点中选取4个参考像素点，并从当前块的8个左侧相邻的像素点中选取另外4个参考像素点。
步骤601的具体过程如下：
帧内预测单元106从块分割单元103获得当前块的宽度和高度，宽度和高度分别由变量cbWidth和cbHeight表示。作为示例，帧内预测单元206调用率失真优化的模式确定方法以确定帧内预测模式，其中当前块被分成一个或多个变换块。令变量nTbW和nTbH分别为变换块的宽度和高度。当MIP模式作为帧内预测模式以获得当前块的帧内预测值时，帧内预测单元106确定MIP的块尺寸索引值，即表示为mipSizeId的变量。
可选地，帧内预测单元106如下确定mipSizeId的取值：
–如果nTbW和nTbH都等于4，则mipSizeId设置为等于0；
–否则，如果cbWidth或cbHeight等于4，则mipSizeId设置为等于1；
–否则，mipSizeId设置为等于2。
具体地，在当前块的尺寸参数为8×8(即cbWidth和cbHeight都等于8)时，mipSizeId设置为等于2。
可选地，帧内预测单元106如下确定mipSizeId的取值：
–如果nTbW和nTbH都等于4，则mipSizeId设置为等于0；
–否则，如果cbWidth或cbHeight等于4，或者cbWidth和cbHeight都等于8，则mipSizeId设置为等于1；
–否则，mipSizeId设置为等于2。
具体地，在当前块的尺寸参数为8×8时，mipSizeId设置为等于1。
进一步地，帧内预测单元106根据mipSizeId获得boundarySize和predSize的取值，具体如下：
–如果mipSizeId等于0，则boundarySize设置为等于2，predSize设置为等于4；
–否则，如果mipSizeId等于1，则boundarySize设置为等于4，predSize设置为等于4；
–否则(如果mipSizeId等于2)，boundarySize设置为等于4，predSize设置为等于8；
其中，boundarySize表示从当前块上侧边相邻的参考像素和左侧边相邻的参考像素中每一边获得的参考像素数量。
帧内预测单元106还可以获得变量isTransposed以指示存储在缓冲区pTemp中的参考像素顺序。例如，isTransposed等于0表示：帧内预测单元106先呈现从当前块上侧边相邻的参考像素获得的参考像素，然后呈现从左侧边相邻的参考像素获得的参考像素；否则，isTransposed等于1表示：帧内预测单元106先呈现从当前块左侧边相邻的参考像素获得的参考像素，然后呈现从上侧边相邻的参考像素获得的参考像素。这里，isTransposed的值将发送到熵编码单元115，作为需要被编码并写入码流的MIP模式的参数之一。
帧内预测单元106获得inSize的取值以指示在使用MIP模式时所使用的参考像素数量，如上述式(1)所示。
帧内预测单元106调用如下过程，以使用当前块相邻的参考像素来获得一组参考像素(存储为阵列p[x]，其中x为0,…,inSize–1)。
帧内预测单元106从当前块上侧边相邻的参考像素(例如，存储在阵列refT中)获得nTbW参考像素，并从当前块左侧边相邻的参考像素(例如，存储在阵列refL中)获得nTbH参考像素。帧内预测单元106对refT调用下采样处理以获得boundarySize参考像素，并将boundarySize参考像素存储在refT中。帧内预测单元106对refL调用下采样处理以获得boundarySize参考像素，并将boundarySize参考像素存储在refL中。
帧内预测单元106还可获得变量isTransposed以指示存储在缓冲区pTemp中的参考像素顺序。例如，isTransposed等于0(或FALSE)表示：帧内预测单元106先呈现从当前块上侧边相邻的参考像素获得的参考像素，然后呈现从左侧边相邻的参考像素获得的参考像素；否则，isTransposed等于1(或TRUE)表示：帧内预测单元106先呈现从当前块左侧边相邻的参考像素获得的参考像素，然后呈现从上侧边相邻的参考像素获得的参考像素。帧内预测单元106可以使用率失真优化方式或基于相邻参考像素与当前块之间的相关性比较来确定isTransposed的取值。isTransposed的取值将发送到熵编码单元115，作为需要被编码并写入码流的MIP模式的参数之一。
帧内预测单元106根据由isTransposed指示的顺序以将refT和refL中的元素放入缓冲区pTemp中。
可选地，帧内预测单元106获得p[x]，其中x＝0,…,inSize–1，如下：
–如果mipSizeId等于2，则p[x]＝pTemp[x+1]-pTemp[0]；
–否则(mipSizeId小于2)，p[0]＝pTemp[0]-(1<<(BitDepth-1))，p[x]＝pTemp[x]-pTemp[0]；
其中，BitDepth是当前块中像素的颜色分量的比特深度。这里，颜色分量可以是RGB分量中的一个、或者YUV分量中的一个、或者YCbCr分量中的一个，例如，Y分量等。
可选地，帧内预测单元106可获得p[x]，其中x＝0,…,inSize–1，如下：
–如果mipSizeId等于2，则p[x]＝pTemp[x+1]-pTemp[0]；
–否则(mipSizeId小于2)，p[0]＝(1<<(BitDepth-1))-pTemp[0]，p[x]＝pTemp[x]-pTemp[0]。
可选地，帧内预测单元106可使用统一的计算方式来获得p[x]的值，而无需判断mipSizeId的取值。例如，在缓冲区pTemp中追加(1<<(BitDepth-1))作为附加元素，这时候帧内预测单元106将p[x]计算为pTemp[x+1]-pTemp[0]。
步骤602，帧内预测单元106使用一组参考像素和MIP加权矩阵获得当前块的MIP预测值。其中，MIP加权矩阵根据与MIP模式相对应的MIP模式索引值(表示为ModeId)和MIP的块尺寸索引值(表 示为mipSizeId)从一组预定义的MIP加权矩阵中选择。
帧内预测单元106获取当前块内一个或多个像素点位置对应的部分预测像素点的MIP预测值(表示为predMip[x][y])。在图6所示步骤602的示例中，部分预测像素点是当前块中标记为灰色填充正方形的像素点。预测模块601的输入是步骤601中获取的参考像素p[x]，预测模块601使用MIP加权矩阵和移位参数计算部分预测像素点；这里，移位参数包括移位数量参数和移位偏移参数。
在一种可能的实现方式中，预测模块601可以使其坐标等于(x,y)而获取预测像素点用predMip[x][y]表示，那么predMip[x][y]的计算公式如式(5)或式(6)所示。
在式(5)或式(6)中，mWeight[i][j]是MIP加权矩阵；这里，矩阵元素可以是预定常数值；或者，也可以使用比如训练方式自适应地更新，该训练方式的输入是一个或多个编码图片或块，或者来自外部装置提供给编码器100的其它码流中的图片；fO是用于确定oW的移位偏移参数；sW是移位数量参数；p[i]是使用参考像素点来计算得到的MIP输入采样值；pTemp[0]表示第一临时参考值中索引序号为0对应的值；“>>”是如VVC中定义的二进制右移运算符。当最终确定MIP模式以对当前块进行帧内预测时，帧内预测单元106还可以将mWeight[i][j]发送到熵编码单元115。熵编码单元115可将mWeight[i][j]写入码流中的一个或多个数据单元。
预测模块601可根据当前块的尺寸和用于当前块的MIP模式确定sW和fO的值。在一个示例中，预测模块601使用查找表获取sW和fO的值。
可选地，预测模块601可以使用上述的表4，根据当前块的尺寸参数和MIP模式确定sW。
可选地，预测模块601还可以使用上述的表5，根据当前块的尺寸参数确定sW。
可选地，预测模块601还可以直接将sW设置为常数值。例如，对于各种尺寸参数的块和不同的MIP模式，预测模块601可以将sW设置为5；或者，对于各种尺寸参数的块和不同的MIP模式，预测模块601将sW设置为6；或者，对于各种尺寸参数的块和不同的MIP模式，预测模块601将sW设置为7。
可选地，预测模块601可以使用上述的表6，根据当前块的尺寸参数和MIP模式确定fO。
可选地，预测模块601还可以使用上述的表7，根据当前块的尺寸参数确定fO。
可选地，预测模块601还可以直接将fO设置为常数值(如0～100)。例如，对于各种尺寸参数的块和不同的MIP模式，预测模块601将fO设置为32；或者，预测模块601将fO设置为46；或者，预测模块601将fO设置为56；或者，预测模块601将fO设置为66。
帧内预测单元106可以对predMip中的MIP预测值执行嵌位处理操作。当isTransposed等于1(或TRUE)时，predSize×predSize的阵列predMip[x][y](其中x＝0,...,predSize-1，y＝0,...,predSize-1)转换为predTemp[y][x]＝predMip[x][y]，然后predMip＝predTemp。
示例性，可选地，在当前块的尺寸参数为8×8(即cbWidth和cbHeight都等于8)时，帧内预测单元106确定mipSizeId等于2并获得8×8的predMip。
可选地，在当前块的尺寸参数为8×8(即cbWidth和cbHeight都等于8)时，帧内预测单元106确定mipSizeId等于1并获得4×4的predMip。
步骤603，帧内预测单元106获取当前块的帧内预测值(作为predSamples[x][y]存储在阵列中，其中x＝0,...,nTbW-1，y＝0,...,nTbH-1)，如下：
–如果帧内预测单元106确定nTbW大于predSize或者nTbH大于predSize，则帧内预测单元106调用上采样过程以使用predMip获得predSamples。帧内预测单元106获得当前块中除了部分像素点之外的剩余像素点对应的预测像素点。在图6中，帧内预测单元106可以使用滤波模块602获取当前块中除了部分像素点之外剩余像素点对应的预测像素点。滤波模块602的输入是步骤602中标记为灰色填充正方形的像素点。滤波模块602可使用一个或多个插值滤波器，利用该输入获取当前块中除了部分像素点之外剩余像素点对应的预测像素点。例如，输入可包括参考像素点和当前块中的一个或多个像素点位置的部分预测像素点；或者，输入可包括相邻像素点和当前块中的一个或多个像素点位置的部分预测像素点；或者，输入可包括参考像素点、相邻像素点和当前块中的一个或多个像素点位置的部分预测像素点。例如，可选地，在当前块的尺寸为8×8(即cbWidth和cbHeight都等于8)时，帧内预测单元106确定mipSizeId等于1，帧内预测单元106将上采样过程应用于4×4 predMip，以获得当前块的8×8的帧内预测块。
–否则，帧内预测单元106将当前块的帧内预测块设置为等于当前块的MIP预测块，即，设置predSamples[x][y](其中x＝0,...,nTbW-1，y＝0,...,nTbH-1)等于predMip[x][y]。例如，具体地，在当前块的尺寸参数为8×8(即cbWidth和cbHeight都等于8)时，帧内预测单元106确定mipSizeId等于2，帧内预测单元106获得当前块的predSamples，其尺寸等于8×8(即cbWidth和cbHeight都等于8)。
如此，根据图6所示的流程框图，在步骤603之后，帧内预测单元106可以获取当前块(即CU) 的帧内预测块，即确定出当前块内至少一个像素的帧内预测值。
预测单元102输出当前块的帧内预测块。第一加法器107计算分割单元101的输出中的当前块和当前块的帧内预测块之间的差值，即残差块(即残差CU)。变换单元108读取残差块并对残差块执行一个或多个变换操作以获得系数。量化单元109对系数进行量化并输出量化系数(即levels)。
变换单元108对残差块执行第一次变换，例如，最初基于DCT设计的整数变换。变换单元108确定是否允许将二次变换应用于该块。当确定允许应用时，变换单元108进一步确定是否将二次变换应用于在执行第一次变换之后获得的系数。LFNST是二次变换的示例。
参见图7，其示出了本申请实施例提供的一种执行LFNST的流程框架示意图。在图7中，块701表示残差块。
步骤701，变换单元108对块701执行第一次变换(即核心变换)，例如，最初基于DCT设计的整数变换，得到包含对应于第一次变换的变换系数(称为“第一变换系数”)的块702。
步骤702，变换单元108确定是否允许对块702中的全部或部分第一变换系数执行二次变换，例如LFNST。
示例性，可选地，如果包含LFNST的变换块的帧内预测模式为MIP模式，且变换块的宽度和高度的最小值大于或等于8，那么允许变换单元108对变换块执行LFNST。
可选地，如果包含LFNST的变换块的帧内预测模式为MIP模式，且mipSizeId的值等于预设值(例如2)或多个预设值(例如1、2)之一，则允许变换单元108对变换块执行LFNST。
当允许变换单元108对变换块执行LFNST时，变换单元108确定LFNST参数，比如lfnst_index。当lfnst_index等于0时，表示变换单元108不对变换块执行LFNST。当lfnst_index大于0时，表示利用LFNST变换核候选集中的变换核对变换块执行LFNST。
变换单元108可采用传统的率失真优化方式来确定lfnst_index的取值。首先，变换单元108在不对变换块执行LFNST的情况下获得第一代价结果。
变换单元108根据MIP参数选择LFNST变换核候选集。MIP参数可包括至少之一：MIP模式索引值(即modeId)和MIP转置指示参数(即isTransposed)。
变换单元108将变量predModeIntra的取值设置为等于modeId的取值。变换单元108在块702中选择第一变换系数的一部分，例如，子块7001、7002和7003中的系数，并确定用于对子块上的系数执行LFNST的变换核。子块的尺寸可以是预设值，例如8×8。变换单元108根据predModeIntra的取值确定LFNST变换核候选集的索引值(即lfnstTrSetIdx)。对于MIP模式下的块编码，变换单元108可以采用如表10的查找表方式来获得lfnstTrSetIdx的取值。
变换单元108可采用传统的率失真优化方式来确定lfnst_index的取值(即，该值大于0表示：由lfnstTrSetIdx指示LFNST变换核候选集中使用的变换核)。变换单元108将lfnst_index的取值设置为变换核在LFNST变换核候选集中的索引序号，这样致使代价函数具有最小值，且还获得第二代价结果。
如果第一代价结果大于第二代价结果，则变换单元108确定对变换块执行LFNST，并将指示变换核的索引序号的lfnst_index发送到熵编码单元115。否则，如果变换单元108确定不对变换块执行LFNST，则将lfnst_value设置为等于0，并将指示不执行LFNST的lfnst_index发送到熵编码单元115。
当lfnst_index不等于0时，变换单元108使用来自子块7001、7002和7003的系数对变换块执行LFNST，获得LFNST系数并将LFNST系数放入块703的子块7011、7012和7013中。假定d[x][y]是第一变换系数的块，其中x＝0,...,nTbW-1，y＝0,...,nTbH-1，其中nTbW和nTbH分别是块703的宽度和高度。假定v[x]是LFNST系数的块，其中x＝0,...,nLfnstOutSize–1，而nLfnstOutSize＝(nTbW>＝8&&nTbH>＝8)？48:16。
可选地，变换单元108采用如下指令得到变换系数的块703，并将d[x][y]中LFNST系数的块703传送到量化单元109，具体如上述式(8)所示。其中，log2LfnstSize的如上述式(9)所示。
可选地，当isTransposed等于0时，变换单元108采用如下指令得到LFNST系数的块703，并将d[x][y]中LFNST系数的块703传送到量化单元109，具体如上述式(10)所示；
而当isTransposed等于1时，变换单元108采用如下指令得到LFNST系数的块703，并将d[x][y]中LFNST系数的块703传送到量化单元109，具体如上述式(11)所示。
可选地，当isTransposed等于1时，变换单元208采用如下指令得到LFNST系数的块703，并将d[x][y]中LFNST系数的块703传送到量化单元109，具体如上述式(10)所示；
而当isTransposed等于0时，变换单元208采用如下指令得到LFNST系数的块703，并将d[x][y]中LFNST系数的块703传送到量化单元109，具体如上述式(11)所示。
其中，log2LfnstSize的确定如上述式(9)所示。
除此之外，变换单元108还可给在传统帧内预测模式(即，角度帧内预测模式，DC模式和PLANAR模式等)中编码的变换块确定lfnst_index的取值。类似于在MIP模式下确定变换块的lfnst_index的取值的率失真优化方式，这时候在不对变换块执行LFNST的情况下确定第一代价结果。
变换单元108使用查找表方式根据传统帧内预测模式的模式索引值(分配给predModeIntra)来确定lfnstTrSetIdx的取值，即确定出当前块所选择的LFNST变换核候选集，具体如上述表11所示。
然后，变换单元108可采用传统的率失真优化方式确定lfnst_index的取值(即，该值大于0表示：由lfnstTrSetIdx指示LFNST变换核候选集中使用的变换核)。变换单元108将lfnst_index的取值设置为变换核在LFNST变换核候选集中的索引序号，这样致使代价函数具有最小值，且还获得第二代价结果。
如果第一代价结果大于第二代价结果，则变换单元108确定对变换块执行LFNST，并将指示变换核的索引序号的lfnst_index发送到熵编码单元115。否则，如果变换单元108确定不对变换块执行LFNST，则将lfnst_value设置为等于0，并将指示不执行LFNST的lfnst_index发送到熵编码单元115。
当lfnst_index不等于0时，变换单元108使用来自子块7001、7002和7003的系数对变换块执行LFNST，获得LFNST系数并将LFNST系数放入块703的子块7011、7012和7013中。假定d[x][y]是第一变换系数的块，其中x＝0,...,nTbW-1，y＝0,...,nTbH-1，其中nTbW和nTbH分别是块703的宽度和高度。假定v[x]是LFNST系数的块，其中x＝0,...,nLfnstOutSize–1，而nLfnstOutSize＝(nTbW>＝8&&nTbH>＝8)？48:16。
可选地，当predModeIntra小于或等于34时，变换单元108采用如下指令得到LFNST系数的块703，并将d[x][y]中LFNST系数的块703传送到量化单元109，具体如上述式(10)所示；
而当predModeIntra大于34时，变换单元108采用如下指令得到LFNST系数的块703，并将d[x][y]中LFNST系数的块703传送到量化单元109，具体如上述式(11)所示。
可选地，当predModeIntra大于34时，变换单元108采用如下指令得到LFNST系数的块703，并将d[x][y]中LFNST系数的块703传送到量化单元109，具体如上述式(10)所示；
而当predModeIntra小于或等于34时，变换单元108采用如下指令得到LFNST系数的块703，并将d[x][y]中LFNST系数的块703传送到量化单元109，具体如上述式(11)所示。
其中，log2LfnstSize的确定如上述式(9)所示。
进一步地，反量化单元110对量化系数执行缩放操作以输出重构系数。逆变换单元111执行对应于变换单元108中的变换的一个或多个逆变换并输出重构残差。第二加法器112通过使重构残差和来自预测单元102的当前块的帧内预测块相加而计算出重构CU。第二加法器112还将其输出发送到预测单元102以用作帧内预测参考。
滤波单元113的输出是解码图片或子图片，这些解码图片或子图片发送到DPB单元114。DPB单元114根据时序和控制信息输出解码图片。其中，存储在DPB单元114中的图片还可用作预测单元102执行帧间预测或帧内预测的参考。
熵编码单元115将来自编码器100中的单元的参数(获得解码图片所必需的)以及控制参数和补充信息转换成二进制表示，并根据每个数据单元的语法结构将这样的二进制表示写入生成的码流(或称为“视频比特流”)，具体如表9A和表9B所示。
使用表9A中的语法结构，当该编码单元中变换块的宽度和高度的最小值大于或等于8(即，宽度和高度都大于或等于8)时，熵编码单元115使用MIP模式对编码单元的lfnst_index进行编码。另一可选条件可以设置为：宽度和高度大于或等于第一值(例如8)，且宽度和高度之和大于或等于第二值(例如16)。
使用表9B中的语法结构，当编码单元的mipSizeId等于2时，熵编码单元115使用MIP模式对编码单元的lfnst_index进行编码。另一可选条件可以设置为：编码单元的mipSizeId等于多个值(例如1和2)之一。
参见图8，其示出了本申请实施例提供的一种对LFNST参数进行编码的流程示意图。如图8所示，该流程可以包括：
S801：熵编码单元115检查上述条件以确定是否对LFNST参数进行编码。
S802：熵编码单元115采用基于上下文的自适应二进制算术编码对LFNST参数进行编码。
需要说明的是，在步骤S801中，熵编码单元115检查确定LFNST参数，如lfnst_index的取值；然后在步骤S802中，可以采用如表9A和表9B所示的描述符为“ae(v)”的CABAC来对lfnst_index的取值进行编码。
在本申请实施例中，编码器100可以是具有处理器和记录编码程序的存储介质的计算设备。当处理器读取并运行编码程序时，编码器100读取输入视频并生成对应的码流。另外，编码器100也可以是具 有一个或多个芯片的计算设备。在芯片上实现为集成电路的单元通过与图1中的对应单元类似的连接以及数据交换而具有类似的功能。
本实施例提供了一种编码方法，通过确定当前块的预测参数，其中，所述预测参数包括预测模式参数；当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；根据所述当前块的色度分量的帧内预测值，确定所述当前块的色度分量的预测残差值；对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；对所述LFNST参数进行编码，并写入码流。这样，针对MIP模式，能够在保证编码性能的基础上，降低复杂度，同时减小编码过程中所需要的存储空间，有效地提高编码效率；另外，LFNST技术在应用于MIP模式预测时，由于引入了MIP参数，使得LFNST变换更加灵活，进一步提高了编码效率。
本申请另一实施例中，参见图9，其示出了本申请实施例提供的一种解码方法的流程示意图。如图9所示，该方法可以包括：
S901：解析码流，获取当前块的预测参数和LFNST参数，其中，所述预测参数包括预测模式参数。
需要说明的是，待解码图像可以划分为多个图像块，每个待解码的图像块可以称为解码块。这里，每个解码块可以包括第一图像分量、第二图像分量和第三图像分量；而当前块为视频图像中当前待进行第一图像分量、第二图像分量或者第三图像分量预测的解码块。
还需要说明的是，预测参数中可以包括有预测模式参数，其中，预测模式参数用于指示当前块采用的预测模式，而且不同的预测模式对应不同的预测模式参数。其中，预测模式通常包括有帧间预测模式、传统帧内预测模式和非传统帧内预测模式等，而传统帧内预测模式可以包括有DC模式、PLANAR模式和角度帧内预测模式等，非传统帧内预测模式可以包括有MIP模式、CCLM模式、IBC模式和PLT模式等。也就是说，编码器会选取最优的预测模式对当前块进行编码，在这过程中可以确定出当前块的预测模式，从而对应得到预测模式参数，然后将包括预测模式参数在内的预测参数写入码流，由编码器传输到解码器。
这样，在解码器中，通过解析码流可以获取到当前块的预测参数，而根据解析获取的预测参数内所包括的预测模式参数可以用来确定当前块是否使用MIP模式。
S902：当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值。
需要说明的是，对于当前块而言，本申请实施例是利用MIP模式对当前块的色度分量进行帧内预测。在这过程中，首先需要获取当前块的相邻采样值；然后根据当前块的相邻采样值，用以确定出当前块的MIP输入采样值。
其中，当前块的相邻采样值可以包括当前块的左相邻采样值和当前块的上相邻采样值。也就是说，可以根据当前块的左相邻采样值和上相邻采样值，用以确定出当前块的MIP输入采样值。
在一些实施例中，所述根据当前块的相邻采样值确定当前块的MIP输入采样值，可以包括：
根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；
对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；
当所述当前块的块尺寸索引值处于预设范围之内时，根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第二常数值与所述第一临时参考值中索引序号为0对应的值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数；
当所述当前块的块尺寸索引值处于预设范围之外时，将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第一临时参考值中索引序号等于j+1对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
进一步地，当所述当前块的块尺寸索引值处于预设范围之内时，该方法还可以包括：
根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第一临时参考值中索引序号为0对应的值与所述第二常数值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数。
需要说明的是，将当前块的相邻采样值进行下采样滤波处理之后，可以得到第一临时参考值。这里， 对于第一临时参考值，具体来讲，可以将滤波处理后的相邻采样值缓存到缓冲区(用pTemp表示)中。其中，第一临时参考值中索引序号为0对应的值即指pTemp[0]，第一临时参考值中索引序号为i对应的值即指pTemp[i]。
在本申请实施例中，预测参数除了预测模式参数之外，还可以包括：当前块的尺寸参数。其中，当前块的尺寸参数可包括：当前块的宽度(用nTbW表示)和高度(用nTbH表示)。而且根据当前块的尺寸参数，可以确定出当前块的块尺寸索引值(即mipSizeId)。
还需要说明的是，根据当前块的块尺寸索引值(用mipSizeId表示)是否处于预设范围之内，可以确定出当前块的尺寸参数取值是否在预设范围之内。具体来讲，当mipSizeId＝0或1时，表明了当前块的块尺寸索引值处于预设范围之内，即当前块的尺寸参数取值在预设范围之内；当mipSizeId＝2时，表明了当前块的块尺寸索引值处于预设范围之外，即当前块的尺寸参数取值不在预设范围之内。
也就是说，MIP输入采样值是由缓冲区(用pTemp表示)、当前块的块尺寸索引值(用MipSizeId表示)、当前块的相邻采样值的比特深度(用BitDepth表示)确定，而且MIP输入采样值中所包含的输入采样个数则仅与当前块的块尺寸索引值有关，最终可以获取MIP输入采样值中索引序号为x对应的值(用p[x]表示)。
进一步地，在一些实施例中，所述根据所述当前块的相邻采样值的比特深度，确定第二常数值，可以包括：
将所述第二常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述当前块的相邻采样值的比特深度减1。
或者，在一些实施例中，所述根据所述当前块的相邻采样值的比特深度，确定第二常数值，可以包括：
对“1”进行二进制比特左移，得到所述第二常数值，其中，所述比特左移的位数等于所述当前块的相邻采样值的比特深度减1。
也就是说，在获取到当前块的相邻采样值的比特深度(用BitDepth表示)之后，第二常数值可以表示为1<<(BitDepth-1)或者2^(BitDepth-1)。这样，针对当前块的尺寸参数取值在预设范围之内的情况，可以结合第二常数值以确定出当前块的MIP输入采样值。
还需要说明的是，MIP输入采样是用于矩阵乘法运算的矩阵向量。目前的相关技术方案是由缓冲区(用pTemp表示)、当前块的种类(即当前块的块尺寸索引值，用mipSizeId表示)、当前块的相邻采样值的比特深度(用BitDepth表示)和MIP输入采样的个数确定，最终获取MIP输入采样值中索引序号为x对应的值(用p[x]表示)。
在一种可能的实施方式中，当mipSizeId＝0或1时，还可以由1<<(BitDepth-1)减去pTemp[0]作为p[0]；然后当x不等于0时，则由pTemp[x]减去pTemp[0]作为p[x]。具体如上述式(2)所示。
在另一种可能的实施方式中，当mipSizeId＝0或1时，还可以由pTemp[0]减去1<<(BitDepth-1)作为p[0]；然后当x不等于0时，则由pTemp[x]减去pTemp[0]作为p[x]。具体如上述式(3)所示。
在又一种可能的实施方式中，当mipSizeId＝2时，由pTemp[x+1]减去pTemp[0]作为p[x]。具体如上述式(4)所示。
除此之外，本申请实施例还可以利用统一的计算方式来获取p[x]的值，而无需判断mipSizeId的取值。这时候，本申请实施例可以通过在缓冲区pTemp的末尾追加1<<(BitDepth-1)作为附加元素，直接将p[x]设置为等于pTemp[x+1]-pTemp[0]，x＝0,…,inSize-1。
这样，在根据当前块的相邻采样值确定出当前块的MIP输入采样值后，可以进一步确定当前块的MIP预测值。
S903：根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值。
其中，MIP预测值是所述当前块的色度分量中部分采样点的预测值。
需要说明的是，移位参数可以包括移位偏移参数和移位数量参数。这时候，对于S903来说，所述根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值，可以包括：
根据所述移位偏移参数的取值，确定所述MIP输入采样值的和值与所述移位偏移参数的乘积，其中，所述移位偏移参数的取值为固定常数；
根据所述移位数量参数的取值，确定第一常数值，其中，所述移位数量参数的取值为固定常数；
将第一偏移量的取值设置为所述第一常数值与所述乘积之差；
根据所述预测参数，确定所述当前块的MIP加权矩阵；
根据所述MIP加权矩阵、所述MIP输入采样值、所述移位数量参数和所述第一偏移量，确定所述 当前块的色度分量的MIP预测值。
需要说明的是，移位偏移参数还可称为移位补偿参数或者偏移因子，可以用fO表示。在本申请实施例中，可以将移位偏移参数的取值设置为固定常数，比如32，或者46，或者56，或者66，或者32等；还可以将移位偏移参数的取值设置为与移位偏移参数表有关，通过查表的方式来确定移位偏移参数的取值，这里并不作任何限定。
还需要说明的是，移位数量参数还可称为移位因子、移位比特数或者权重移位值等，可以用sW、shift或者weight shift表示。在本申请实施例中，移位数量参数通常用sW表示。在本申请实施例中，可以将移位数量参数的取值设置为固定常数，比如5，或者6，或者7等。还可以将移位数量参数的取值设置为与移位数量参数表有关，通过查表的方式来确定移位数量参数的取值，这里并不作任何限定。
另外，第一偏移量可以用oW表示，而且第一偏移量与移位数量参数和移位偏移参数均有关。
还应理解，预测参数可以包括预测模式参数和当前块的尺寸参数。当预测模式参数指示当前块的色度分量使用MIP模式确定帧内预测值时，这时候可以预先建立有权重矩阵表，并且该权重矩阵表存储在存储器或者存储单元中，这里的存储器或者存储单元可以是集成在解码器中，或者也可以单独设置。这样，根据当前块的块尺寸索引值(mipSizeId)和MIP模式索引值(modeId)，可以通过查表的方式确定出当前块所需要使用的MIP加权矩阵(或者称为MIP权重矩阵，也可简称为MIP矩阵)，用mWeight[x][y]表示。
还需要说明的是，在解码器中，还可以预先建立有移位数量参数表，并且该移位数量参数表也存储在存储器或者存储单元中，这里的存储器或者存储单元可以是集成在解码器中，或者也可以单独设置。本申请实施例中，针对移位数量参数(sW)的确定可以包括如下几种方式：
一种可能的实施方式中，对于不同的块尺寸和不同的MIP模式索引值，移位数量参数可以是不同的。在一些实施例中，该方法还可以包括：
在所述当前块的色度分量使用MIP模式确定帧内预测值时，解析码流，获取所述当前块的MIP模式索引值；
根据所述MIP模式索引值，从第一预设查找表中查询与所述MIP模式索引值对应的取值，其中，所述第一预设查找表用于记录MIP模式索引值与移位数量参数的取值之间的对应关系；
将查询到的取值确定为所述移位数量参数的取值。
需要说明的是，可以根据当前块的块尺寸索引值(用mipSizeId表示)和MIP模式索引值(modeId)来查询移位数量参数的取值。如上述表4所示的第一预设查找表，针对不同的mipSizeId和modeId，可以通过查找表方式确定出矩阵乘法中所需要使用的移位数量参数。
但是在解码器侧，表4是需要以查找表形式存储入存储器或者存储单元中；然而存储是需要代价的，而且查找过程也是需要代价的；由于表4中移位因子与当前块的块尺寸大小和MIP模式索引值均相关，从而增加了内存占用，而且还增加了计算复杂度。
为了减少内存占用和降低计算复杂度，本申请实施例针对移位因子的确定方式进行简化。
在另一种可能的实施方式中，移位数量参数的取值可以设置为一个固定常数，而且是与块尺寸索引值和MIP模式索引值无关的固定常数。例如，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位数量参数的取值均可以设置为5；或者，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位数量参数的取值均可以设置为6；或者，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位数量参数的取值均可以设置为7。在本申请实施例中，优选地，移位数量参数的取值等于6，但对此不作任何限定。
在又一种可能的实施方式中，对于移位数量参数的取值来说，该方法还可以包括：
根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；
根据所述当前块的块尺寸索引值，确定所述移位数量参数的取值。
需要说明的是，根据所述当前块的尺寸参数，可以确定出当前块的块尺寸索引值。一种可能的实施方式中，所述根据所述当前块的大小参数，确定所述当前块的块尺寸索引值，可以包括：
当所述当前块的宽度和高度均等于4时，将所述当前块的块尺寸索引值设置为0；
当所述当前块的宽度和高度均等于8、或者所述当前块的宽度和高度之一等于4时，将所述当前块的块尺寸索引值设置为1；
当所述当前块的宽度和高度不满足前述条件时，将所述当前块的块尺寸索引值设置为2。
这样，在确定出当前块的块尺寸索引值之后，根据当前块的块尺寸索引值，可以进一步确定出移位数量参数的取值。
可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位数量参数的取值，可以包括：
当所述块尺寸索引值分别等于0、1和2时，确定所述当前块的块尺寸索引值对应的所述移位数量参数的取值分别等于5、6和5。
可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位数量参数的取值，可以包括：
将所述移位数量参数的取值设置为等于所述当前块的宽度或高度与对应于所述当前块的块尺寸索引值的第一预设值之间的比值。
这里，第一预设值表示从所述当前块的边界获取的MIP输入采样值的数量。在这种情况下，该方法还可以包括：
当所述当前块的块尺寸索引值分别等于0、1和2时，确定所述当前块的块尺寸索引值对应的所述第一预设值分别等于2、4和4。
也就是说，当第一预设值表示从当前块的边界获取的MIP输入采样值的数量时，这时候如果当前块的块尺寸索引值等于0，那么对应的第一预设值等于2；如果当前块的块尺寸索引值等于1，那么对应的第一预设值等于4；如果当前块的块尺寸索引值等于2，那么对应的第一预设值等于4，从而可以根据当前块的宽度或高度与对应的第一预设值之间的比值来确定出移位数量参数的取值。
可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位数量参数的取值，可以包括：
将所述移位数量参数的取值设置为等于所述当前块的宽度或高度与对应于所述当前块的块尺寸索引值的第二预设值之间的比值。
这里，第二预设值表示直接使用所述MIP加权矩阵计算获取的所述当前块的MIP预测块的大小。在这种情况下，该方法还可以包括：
当所述当前块的块尺寸索引值分别等于0、1和2时，确定所述当前块的块尺寸索引值对应的所述第二预设值分别等于4、4和8。
也就是说，当第二预设值表示直接使用MIP加权矩阵计算获取的当前块的MIP预测块的大小时，这时候如果当前块的块尺寸索引值等于0，那么对应的第二预设值等于4；如果当前块的块尺寸索引值等于1，那么对应的第二预设值等于4；如果当前块的块尺寸索引值等于2，那么对应的第二预设值等于8，从而可以根据当前块的宽度或高度与对应的第二预设值之间的比值来确定出移位数量参数的取值。
在又一种可能的实施方式中，可以最小化移位数量参数表，并且仍然使用查找表的方式来确定移位数量参数的取值。可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位数量参数的取值，可以包括：
根据所述块尺寸索引值，从第二预设查找表中查询与所述块尺寸索引值对应的取值，其中，所述第二预设查找表用于记录块尺寸索引值与移位数量参数的取值之间的对应关系；
将查询到的取值确定为所述移位数量参数的取值。
需要说明的是，可以仅根据当前块的块尺寸索引值(用mipSizeId表示)来查询移位数量参数的取值。如上述表5所示的第二预设查找表，针对每一种的块尺寸索引值，可以对应固定的取值，也就是说，每个块的尺寸大小或者每个块的尺寸集合可以具有如表5所示固定的移位数量参数的取值。
根据上述的表5，当所述当前块的块尺寸索引值分别等于0、1和2时，可以确定出所述块尺寸索引值对应的移位数量参数的取值分别等于5、6和5。
这样，通过简化移位数量参数的确定方式，尤其是最小化移位数量参数表或者固定移位数量参数的取值，可以实现最小化查找表的存储，能够在不增加计算复杂度的情况下，减小MIP模式下移位因子表存储所占用的内存。
进一步地，在解码器中，还可以预先建立有移位偏移参数表，并且该移位偏移参数表也存储在存储器或者存储单元中，这里的存储器或者存储单元可以是集成在解码器中，或者也可以单独设置。本申请实施例中，针对移位偏移参数(fO)的确定可以包括如下几种方式：
在一种可能的实施方式中，对于不同的块尺寸和不同的MIP模式索引值，移位偏移参数也可以是不同的。在一些实施例中，该方法还可以包括：
在所述当前块的色度分量使用MIP模式确定帧内预测值时，解析码流，确定当前块的MIP模式索引值；
根据所述MIP模式索引值，从第三预设查找表中查询与所述MIP模式索引值对应的取值，其中，所述第三预设查找表用于记录MIP模式索引值与移位偏移参数的取值之间的对应关系；
将查询到的取值确定为所述移位偏移参数的取值。
需要说明的是，可以根据当前块的块尺寸索引值(用mipSizeId表示)和MIP模式索引值(modeId)来查询移位偏移参数的取值。如上述表6所示的第三预设查找表，针对不同的mipSizeId和modeId，可 以通过查找表方式确定出矩阵乘法中所需要使用的移位偏移参数。
但是在解码器侧，表6也是需要以查找表形式存储入存储器或者存储单元中；然而存储是需要代价的，而且查找过程也是需要代价的；由于表6中移位偏移参数的取值与当前块的块尺寸大小和MIP模式索引值均相关，从而增加了内存占用，而且还增加了计算复杂度。
为了减少内存占用和降低计算复杂度，本申请实施例针对移位偏移参数的确定方式也进行简化。
在另一种可能的实施方式中，移位偏移参数的取值可以设置为一个固定常数，而且是与块尺寸索引值和MIP模式索引值无关的固定常数。通常情况下，该固定常数的取值范围为0～100。例如，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位偏移参数的取值均可以设置为32；或者，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位偏移参数的取值均可以设置为46；或者，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位偏移参数的取值均可以设置为56；或者，针对不同的块尺寸索引值以及不同的MIP模式索引值，移位偏移参数的取值均可以设置为66。在本申请实施例中，优选地，移位偏移参数的取值等于32，但对此不作任何限定。
在又一种可能的实施方式中，对于移位偏移参数的取值来说，该方法还可以包括：
根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；
根据所述当前块的块尺寸索引值，确定所述移位偏移参数的取值。
需要说明的是，根据所述当前块的尺寸参数，可以确定出当前块的块尺寸索引值；然后根据当前块的块尺寸索引值，可以进一步确定出移位偏移参数的取值。
可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位偏移参数的取值，可以包括：
当所述块尺寸索引值分别等于0、1和2时，确定所述当前块的块尺寸索引值对应的所述移位偏移参数的取值分别等于34、23和46。
可选地，在一些实施例中，还可以最小化移位偏移参数表，并且仍然使用查找表的方式来确定偏移因子。可选地，在一些实施例中，所述根据所述当前块的块尺寸索引值，确定所述移位偏移参数的取值，可以包括：
根据所述块尺寸索引值，从第四预设查找表中查询与所述块尺寸索引值对应的取值，其中，所述第四预设查找表用于记录块尺寸索引值与移位偏移参数的取值之间的对应关系；
将查询到的取值确定为所述移位偏移参数的取值。
需要说明的是，可以仅根据当前块的块尺寸索引值(用mipSizeId表示)来查询移位偏移参数的取值。如上述表7所示的第四预设查找表，针对每一种的块尺寸索引值，可以对应固定的取值，也就是说，每个块的尺寸大小或者每个块的尺寸集合可以具有如表7所示固定的移位偏移参数的取值。
根据上述的表7，当所述当前块的块尺寸索引值分别等于0、1和2时，可以确定块尺寸索引值对应的移位偏移参数的取值分别等于34、23和46。
这样，通过简化移位偏移参数的确定方式，尤其是最小化移位偏移参数表或者固定移位偏移参数的取值，可以实现最小化查找表的存储，从而能够在不增加计算复杂度的情况下，减小MIP模式下移位偏移参数表存储所占用的内存。
如此，如此，在确定出移位偏移参数(fO)和移位数量参数(sW)之后，针对第一常数值而言，可选地，在一些实施例中，所述根据移位数量参数的取值，确定第一常数值，可以包括：
将所述第一常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述移位数量参数的取值减1。
可选地，在一些实施例中，所述根据移位数量参数的取值，确定第一常数值，可以包括：
对“1”进行二进制比特左移，得到所述第一常数值，其中，所述比特左移的位数等于所述移位数量参数的取值减1。
也就是说，在得到移位数量参数(sW)之后，第一常数值可以表示为1<<(sW-1)，也可以表示为2^(sW-1)。这时候当移位数量参数的取值设置为6时，可以得到第一常数值等于32。
这里，如果第一偏移量用oW表示，根据移位偏移参数(fO)可以计算得到MIP输入采样值的和值与移位偏移参数的乘积，即为
Figure PCTCN2020141080-appb-000013
那么oW的取值可以设置为
Figure PCTCN2020141080-appb-000014
这样，在MIP模式下，可以得到MIP加权矩阵、MIP输入采样值、移位数量参数和第一偏移量，以便后续确定当前块的MIP预测值。具体地，在一些实施例中，所述根据所述MIP加权矩阵、所述MIP输入采样值、所述移位数量参数和所述第一偏移量，确定所述当前块的MIP预测值，可以包括：
计算所述MIP加权矩阵与所述MIP输入采样值的第一加权和；
计算所述第一加权和与所述第一偏移量的第一和值；
对所述第一和值进行二进制比特右移，得到第一右移值，其中，所述右移的位数等于所述移位数量参数的取值；
将所述当前块的MIP预测值设置为等于所述第一右移值与所述第一临时参考值中索引序号为0对应的值的和值；其中，所述第一临时参考值是基于对所述当前块的相邻采样值进行下采样滤波处理得到的。
也就是说，在MIP模式下，可根据当前块的块尺寸索引值(用mipSizeId表示)和MIP模式索引值(用modeId表示)来确定出MIP加权矩阵(用mWeight表示)、移位数量参数(用sW表示)以及移位偏移参数(用fO表示)；然后将MIP输入采样值(用p[x]表示)、mWeight、sW以及fO输入矩阵乘法的过程，以获取矩阵乘法输出的MIP预测值(可以用predMip[x][y]表示)，且将predMip[x][y]中的采样点按照predSize×predSize排列成矩阵/阵列形式。其中，其计算公式如上述式(5)或式(6)所示。
S904：对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值。
需要说明的是，MIP预测块是由MIP预测值组成的。在所得到的MIP预测块之后，可以通过判断MIP预测块的尺寸参数与当前块的尺寸参数是否相同，以便进一步确定出当前块的帧内预测值。具体地，可以根据判断结果，当所述MIP预测块的尺寸参数与所述当前块的尺寸参数相同时，将所述当前块的帧内预测块设置为等于所述MIP预测块；这时候所述MIP预测块中包含所述当前块中全部像素位置的预测值；当所述MIP预测块的尺寸参数与所述当前块的尺寸参数不相同时，对所述MIP预测块进行滤波处理，得到滤波预测块，将所述滤波预测块设置为所述当前块的帧内预测块。这里，滤波处理可以包括上采样滤波处理或低通滤波处理。
S905：当所述LFNST参数指示对所述当前块执行LFNST时，确定所述当前块的重构变换系数块，对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块。
需要说明的是，并不是任意的当前块都可以执行LFNST，也就意味着码流中不一定存在LFNST参数；这时候需要检测(check)一些条件(比如当前块的尺寸参数中的最小值、当前块的块尺寸索引值等)以确定码流中是否存在LFNST参数。因此，在一些实施例中，所述解析码流，获取LFNST参数，可以包括：
确定所述码流中是否存在LFNST参数；
在确定所述码流中存在LFNST参数的情况下，解析所述码流，获取LFNST参数。
也就是说，首先需要确定码流中是否存在LFNST参数，只有在确定结果为是，即码流中存在LFNST参数的情况下，可以执行解析码流，获取LFNST参数的步骤。
可选地，在一些实施例中，所述确定所述码流中是否存在LFNST参数，可以包括：
在所述当前块的色度分量使用MIP模式确定帧内预测值时，确定当前块的尺寸参数中的最小值；
根据所述最小值，确定所述码流中是否存在LFNST参数。
进一步地，所述根据所述最小值，确定所述码流中是否存在LFNST参数，可以包括：
若所述最小值大于或等于第一预设阈值，则确定所述码流中存在LFNST参数。
这里，第一预设阈值可以用于表示预先设定的衡量是否执行LFNST的门限值。在本申请实施例中，第一预设阈值可以设置为8，但是不作具体限定。
也就是说，当对当前块的色度分量使用MIP模式进行帧内预测时，这时候可以确定当前块的尺寸参数中的最小值；比如当宽度和高度的最小值大于或等于8时，此时确定出码流中存在LFNST参数，可以通过解析码流，以获取LFNST参数，比如lfnst_index的取值。
可选地，在一些实施例中，所述确定所述码流中是否存在LFNST参数，可以包括：
在所述当前块的色度分量使用MIP模式确定帧内预测值时，根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；
根据所述块尺寸索引值，确定所述码流中是否存在LFNST参数。
进一步地，所述根据所述块尺寸索引值，确定所述码流中是否存在LFNST参数，可以包括：
若所述块尺寸索引值等于第二预设阈值，则确定所述码流中存在LFNST参数。
这里，第二预设阈值也可以用于表示预先设定的衡量是否执行LFNST的门限值。在本申请实施例中，第二预设阈值可以设置为预设值(例如2)，或者多个预设值(例如1、2)之一，但是不作具体限定。
也就是说，当对当前块的色度分量使用MIP模式进行帧内预测时，这时候可以根据当前块的尺寸参数确定当前块的块尺寸索引值(mipSizeId)；比如当mipSizeId的取值等于2时，此时确定出码流中 存在LFNST参数，可以通过解析码流，以获取LFNST参数，比如lfnst_index的取值。
在一些实施例中，所述解析码流，获取LFNST参数，可以包括：
解析所述码流，获取LFNST索引序号的取值。
需要说明的是，LFNST索引序号可以用lfnst_index表示，而且LFNST索引序号的取值用于确定是否对当前块执行LFNST。具体地，在一些实施例中，该方法还可以包括：
若所述LFNST索引序号的取值为大于零，则确定对所述当前块执行LFNST；
若所述LFNST索引序号的取值为等于零，则确定不对所述当前块执行LFNST。
也就是说，在获取到lfnst_index的取值之后，如果lfnst_index的取值为大于零，那么确定对当前块执行LFNST；如果lfnst_index的取值为等于零，那么确定不对当前块执行LFNST。
还需要说明的是，在获取到LFNST参数，比如lfnst_index的取值之后，如果lfnst_index大于零，那么表明LFNST参数指示对当前块执行LFNST。这时候，在一些实施例中，当所述LFNST参数指示对所述当前块执行LFNST时，所述确定所述当前块的重构变换系数块，可以包括：
解析所述码流，获取所述当前块的量化系数；
对所述量化系数进行反量化，得到所述当前块的重构变换系数块。
具体来讲，反量化处理即为缩放处理。在一种具体的示例中，所述对所述量化系数进行反量化，得到所述当前块的重构变换系数块，可以包括：对所述量化系数进行缩放操作，得到所述当前块的重构变换系数块。
这样，当LFNST参数指示对当前块执行LFNST时，这时候可以对该重构变换系数块中全部或者部分重构变换系数执行LFNST，以得到第二变换系数块。
还需要说明的是，在确定当前块可以执行LFNST时，这时候还需要确定当前块使用的LFNST变换核(用kernel表示)。在一些实施例中，所述对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块，可以包括：利用LFNST变换核对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到所述第二变换系数块。
在本申请实施例中，LFNST中共有4个变换核候选集，这四个变换核候选集可以包括有set0、set1、set2和set3。这里，本申请实施例可以利用MIP参数来确定LFNST变换核候选集，然后从LFNST变换核候选集中选择当前块使用的LFNST变换核。因此，在一些实施例中，该方法还可以包括：
在所述当前块的色度分量使用MIP模式确定帧内预测值时，从当前块的预测模式参数中确定MIP参数；
根据所述MIP参数，确定LFNST变换核候选集；
在所述LFNST变换核候选集中，将所述LFNST索引序号的取值指示的变换核确定为所述当前块使用的LFNST变换核；其中，所述LFNST变换核候选集包含预设的两个或多个LFNST变换核。
需要说明的是，MIP参数可以包括下述至少之一：MIP模式索引值(用modeId表示)和MIP转置指示参数(用isTransposed表示)。其中，针对当前块使用的LFNST变换核，当所述MIP转置指示参数的取值指示对MIP模式使用的采样点输入向量进行转置处理时，这时候还需要对所述选择的变换核进行矩阵转置处理，以得到所述当前块使用的LFNST变换核。
进一步地，在一些实施例中，所述根据所述MIP参数，确定LFNST变换核候选集，可以包括：
根据所述MIP模式索引值，确定LFNST帧内预测模式索引序号的取值；
根据所述LFNST帧内预测模式索引序号的取值，通过第五预设查找表确定LFNST变换核候选集的索引值；
根据所述LFNST变换核候选集的索引值，从多个候选LFNST变换核候选集中选择所述LFNST变换核候选集。
这里，LFNST帧内预测模式索引序号可以用predModeIntra表示，LFNST变换核候选集的索引值可以用lfnstTrSetIdx表示。
需要说明的是，第五预设查找表如上述表10所示。根据MIP模式索引值(即modeId)，可以确定出predModeIntra的取值；然后根据predModeIntra的取值，可以结合表10直接确定出lfnstTrSetIdx的取值，即确定出当前块所选择的LFNST变换核候选集。这里，lfnstTrSetIdx的取值指示了LFNST时所使用的变换核候选集；由于modeId的取值可以包括有0、1、2、3、4、5，那么predModeIntra的取值也为0、1、2、3、4、5；其与lfnstTrSetIdx的对应关系具体如表10所示。
在一种具体的实施例中，所述根据所述MIP模式索引值，确定LFNST帧内预测模式索引序号的取值，可以包括：将LFNST帧内预测模式索引序号的取值设置为等于所述MIP模式索引值。
也就是说，本申请实施例可以将predModeIntra的取值设置为等于modeId的取值，然后根据predModeIntra的取值，结合表10直接确定出lfnstTrSetIdx的取值，即确定出当前块所选择的LFNST 变换核候选集。还需要注意的是，在另一种具体的实施例中，本申请实施例还可以将MIP模式索引值直接映射为PLANAR模式的取值，然后使用PLANAR模式所对应的predModeIntra的取值来确定出lfnstTrSetIdx的取值，即确定出当前块所选择的LFNST变换核候选集。
进一步地，在执行LFNST的过程中，扫描顺序可以包括水平扫描顺序和垂直扫描顺序。这里，扫描顺序的确定与MIP转置指示参数(用isTransposed表示)的取值有关。
可选地，在一些实施例中，所述对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块，可以包括：
当所述MIP转置指示参数的取值指示对MIP模式使用的采样点输入向量进行转置处理时，根据垂直扫描顺序对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到所述第二变换系数块；
当所述MIP转置指示参数的取值指示不对MIP模式使用的采样点输入向量进行转置处理时，根据水平扫描顺序对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到所述第二变换系数块。
具体来讲，当isTransposed的取值等于0时，这时候对MIP模式使用的采样点输入向量进行转置处理，那么可以根据水平扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，具体如上述式(10)所示。
而当isTransposed的取值等于1时，这时候不对MIP模式使用的采样点输入向量进行转置处理，那么可以根据垂直扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，具体如上述式(11)所示。
其中，log2LfnstSize的确定如上述式(9)所示。
可选地，在一些实施例中，所述对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块，可以包括：
当所述MIP转置指示参数的取值指示对MIP模式使用的采样点输入向量进行转置处理时，根据水平扫描顺序对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到所述第二变换系数块；
当所述MIP转置指示参数的取值指示不对MIP模式使用的采样点输入向量进行转置处理时，根据垂直扫描顺序对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到所述第二变换系数块。
具体来讲，当isTransposed的取值等于1时，这时候对MIP模式使用的采样点输入向量进行转置处理，那么可以根据水平扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，具体如上述式(10)所示。
而当isTransposed的取值等于0时，这时候不对MIP模式使用的采样点输入向量进行转置处理，那么可以根据垂直扫描顺序对所述第一变换系数块中至少部分变换系数执行LFNST，具体如上述式(11)所示。
其中，log2LfnstSize的确定如上述式(9)所示。
S906：对所述第二变换系数块进行第一次变换，得到所述当前块的色度分量的重构残差块。
S907：根据所述当前块的色度分量的帧内预测值以及所述重构残差块，确定所述当前块的色度分量的重构块。
需要说明的是，在解码器中，通过解析码流，可以获得LFNST系数的量化值，通过反量化单元对该量化值进行反量化处理(可以称之为“Scaling”)，得到重构变换系数块，对该重构变换系数块中的至少部分重构变换系数进行反向的LFNST变换，可以得到第二变换系数块；然后再通过反向的第一次变换，可以得到恢复出当前块的色度分量的重构残差块。这里，反向的第一次变换是与编码器中“第一次变换(Core Transform)”对应的反变换。需要注意的是，标准中仅定义了解码器中的“反变换”操作，因此在标准中针对“反向LFNST变换”也可以称之为“LFNST变换”。
还需要说明的是，在恢复出当前块的色度分量的重构残差块之后，可以将当前块的色度分量的帧内预测值和所述重构残差块进行加法处理，从而确定出当前块的色度分量的重构块。
下面将结合图2所示的解码器200对本申请实施例的解码方法进行具体说明。
解码器200的输入码流可以是由编码器100生成的码流。解析单元201对输入码流进行解析并从输入码流获得语法元素的值。解析单元201将语法元素的二进制表示转换成数字值并将数字值发送到解码器200中的单元以获得一个或多个解码图片。解析单元201还可从输入码流解析出一个或多个语法元素以渲染解码图片。
参见图10，其示出了本申请实施例提供的一种对LFNST参数进行解析的流程示意图。如图10所示，该流程可以包括：
S1001：解析单元201检查用于确定输入码流中是否存在LFNST参数。
需要说明的是，LFNST参数可以是指lfnst_index的取值。这里，用于表示LFNST参数的语法结构的示例可以与上述表8中所示的语法结构的示例相同，也可以与上述表9A和表9B中所示的语法结构的示例相同。
具体来讲，在当前块的帧内预测模式是MIP模式时，解析单元201使用如下方式从输入码流获得lfnst_index的编码比特。可选地，使用表9A中的语法结构，当该当前块中变换块的宽度和高度的最小值大于或等于8(即，宽度和高度都大于或等于8)时，解析单元201使用MIP模式读取lfnst_index位。另一可选条件可以设置为：宽度和高度大于或等于第一值(例如8)，且宽度和高度之和大于或等于第二值(例如16)。
可选地，使用表9B中的语法结构，当该当前块的mipSizeId等于2时，解析单元201使用MIP模式读取lfnst_index位。另一可选条件可以设置为：该当前块的mipSizeId等于多个值(例如1和2)之一。
S1002：解析单元201对LFNST参数进行自适应二进制算术解码，获取LFNST参数。
这里，由于编码器100侧，熵编码单元115采用如表9A和表9B所示的描述符为“ae(v)”的CABAC来对LFNST参数(例如lfnst_index的取值)进行编码，那么在解码器200侧，通过解析单元201进行自适应二进制算术解码，可以从lfnst_index位获得lfnst_index的取值。
解析单元201将语法元素的值以及根据语法元素的值设置或确定的、用于获得一个或多个解码图片的一个或多个变量发送到解码器200中的单元。预测单元202确定当前块(例如CU)的帧内预测块。当指示帧间解码模式用于对当前解码块进行解码时，预测单元202将来自解析单元201的相对参数传送到MC单元203以获得帧间预测块。当指示帧内预测模式(包括基于MIP模式索引值指示的MIP模式)用于对当前块进行解码时，预测单元202将来自解析单元201的相对参数传送到帧内预测单元204以获得帧内预测块。
应理解，本申请实施例中的当前块(或称为“解码块”)可以是CU或CU的分区(例如，变换块)。具体地，当指示MIP模式用于对当前块进行解码时，帧内预测单元204将获得当前块的帧内预测块，其中当前块的帧内预测块包括当前块内至少一个像素(samples)的帧内预测值。
具体地，对于帧内预测单元204而言，其使用MIP模式获取帧内预测块，步骤如下：
首先，帧内预测单元204从当前块的相邻像素点获取一个或多个参考像素点，例如通过对相邻像素点进行下采样获取，或者直接从相邻像素点进行提取。
然后，帧内预测单元204使用所获取的参考像素点、MIP矩阵和移位参数以确定当前块内的像素点位置对应的一个或多个部分预测像素点。这里，像素点位置可以是当前块中的预设像素点位置。例如，像素点位置具有均匀的水平和垂直坐标值。移位参数包括移位数量参数和移位偏移参数，而且移位参数可以在获取部分像素点的帧内预测值的过程中用于偏移操作。
最后，如果获取到当前块的部分像素点对应的预测像素点，那么帧内预测单元204还需要获取当前块中除了部分像素点之外的剩余像素点对应的预测像素点。例如，帧内预测单元204可以使用插值滤波器获取剩余像素点对应的预测像素点，这里插值滤波器的输入可以是部分像素点和相邻像素点。
如图6所示，其示出的使用MIP模式获取帧内预测块的流程框图示例，也可以在解码器200上实现，其中，这里的“当前块”指的是“解码块”。
步骤601，帧内预测单元204获取当前块的相邻像素点，例如相邻像素点被标记为与图6所示步骤601的当前块相邻的灰色填充正方形。帧内预测单元204从相邻像素点获取一个或多个参考像素点。在图6所示步骤601的示例中，可选地，帧内预测单元204可计算两个相邻像素点的平均值并将该平均值用作参考像素点。可选地，帧内预测单元204每隔一个相邻像素点就选择一个相邻像素点作为参考像素点。例如，在图6所示步骤601中的示例中，帧内预测单元204从当前块的8个上侧相邻的像素点中选取4个参考像素点，并从当前块的8个左侧相邻的像素点中选取另外4个参考像素点。
步骤601的具体过程如下：
帧内预测单元204从解析单元201获得当前块的宽度和高度，宽度和高度分别由变量cbWidth和cbHeight表示。根据来自解析单元201的参数，当前块被分成一个或多个变换块。令变量nTbW和nTbH分别为变换块的宽度和高度。当MIP模式用作帧内预测模式以获得当前块的预测时，帧内预测单元204确定MIP的块尺寸索引值，即表示为mipSizeId的变量。
可选地，帧内预测单元204如下确定mipSizeId的取值：
–如果nTbW和nTbH都等于4，则mipSizeId设置为等于0；
–否则，如果cbWidth或cbHeight等于4，则mipSizeId设置为等于1；
–否则，mipSizeId设置为等于2。
具体地，在当前块的尺寸参数为8×8(即cbWidth和cbHeight都等于8)时，mipSizeId设置为等于2。
可选地，帧内预测单元204如下确定mipSizeId的取值：
–如果nTbW和nTbH都等于4，则mipSizeId设置为等于0；
–否则，如果cbWidth或cbHeight等于4，或者cbWidth和cbHeight都等于8，则mipSizeId设置为等于1；
–否则，mipSizeId设置为等于2。
具体地，在当前块的尺寸参数为8×8时，mipSizeId设置为等于1。
进一步地，帧内预测单元204根据mipSizeId获得boundarySize和predSize的取值，具体如下：
–如果mipSizeId等于0，则boundarySize设置为等于2，predSize设置为等于4；
–否则，如果mipSizeId等于1，则boundarySize设置为等于4，predSize设置为等于4；
–否则(如果mipSizeId等于2)，boundarySize设置为等于4，predSize设置为等于8；
其中，boundarySize表示从当前块上侧边相邻的参考像素和左侧边相邻的参考像素中每一边获得的参考像素数量。
帧内预测单元204还可以来自解析单元701的、指示存储在缓冲区pTemp中的参考像素顺序的参数，以用于获得用于计算MIP预测值的参考像素。帧内预测单元204将变量isTransposed设置为等于来自解析单元701的、指示存储在缓冲区pTemp中的参考像素顺序的参数。例如，isTransposed等于0(或FALSE)表示：帧内预测单元204先呈现从当前块上侧边相邻的参考像素获得的参考像素，然后呈现从左侧边相邻的参考像素获得的参考像素；否则，isTransposed等于1(或TRUE)表示：帧内预测单元204先呈现从当前块左侧边相邻的参考像素获得的参考像素，然后呈现从上侧边相邻的参考像素获得的参考像素。
帧内预测单元204获得变量inSize的取值以指示在使用MIP模式时所使用的参考像素数量，如上述式(1)所示。
帧内预测单元204调用如下过程，以使用当前块相邻的参考像素来获得一组参考像素(存储为阵列p[x]，其中x为0,…,inSize–1)。
帧内预测单元204从当前块上侧边相邻的参考像素(例如，存储在阵列refT中)获得nTbW参考像素，并从当前块左侧边相邻的参考像素(例如，存储在阵列refL中)获得nTbH参考像素。帧内预测单元204对refT调用下采样处理以获得boundarySize参考像素，并将boundarySize参考像素存储在refT中。帧内预测单元204对refL调用下采样处理以获得boundarySize参考像素，并将boundarySize参考像素存储在refL中。
帧内预测单元204还可获得由isTransposed指示的顺序以将refT和refL中的元素放入缓冲区pTemp中。
可选地，帧内预测单元204获得p[x]，其中x＝0,…,inSize–1，如下：
–如果mipSizeId等于2，则p[x]＝pTemp[x+1]-pTemp[0]；
–否则(mipSizeId小于2)，p[0]＝pTemp[0]-(1<<(BitDepth-1))，p[x]＝pTemp[x]-pTemp[0]；
其中，BitDepth是当前块中像素的颜色分量的比特深度。这里，颜色分量可以是RGB分量中的一个、或者YUV分量中的一个、或者YCbCr分量中的一个，例如，Y分量等。
可选地，帧内预测单元204可获得p[x]，其中x＝0,…,inSize–1，如下：
–如果mipSizeId等于2，则p[x]＝pTemp[x+1]-pTemp[0]；
–否则(mipSizeId小于2)，p[0]＝(1<<(BitDepth-1))-pTemp[0]，p[x]＝pTemp[x]-pTemp[0]。
可选地，帧内预测单元204可使用统一的计算方式来获得p[x]的值，而无需判断mipSizeId的取值。例如，在缓冲区pTemp中追加(1<<(BitDepth-1))作为附加元素，这时候帧内预测单元204将p[x]计算为pTemp[x+1]-pTemp[0]。
步骤602，帧内预测单元204使用一组参考像素和MIP加权矩阵获得当前块的MIP预测值。其中，MIP加权矩阵根据与MIP模式相对应的MIP模式索引值(表示为ModeId)和MIP的块尺寸索引值(表示为mipSizeId)从一组预定义的MIP加权矩阵中选择。帧内预测单元204从解析单元201获得ModeId的取值。
帧内预测单元204获取当前块内一个或多个像素点位置对应的部分预测像素点的MIP预测值(表示为predMip[x][y])。在图6所示步骤602的示例中，部分预测像素点是当前块中标记为灰色填充正方形的像素点。预测模块601的输入是步骤601中获取的参考像素p[x]。预测模块601使用MIP加权矩阵和移位参数计算部分预测像素点；这里，移位参数包括移位数量参数和移位偏移参数。
在一种可能的实现方式中，预测模块601可用坐标(x,y)表示像素点，而预测像素点用predMip[x][y] 表示，那么predMip[x][y]的计算公式如式(5)或式(6)所示。
在式(5)或式(6)中，mWeight[i][j]是MIP加权矩阵；这里，矩阵元素可以是预定常数值；或者，也可以使用如训练方式自适应地更新，该训练方式的输入是一个或多个编码图片或块，或者来自外部装置提供给解码器100的其它码流中的图片，或者从解析单元201通过在包含MIP加权矩阵的输入码流中解析特殊数据单元而获得的图片；mWeight[i][j]可以根据来自解析单元201的相应的一个或多个参数所指示的MIP模式来确定；fO是用于确定oW的移位偏移参数；sW是移位数量参数；p[i]是使用参考像素来计算的，即本申请实施例中的MIP输入采样值；pTemp[0]表示参考像素中的首位像素，即本申请实施例中的第一临时参考值中索引序号为0对应的值；“>>”是如VVC中定义的二进制右移运算符。
预测模块601可根据当前块的尺寸和用于当前块的MIP模式确定sW和fO的值。在一个示例中，预测模块601使用查找表获取sW和fO的值。
可选地，预测模块601可以使用上述的表4，根据当前块的尺寸参数和MIP模式确定sW。
可选地，预测模块601还可以使用上述的表5，根据当前块的尺寸参数确定sW。
可选地，预测模块601还可以直接将sW设置为常数值。例如，对于各种尺寸参数的块和不同的MIP模式，预测模块601可以将sW设置为5；或者，对于各种尺寸参数的块和不同的MIP模式，预测模块601将sW设置为6；或者，对于各种尺寸参数的块和不同的MIP模式，预测模块601将sW设置为7。
可选地，预测模块601可以使用上述的表6，根据当前块的尺寸参数和MIP模式确定fO。
可选地，预测模块601还可以使用上述的表7，根据当前块的尺寸参数确定fO。
可选地，预测模块601还可以直接将fO设置为常数值(如0～100)。例如，对于各种尺寸参数的块和不同的MIP模式，预测模块601将fO设置为32；或者，预测模块601将fO设置为46；或者，预测模块601将fO设置为56；或者，预测模块601将fO设置为66。
帧内预测单元204可以对predMip中的MIP预测值执行嵌位处理操作。当isTransposed等于1(或TRUE)时，predSize×predSize的阵列predMip[x][y](其中x＝0,...,predSize-1，y＝0,...,predSize-1)转换为predTemp[y][x]＝predMip[x][y]，然后predMip＝predTemp。
示例性，可选地，在当前块的尺寸参数为8×8(即cbWidth和cbHeight都等于8)时，帧内预测单元204确定mipSizeId等于2并获得8×8的predMip。
可选地，在当前块的尺寸参数为8×8(即cbWidth和cbHeight都等于8)时，帧内预测单元204确定mipSizeId等于1并获得4×4的predMip。
步骤603，帧内预测单元204获取当前块的帧内预测值(作为predSamples[x][y]存储在阵列中，其中x＝0,...,nTbW-1，y＝0,...,nTbH-1)，如下：
–如果帧内预测单元204确定nTbW大于predSize或者nTbH大于predSize，则帧内预测单元204调用上采样过程以使用predMip获得predSamples。帧内预测单元204获得当前块中除了部分像素点之外的剩余像素点对应的预测像素点。在图6中，帧内预测单元204可以使用滤波模块602获取当前块中除了部分像素点之外剩余像素点对应的预测像素点。滤波模块602的输入是步骤602中标记为灰色填充正方形的像素点。滤波模块602可使用一个或多个插值滤波器，利用该输入获取当前块中除了部分像素点之外剩余像素点对应的预测像素点。例如，输入可包括参考像素点和当前块中的一个或多个像素点位置的部分预测像素点；或者，输入可包括相邻像素点和当前块中的一个或多个像素点位置的部分预测像素点；或者，输入可包括参考像素点、相邻像素点和当前块中的一个或多个像素点位置的部分预测像素点。例如，可选地，在当前块的尺寸为8×8(即cbWidth和cbHeight都等于8)时，帧内预测单元204确定mipSizeId等于1，帧内预测单元204将上采样过程应用于4×4predMip，以获得当前块的8×8的帧内预测块。
–否则，帧内预测单元204将当前块的帧内预测块设置为等于当前块的MIP预测块，即，设置predSamples[x][y](其中x＝0,...,nTbW-1，y＝0,...,nTbH-1)等于predMip[x][y]。例如，在当前块的尺寸参数为8×8(即cbWidth和cbHeight都等于8)时，帧内预测单元204获得当前块的predSamples，其尺寸等于8×8(即cbWidth和cbHeight都等于8)。
如此，根据图6所示的流程框图，在步骤603之后，帧内预测单元204可以获取当前块(即CU)的帧内预测块，即确定出当前块内至少一个像素的帧内预测值。
缩放单元205具有与编码器100中的反量化单元110的功能相同的功能。缩放单元205对来自解析单元201的量化系数(即levels)执行缩放操作以获得重构系数。
变换单元206具有与编码器100中的逆变换单元111的功能相同的功能。变换单元206执行一个或多个变换操作(即编码器100中的逆变换单元111执行的一个或多个变换操作的反操作)以获得重构残差。
当lfnst_index不等于0时，变换单元206对重构系数执行二次变换以获得二次变换系数块，然后对二次变换系数块执行第一次变换，例如，最初基于DCT设计的整数变换，以获得重构残差块。LFNST是二次变换的示例。
参见图11，其示出了本申请实施例提供的另一种执行LFNST的流程框架示意图。在图11中，块1101是包含重构变换系数的重构变换系数块。
步骤1101，变换单元206对块1101中的全部或部分重构变换系数执行二次变换LFNST。
在当前块的帧内预测模式是MIP模式时，变换单元206根据MIP参数选择LFNST变换核候选集。MIP参数可包括下述至少之一：MIP模式索引值(即modeId)和MIP转置指示参数(即isTransposed)。
变换单元206将变量predModeIntra的取值设置为等于modeId的取值。变换单元206确定用于对子块11001、11002和11003上的系数执行LFNST的变换核。子块的尺寸可以是预设值，例如8×8。变换单元206根据predModeIntra的取值确定LFNST变换核候选集的索引值(即lfnstTrSetIdx)。对于MIP模式下的块解码，变换单元206采用如表10的查找表方式来获得lfnstTrSetIdx的取值。变换单元206将用于执行LFNST的变换核确定为由lfnstTrSetIdx指示的LFNST变换核候选集中的、由lfnst_index指示的变换核。
当lfnst_index不等于0时，变换单元206使用来自子块11001、11002和11003的系数对变换块执行LFNST，获得LFNST系数并将LFNST系数放入块1102的子块11011、11012和11013中。假定d[x][y]是第一变换系数的块，其中x＝0,...,nTbW-1，y＝0,...,nTbH-1，其中nTbW和nTbH分别是块1102的宽度和高度。假定v[x]是LFNST系数的块，其中x＝0,...,nLfnstOutSize–1，而nLfnstOutSize＝(nTbW>＝8&&nTbH>＝8)？48:16。
可选地，变换单元206采用如下指令得到变换系数的块1102，具体如上述式(8)所示。其中，log2LfnstSize的如上述式(9)所示。
可选地，当isTransposed等于0时，变换单元206采用如下指令得到变换系数的块1102，具体如上述式(10)所示；
而当isTransposed等于1时，变换单元206采用如下指令得到变换系数的块1102，具体如上述式(11)所示。
可选地，当isTransposed等于1时，变换单元206采用如下指令得到变换系数的块1102，具体如上述式(10)所示；
而当isTransposed等于0时，变换单元706采用如下指令得到变换系数的块902，具体如上述式(11)所示。
其中，log2LfnstSize的确定如上述式(9)所示。
除此之外，在当前块的帧内预测模式是传统帧内预测模式，即角度帧内预测模式，DC模式和PLANAR模式时，变换单元206使用如表11所示的查找表，根据传统帧内预测模式的模式索引值(分配给predModeIntra)来确定lfnstTrSetIdx的取值，即确定出当前块所选择的LFNST变换核候选集。变换单元206将用于执行LFNST的变换核确定为由lfnstTrSetIdx指示的LFNST变换核候选集中的、由lfnst_index指示的变换核。
当lfnst_index不等于0时，变换单元206使用来自子块11001、11002和11003的系数对变换块执行LFNST，获得LFNST系数并将LFNST系数放入块1102的子块11011、11012和11013中。假定d[x][y]是第一变换系数的块，其中x＝0,...,nTbW-1，y＝0,...,nTbH-1，其中nTbW和nTbH分别是块1102的宽度和高度。假定v[x]是LFNST系数的块，其中x＝0,...,nLfnstOutSize–1，而nLfnstOutSize＝(nTbW>＝8&&nTbH>＝8)？48:16。
可选地，当predModeIntra小于或等于34时，变换单元206采用如下指令得到变换系数的块1102，具体如上述式(10)所示；
而当predModeIntra大于34时，变换单元206采用如下指令得到变换系数的块1102，具体如上述式(11)所示。
可选地，当predModeIntra大于34时，变换单元206采用如下指令得到变换系数的块1102，具体如上述式(10)所示；
而当predModeIntra小于或等于34时，变换单元706采用如下指令得到变换系数的块902，具体如上述式(11)所示。
其中，log2LfnstSize的确定如上述式(9)所示。
步骤902，变换单元206对块1102中的变换系数执行第一次变换(即核心变换)，例如，最初基于DCT设计的整数变换，得到当前块的重构残差块1103。
进一步地，加法器207对其输入(来自预测单元202的帧内预测块和来自变换单元206的重构残差块)执行相加操作以获得当前块的重构块。重构块还发送到预测单元202以用作在帧内预测模式下编码的其它块的参考。
在图片或子图片中的所有CU被重构之后，滤波单元208对重构图片或子图片执行环路滤波。滤波单元208包含一个或多个滤波器，例如去方块滤波器、采样自适应偏移滤波器、自适应环路滤波器、亮度映射和色度缩放滤波器以及基于神经网络的滤波器等。替代地，当滤波单元208确定重构块不用作对其它块进行解码的参考时，滤波单元208对重构块中的一个或多个目标像素执行环路滤波。
滤波单元208的输出是解码图片或子图片，解码图片或子图片发送到DPB单元209。DPB单元209根据时序和控制信息输出解码图片。存储在DPB单元209中的图片还可用作通过预测单元202执行帧间预测或帧内预测的参考。
在本申请实施例中，解码器200可以是具有处理器和记录解码程序的存储介质的计算设备。当处理器读取并运行解码程序时，解码器200读取输入码流并生成对应的解码视频。另外，解码器200可以是具有一个或多个芯片的计算设备。在芯片上实现为集成电路的单元通过与图2中的对应单元类似的连接以及数据交换而具有类似的功能。
本实施例提供了一种编码方法，通过解析码流，获取当前块的预测参数和LFNST参数，其中，所述预测参数包括预测模式参数；当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；当所述LFNST参数指示对所述当前块执行LFNST时，确定所述当前块的重构变换系数块，对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块；对所述第二变换系数块进行第一次变换，得到所述当前块的色度分量的重构残差块；根据所述当前块的色度分量的帧内预测值以及所述重构残差块，确定所述当前块的色度分量的重构块。这样，针对MIP模式，能够在保证解码性能的基础上，降低复杂度，同时减小解码过程中所需要的存储空间，有效地提高解码效率；另外，LFNST技术在应用于MIP模式预测时，由于引入了MIP参数，使得LFNST变换更加灵活，进一步提高了解码效率。
本申请的又一实施例中，基于前述实施例相同的发明构思，参见图12，其示出了本申请实施例提供的一种编码器120的组成结构示意图。如图12所示，该编码器120可以包括：第一确定单元1201、第一预测单元1202、第一变换单元1203和编码单元1204；其中，
第一确定单元1201，配置为确定当前块的预测参数，其中，所述预测参数包括预测模式参数；
第一确定单元1201，还配置为当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
第一预测单元1202，配置为根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；以及对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
第一变换单元1203，配置为根据所述当前块的色度分量的帧内预测值，确定所述当前块的色度分量的预测残差值；以及对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；
编码单元1204，配置为对所述LFNST参数进行编码，并写入码流。
在一些实施例中，所述当前块的相邻采样值包括所述当前块的左相邻采样值和上相邻采样值。
在一些实施例中，所述移位参数包括移位偏移参数和移位数量参数；
第一确定单元1201，具体配置为根据移位偏移参数的取值，确定所述MIP输入采样值的和值与所述移位偏移参数的乘积，其中，所述移位偏移参数的取值为固定常数；以及根据移位数量参数的取值，确定第一常数值，其中，所述移位数量参数的取值为固定常数；以及将第一偏移量的取值设置为所述第一常数值与所述乘积之差；以及根据所述预测参数，确定所述当前块的MIP加权矩阵；以及根据所述MIP加权矩阵、所述MIP输入采样值、所述移位数量参数和所述第一偏移量，确定所述当前块的色度分量的MIP预测值。
在一些实施例中，所述预测参数还包括：所述当前块的尺寸参数。
在一些实施例中，第一确定单元1201，还配置为根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；以及对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；以及当所述当前块的块尺寸索引值处于预设范围之内时，根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第二常数值与所述第一临时参 考值中索引序号为0对应的值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数；当所述当前块的块尺寸索引值处于预设范围之外时，将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第一临时参考值中索引序号等于j+1对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
在一些实施例中，第一确定单元1201，还配置为当所述当前块的块尺寸索引值处于预设范围之内时，根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第一临时参考值中索引序号为0对应的值与所述第二常数值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数。
在一些实施例中，第一确定单元1201，还配置为当所述当前块的宽度和高度均等于4时，将所述当前块的块尺寸索引值设置为0；当所述当前块的宽度和高度均等于8、或者所述当前块的宽度和高度之一等于4时，将所述当前块的块尺寸索引值设置为1；当所述当前块的宽度和高度不满足前述条件时，将所述当前块的块尺寸索引值设置为2。
在一些实施例中，第一确定单元1201，还配置为对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；以及根据所述当前块的相邻采样值的比特深度，确定第二常数值，将所述第二常数值缓存在所述第一临时参考值之后的一个数据单元中，得到第二临时参考值；以及将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第二临时参考值中索引序号等于j+1对应的值与所述第二临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
在一些实施例中，第一确定单元1201，还配置为将所述第二常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述当前块的相邻采样值的比特深度减1。
在一些实施例中，第一确定单元1201，还配置为对“1”进行二进制比特左移，得到所述第二常数值，其中，所述比特左移的位数等于所述当前块的相邻采样值的比特深度减1。
在一些实施例中，所述移位数量参数的取值为与块尺寸索引值和MIP模式索引值无关的固定常数。
在一些实施例中，所述移位数量参数的取值设置为等于6。
在一些实施例中，所述移位偏移参数的取值为与块尺寸索引值和MIP模式索引值无关的固定常数。
在一些实施例中，所述移位偏移参数的取值设置为等于32。
在一些实施例中，第一确定单元1201，还配置为将所述第一常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述移位数量参数的取值减1。
在一些实施例中，第一确定单元1201，还配置为对“1”进行二进制比特左移，得到所述第一常数值，其中，所述比特左移的位数等于所述移位数量参数的取值减1。
在一些实施例中，参见图12，编码器120还可以包括第一计算单元1205，配置为计算所述MIP加权矩阵与所述MIP输入采样值的第一加权和；以及计算所述第一加权和与所述第一偏移量的第一和值；以及对所述第一和值进行二进制比特右移，得到第一右移值，其中，所述右移的位数等于所述移位数量参数的取值；
第一确定单元1201，还配置为将所述当前块的色度分量的MIP预测值设置为等于所述第一右移值与第一临时参考值中索引序号为0对应的值的和值；其中，所述第一临时参考值是基于对所述当前块的相邻采样值进行下采样滤波处理得到的。
在一些实施例中，第一变换单元1203，配置为根据所述预测残差值确定当前块的残差块，对所述残差块进行第一次变换，得到第一变换系数块；
第一确定单元1201，还配置为确定是否对所述第一变换系数块中至少部分变换系数执行LFNST；
第一变换单元1203，还配置为在确定结果为是的情况下，对所述第一变换系数块中至少部分变换系数执行LFNST，确定所述LFNST参数。
在一些实施例中，第一确定单元1201，还配置为在所述当前块的色度分量使用MIP模式确定帧内预测值时，确定所述当前块的尺寸参数中的最小值；以及若所述最小值大于或等于第一预设阈值，则确定对所述第一变换系数块中至少部分变换系数执行LFNST。
在一些实施例中，所述LFNST参数包括LFNST索引序号，第一确定单元1201，还配置为若确定对所述第一变换系数块中至少部分变换系数执行LFNST，则设置所述LFNST索引序号的取值为大于零；若确定不对所述第一变换系数块中至少部分变换系数执行LFNST，则设置所述LFNST索引序号的取值为等于零；
编码单元1204，还配置为对所述LFNST索引序号进行编码，并写入码流。
在一些实施例中，所述LFNST参数还包括LFNST系数，参见图12，编码器120还可以包括量化 单元1206；其中，
第一变换单元1203，配置为对所述第一变换系数块中至少部分变换系数执行LFNST，得到所述LFNST系数；
量化单元1206，配置为对所述LFNST系数进行量化，得到量化系数；
编码单元1204，还配置为对所述量化系数进行编码，并写入码流。
可以理解地，在本申请实施例中，“单元”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是模块，还可以是非模块化的。而且在本实施例中的各组成部分可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
所述集成的单元如果以软件功能模块的形式实现并非作为独立的产品进行销售或使用时，可以存储在一个计算机可读取存储介质中，基于这样的理解，本实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或processor(处理器)执行本实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
因此，本申请实施例提供了一种计算机存储介质，应用于编码器120，该计算机存储介质存储有计算机程序，所述计算机程序被第一处理器执行时实现前述实施例中任一项所述的方法。
基于上述编码器120的组成以及计算机存储介质，参见图13，其示出了本申请实施例提供的编码器120的具体硬件结构示意图。如图13所示，可以包括：第一通信接口1301、第一存储器1302和第一处理器1303；各个组件通过第一总线系统1304耦合在一起。可理解，第一总线系统1304用于实现这些组件之间的连接通信。第一总线系统1304除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图13中将各种总线都标为第一总线系统1304。其中，
第一通信接口1301，用于在与其他外部网元之间进行收发信息过程中，信号的接收和发送；
第一存储器1302，用于存储能够在第一处理器1303上运行的计算机程序；
第一处理器1303，用于在运行所述计算机程序时，执行：
确定当前块的预测参数，其中，所述预测参数包括预测模式参数；
当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；
对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
根据所述当前块的色度分量的帧内预测值，确定所述当前块的色度分量的预测残差值；
对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；
对所述LFNST参数进行编码，并写入码流。
可以理解，本申请实施例中的第一存储器1302可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本申请描述的系统和方法的第一存储器1302旨在包括但不限于这些和任意其它适合类型的存储器。
而第一处理器1303可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过第一处理器1303中的硬件的集成逻辑电路或者软件形式的指令完成。上述的第一处理器1303可以是通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的 公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于第一存储器1302，第一处理器1303读取第一存储器1302中的信息，结合其硬件完成上述方法的步骤。
可以理解的是，本申请描述的这些实施例可以用硬件、软件、固件、中间件、微码或其组合来实现。对于硬件实现，处理单元可以实现在一个或多个专用集成电路(Application Specific Integrated Circuits，ASIC)、数字信号处理器(Digital Signal Processing，DSP)、数字信号处理设备(DSP Device，DSPD)、可编程逻辑设备(Programmable Logic Device，PLD)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)、通用处理器、控制器、微控制器、微处理器、用于执行本申请所述功能的其它电子单元或其组合中。对于软件实现，可通过执行本申请所述功能的模块(例如过程、函数等)来实现本申请所述的技术。软件代码可存储在存储器中并通过处理器执行。存储器可以在处理器中或在处理器外部实现。
可选地，作为另一个实施例，第一处理器1303还配置为在运行所述计算机程序时，执行前述实施例中任一项所述的方法。
本实施例提供了一种编码器，该编码器可以包括第一确定单元、第一预测单元、第一变换单元和编码单元。这样，针对MIP模式，能够在保证编码性能的基础上，降低复杂度，同时减小编码过程中所需要的存储空间，有效地提高编码效率；另外，LFNST技术在应用于MIP模式预测时，由于引入了MIP参数，使得LFNST变换更加灵活，进一步提高了编码效率。
本申请的再一实施例中，基于前述实施例相同的发明构思，参见图14，其示出了本申请实施例提供的一种解码器140的组成结构示意图。如图14所示，该解码器140可以包括：解析单元1401、第二确定单元1402、第二预测单元1403和第二变换单元1404；其中，
解析单元1401，配置为解析码流，获取当前块的预测参数和LFNST参数，其中，所述预测参数包括预测模式参数；
第二确定单元1402，配置为当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
第二预测单元1403，配置为根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；以及为对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
第二变换单元1404，配置为当所述LFNST参数指示对所述当前块执行LFNST时，确定所述当前块的重构变换系数块，对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块；以及对所述第二变换系数块进行第一次变换，得到所述当前块的色度分量的重构残差块；
第二确定单元1402，还配置为根据所述当前块的色度分量的帧内预测值以及所述重构残差块，确定所述当前块的色度分量的重构块。
在一些实施例中，所述当前块的相邻采样值包括所述当前块的左相邻采样值和上相邻采样值。
在一些实施例中，所述移位参数包括移位偏移参数和移位数量参数；
第二确定单元1402，具体配置为根据移位偏移参数的取值，确定所述MIP输入采样值的和值与所述移位偏移参数的乘积，其中，所述移位偏移参数的取值为固定常数；以及根据移位数量参数的取值，确定第一常数值，其中，所述移位数量参数的取值为固定常数；以及将第一偏移量的取值设置为所述第一常数值与所述乘积之差；以及根据所述预测参数，确定所述当前块的MIP加权矩阵；以及根据所述MIP加权矩阵、所述MIP输入采样值、所述移位数量参数和所述第一偏移量，确定所述当前块的色度分量的MIP预测值。
在一些实施例中，所述预测参数还包括：所述当前块的尺寸参数。
在一些实施例中，第二确定单元1402，还配置为根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；以及对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；以及当所述当前块的块尺寸索引值处于预设范围之内时，根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第二常数值与所述第一临时参考值中索引序号为0对应的值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数；当所述当前块的块尺寸索引值处于预设范围之外时，将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第一临时参考值中索引序号等于j+1对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
在一些实施例中，第二确定单元1402，还配置为当所述当前块的块尺寸索引值处于预设范围之内时，根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第一临时参考值中索引序号为0对应的值与所述第二常数值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数。
在一些实施例中，第二确定单元1402，还配置为当所述当前块的宽度和高度均等于4时，将所述当前块的块尺寸索引值设置为0；当所述当前块的宽度和高度均等于8、或者所述当前块的宽度和高度之一等于4时，将所述当前块的块尺寸索引值设置为1；当所述当前块的宽度和高度不满足前述条件时，将所述当前块的块尺寸索引值设置为2。
在一些实施例中，第二确定单元1402，还配置为对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；以及根据所述当前块的相邻采样值的比特深度，确定第二常数值，将所述第二常数值缓存在所述第一临时参考值之后的一个数据单元中，得到第二临时参考值；以及将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第二临时参考值中索引序号等于j+1对应的值与所述第二临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
在一些实施例中，第二确定单元1402，还配置为将所述第二常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述当前块的相邻采样值的比特深度减1。
在一些实施例中，第二确定单元1402，还配置为对“1”进行二进制比特左移，得到所述第二常数值，其中，所述比特左移的位数等于所述当前块的相邻采样值的比特深度减1。
在一些实施例中，所述移位数量参数的取值为与块尺寸索引值和MIP模式索引值无关的固定常数。
在一些实施例中，所述移位数量参数的取值设置为等于6。
在一些实施例中，所述移位偏移参数的取值为与块尺寸索引值和MIP模式索引值无关的固定常数。
在一些实施例中，所述移位偏移参数的取值设置为等于32。
在一些实施例中，第二确定单元1402，还配置为将所述第一常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述移位数量参数的取值减1。
在一些实施例中，第二确定单元1402，还配置为对“1”进行二进制比特左移，得到所述第一常数值，其中，所述比特左移的位数等于所述移位数量参数的取值减1。
在一些实施例中，参见图14，解码器140还可以包括第二计算单元1405，配置为计算所述MIP加权矩阵与所述MIP输入采样值的第一加权和；以及计算所述第一加权和与所述第一偏移量的第一和值；以及对所述第一和值进行二进制比特右移，得到第一右移值，其中，所述右移的位数等于所述移位数量参数的取值；
第二确定单元1402，还配置为将所述当前块的色度分量的MIP预测值设置为等于所述第一右移值与第一临时参考值中索引序号为0对应的值的和值；其中，所述第一临时参考值是基于对所述当前块的相邻采样值进行下采样滤波处理得到的。
在一些实施例中，第二确定单元1402，还配置为确定所述码流中是否存在LFNST参数；
解析单元1401，还配置为在确定所述码流中存在LFNST参数的情况下，解析所述码流，获取LFNST参数。
在一些实施例中，第二确定单元1402，还配置为在所述当前块的色度分量使用MIP模式确定帧内预测值时，确定所述当前块的尺寸参数中的最小值；以及若所述最小值大于或等于第一预设阈值，则确定所述码流中存在LFNST参数。
在一些实施例中，参见图14，解码器140还可以包括反量化单元1406；
解析单元1401，还配置为当所述LFNST参数指示对所述当前块执行LFNST时，解析所述码流，获取所述当前块的量化系数；
反量化单元1406，配置为对所述量化系数进行反量化，得到所述当前块的重构变换系数块。
在一些实施例中，反量化单元1406，具体配置为对所述量化系数进行缩放操作，得到所述当前块的重构变换系数块。
在一些实施例中，解析单元1401，具体配置为解析所述码流，获取LFNST索引序号的取值；以及若所述LFNST索引序号的取值为大于零，则确定对所述当前块执行LFNST；若所述LFNST索引序号的取值为等于零，则确定不对所述当前块执行LFNST。
可以理解地，在本实施例中，“单元”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是模块，还可以是非模块化的。而且在本实施例中的各组成部分可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
所述集成的单元如果以软件功能模块的形式实现并非作为独立的产品进行销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本实施例提供了一种计算机存储介质，应用于解码器140，该计算机存储介质存储有计算机程序，所述计算机程序被第二处理器执行时实现前述实施例中任一项所述的方法。
基于上述解码器140的组成以及计算机存储介质，参见图15，其示出了本申请实施例提供的解码器140的具体硬件结构示意图。如图15所示，可以包括：第二通信接口1501、第二存储器1502和第二处理器1503；各个组件通过第二总线系统1504耦合在一起。可理解，第二总线系统1504用于实现这些组件之间的连接通信。第二总线系统1504除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图15中将各种总线都标为第二总线系统1504。其中，
第二通信接口1501，用于在与其他外部网元之间进行收发信息过程中，信号的接收和发送；
第二存储器1502，用于存储能够在第二处理器1503上运行的计算机程序；
第二处理器1503，用于在运行所述计算机程序时，执行：
解析码流，获取当前块的预测参数和LFNST参数，其中，所述预测参数包括预测模式参数；
当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；
对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
当所述LFNST参数指示对所述当前块执行LFNST时，确定所述当前块的重构变换系数块，对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块；
对所述第二变换系数块进行第一次变换，得到所述当前块的色度分量的重构残差块；
根据所述当前块的色度分量的帧内预测值以及所述重构残差块，确定当前块的色度分量的重构块。
可选地，作为另一个实施例，第二处理器1503还配置为在运行所述计算机程序时，执行前述实施例中任一项所述的方法。
可以理解，第二存储器1502与第一存储器1302的硬件功能类似，第二处理器1503与第一处理器1303的硬件功能类似；这里不再详述。
本实施例提供了一种解码器，该解码器可以包括解析单元、第二确定单元、第二预测单元和第二变换单元。这样，针对MIP模式，能够在保证解码性能的基础上，降低复杂度，同时减小解码过程中所需要的存储空间，有效地提高解码效率；另外，LFNST技术在应用于MIP模式预测时，由于引入了MIP参数，使得LFNST变换更加灵活，进一步提高了解码效率。
本申请的再一实施例中，参见图16，其示出了本申请实施例提供的一种发送设备的组成结构示意图。如图16所示，其展示了示例性的发送设备1600。其中，采集单元1601获取视频信号并将视频信号发送到编码器1602。采集单元1601可以是包含一个或多个相机(包括深度相机)的设备。采集单元1601可以是对码流进行部分或完全解码以获取视频的设备。采集单元1601还可包含一个或多个元件以捕捉音频信号。编码器1602的一种具体实现方式为前述实施例所述的编码器100或者编码器120，对来自采集单元1601的作为其输入视频的视频信号进行编码并生成视频码流。编码器1602还可包含一个或多个音频编码器以对音频信号进行编码而生成音频码流。存储/发送单元1603从编码器1602接收视频码流。存储/发送单元1603还可从编码器1602接收音频码流并将视频码流和音频码流压缩在一起以形成媒体文件(例如基于ISO的媒体文件格式)或传输流。
可选地，存储/发送单元1603将媒体文件或传输流写入存储单元，例如硬盘、DVD盘、云、便携式存储设备。
可选地，存储/发送单元1603将比特流发送到传输网络，例如因特网、有线网络、蜂窝网络、无线局域网等。
本申请的再一实施例中，参见图17，其示出了本申请实施例提供的一种目标设备的组成结构示意图。如图17所示，其展示了示例性的目标设备1700。其中，接收单元1701从网络接收媒体文件或传输流，或者从存储设备读取媒体文件或传输流。接收单元1701将来自媒体文件或传输流的视频码流和音频码流分开。接收单元1701还可通过提取视频码流而生成新的视频码流。接收单元1701还可通过提取音频码流而生成新的音频码流。解码器1702包括一个或多个视频解码器，例如一种具体实施方式为前述实施例所述的解码器200或者解码器140。
解码器1702还可包含一个或多个音频解码器。解码器1702对来自接收单元1701的视频码流和音频码流进行解码，以获取解码视频以及对应于一个或多个声道的一个或多个解码音频。渲染单元1703 对重构视频执行操作，使之适合于显示出来。这样的操作可包括提高感知质量的如下操作中的一个或多个：降噪、合成、颜色空间的转换、上采样、下采样等。渲染单元1703还可对解码音频执行操作，以提高显示出来的音频信号的感知质量。
本申请的再一实施例中，参见图18，其示出了本申请实施例提供的一种通信系统的组成结构示意图。如图18所示，其展示了示例性的通信系统1800。其中，源设备1801可以是图16所示的发送设备1600。存储/发送单元1603的输出被存储介质/传输网络1802处理以存储或传输码流。目标设备1803可以是图17所示的目标设备1700。这里，接收单元1701可从存储介质/传输网络1802获取码流。接收单元1701可从媒体文件或传输流提取新的视频码流。接收单元1701还可从媒体文件或传输流提取新的音频码流。
需要说明的是，在本申请中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
本申请所提供的几个方法实施例中所揭露的方法，在不冲突的情况下可以任意组合，得到新的方法实施例。
本申请所提供的几个产品实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的产品实施例。
本申请所提供的几个方法或设备实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的方法实施例或设备实施例。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。
工业实用性
本申请实施例中，编码器侧，在确定当前块的预测参数后，当预测模式参数指示对当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取当前块的相邻采样值，并根据所述当前块的相邻采样值确定当前块的MIP输入采样值；根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定当前块的色度分量的MIP预测值；对所述MIP预测值进行滤波处理，确定当前块的色度分量的帧内预测值；根据当前块的色度分量的帧内预测值，确定当前块的色度分量的预测残差值；对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；对所述LFNST参数进行编码，并写入码流。解码器侧，在解析码流，获取当前块的预测参数和LFNST参数后，当预测模式参数指示对当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取当前块的相邻采样值，并根据所述当前块的相邻采样值确定当前块的MIP输入采样值；根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定当前块的色度分量的MIP预测值；对MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；当LFNST参数指示对当前块执行LFNST时，确定当前块的重构变换系数块，对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块；对所述第二变换系数块进行第一次变换，得到当前块的色度分量的重构残差块；根据当前块的色度分量的帧内预测值以及所述重构残差块，确定当前块的色度分量的重构块。这样，针对MIP模式，能够在保证编解码性能的基础上，降低复杂度，同时减小编解码过程中所需要的存储空间，有效地提高编解码效率；另外，LFNST技术在应用于MIP模式预测时，由于引入了MIP参数，使得LFNST变换更加灵活，进一步提高了编解码效率。

Claims (48)

  1. 一种编码方法，应用于编码器，所述方法包括：
    确定当前块的预测参数，其中，所述预测参数包括预测模式参数；
    当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
    根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；
    对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
    根据所述当前块的色度分量的帧内预测值，确定所述当前块的色度分量的预测残差值；
    对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；
    对所述LFNST参数进行编码，并写入码流。
  2. 根据权利要求1所述的方法，其中，所述当前块的相邻采样值包括所述当前块的左相邻采样值和上相邻采样值。
  3. 根据权利要求1所述的方法，其中，所述移位参数包括移位偏移参数和移位数量参数；
    相应地，所述根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值，包括：
    根据所述移位偏移参数的取值，确定所述MIP输入采样值的和值与所述移位偏移参数的乘积，其中，所述移位偏移参数的取值为固定常数；
    根据所述移位数量参数的取值，确定第一常数值，其中，所述移位数量参数的取值为固定常数；
    将第一偏移量的取值设置为所述第一常数值与所述乘积之差；
    根据所述预测参数，确定所述当前块的MIP加权矩阵；
    根据所述MIP加权矩阵、所述MIP输入采样值、所述移位数量参数和所述第一偏移量，确定所述当前块的色度分量的MIP预测值。
  4. 根据权利要求3所述的方法，其特征在于，所述预测参数还包括：所述当前块的尺寸参数。
  5. 根据权利要求4所述的方法，其中，所述根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值，包括：
    根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；
    对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；
    当所述当前块的块尺寸索引值处于预设范围之内时，根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第二常数值与所述第一临时参考值中索引序号为0对应的值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数；
    当所述当前块的块尺寸索引值处于预设范围之外时，将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第一临时参考值中索引序号等于j+1对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
  6. 根据权利要求5所述的方法，其中，当所述当前块的块尺寸索引值处于预设范围之内时，所述方法还包括：
    根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第一临时参考值中索引序号为0对应的值与所述第二常数值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数。
  7. 根据权利要求5所述的方法，其中，所述根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值，包括：
    当所述当前块的宽度和高度均等于4时，将所述当前块的块尺寸索引值设置为0；
    当所述当前块的宽度和高度均等于8、或者所述当前块的宽度和高度之一等于4时，将所述当前块的块尺寸索引值设置为1；
    当所述当前块的宽度和高度不满足前述条件时，将所述当前块的块尺寸索引值设置为2。
  8. 根据权利要求3所述的方法，其中，所述根据所述当前块的相邻采样值确定所述当前块的MIP 输入采样值，包括：
    对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；
    根据所述当前块的相邻采样值的比特深度，确定第二常数值，将所述第二常数值缓存在所述第一临时参考值之后的一个数据单元中，得到第二临时参考值；
    将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第二临时参考值中索引序号等于j+1对应的值与所述第二临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
  9. 根据权利要求5至8任一项所述的方法，其中，所述根据所述当前块的相邻采样值的比特深度，确定第二常数值，包括：
    将所述第二常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述当前块的相邻采样值的比特深度减1。
  10. 根据权利要求5至8任一项所述的方法，其中，所述根据所述当前块的相邻采样值的比特深度，确定第二常数值，包括：
    对“1”进行二进制比特左移，得到所述第二常数值，其中，所述比特左移的位数等于所述当前块的相邻采样值的比特深度减1。
  11. 根据权利要求3所述的方法，其中，所述移位数量参数的取值为与块尺寸索引值和MIP模式索引值无关的固定常数。
  12. 根据权利要求11所述的方法，其中，所述移位数量参数的取值设置为等于6。
  13. 根据权利要求3所述的方法，其中，所述移位偏移参数的取值为与块尺寸索引值和MIP模式索引值无关的固定常数。
  14. 根据权利要求13所述的方法，其中，所述移位偏移参数的取值设置为等于32。
  15. 根据权利要求3所述的方法，其中，所述根据移位数量参数的取值，确定第一常数值，包括：
    将所述第一常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述移位数量参数的取值减1。
  16. 根据权利要求3所述的方法，其中，所述根据移位数量参数的取值，确定第一常数值，包括：
    对“1”进行二进制比特左移，得到所述第一常数值，其中，所述比特左移的位数等于所述移位数量参数的取值减1。
  17. 根据权利要求3所述的方法，其中，所述根据所述MIP加权矩阵、所述MIP输入采样值、所述移位数量参数和所述第一偏移量，确定所述当前块的色度分量的MIP预测值，包括：
    计算所述MIP加权矩阵与所述MIP输入采样值的第一加权和；
    计算所述第一加权和与所述第一偏移量的第一和值；
    对所述第一和值进行二进制比特右移，得到第一右移值，其中，所述右移的位数等于所述移位数量参数的取值；
    将所述当前块的色度分量的MIP预测值设置为等于所述第一右移值与第一临时参考值中索引序号为0对应的值的和值；其中，所述第一临时参考值是基于对所述当前块的相邻采样值进行下采样滤波处理得到的。
  18. 根据权利要求1所述的方法，其中，所述对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数，包括：
    根据所述预测残差值确定所述当前块的残差块，对所述残差块进行第一次变换，得到第一变换系数块；
    确定是否对所述第一变换系数块中至少部分变换系数执行LFNST；
    在确定结果为是的情况下，对所述第一变换系数块中至少部分变换系数执行LFNST，确定所述LFNST参数。
  19. 根据权利要求18所述的方法，其中，所述确定是否对所述第一变换系数块中至少部分变换系数执行LFNST，包括：
    在所述当前块的色度分量使用MIP模式确定帧内预测值时，确定所述当前块的尺寸参数中的最小值；
    若所述最小值大于或等于第一预设阈值，则确定对所述第一变换系数块中至少部分变换系数执行LFNST。
  20. 根据权利要求18所述的方法，其中，所述LFNST参数包括LFNST索引序号，所述确定LFNST参数，包括：
    若确定对所述第一变换系数块中至少部分变换系数执行LFNST，则设置所述LFNST索引序号的取 值为大于零；
    若确定不对所述第一变换系数块中至少部分变换系数执行LFNST，则设置所述LFNST索引序号的取值为等于零；
    相应地，所述对所述LFNST参数进行编码，并写入码流，包括：
    对所述LFNST索引序号进行编码，并写入码流。
  21. 根据权利要求18所述的方法，其中，所述LFNST参数还包括LFNST系数，所述确定LFNST参数，包括：
    对所述第一变换系数块中至少部分变换系数执行LFNST，得到所述LFNST系数；
    相应地，所述对所述LFNST参数进行编码，并写入码流，包括：
    对所述LFNST系数进行量化，得到量化系数；
    对所述量化系数进行编码，并写入码流。
  22. 一种解码方法，应用于解码器，所述方法包括：
    解析码流，获取当前块的预测参数和LFNST参数，其中，所述预测参数包括预测模式参数；
    当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
    根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；
    对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
    当所述LFNST参数指示对所述当前块执行LFNST时，确定所述当前块的重构变换系数块，对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块；
    对所述第二变换系数块进行第一次变换，得到所述当前块的色度分量的重构残差块；
    根据所述当前块的色度分量的帧内预测值以及所述重构残差块，确定所述当前块的色度分量的重构块。
  23. 根据权利要求22所述的方法，其中，所述当前块的相邻采样值包括所述当前块的左相邻采样值和上相邻采样值。
  24. 根据权利要求22所述的方法，其中，所述移位参数包括移位偏移参数和移位数量参数；
    相应地，所述根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值，包括：
    根据所述移位偏移参数的取值，确定所述MIP输入采样值的和值与所述移位偏移参数的乘积，其中，所述移位偏移参数的取值为固定常数；
    根据所述移位数量参数的取值，确定第一常数值，其中，所述移位数量参数的取值为固定常数；
    将第一偏移量的取值设置为所述第一常数值与所述乘积之差；
    根据所述预测参数，确定所述当前块的MIP加权矩阵；
    根据所述MIP加权矩阵、所述MIP输入采样值、所述移位数量参数和所述第一偏移量，确定所述当前块的色度分量的MIP预测值。
  25. 根据权利要求24所述的方法，其特征在于，所述预测参数还包括：所述当前块的尺寸参数。
  26. 根据权利要求25所述的方法，其中，所述根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值，包括：
    根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值；
    对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；
    当所述当前块的块尺寸索引值处于预设范围之内时，根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号为0对应的值设置为等于所述第二常数值与所述第一临时参考值中索引序号为0对应的值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数；
    当所述当前块的块尺寸索引值处于预设范围之外时，将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第一临时参考值中索引序号等于j+1对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
  27. 根据权利要求26所述的方法，其中，当所述当前块的块尺寸索引值处于预设范围之内时，所述方法还包括：
    根据所述当前块的相邻采样值的比特深度，确定第二常数值；将所述MIP输入采样值中索引序号 为0对应的值设置为等于所述第一临时参考值中索引序号为0对应的值与所述第二常数值之间的差值；将所述MIP输入采样值中索引序号为i对应的值设置为等于所述第一临时参考值中索引序号为i对应的值与所述第一临时参考值中索引序号等于0对应的值之间的差值，其中，i是大于0的整数。
  28. 根据权利要求26所述的方法，其中，所述根据所述当前块的尺寸参数，确定所述当前块的块尺寸索引值，包括：
    当所述当前块的宽度和高度均等于4时，将所述当前块的块尺寸索引值设置为0；
    当所述当前块的宽度和高度均等于8、或者所述当前块的宽度和高度之一等于4时，将所述当前块的块尺寸索引值设置为1；
    当所述当前块的宽度和高度不满足前述条件时，将所述当前块的块尺寸索引值设置为2。
  29. 根据权利要求24所述的方法，其中，所述根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值，包括：
    对所述当前块的相邻采样值进行下采样滤波处理，得到第一临时参考值；
    根据所述当前块的相邻采样值的比特深度，确定第二常数值，将所述第二常数值缓存在所述第一临时参考值之后的一个数据单元中，得到第二临时参考值；
    将所述MIP输入采样值中索引序号等于j对应的值设置为等于所述第二临时参考值中索引序号等于j+1对应的值与所述第二临时参考值中索引序号等于0对应的值之间的差值，其中，j是大于或等于0的整数。
  30. 根据权利要求26至29任一项所述的方法，其中，所述根据所述当前块的相邻采样值的比特深度，确定第二常数值，包括：
    将所述第二常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述当前块的相邻采样值的比特深度减1。
  31. 根据权利要求26至29任一项所述的方法，其中，所述根据所述当前块的相邻采样值的比特深度，确定第二常数值，包括：
    对“1”进行二进制比特左移，得到所述第二常数值，其中，所述比特左移的位数等于所述当前块的相邻采样值的比特深度减1。
  32. 根据权利要求24所述的方法，其中，所述移位数量参数的取值为与块尺寸索引值和MIP模式索引值无关的固定常数。
  33. 根据权利要求32所述的方法，其中，所述移位数量参数的取值设置为等于6。
  34. 根据权利要求24所述的方法，其中，所述移位偏移参数的取值为与块尺寸索引值和MIP模式索引值无关的固定常数。
  35. 根据权利要求34所述的方法，其中，所述移位偏移参数的取值设置为等于32。
  36. 根据权利要求24所述的方法，其中，所述根据移位数量参数的取值，确定第一常数值，包括：
    将所述第一常数值设置为等于2的整数指数幂，其中，所述幂的指数等于所述移位数量参数的取值减1。
  37. 根据权利要求24所述的方法，其中，所述根据移位数量参数的取值，确定第一常数值，包括：
    对“1”进行二进制比特左移，得到所述第一常数值，其中，所述比特左移的位数等于所述移位数量参数的取值减1。
  38. 根据权利要求24所述的方法，其中，所述根据所述MIP加权矩阵、所述MIP输入采样值、所述移位数量参数和所述第一偏移量，确定所述当前块的色度分量的MIP预测值，包括：
    计算所述MIP加权矩阵与所述MIP输入采样值的第一加权和；
    计算所述第一加权和与所述第一偏移量的第一和值；
    对所述第一和值进行二进制比特右移，得到第一右移值，其中，所述右移的位数等于所述移位数量参数的取值；
    将所述当前块的色度分量的MIP预测值设置为等于所述第一右移值与第一临时参考值中索引序号为0对应的值的和值；其中，所述第一临时参考值是基于对所述当前块的相邻采样值进行下采样滤波处理得到的。
  39. 根据权利要求22所述的方法，其中，所述解析码流，获取LFNST参数，包括：
    确定所述码流中是否存在LFNST参数；
    在确定所述码流中存在LFNST参数的情况下，解析所述码流，获取LFNST参数。
  40. 根据权利要求39所述的方法，其中，所述确定所述码流中是否存在LFNST参数，包括：
    在所述当前块的色度分量使用MIP模式确定帧内预测值时，确定所述当前块的尺寸参数中的最小值；
    若所述最小值大于或等于第一预设阈值，则确定所述码流中存在LFNST参数。
  41. 根据权利要求22所述的方法，其中，当所述LFNST参数指示对所述当前块执行LFNST时，所述确定所述当前块的重构变换系数块，包括：
    解析所述码流，获取所述当前块的量化系数；
    对所述量化系数进行反量化，得到所述当前块的重构变换系数块。
  42. 根据权利要求41所述的方法，其中，所述对所述量化系数进行反量化，得到所述当前块的重构变换系数块，包括：
    对所述量化系数进行缩放操作，得到所述当前块的重构变换系数块。
  43. 根据权利要求22所述的方法，其中，所述解析码流，获取LFNST参数，包括：
    解析所述码流，获取LFNST索引序号的取值；
    若所述LFNST索引序号的取值为大于零，则确定对所述当前块执行LFNST；
    若所述LFNST索引序号的取值为等于零，则确定不对所述当前块执行LFNST。
  44. 一种编码器，所述编码器包括第一确定单元、第一预测单元、第一变换单元和编码单元；其中，
    所述第一确定单元，配置为确定当前块的预测参数，其中，所述预测参数包括预测模式参数；
    所述第一确定单元，还配置为当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
    所述第一预测单元，配置为根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；以及对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
    所述第一变换单元，配置为根据所述当前块的色度分量的帧内预测值，确定所述当前块的色度分量的预测残差值；以及对所述预测残差值执行低频不可分离二次变换LFNST，确定LFNST参数；
    所述编码单元，配置为对所述LFNST参数进行编码，并写入码流。
  45. 一种编码器，所述编码器包括第一存储器和第一处理器；其中，
    所述第一存储器，用于存储能够在所述第一处理器上运行的计算机程序；
    所述第一处理器，用于在运行所述计算机程序时，执行如权利要求1至21任一项所述的方法。
  46. 一种解码器，所述解码器包括解析单元、第二确定单元、第二预测单元和第二变换单元；其中，
    所述解析单元，配置为解析码流，获取当前块的预测参数和LFNST参数，其中，所述预测参数包括预测模式参数；
    所述第二确定单元，配置为当所述预测模式参数指示对所述当前块的色度分量使用基于矩阵的帧内预测MIP模式确定帧内预测值时，获取所述当前块的相邻采样值，并根据所述当前块的相邻采样值确定所述当前块的MIP输入采样值；
    所述第二预测单元，配置为根据所述MIP输入采样值、MIP加权矩阵和移位参数，确定所述当前块的色度分量的MIP预测值；其中，所述MIP预测值是所述当前块的色度分量中部分采样点的预测值；以及为对所述MIP预测值进行滤波处理，确定所述当前块的色度分量的帧内预测值；
    所述第二变换单元，配置为当所述LFNST参数指示对所述当前块执行LFNST时，确定所述当前块的重构变换系数块，对所述重构变换系数块中至少部分重构变换系数执行LFNST，得到第二变换系数块；以及对所述第二变换系数块进行第一次变换，得到所述当前块的色度分量的重构残差块；
    所述第二确定单元，还配置为根据所述当前块的色度分量的帧内预测值以及所述重构残差块，确定所述当前块的色度分量的重构块。
  47. 一种解码器，所述解码器包括第二存储器和第二处理器；其中，
    所述第二存储器，用于存储能够在所述第二处理器上运行的计算机程序；
    所述第二处理器，用于在运行所述计算机程序时，执行如权利要求22至43任一项所述的方法。
  48. 一种计算机存储介质，其中，所述计算机存储介质存储有计算机程序，所述计算机程序被第一处理器执行时实现如权利要求1至21任一项所述的方法、或者被第二处理器执行时实现如权利要求22至43任一项所述的方法。
PCT/CN2020/141080 2020-01-08 2020-12-29 编码方法、解码方法、编码器、解码器以及存储介质 WO2021139572A1 (zh)

Priority Applications (8)

Application Number Priority Date Filing Date Title
CN202210689703.4A CN115314710A (zh) 2020-01-08 2020-12-29 编码方法、解码方法、编码器、解码器以及存储介质
EP20912114.4A EP4090014A4 (en) 2020-01-08 2020-12-29 ENCODING METHOD, DECODING METHOD, ENCODER, DECODER, AND STORAGE MEDIUM
JP2022541305A JP2023509951A (ja) 2020-01-08 2020-12-29 符号化方法、復号化方法、符号器、復号器及び記憶媒体
CN202080068055.XA CN114450945A (zh) 2020-01-08 2020-12-29 编码方法、解码方法、编码器、解码器以及存储介质
KR1020227023785A KR20220122666A (ko) 2020-01-08 2020-12-29 인코딩 방법, 디코딩 방법, 인코더, 디코더 및 저장 매체
US17/858,161 US11843781B2 (en) 2020-01-08 2022-07-06 Encoding method, decoding method, and decoder
US18/494,947 US20240056582A1 (en) 2020-01-08 2023-10-26 Encoding method, decoding method, and decoder
US18/494,880 US20240056581A1 (en) 2020-01-08 2023-10-26 Encoding method, decoding method, and decoder

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202062958582P 2020-01-08 2020-01-08
US62/958,582 2020-01-08

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/858,161 Continuation US11843781B2 (en) 2020-01-08 2022-07-06 Encoding method, decoding method, and decoder

Publications (1)

Publication Number Publication Date
WO2021139572A1 true WO2021139572A1 (zh) 2021-07-15

Family

ID=76788400

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/141080 WO2021139572A1 (zh) 2020-01-08 2020-12-29 编码方法、解码方法、编码器、解码器以及存储介质

Country Status (6)

Country Link
US (3) US11843781B2 (zh)
EP (1) EP4090014A4 (zh)
JP (1) JP2023509951A (zh)
KR (1) KR20220122666A (zh)
CN (2) CN115314710A (zh)
WO (1) WO2021139572A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023044919A1 (zh) * 2021-09-27 2023-03-30 Oppo广东移动通信有限公司 视频编解码方法、设备、系统、及存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021165018A1 (en) * 2020-02-21 2021-08-26 Interdigital Vc Holdings France Neural network-based intra prediction for video encoding or decoding
WO2024007120A1 (zh) * 2022-07-04 2024-01-11 Oppo广东移动通信有限公司 编解码方法、编码器、解码器以及存储介质
CN116600107B (zh) * 2023-07-20 2023-11-21 华侨大学 基于ipms-cnn和空域相邻cu编码模式的hevc-scc快速编码方法及装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019172676A1 (ko) * 2018-03-08 2019-09-12 삼성전자 주식회사 비디오 복호화 방법 및 장치, 비디오 부호화 방법 및 장치
CN110519591A (zh) * 2019-08-29 2019-11-29 中南大学 一种基于多用途编码中帧内编码的预测模式快速选择方法
CN110546952A (zh) * 2017-03-21 2019-12-06 Lg电子株式会社 图像编码系统中的变换方法和用于该变换方法的设备

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102695061B (zh) 2011-03-20 2015-01-21 华为技术有限公司 一种权重因子的确定方法和装置,以及一种帧内加权预测方法和装置
US20130003856A1 (en) * 2011-07-01 2013-01-03 Samsung Electronics Co. Ltd. Mode-dependent transforms for residual coding with low latency
CN114339227B (zh) * 2016-05-04 2024-04-12 夏普株式会社 用于对变换数据进行编码的系统和方法
CN106254883B (zh) 2016-08-02 2021-01-22 海信视像科技股份有限公司 一种视频解码中的反变换方法和装置
US11818340B2 (en) * 2018-07-06 2023-11-14 Electronics And Telecommunications Research Institute Image encoding/decoding method and device, and recording medium in which bitstream is stored
WO2020091213A1 (ko) * 2018-11-04 2020-05-07 엘지전자 주식회사 영상 코딩 시스템에서 인트라 예측 방법 및 장치

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110546952A (zh) * 2017-03-21 2019-12-06 Lg电子株式会社 图像编码系统中的变换方法和用于该变换方法的设备
WO2019172676A1 (ko) * 2018-03-08 2019-09-12 삼성전자 주식회사 비디오 복호화 방법 및 장치, 비디오 부호화 방법 및 장치
CN110519591A (zh) * 2019-08-29 2019-11-29 中南大学 一种基于多用途编码中帧内编码的预测模式快速选择方法

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
A. K. RAMASUBRAMONIAN (QUALCOMM), G. VAN DER AUWERA (QUALCOMM), T. HSIEH (QUALCOMM), T. BIATEK (QUALCOMM), L. PHAM VAN (QUALCOMM),: "Non-CE3: On signalling of MIP parameters", 15. JVET MEETING; 20190703 - 20190712; GOTHENBURG; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ), no. JVET-O0755, 29 June 2019 (2019-06-29), XP030220324 *
AKLOUF MOURAD; LENY MARC; DUFAUX FREDERIC; KIEFFER MICHEL: "Low Complexity Versatile Video Coding (VVC) for Low Bitrate Applications", 2019 8TH EUROPEAN WORKSHOP ON VISUAL INFORMATION PROCESSING (EUVIP), IEEE, 28 October 2019 (2019-10-28), pages 22 - 27, XP033682775, DOI: 10.1109/EUVIP47703.2019.8946261 *
See also references of EP4090014A4

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023044919A1 (zh) * 2021-09-27 2023-03-30 Oppo广东移动通信有限公司 视频编解码方法、设备、系统、及存储介质

Also Published As

Publication number Publication date
US20240056582A1 (en) 2024-02-15
US20220360785A1 (en) 2022-11-10
CN114450945A (zh) 2022-05-06
KR20220122666A (ko) 2022-09-02
EP4090014A1 (en) 2022-11-16
US20240056581A1 (en) 2024-02-15
US11843781B2 (en) 2023-12-12
EP4090014A4 (en) 2024-01-03
JP2023509951A (ja) 2023-03-10
CN115314710A (zh) 2022-11-08

Similar Documents

Publication Publication Date Title
CN113411577B (zh) 编码方法及装置
WO2021139572A1 (zh) 编码方法、解码方法、编码器、解码器以及存储介质
WO2021004152A1 (zh) 图像分量的预测方法、编码器、解码器以及存储介质
JP2017538381A (ja) ビデオ符号化における成分間予測
CN111819852A (zh) 用于变换域中残差符号预测的方法及装置
WO2020103800A1 (zh) 视频解码方法和视频解码器
CN110754085B (zh) 用于非4:4:4格式视频内容的颜色重映射
CN111327904B (zh) 图像重建方法和装置
WO2021004155A1 (zh) 图像分量预测方法、编码器、解码器以及存储介质
TW202112135A (zh) 用於視訊寫碼之色度內預測單元
CN117241014A (zh) Mpm列表构建方法、色度块的帧内预测模式获取方法及装置
JP7393366B2 (ja) 画像の符号化方法、復号化方法、エンコーダおよびデコーダ
CN112616057A (zh) 图像预测方法、编码器、解码器以及存储介质
WO2023044868A1 (zh) 视频编解码方法、设备、系统、及存储介质
CN112532997B (zh) 图像预测方法、编码器、解码器以及存储介质
WO2021134303A1 (zh) 变换方法、编码器、解码器以及存储介质
WO2021134327A1 (zh) 变换方法、编码器、解码器以及存储介质
WO2024098263A1 (zh) 编解码方法、码流、编码器、解码器以及存储介质
RU2815738C2 (ru) Определение режима кодирования цветности на основе внутрикадрового предсказания на основе матрицы
WO2023184747A1 (zh) 视频编解码方法、装置、设备、系统及存储介质
WO2022155922A1 (zh) 视频编解码方法与系统、及视频编码器与视频解码器
WO2022116105A1 (zh) 视频编解码方法与系统、及视频编码器与视频解码器
WO2023044919A1 (zh) 视频编解码方法、设备、系统、及存储介质
WO2023197195A1 (zh) 视频编解码方法、编码器、解码器及存储介质
WO2021134637A1 (zh) 变换方法、编码器、解码器以及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20912114

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022541305

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20227023785

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020912114

Country of ref document: EP

Effective date: 20220808